Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Kaoru0822
|
7aee9900da | ||
|
Kaoru0822andClaude Opus 5.5
|
e1def2bf1a |
@@ -25,7 +25,6 @@ node scripts/release-notes.mjs inventory \
|
||||
--manifest /tmp/rea-VERSION.contributions.json
|
||||
node scripts/release-notes.mjs record \
|
||||
--manifest /tmp/rea-VERSION.contributions.json \
|
||||
--summary \
|
||||
--output /tmp/rea-VERSION.record.md
|
||||
```
|
||||
|
||||
@@ -57,31 +56,25 @@ version/date/link heading. Use single-line Markdown list items under:
|
||||
PR/commit references required by the existing checkpoint validator.
|
||||
|
||||
Group related work by analyst workflow or user impact. Explain the resulting
|
||||
behavior in short, concrete entries. Highlights summarize the release; avoid
|
||||
repeating their full descriptions in Changes. Internal cleanup, tests, CI, and
|
||||
routine documentation generally belong only in the provenance file. Read ambiguous PRs/diffs before making
|
||||
behavior; internal cleanup, tests, CI, and routine documentation generally
|
||||
belong only in the contribution record. Read ambiguous PRs/diffs before making
|
||||
claims. Distinguish simulated-provider checks from real-provider verification.
|
||||
|
||||
Credit each represented contribution once in the public notes with its link
|
||||
and `Thanks @...` for its verified human author, known co-authors, and confirmed
|
||||
closing-issue reporters. Grouping preserves all those credits in one line.
|
||||
When Highlights, detailed entries or migrations reference the same work,
|
||||
put its thanks in one of those entries, not all of them. Include issue links.
|
||||
Direct-commit bullets carry the commit link and known human credit.
|
||||
Each bullet covering a PR carries its link and `Thanks @...` for its verified
|
||||
human author, known co-authors, and confirmed closing-issue reporters. Grouping
|
||||
must preserve **all** those credits in the same bullet. Include the issue
|
||||
links. Direct-commit bullets carry the commit link and known human credit.
|
||||
Exclude GitHub bots and known agent accounts. REA's observed Anthropic and
|
||||
Cursor co-author trailers resolve to `@claude` and `@cursoragent` with GitHub
|
||||
type `User`; the credit policy excludes them while retaining their metadata.
|
||||
At the REA maintainer's request, omit `@morluto` self-thanks while preserving
|
||||
authorship in the inventory. Other human contributors remain eligible.
|
||||
Do not copy OpenClaw's maintainer-specific exclusions.
|
||||
Use explicit GitHub links for cross-repository references to avoid ambiguous
|
||||
`#NNN` numbers. Unknown handles stay unknown; they are not fabricated credit.
|
||||
|
||||
Append the generated `record --summary` provenance link verbatim at the end
|
||||
of this release. The linked inventory accounts for every discovered in-range
|
||||
PR and direct contribution, including internal work omitted from the prose.
|
||||
Do not paste the full `record` output into public notes: it duplicates thanks
|
||||
and obscures the user-facing changes. Full records remain available for review.
|
||||
Do not hand-edit generated credits to make validation pass. Refresh the inventory
|
||||
Append the generated `### Complete contribution record` verbatim as the last
|
||||
section of this release. It accounts for every discovered in-range PR and
|
||||
direct contribution, including internal work omitted from the prose. Do not
|
||||
hand-edit generated credits to make validation pass. Refresh the inventory
|
||||
when product history changes; a prose-only correction can reuse the snapshot.
|
||||
|
||||
## Finalize with Release Please
|
||||
|
||||
@@ -135,16 +135,6 @@ session to a selected path. The analysis remains complete in the ledger; avoid
|
||||
repeating it merely to request the same oversized response. Complete CLI JSON
|
||||
output also streams without a single MCP frame.
|
||||
|
||||
When the retained parent ID is unknown, use `get_evidence_bundle` with
|
||||
`detail: "summary"` and exact filters for operation, target SHA-256,
|
||||
analysis-profile digest or canonical procedure address. Select its returned
|
||||
Evidence ID for the next view instead of requesting the complete bundle.
|
||||
`retention: "complete-record"` means the whole record is stored; native
|
||||
value-flow truncation and unknown metadata remain separate. For an exported
|
||||
portable bundle, use `rea inspect-evidence-bundle` with JSON containing `path`,
|
||||
`detail: "summary"` and the same `filters`. Reading that file still requires
|
||||
validating its full contents within the local memory budget.
|
||||
|
||||
A client can also truncate a successful tool result without a REA transport
|
||||
error. Treat that preview as incomplete; use the retained parent ID for smaller
|
||||
views or export when the full record is needed. Do not parse the preview as
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
{
|
||||
".": "6.4.0"
|
||||
".": "6.3.0"
|
||||
}
|
||||
|
||||
@@ -1,69 +1,5 @@
|
||||
# Changelog
|
||||
|
||||
## [6.4.0](https://github.com/morluto/rea/compare/rea-agents-6.3.0...rea-agents-6.4.0) (2026-10-11)
|
||||
|
||||
### Highlights
|
||||
|
||||
- Android: added device inspection, split-APK acquisition, logs, screenshots, file transfers and explicit app lifecycle actions through ADB, plus manifest and localized-string decoding with Apktool. ([#1552](https://github.com/morluto/rea/pull/1552), [#1556](https://github.com/morluto/rea/pull/1556)) Thanks @3QL.
|
||||
- JEB: browse project units and decompile selected items through a caller-started local JEB MCP server. ([#1536](https://github.com/morluto/rea/pull/1536)) Thanks @3QL.
|
||||
- Large analyses: discover retained Evidence without loading every payload, and inspect JavaScript graphs, binary layouts and function dossiers through summaries and selected views. ([#1682](https://github.com/morluto/rea/pull/1682), [#1568](https://github.com/morluto/rea/pull/1568), [#1570](https://github.com/morluto/rea/pull/1570), [#1345](https://github.com/morluto/rea/pull/1345), [#1507](https://github.com/morluto/rea/pull/1507), [#1247](https://github.com/morluto/rea/issues/1247)) Thanks @jhuang-tw, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- JavaScript: isolate source analysis in an owned process with configurable heap and timeout limits; retain completed facts and explicit unknowns when analysis runs out of resources. ([#1651](https://github.com/morluto/rea/pull/1651), [#1563](https://github.com/morluto/rea/issues/1563), [#1565](https://github.com/morluto/rea/issues/1565)) Thanks @N0zoM1z0, @peter-research.
|
||||
- Ghidra: configure import language/compiler specifications and seed functions, code and labels before auto-analysis. ([#1667](https://github.com/morluto/rea/pull/1667)) Thanks @sqmagellan.
|
||||
- RPC discovery: recognize supported tRPC calls, HTTP-link URLs and named GraphQL operations in JavaScript applications and web bundles. ([#1670](https://github.com/morluto/rea/pull/1670)) Thanks @pompiliodonatoapp.
|
||||
|
||||
### Changes
|
||||
|
||||
- Added Ghidra support for Windows PE DLLs and bounded MIPS ELF analysis, including carved MIPS inputs with incomplete section tables. Windows P0 remains read-only. ([#1303](https://github.com/morluto/rea/pull/1303), [#1362](https://github.com/morluto/rea/pull/1362), [#1629](https://github.com/morluto/rea/pull/1629), [#1290](https://github.com/morluto/rea/issues/1290), [#718](https://github.com/morluto/rea/issues/718), [#1619](https://github.com/morluto/rea/issues/1619)) Thanks @bqw371, @jaelliot, @kvalls, @N0zoM1z0.
|
||||
- Enabled `set_address_name` and `set_addresses_names` for Ghidra sessions with mutation authority, including namespace-aware data labels and per-address batch results. ([#1635](https://github.com/morluto/rea/pull/1635), [#1634](https://github.com/morluto/rea/issues/1634)) Thanks @giri256.
|
||||
- Recovered additional signed AArch64 relative jump-table patterns. ([#1592](https://github.com/morluto/rea/pull/1592)) Thanks @RishTheFish21.
|
||||
- Added PE resource inspection and read-only NativeAOT metadata parsing for supported x64 PE32+/.NET 8 RTR 9.1 inputs. The NativeAOT parser’s Windows-host Ghidra workflow remains unverified on a real Windows host. ([#1482](https://github.com/morluto/rea/pull/1482), [76d8f68](https://github.com/morluto/rea/commit/76d8f68cb918dfb43869f2b649a0b6c0c1cf266d), [#1254](https://github.com/morluto/rea/issues/1254)) Thanks @N0zoM1z0, @thangldw.
|
||||
- Added Pi, Hermes and Qwen Code setup, including Hermes active-profile support. ([#1376](https://github.com/morluto/rea/pull/1376), [#1403](https://github.com/morluto/rea/pull/1403), [#1415](https://github.com/morluto/rea/pull/1415), [#1409](https://github.com/morluto/rea/issues/1409)) Thanks @rasatpetabit, @yiliang114, @zaydiscold.
|
||||
- Added `REA_MCP_INPUT_SCHEMA_PROFILE=compact` for size-capped MCP clients such as Kimi/Moonshot, while retaining canonical runtime validation. ([#1514](https://github.com/morluto/rea/pull/1514), [#1580](https://github.com/morluto/rea/pull/1580), [#1484](https://github.com/morluto/rea/issues/1484), [#1581](https://github.com/morluto/rea/issues/1581)) Thanks @haythemsellami, @N0zoM1z0.
|
||||
- Added live process-capture progress and optional `finalization_ms` to capture shutdown output before timeout escalation; cancellation remains immediate. ([#1483](https://github.com/morluto/rea/pull/1483), [#1583](https://github.com/morluto/rea/pull/1583), [#1252](https://github.com/morluto/rea/issues/1252), [#1253](https://github.com/morluto/rea/issues/1253), [modelcontextprotocol/typescript-sdk#2997](https://github.com/modelcontextprotocol/typescript-sdk/issues/2997)) Thanks @N0zoM1z0, @nikolay-dementiev, @sathariels.
|
||||
- Improved Codex onboarding and recovery from observed client, PATH and provider blockers. ([#1613](https://github.com/morluto/rea/pull/1613)) Thanks @N0zoM1z0.
|
||||
- Reduced redirected JSON and terminal output size, and deferred heavy imports until their commands need them. ([#1413](https://github.com/morluto/rea/pull/1413), [#1450](https://github.com/morluto/rea/pull/1450), [#1439](https://github.com/morluto/rea/pull/1439), [21dbcf7](https://github.com/morluto/rea/commit/21dbcf7a1458219989ad291e44ce7d017378b727)) Thanks @kaoru0822-kitauji.
|
||||
|
||||
### Fixes
|
||||
|
||||
- Electron analysis: follow renamed and default-imported APIs and optional calls, ignore overwritten identities and media preload attributes, and resolve packaged `app.asar` pages and unpacked companions. ([#1391](https://github.com/morluto/rea/pull/1391), [#1393](https://github.com/morluto/rea/pull/1393), [#1400](https://github.com/morluto/rea/pull/1400), [#1664](https://github.com/morluto/rea/pull/1664), [#1673](https://github.com/morluto/rea/pull/1673)) Thanks @kaoru0822-kitauji, @marnixpluim, @N0zoM1z0, @suninrain086.
|
||||
- JavaScript data flow: preserve returned/yielded reference uncertainty, invalidate property-stored aliases correctly, and retain bundled-module runtime scope. ([#1437](https://github.com/morluto/rea/pull/1437), [#1445](https://github.com/morluto/rea/pull/1445), [#1471](https://github.com/morluto/rea/pull/1471), [#1577](https://github.com/morluto/rea/pull/1577), [#1428](https://github.com/morluto/rea/issues/1428), [#1442](https://github.com/morluto/rea/issues/1442), [#1449](https://github.com/morluto/rea/issues/1449), [#1576](https://github.com/morluto/rea/issues/1576)) Thanks @CassieFox23, @dakjdakd, @N0zoM1z0.
|
||||
- JavaScript packages: resolve exact exports subpaths, preserve trailing-slash directory requests and exact CommonJS keys, and respect shadowed `require` bindings. ([#1424](https://github.com/morluto/rea/pull/1424), [#1615](https://github.com/morluto/rea/pull/1615), [5485483](https://github.com/morluto/rea/commit/5485483f7da2f6cb66441bb784aaf2a1a5f8731a), [8b2f851](https://github.com/morluto/rea/commit/8b2f8514c50e469c3add8ce9ba7ebebef888b14f), [f998fcb](https://github.com/morluto/rea/commit/f998fcb1cca9994c3589fb70d6e0127fc215f76f), [1e51c73](https://github.com/morluto/rea/commit/1e51c732bbf08191688d6526bf58ab2650b601e3), [#1365](https://github.com/morluto/rea/issues/1365)) Thanks @akshitkhokhani, @AWUQAQ, @immanuwell, @rudycelekli.
|
||||
- Large JavaScript inputs: handle oversized inline source maps, deep returns and reassignment loops; avoid argument expansion limits and yield during large analysis passes. ([#1386](https://github.com/morluto/rea/pull/1386), [#1368](https://github.com/morluto/rea/pull/1368), [#1390](https://github.com/morluto/rea/pull/1390), [#1539](https://github.com/morluto/rea/pull/1539), [#1525](https://github.com/morluto/rea/pull/1525), [#1542](https://github.com/morluto/rea/pull/1542), [#1545](https://github.com/morluto/rea/pull/1545), [#1557](https://github.com/morluto/rea/pull/1557), [#1380](https://github.com/morluto/rea/issues/1380), [#1401](https://github.com/morluto/rea/issues/1401), [#1538](https://github.com/morluto/rea/issues/1538), [#1462](https://github.com/morluto/rea/issues/1462)) Thanks @cold-atom, @KadirJeager, @kaoru0822-kitauji, @marnixpluim, @N0zoM1z0, @rudycelekli.
|
||||
- JavaScript receivers: expand escaped methods once, index returned-object targets, and avoid replaying earlier writes or duplicating retained graph allocations. ([#1517](https://github.com/morluto/rea/pull/1517), [#1537](https://github.com/morluto/rea/pull/1537), [#1444](https://github.com/morluto/rea/pull/1444), [#1601](https://github.com/morluto/rea/pull/1601), [#1495](https://github.com/morluto/rea/issues/1495)) Thanks @CassieFox23, @geekwolf, @kaoru0822-kitauji, @N0zoM1z0, @zeynelacikgoez.
|
||||
- Browser captures: retain WebSocket URLs, empty connections and frame order; compare only common known payload coverage and preserve incomplete WebMCP replay coverage. ([db4e360](https://github.com/morluto/rea/commit/db4e3608465486466591b594cf77037ac150c988), [aa257ea](https://github.com/morluto/rea/commit/aa257ea4b16f2d2a35c12df63359d650b48b1154), [e52c0a7](https://github.com/morluto/rea/commit/e52c0a72c59c740315e89149881a27ce4d89a11c))
|
||||
- Browser scenarios: seed storage in the correct partition, normalize loopback attach identities, retain declared-secret redaction and require verified lineage before Windows Electron signals. ([#1604](https://github.com/morluto/rea/pull/1604), [#1590](https://github.com/morluto/rea/pull/1590), [#1588](https://github.com/morluto/rea/pull/1588), [8b4c3de](https://github.com/morluto/rea/commit/8b4c3de3b3a3b580adec266f94f29ccedf1da228), [#1497](https://github.com/morluto/rea/issues/1497), [#1591](https://github.com/morluto/rea/issues/1591), [#1589](https://github.com/morluto/rea/issues/1589)) Thanks @Adolanium, @akushonkamen, @dakjdakd.
|
||||
- Added Termux Android browser-scenario support. ([#1655](https://github.com/morluto/rea/pull/1655)) Thanks @tyloryang.
|
||||
- Web inputs: accept BOM-prefixed HAR, preserve the first duplicate DOM attribute and resolve SVG/XLink URLs in HTML and XHTML. ([#1509](https://github.com/morluto/rea/pull/1509), [#1516](https://github.com/morluto/rea/pull/1516), [#1602](https://github.com/morluto/rea/pull/1602), [#1683](https://github.com/morluto/rea/pull/1683), [#1496](https://github.com/morluto/rea/issues/1496), [#1586](https://github.com/morluto/rea/issues/1586)) Thanks @CassieFox23, @kaoru0822-kitauji, @sathariels, @taljeri.
|
||||
- Screenshot comparisons: validate PNG chunks, distinguish corruption from unsupported encodings and bound comparison memory. ([4e4b15b](https://github.com/morluto/rea/commit/4e4b15b8250b29e176d4191c9610a3931b693474), [b3e977e](https://github.com/morluto/rea/commit/b3e977e6d073ab1783cece8581279f87fc360411)) Thanks @rudycelekli.
|
||||
- Archives: preserve case-distinct names, report collisions at their diverging segment, detect nested formats from bytes and reject Windows alternate-stream extraction paths. ([#1480](https://github.com/morluto/rea/pull/1480), [#1505](https://github.com/morluto/rea/pull/1505), [#1553](https://github.com/morluto/rea/pull/1553), [573598c](https://github.com/morluto/rea/commit/573598c6324698cc48ceafd24ed3864670e08d00), [#1470](https://github.com/morluto/rea/issues/1470), [#1491](https://github.com/morluto/rea/issues/1491)) Thanks @CassieFox23, @kaoru0822-kitauji.
|
||||
- Cleanup: retain failed readers, ASAR handles, output trees and provider workers for retry without discarding completed output or partial observations. ([ffe0574](https://github.com/morluto/rea/commit/ffe0574e17b9d973860371083cb1a351be912131), [de0d341](https://github.com/morluto/rea/commit/de0d341524dc68250a45b41eb7e49fcecc1e32a9), [3f8c51e](https://github.com/morluto/rea/commit/3f8c51e9b506d88fc41aded7b1a22b9a09037c26), [d1235fa](https://github.com/morluto/rea/commit/d1235fadc901e1fb9ab3149d53063306c8545414), [026126e](https://github.com/morluto/rea/commit/026126eafe741e5a8b857433587fd615cdd44c19))
|
||||
- Providers: canonicalize IDA workspace roots before identity checks, respect IDA cancellation, serialize JEB acquisition and correct Ghidra no-return/imported-symbol recovery. ([#1481](https://github.com/morluto/rea/pull/1481), [#1419](https://github.com/morluto/rea/pull/1419), [5f30ea6](https://github.com/morluto/rea/commit/5f30ea6ff2dd89aab7af6da6e8dfa5e508ed459b), [f88c990](https://github.com/morluto/rea/commit/f88c99021b88ce67e513025901f57478d8107d0c), [49e6f00](https://github.com/morluto/rea/commit/49e6f00ed08b225abf9ddb7c762553607fdab115)) Thanks @AS9929, @JonahSimon, @Y0UW1SH.
|
||||
- Binary metadata: respect PE/.NET section extents, bound binary-plist expansion, scope keyed-archive traversal and decode app `Info.plist` files without `plutil`. ([#1678](https://github.com/morluto/rea/pull/1678), [#1515](https://github.com/morluto/rea/pull/1515), [#1378](https://github.com/morluto/rea/pull/1378), [#1476](https://github.com/morluto/rea/pull/1476), [975dff2](https://github.com/morluto/rea/commit/975dff2e25b14b634e14dd5e0db1537d0ca9815b), [76296c5](https://github.com/morluto/rea/commit/76296c5a3868ba3b54be6ed3896719283d6de5f8)) Thanks @jayhemnani9910, @kaoru0822-kitauji, @rudycelekli.
|
||||
- Windows runtimes: preserve drive-root paths when creating private provider directories. ([ee46323](https://github.com/morluto/rea/commit/ee46323b493994902b04824a7dc8aae2a8117903))
|
||||
- Compatibility: load Windows native controls under Node-API hosts with renamed executables, preserve dylib directory requirements, support Wakaru 1.x and reject directory-valued Hopper launchers. ([#1531](https://github.com/morluto/rea/pull/1531), [#1623](https://github.com/morluto/rea/pull/1623), [#1540](https://github.com/morluto/rea/pull/1540), [#1426](https://github.com/morluto/rea/pull/1426)) Thanks @evenafterallisaid, @Fazmin, @pionxzh, @xeropresence.
|
||||
- Reference imports: normalize internal symlink lookup consistently, preserve exact paths and exclusion provenance, report root access failures and reject FIFO replacements without blocking. ([#1535](https://github.com/morluto/rea/pull/1535), [2ca3ad2](https://github.com/morluto/rea/commit/2ca3ad238608adfca80a7ff1aaadea6e59a58225), [6636045](https://github.com/morluto/rea/commit/6636045c9c5b472bba51afee9d434394861e93a4), [3967cfa](https://github.com/morluto/rea/commit/3967cfa43e786c65a57e89c265343983d19ef9d0), [f1c58cc](https://github.com/morluto/rea/commit/f1c58cc53fbd83b968e11b3e2eccebca51299c54)) Thanks @anujsingh-cse.
|
||||
- Setup: honor active configuration and skill roots, preserve configured MCP environments, validate Hermes/OMP profiles before writing, recognize Omarchy and retain recovery backups. ([#1504](https://github.com/morluto/rea/pull/1504), [#1607](https://github.com/morluto/rea/pull/1607), [#1528](https://github.com/morluto/rea/pull/1528), [#1562](https://github.com/morluto/rea/pull/1562), [#1662](https://github.com/morluto/rea/pull/1662), [a011485](https://github.com/morluto/rea/commit/a01148553e2a37c72ae78405418ab648876a96a6), [#1606](https://github.com/morluto/rea/issues/1606), [#1521](https://github.com/morluto/rea/issues/1521), [#1561](https://github.com/morluto/rea/issues/1561)) Thanks @Adkid-Zephyr, @CassieFox23, @LuisTaffelli, @N0zoM1z0, @peter-research.
|
||||
- Process capture: retain descendant lineage after launcher exit, preserve identity uncertainty before signaling and release terminals after observation failures. ([80896a1](https://github.com/morluto/rea/commit/80896a1d64b216ef893907249cd4252f86215d49), [c916850](https://github.com/morluto/rea/commit/c9168508527302ac57bc86385ccf0d824697b5dc)) Thanks @Y0UW1SH.
|
||||
- CLI/MCP errors: accept JSON files for web comparisons, preserve native option help and union correction details, remove invalid aliases and validate doctor against its discovered output contract. ([#1397](https://github.com/morluto/rea/pull/1397), [#1395](https://github.com/morluto/rea/pull/1395), [#1425](https://github.com/morluto/rea/pull/1425), [#1461](https://github.com/morluto/rea/pull/1461), [8b4c96e](https://github.com/morluto/rea/commit/8b4c96e6c6b3ed6806e4cd72b556b5db6114304a), [#1421](https://github.com/morluto/rea/issues/1421), [#1454](https://github.com/morluto/rea/issues/1454)) Thanks @Fazmin, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
- WebSocket results: use `network.websocket_connections` instead of `network.websocket_events`. Connections contain `request_id`, `url` and `events`; allow `payload_bytes: null` when size is unknown. No legacy alias remains. ([db4e360](https://github.com/morluto/rea/commit/db4e3608465486466591b594cf77037ac150c988))
|
||||
- Direct resource APIs: pass a caller-owned `ArtifactResourceScope` to inventory/extraction, JavaScript analysis, reference imports and web exports; retain it through cleanup/shutdown. For `readJavaScriptArtifactFiles`, scope is argument three and abort signal is argument four. Keep each root inventory source’s single cleanup owner. Use `OwnedFileHandle` owners, injected ASAR `fd`/`read` handles instead of `createReadStream`, VCS `Result` values, and `WebScriptExportError`’s options object. CLI/MCP inputs for the scoped operations are unchanged. ([ffe0574](https://github.com/morluto/rea/commit/ffe0574e17b9d973860371083cb1a351be912131), [53e7fad](https://github.com/morluto/rea/commit/53e7fadbd078422ba3d55b5499a70d237c5f4908), [684199b](https://github.com/morluto/rea/commit/684199ba0be7822266d57ddb8a40e95f087fdc42), [be75ed5](https://github.com/morluto/rea/commit/be75ed5e4b4bfb146af497f8fa5d172a1ea19462), [be6272f](https://github.com/morluto/rea/commit/be6272f26115bd8e94f2ab0df4cf092004587a18), [e0e49e9](https://github.com/morluto/rea/commit/e0e49e9dd53d9d1f1d2330b23f7f880dc1c81774), [95a77b6](https://github.com/morluto/rea/commit/95a77b6e25420b3879bd0e32bdbb14d452b38776))
|
||||
- Saved reference graphs: exclusions require their matching `pattern` and distinguish `project-ignored` from `default-ignored`. Re-import graphs missing that provenance. ([6585293](https://github.com/morluto/rea/commit/6585293c562853b95c3a70df02c12ad8cc17afc4))
|
||||
- Terminal captures: keep initial and resized buffers within 1,000,000 cells using `columns × (rows + scrollback)`; oversized scenarios are rejected before launch. ([f9ea9a9](https://github.com/morluto/rea/commit/f9ea9a9268c854d31048da9f8a50cfa37703fe5a))
|
||||
- Optional agent-verification reports: accept schema version 4, nullable `expectedFirstTool` and `targetClarificationPassed`. Production CLI/MCP contracts are unchanged. ([#1613](https://github.com/morluto/rea/pull/1613))
|
||||
- Capture comparisons: recapture both sides with matching `literal-replacements-v1` semantics when custom normalization patterns are used. Legacy unversioned pattern captures remain readable but cannot be compared with new pattern captures; captures without patterns are unaffected. ([#1456](https://github.com/morluto/rea/pull/1456), [2b6f14f](https://github.com/morluto/rea/commit/2b6f14f528297cf12c776c960af0859f5cfbefbf)) Thanks @Madhavan006, @N0zoM1z0.
|
||||
- Artifact graph edges: read child artifact IDs and paths from the occurrence selected by `occurrence_id`, not edge fields `child_artifact_id`/`logical_path`. Earlier inventory Evidence no longer validates against the new commitment. ([#1493](https://github.com/morluto/rea/pull/1493)) Thanks @kaoru0822-kitauji.
|
||||
- Semantic graph schema: nodes and relations no longer include `identifier_strategy`, and relation IDs no longer hash it. Earlier Evidence no longer validates against this schema. ([#1474](https://github.com/morluto/rea/pull/1474)) Thanks @kaoru0822-kitauji.
|
||||
- Semantic identities: select IDs from the parent graph; read exact event names from `event_name`, endpoints from `endpoint` and literal values from `properties.value`. New analyses use digest-based IDs for long/reserved event and endpoint names and `value-sha256` literal role keys, with shorter labels; string literal labels now read `string literal`. Previously stored graphs retain their original IDs and remain valid workflow inputs. ([#1427](https://github.com/morluto/rea/pull/1427), [#1423](https://github.com/morluto/rea/pull/1423), [#1407](https://github.com/morluto/rea/pull/1407)) Thanks @N0zoM1z0.
|
||||
|
||||
### Contributors
|
||||
|
||||
- Thanks also to @BaaRai, @commit-regrets, @coygeek, @Franky100-pig, @inii-sayaa, @jopemachine, @JosephDemarest, @keemsisi, @Totopo27 for contributions included in this release.
|
||||
|
||||
Full contribution record: [336 commits and 163 merged PRs](https://github.com/morluto/rea/blob/rea-agents-6.4.0/docs/releases/6.4.0.contributions.json).
|
||||
|
||||
## [6.3.0](https://github.com/morluto/rea/compare/rea-agents-6.2.0...rea-agents-6.3.0) (2026-10-09)
|
||||
|
||||
|
||||
|
||||
+183
-388
@@ -2004,20 +2004,9 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
);
|
||||
JumpTable.LoadTable[] loadTables = jumpTable.getLoadTables();
|
||||
RecoveredAArch64RelativeTable recoveredAArch64 =
|
||||
recoverAArch64RelativeTable(jumpTable);
|
||||
if (recoveredAArch64 != null && loadTables.length > 0) {
|
||||
// A decoded table only adds evidence when Ghidra's own load table
|
||||
// describes the same bytes; any disagreement keeps Ghidra's view.
|
||||
boolean agrees = false;
|
||||
for (JumpTable.LoadTable loadTable : loadTables) {
|
||||
if (loadTable.getAddress().equals(recoveredAArch64.tableAddress) &&
|
||||
loadTable.getSize() == recoveredAArch64.entrySize &&
|
||||
loadTable.getNum() == recoveredAArch64.mappings.size()) {
|
||||
agrees = true;
|
||||
}
|
||||
}
|
||||
if (!agrees) recoveredAArch64 = null;
|
||||
}
|
||||
loadTables.length == 0
|
||||
? recoverAArch64RelativeTable(jumpTable)
|
||||
: null;
|
||||
List<Address> referencedDataAddresses = loadTables.length == 0
|
||||
? dispatchPathDataReferences(jumpTable)
|
||||
: List.of();
|
||||
@@ -2051,7 +2040,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
source.add("evidence", evidence);
|
||||
dataSources.add(source);
|
||||
}
|
||||
if (recoveredAArch64 != null && loadTables.length == 0) {
|
||||
if (recoveredAArch64 != null) {
|
||||
JsonObject source = new JsonObject();
|
||||
source.addProperty(
|
||||
"address",
|
||||
@@ -2066,16 +2055,9 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
inferenceEvidence(
|
||||
"jump-table",
|
||||
"ghidra-instruction-and-memory",
|
||||
"Decoded " + recoveredAArch64.mappings.size() + " " +
|
||||
(recoveredAArch64.signedEntries ? "signed" : "unsigned") +
|
||||
"Decoded " + recoveredAArch64.mappings.size() +
|
||||
" entries of size " + recoveredAArch64.entrySize + " bytes from " + canonicalAddress(recoveredAArch64.tableAddress) +
|
||||
"; the AArch64 ADRP/ADD table, indexed load, ADR base, ADD" +
|
||||
(recoveredAArch64.offsetShift == 0 ? "" : " LSL #" + recoveredAArch64.offsetShift) +
|
||||
"/BR chain uses a selector bounded by CMP #" + recoveredAArch64.compareImmediate +
|
||||
" and " + recoveredAArch64.guardMnemonic + " on one straight-line path" +
|
||||
(recoveredAArch64.nullTarget == null
|
||||
? "."
|
||||
: "; zero entries are routed by CBZ to " + canonicalAddress(recoveredAArch64.nullTarget) + ".")
|
||||
"; the AArch64 LDRB-or-LDRH/ADR/ADD/BR chain uses a selector guarded by an unsigned upper-bound branch."
|
||||
)
|
||||
);
|
||||
source.add("evidence", evidence);
|
||||
@@ -2114,33 +2096,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
result.add("data_sources", dataSources);
|
||||
Address[] targets = jumpTable.getCases();
|
||||
TypedJumpTableCases typedCases = typedJumpTableCases(jumpTable, caseTokens);
|
||||
// Table indices identify the register bounded by CMP. Typed labels may
|
||||
// instead identify a selector before normalization (for example, SUB
|
||||
// index, selector, #10). Do not merge those distinct value identities.
|
||||
if (recoveredAArch64 != null && recoveredAArch64.priorSelectorDefinition != null) {
|
||||
typedCases.limitations.add(
|
||||
"REA decoded the relative table, but the compared selector has an earlier register definition at " +
|
||||
canonicalAddress(recoveredAArch64.priorSelectorDefinition) + "; its table indices are not proven to use the typed switch selector's case values. Ghidra's typed labels are retained and additional case labels remain unknown."
|
||||
);
|
||||
recoveredAArch64 = null;
|
||||
}
|
||||
if (recoveredAArch64 != null) {
|
||||
for (Map.Entry<Long, Address> typed : typedCases.cases.entrySet()) {
|
||||
long value = typed.getKey();
|
||||
if (value < 0 || value >= recoveredAArch64.mappings.size()) {
|
||||
typedCases.limitations.add("Typed case labels and decoded table indices do not use a verified common selector; Ghidra's typed labels are retained.");
|
||||
recoveredAArch64 = null;
|
||||
break;
|
||||
}
|
||||
RecoveredAArch64Mapping decoded = recoveredAArch64.mappings.get((int) value);
|
||||
boolean divertedZero = decoded.nullEntry && typed.getValue().equals(recoveredAArch64.unreachableZeroTarget);
|
||||
if (!typed.getValue().equals(decoded.targetAddress) && !divertedZero) {
|
||||
typedCases.limitations.add("Typed case evidence and the decoded relative table disagree on a destination; Ghidra's typed labels are retained.");
|
||||
recoveredAArch64 = null;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
JsonArray mappings = new JsonArray();
|
||||
JsonArray defaultTargets = new JsonArray();
|
||||
for (Address target : typedCases.defaults) {
|
||||
@@ -2148,30 +2103,27 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
}
|
||||
Set<Address> coveredTargets = new HashSet<>(typedCases.cases.values());
|
||||
coveredTargets.addAll(typedCases.defaults);
|
||||
if (recoveredAArch64 != null && recoveredAArch64.unreachableZeroTarget != null) {
|
||||
Address unreachable = recoveredAArch64.unreachableZeroTarget;
|
||||
coveredTargets.add(unreachable);
|
||||
for (RecoveredAArch64Mapping recovered : recoveredAArch64.mappings) {
|
||||
if (recovered.nullEntry && unreachable.equals(typedCases.cases.get((long) recovered.tableIndex))) {
|
||||
typedCases.cases.remove((long) recovered.tableIndex);
|
||||
}
|
||||
}
|
||||
typedCases.limitations.add(
|
||||
"Ghidra recovered destination " + canonicalAddress(unreachable) +
|
||||
" as the zero-offset decode of the relative table, but the CBZ before the dispatch diverts zero entries to " +
|
||||
canonicalAddress(recoveredAArch64.nullTarget) + ", so the indirect branch never reaches it for an in-range case."
|
||||
);
|
||||
}
|
||||
if (recoveredAArch64 != null) {
|
||||
for (int index = 0; index < recoveredAArch64.mappings.size(); index += 1) {
|
||||
monitor.checkCancelled();
|
||||
RecoveredAArch64Mapping recovered = recoveredAArch64.mappings.get(index);
|
||||
if (
|
||||
typedCases.unknownTargets.contains(recovered.targetAddress) ||
|
||||
typedCases.ambiguousValues.contains((long) recovered.tableIndex)
|
||||
typedCases.ambiguousValues.contains((long) recovered.caseValue)
|
||||
) continue;
|
||||
Address typedTarget = typedCases.cases.get((long) recovered.caseValue);
|
||||
if (typedTarget != null && !typedTarget.equals(recovered.targetAddress)) {
|
||||
typedCases.cases.remove((long) recovered.caseValue);
|
||||
typedCases.ambiguousValues.add((long) recovered.caseValue);
|
||||
typedCases.unknownTargets.add(typedTarget);
|
||||
typedCases.unknownTargets.add(recovered.targetAddress);
|
||||
typedCases.limitations.add(
|
||||
"Typed case evidence and the verified relative table disagree on a destination; both destinations remain unknown for that case."
|
||||
);
|
||||
continue;
|
||||
}
|
||||
JsonObject mapping = new JsonObject();
|
||||
mapping.addProperty("case_value", recovered.tableIndex);
|
||||
mapping.addProperty("case_value", recovered.caseValue);
|
||||
String targetAddress = canonicalAddress(recovered.targetAddress);
|
||||
mapping.addProperty("target_address", targetAddress);
|
||||
mapping.addProperty("confidence", "high");
|
||||
@@ -2180,18 +2132,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
inferenceEvidence(
|
||||
"jump-table",
|
||||
"ghidra-instruction-and-memory",
|
||||
"Case value " + recovered.tableIndex + (recoveredAArch64.entrySize == 1 ? " indexes byte " : " indexes halfword at byte offset ") +
|
||||
(recovered.tableIndex * recoveredAArch64.entrySize) + " at " + canonicalAddress(recoveredAArch64.tableAddress) +
|
||||
(recovered.nullEntry
|
||||
? "; the entry is zero, so the CBZ before the dispatch branches to " + targetAddress + "."
|
||||
: "; the loaded " + (recoveredAArch64.signedEntries ? "signed" : "unsigned") +
|
||||
" entry " + recovered.encodedEntry + " selects a " +
|
||||
(recoveredAArch64.offsetShift == 2 ? "four-byte instruction" : "byte") +
|
||||
" offset from " + canonicalAddress(recoveredAArch64.branchBase) + ", reaching " + targetAddress + ".")
|
||||
"Case value " + recovered.caseValue + (recoveredAArch64.entrySize == 1 ? " indexes byte " : " indexes halfword at byte offset ") +
|
||||
(recovered.caseValue * recoveredAArch64.entrySize) + " at " + canonicalAddress(recoveredAArch64.tableAddress) +
|
||||
"; the loaded unsigned entry selects a four-byte instruction offset from " +
|
||||
canonicalAddress(recoveredAArch64.branchBase) + ", reaching " + targetAddress + "."
|
||||
)
|
||||
);
|
||||
mapping.add("evidence", evidence);
|
||||
if (!typedCases.cases.containsKey((long) recovered.tableIndex)) {
|
||||
if (!typedCases.cases.containsKey((long) recovered.caseValue)) {
|
||||
mappings.add(mapping);
|
||||
}
|
||||
coveredTargets.add(recovered.targetAddress);
|
||||
@@ -2265,7 +2213,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
);
|
||||
}
|
||||
if (recoveredAArch64 == null && loadTables.length == 0 && currentProgram.getLanguageID().getIdAsString().startsWith("AARCH64:")) {
|
||||
limitations.add("REA did not verify a supported signed or unsigned byte/halfword relative dispatch chain with exact unsigned bounds and register definitions; unsupported encodings and ambiguous bounds remain unknown.");
|
||||
limitations.add("REA did not verify a supported byte/halfword relative dispatch chain with exact unsigned bounds and register definitions; unsupported encodings and ambiguous bounds remain unknown.");
|
||||
}
|
||||
if (targets.length == 0) {
|
||||
limitations.add(
|
||||
@@ -2395,21 +2343,12 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
}
|
||||
|
||||
private static final class RecoveredAArch64Mapping {
|
||||
private final int tableIndex;
|
||||
private final int caseValue;
|
||||
private final Address targetAddress;
|
||||
private final long encodedEntry;
|
||||
private final boolean nullEntry;
|
||||
|
||||
private RecoveredAArch64Mapping(
|
||||
int tableIndex,
|
||||
Address targetAddress,
|
||||
long encodedEntry,
|
||||
boolean nullEntry
|
||||
) {
|
||||
this.tableIndex = tableIndex;
|
||||
private RecoveredAArch64Mapping(int caseValue, Address targetAddress) {
|
||||
this.caseValue = caseValue;
|
||||
this.targetAddress = targetAddress;
|
||||
this.encodedEntry = encodedEntry;
|
||||
this.nullEntry = nullEntry;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2417,70 +2356,24 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
private final Address tableAddress;
|
||||
private final Address branchBase;
|
||||
private final int entrySize;
|
||||
private final Address priorSelectorDefinition;
|
||||
private final boolean signedEntries;
|
||||
private final int offsetShift;
|
||||
private final String guardMnemonic;
|
||||
private final long compareImmediate;
|
||||
private final Address nullTarget;
|
||||
private final Address unreachableZeroTarget;
|
||||
private final List<RecoveredAArch64Mapping> mappings;
|
||||
|
||||
private RecoveredAArch64RelativeTable(
|
||||
Address tableAddress,
|
||||
Address branchBase,
|
||||
int entrySize,
|
||||
Address priorSelectorDefinition,
|
||||
boolean signedEntries,
|
||||
int offsetShift,
|
||||
String guardMnemonic,
|
||||
long compareImmediate,
|
||||
Address nullTarget,
|
||||
Address unreachableZeroTarget,
|
||||
List<RecoveredAArch64Mapping> mappings
|
||||
) {
|
||||
this.tableAddress = tableAddress;
|
||||
this.branchBase = branchBase;
|
||||
this.entrySize = entrySize;
|
||||
this.priorSelectorDefinition = priorSelectorDefinition;
|
||||
this.signedEntries = signedEntries;
|
||||
this.offsetShift = offsetShift;
|
||||
this.guardMnemonic = guardMnemonic;
|
||||
this.compareImmediate = compareImmediate;
|
||||
this.nullTarget = nullTarget;
|
||||
this.unreachableZeroTarget = unreachableZeroTarget;
|
||||
this.mappings = mappings;
|
||||
}
|
||||
}
|
||||
|
||||
private static final Pattern AARCH64_DISPATCH_ADD = Pattern.compile(
|
||||
"(?i)^([wx]\\d+),\\s*([wx]\\d+),\\s*([wx]\\d+)(?:,\\s*LSL\\s*#(?:0x)?([0-9a-f]+))?$"
|
||||
);
|
||||
private static final Pattern AARCH64_TABLE_LOAD = Pattern.compile(
|
||||
"(?i)^([wx]\\d+),\\s*\\[([wx]\\d+),\\s*([wx]\\d+)(?:,\\s*(LSL|UXTW)\\s*(?:#(?:0x)?([0-9a-f]+))?)?\\s*\\]$"
|
||||
);
|
||||
private static final Pattern AARCH64_REGISTER_AND_ADDRESS = Pattern.compile(
|
||||
"(?i)^([wx]\\d+),\\s*(0x[0-9a-f]+)$"
|
||||
);
|
||||
private static final Pattern AARCH64_ADD_IMMEDIATE = Pattern.compile(
|
||||
"(?i)^([wx]\\d+),\\s*([wx]\\d+),\\s*#?(0x[0-9a-f]+|[0-9]+)$"
|
||||
);
|
||||
private static final Pattern AARCH64_COMPARE_IMMEDIATE = Pattern.compile(
|
||||
"(?i)^([wx]\\d+),\\s*#?(0x[0-9a-f]+|[0-9]+)$"
|
||||
);
|
||||
private static final Pattern AARCH64_MOVE_REGISTER = Pattern.compile(
|
||||
"(?i)^([wx]\\d+),\\s*([wx]\\d+)$"
|
||||
);
|
||||
private static final Set<String> AARCH64_FLAG_PRESERVING =
|
||||
Set.of("MOV", "ADRP", "ADR", "ADD");
|
||||
|
||||
/**
|
||||
* Recover only directly verified AArch64 relative branch tables:
|
||||
* ADRP/ADD table base, LDRB/LDRH/LDRSB/LDRSH entry load with an exact
|
||||
* index scale, ADR branch base, optional CBZ on the loaded entry, and
|
||||
* ADD (byte or LSL #2 scale)/BR, with a selector bounded by CMP #imm and an
|
||||
* unsigned B.HI or B.CS/B.HS guard on one straight-line path. Any other
|
||||
* instruction sequence keeps its case labels unknown.
|
||||
* Recover only the directly verified AArch64 byte/halfword-indexed relative branch
|
||||
* table form. Other instruction sequences keep their case labels unknown.
|
||||
*/
|
||||
private RecoveredAArch64RelativeTable recoverAArch64RelativeTable(
|
||||
JumpTable jumpTable
|
||||
@@ -2502,265 +2395,167 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
if (instruction.getAddress().equals(jumpTable.getSwitchAddress())) break;
|
||||
}
|
||||
int branchIndex = instructions.size() - 1;
|
||||
if (branchIndex < 1 || !mnemonic(instructions.get(branchIndex)).equals("BR")) {
|
||||
if (branchIndex < 1 ||
|
||||
!instructions.get(branchIndex).getMnemonicString().equalsIgnoreCase("BR")) {
|
||||
return null;
|
||||
}
|
||||
|
||||
Address[] recoveredTargets = jumpTable.getCases();
|
||||
Set<String> recoveredTargetSet = new HashSet<>();
|
||||
for (Address target : jumpTable.getCases()) recoveredTargetSet.add(canonicalAddress(target));
|
||||
if (recoveredTargetSet.isEmpty()) return null;
|
||||
return matchAArch64RelativeDispatch(
|
||||
instructions,
|
||||
branchIndex,
|
||||
recoveredTargetSet,
|
||||
dispatchPathDataReferences(jumpTable)
|
||||
);
|
||||
}
|
||||
for (Address target : recoveredTargets) recoveredTargetSet.add(canonicalAddress(target));
|
||||
List<Address> dataAddresses = dispatchPathDataReferences(jumpTable);
|
||||
|
||||
private RecoveredAArch64RelativeTable matchAArch64RelativeDispatch(
|
||||
List<Instruction> instructions,
|
||||
int branchIndex,
|
||||
Set<String> recoveredTargetSet,
|
||||
List<Address> dataAddresses
|
||||
) throws Exception {
|
||||
String branchRegister = normalizedRegister(operands(instructions.get(branchIndex)));
|
||||
for (int loadIndex = 0; loadIndex < branchIndex; loadIndex += 1) {
|
||||
Instruction load = instructions.get(loadIndex);
|
||||
int entrySize = load.getMnemonicString().equalsIgnoreCase("LDRB") ? 1 : load.getMnemonicString().equalsIgnoreCase("LDRH") ? 2 : 0;
|
||||
if (entrySize == 0 || branchIndex != loadIndex + 2 || loadIndex < 3 ||
|
||||
!instructions.get(loadIndex - 1).getMnemonicString().equalsIgnoreCase("ADR") ||
|
||||
!instructions.get(loadIndex - 2).getMnemonicString().equalsIgnoreCase("ADD") ||
|
||||
!instructions.get(loadIndex - 3).getMnemonicString().equalsIgnoreCase("ADRP")) continue;
|
||||
String loadOperands = operands(load);
|
||||
java.util.regex.Matcher loadMatcher = java.util.regex.Pattern
|
||||
.compile(entrySize == 1 ? "(?i)^[wx]\\d+,\\s*\\[([wx]\\d+),\\s*([wx]\\d+)\\]$" : "(?i)^[wx]\\d+,\\s*\\[([wx]\\d+),\\s*([wx]\\d+),\\s*LSL\\s*#(?:0x)?1\\]$")
|
||||
.matcher(loadOperands);
|
||||
if (!loadMatcher.matches()) continue;
|
||||
String loadedRegister = normalizedRegister(loadOperands.substring(0, loadOperands.indexOf(',')));
|
||||
String tableBaseRegister = normalizedRegister(loadMatcher.group(1));
|
||||
String indexRegister = normalizedRegister(loadMatcher.group(2));
|
||||
|
||||
// ADD Xd, Xn, Xm{, LSL #s}; BR Xd. LSL applies to Xm only.
|
||||
Instruction dispatchAdd = instructions.get(branchIndex - 1);
|
||||
if (!mnemonic(dispatchAdd).equals("ADD")) return null;
|
||||
java.util.regex.Matcher addMatcher = AARCH64_DISPATCH_ADD.matcher(operands(dispatchAdd));
|
||||
if (!addMatcher.matches() || !addMatcher.group(1).toLowerCase(Locale.ROOT).startsWith("x")) return null;
|
||||
if (!normalizedRegister(addMatcher.group(1)).equals(branchRegister)) return null;
|
||||
int offsetShift = addMatcher.group(4) == null ? 0 : Integer.parseInt(addMatcher.group(4), 16);
|
||||
if (offsetShift != 0 && offsetShift != 2) return null;
|
||||
String firstSource = normalizedRegister(addMatcher.group(2));
|
||||
String secondSource = normalizedRegister(addMatcher.group(3));
|
||||
if (firstSource.equals(secondSource)) return null;
|
||||
|
||||
// The entry register is the source whose latest definition is the
|
||||
// table load; the other source must be defined by ADR.
|
||||
int loadIndex = -1;
|
||||
String entryRegister = null;
|
||||
String baseRegister = null;
|
||||
for (String candidate : offsetShift == 0 ? List.of(secondSource, firstSource) : List.of(secondSource)) {
|
||||
int writer = latestWriter(instructions, branchIndex - 1, candidate);
|
||||
if (writer >= 0 && Set.of("LDRB", "LDRH", "LDRSB", "LDRSH").contains(mnemonic(instructions.get(writer)))) {
|
||||
loadIndex = writer;
|
||||
entryRegister = candidate;
|
||||
baseRegister = candidate.equals(firstSource) ? secondSource : firstSource;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (loadIndex < 0) return null;
|
||||
|
||||
Instruction load = instructions.get(loadIndex);
|
||||
String loadMnemonic = mnemonic(load);
|
||||
int entrySize = loadMnemonic.endsWith("B") ? 1 : 2;
|
||||
boolean signedEntries = loadMnemonic.startsWith("LDRS");
|
||||
java.util.regex.Matcher loadMatcher = AARCH64_TABLE_LOAD.matcher(operands(load));
|
||||
if (!loadMatcher.matches()) return null;
|
||||
// A signed load into a W register sign-extends only to 32 bits; the
|
||||
// 64-bit dispatch ADD would then read a zero-extended value.
|
||||
if (signedEntries && !loadMatcher.group(1).toLowerCase(Locale.ROOT).startsWith("x")) return null;
|
||||
String tableBaseRegister = normalizedRegister(loadMatcher.group(2));
|
||||
String indexRegister = normalizedRegister(loadMatcher.group(3));
|
||||
boolean wordIndex = loadMatcher.group(3).toLowerCase(Locale.ROOT).startsWith("w");
|
||||
String extend = loadMatcher.group(4) == null ? null : loadMatcher.group(4).toUpperCase(Locale.ROOT);
|
||||
int indexShift = loadMatcher.group(5) == null ? 0 : Integer.parseInt(loadMatcher.group(5), 16);
|
||||
if (wordIndex ? !"UXTW".equals(extend) : "UXTW".equals(extend)) return null;
|
||||
if (indexShift != (entrySize == 1 ? 0 : 1)) return null;
|
||||
|
||||
// Only ADR and one CBZ on the entry may sit between load and ADD.
|
||||
Address nullTarget = null;
|
||||
for (int index = loadIndex + 1; index < branchIndex - 1; index += 1) {
|
||||
Instruction instruction = instructions.get(index);
|
||||
String name = mnemonic(instruction);
|
||||
if (name.equals("ADR")) continue;
|
||||
if (name.equals("CBZ") && nullTarget == null) {
|
||||
String[] parts = operands(instruction).split(",");
|
||||
Address[] flows = instruction.getFlows();
|
||||
if (parts.length != 2 || flows.length != 1 ||
|
||||
!normalizedRegister(parts[0]).equals(entryRegister)) return null;
|
||||
nullTarget = flows[0];
|
||||
continue;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
int baseIndex = latestWriter(instructions, branchIndex - 1, baseRegister);
|
||||
if (baseIndex < 0 || !mnemonic(instructions.get(baseIndex)).equals("ADR")) return null;
|
||||
java.util.regex.Matcher baseMatcher = AARCH64_REGISTER_AND_ADDRESS.matcher(operands(instructions.get(baseIndex)));
|
||||
if (!baseMatcher.matches()) return null;
|
||||
Address branchBase = addressFromOperand(baseMatcher.group(2));
|
||||
|
||||
int tableAddIndex = latestWriter(instructions, loadIndex, tableBaseRegister);
|
||||
if (tableAddIndex < 0 || !mnemonic(instructions.get(tableAddIndex)).equals("ADD")) return null;
|
||||
java.util.regex.Matcher tableAddMatcher = AARCH64_ADD_IMMEDIATE.matcher(operands(instructions.get(tableAddIndex)));
|
||||
if (!tableAddMatcher.matches() ||
|
||||
!normalizedRegister(tableAddMatcher.group(1)).equals(tableBaseRegister) ||
|
||||
!normalizedRegister(tableAddMatcher.group(2)).equals(tableBaseRegister)) return null;
|
||||
int tablePageIndex = latestWriter(instructions, tableAddIndex, tableBaseRegister);
|
||||
if (tablePageIndex < 0 || !mnemonic(instructions.get(tablePageIndex)).equals("ADRP")) return null;
|
||||
java.util.regex.Matcher pageMatcher = AARCH64_REGISTER_AND_ADDRESS.matcher(operands(instructions.get(tablePageIndex)));
|
||||
if (!pageMatcher.matches() || !normalizedRegister(pageMatcher.group(1)).equals(tableBaseRegister)) return null;
|
||||
Address tableAddress = addressFromOperand(pageMatcher.group(2)).add(parseInteger(tableAddMatcher.group(3)));
|
||||
if (!currentProgram.getMemory().contains(tableAddress) ||
|
||||
(!dataAddresses.isEmpty() && !dataAddresses.contains(tableAddress))) return null;
|
||||
|
||||
// Unsigned guard: CMP Rs, #imm, then B.HI (imm + 1 entries) or
|
||||
// B.CS/B.HS (imm entries), with only flag-preserving moves and
|
||||
// address arithmetic between the compare and the load.
|
||||
int guardIndex = -1;
|
||||
for (int index = loadIndex - 1; index >= 0; index -= 1) {
|
||||
String name = mnemonic(instructions.get(index));
|
||||
if (name.startsWith("B.")) {
|
||||
guardIndex = index;
|
||||
break;
|
||||
}
|
||||
if (!AARCH64_FLAG_PRESERVING.contains(name)) return null;
|
||||
}
|
||||
if (guardIndex < 1) return null;
|
||||
Instruction guard = instructions.get(guardIndex);
|
||||
String guardMnemonic = mnemonic(guard);
|
||||
if (!Set.of("B.HI", "B.CS", "B.HS").contains(guardMnemonic)) return null;
|
||||
Address[] guardFlows = guard.getFlows();
|
||||
if (guardFlows.length != 1) return null;
|
||||
Address defaultTarget = guardFlows[0];
|
||||
int compareIndex = -1;
|
||||
for (int index = guardIndex - 1; index >= 0; index -= 1) {
|
||||
String name = mnemonic(instructions.get(index));
|
||||
if (name.equals("CMP")) {
|
||||
compareIndex = index;
|
||||
break;
|
||||
}
|
||||
if (!AARCH64_FLAG_PRESERVING.contains(name)) return null;
|
||||
}
|
||||
if (compareIndex < 0) return null;
|
||||
java.util.regex.Matcher compareMatcher = AARCH64_COMPARE_IMMEDIATE.matcher(operands(instructions.get(compareIndex)));
|
||||
if (!compareMatcher.matches()) return null;
|
||||
String comparedRegister = normalizedRegister(compareMatcher.group(1));
|
||||
boolean comparedFullWidth = compareMatcher.group(1).toLowerCase(Locale.ROOT).startsWith("x");
|
||||
long compareImmediate = parseInteger(compareMatcher.group(2));
|
||||
long entryCountLong = guardMnemonic.equals("B.HI") ? compareImmediate + 1 : compareImmediate;
|
||||
if (entryCountLong <= 0 || entryCountLong > 4096) return null;
|
||||
int entryCount = (int) entryCountLong;
|
||||
|
||||
// The loaded index must carry the compared value: either the compared
|
||||
// register itself, unchanged since CMP, or a MOV copy made after CMP.
|
||||
// A W compare bounds only the low 32 bits, so an X index needs a
|
||||
// zero-extending W copy; a W index is zero-extended by UXTW.
|
||||
int indexWriter = latestWriter(instructions, loadIndex, indexRegister);
|
||||
boolean bounded = false;
|
||||
if (indexWriter < compareIndex) {
|
||||
bounded = indexRegister.equals(comparedRegister) && (wordIndex || comparedFullWidth);
|
||||
}
|
||||
else if (mnemonic(instructions.get(indexWriter)).equals("MOV")) {
|
||||
java.util.regex.Matcher moveMatcher = AARCH64_MOVE_REGISTER.matcher(operands(instructions.get(indexWriter)));
|
||||
if (moveMatcher.matches() &&
|
||||
normalizedRegister(moveMatcher.group(2)).equals(comparedRegister) &&
|
||||
latestWriter(instructions, indexWriter, comparedRegister) < compareIndex) {
|
||||
boolean wordCopy = moveMatcher.group(1).toLowerCase(Locale.ROOT).startsWith("w");
|
||||
bounded = wordCopy || comparedFullWidth;
|
||||
}
|
||||
}
|
||||
if (!bounded) return null;
|
||||
|
||||
// Every fact above holds only on one straight-line fall-through path
|
||||
// with no other entry into the window.
|
||||
int windowStart = Math.min(Math.min(tablePageIndex, baseIndex), compareIndex);
|
||||
for (int index = windowStart + 1; index <= branchIndex; index += 1) {
|
||||
monitor.checkCancelled();
|
||||
Instruction previous = instructions.get(index - 1);
|
||||
Instruction current = instructions.get(index);
|
||||
Address fallThrough = previous.getFallThrough();
|
||||
if (fallThrough == null || !fallThrough.equals(current.getAddress())) return null;
|
||||
ReferenceIterator incoming = currentProgram.getReferenceManager().getReferencesTo(current.getAddress());
|
||||
while (incoming.hasNext()) {
|
||||
if (incoming.next().getReferenceType().isFlow()) return null;
|
||||
}
|
||||
}
|
||||
|
||||
List<RecoveredAArch64Mapping> mappings = new ArrayList<>();
|
||||
Set<String> mappedTargets = new HashSet<>();
|
||||
for (int tableIndex = 0; tableIndex < entryCount; tableIndex += 1) {
|
||||
monitor.checkCancelled();
|
||||
Address entry = tableAddress.add((long) tableIndex * entrySize);
|
||||
if (!currentProgram.getMemory().contains(entry)) return null;
|
||||
long encoded;
|
||||
if (entrySize == 1) {
|
||||
byte value = currentProgram.getMemory().getByte(entry);
|
||||
encoded = signedEntries ? value : value & 0xffL;
|
||||
}
|
||||
else {
|
||||
short value = currentProgram.getMemory().getShort(entry);
|
||||
encoded = signedEntries ? value : value & 0xffffL;
|
||||
}
|
||||
if (encoded == 0 && nullTarget != null) {
|
||||
mappings.add(new RecoveredAArch64Mapping(tableIndex, nullTarget, encoded, true));
|
||||
continue;
|
||||
}
|
||||
Address target;
|
||||
try {
|
||||
target = branchBase.addNoWrap(encoded << offsetShift);
|
||||
}
|
||||
catch (ghidra.program.model.address.AddressOverflowException overflow) {
|
||||
return null;
|
||||
}
|
||||
String canonicalTarget = canonicalAddress(target);
|
||||
if (!recoveredTargetSet.contains(canonicalTarget)) return null;
|
||||
mappedTargets.add(canonicalTarget);
|
||||
mappings.add(new RecoveredAArch64Mapping(tableIndex, target, encoded, false));
|
||||
}
|
||||
if (mappedTargets.contains(canonicalAddress(defaultTarget))) return null;
|
||||
// Ghidra's recovered destinations must be fully accounted for by the
|
||||
// decoded entries, the guard default, and the CBZ null path.
|
||||
Set<String> accountedTargets = new HashSet<>(mappedTargets);
|
||||
accountedTargets.add(canonicalAddress(defaultTarget));
|
||||
if (nullTarget != null) accountedTargets.add(canonicalAddress(nullTarget));
|
||||
// Ghidra does not model the CBZ, so it may decode a zero entry as a
|
||||
// branch to the base itself. Only a zero entry encodes offset 0, and
|
||||
// the CBZ diverts every zero entry, so the BR never reaches the base.
|
||||
Address unreachableZeroTarget = null;
|
||||
boolean divertedZeroEntry = mappings.stream().anyMatch(mapping -> mapping.nullEntry);
|
||||
if (divertedZeroEntry &&
|
||||
recoveredTargetSet.contains(canonicalAddress(branchBase)) &&
|
||||
!mappedTargets.contains(canonicalAddress(branchBase)) &&
|
||||
!branchBase.equals(defaultTarget) &&
|
||||
!branchBase.equals(nullTarget)) {
|
||||
unreachableZeroTarget = branchBase;
|
||||
accountedTargets.add(canonicalAddress(branchBase));
|
||||
}
|
||||
if (!accountedTargets.containsAll(recoveredTargetSet)) return null;
|
||||
int selectorWriter = latestWriter(instructions, compareIndex, comparedRegister);
|
||||
return new RecoveredAArch64RelativeTable(
|
||||
tableAddress,
|
||||
branchBase,
|
||||
entrySize,
|
||||
selectorWriter < 0 ? null : instructions.get(selectorWriter).getAddress(),
|
||||
signedEntries,
|
||||
offsetShift,
|
||||
guardMnemonic,
|
||||
compareImmediate,
|
||||
nullTarget,
|
||||
unreachableZeroTarget,
|
||||
List.copyOf(mappings)
|
||||
);
|
||||
}
|
||||
|
||||
private static String mnemonic(Instruction instruction) {
|
||||
return instruction.getMnemonicString().toUpperCase(Locale.ROOT);
|
||||
}
|
||||
|
||||
/** Index of the last instruction before {@code before} that writes {@code register}, or -1. */
|
||||
private static int latestWriter(List<Instruction> instructions, int before, String register) {
|
||||
for (int index = before - 1; index >= 0; index -= 1) {
|
||||
for (Object result : instructions.get(index).getResultObjects()) {
|
||||
if (result instanceof Register written &&
|
||||
normalizedRegister(written.getBaseRegister().getName()).equals(register)) {
|
||||
return index;
|
||||
Address resolvedTableAddress = null;
|
||||
for (int index = loadIndex - 3; index < loadIndex - 1; index += 1) {
|
||||
Instruction instruction = instructions.get(index);
|
||||
String mnemonic = instruction.getMnemonicString().toUpperCase(Locale.ROOT);
|
||||
String text = operands(instruction);
|
||||
if (mnemonic.equals("ADRP") &&
|
||||
text.matches("(?i)^[wx]\\d+,\\s*0x[0-9a-f]+$")) {
|
||||
String[] parts = text.split(",");
|
||||
if (normalizedRegister(parts[0]).equals(tableBaseRegister)) {
|
||||
resolvedTableAddress = addressFromOperand(parts[1]);
|
||||
}
|
||||
}
|
||||
else if (mnemonic.equals("ADD") && resolvedTableAddress != null &&
|
||||
text.matches("(?i)^[wx]\\d+,\\s*[wx]\\d+,\\s*#?(?:0x[0-9a-f]+|[0-9]+)$")) {
|
||||
String[] parts = text.split(",");
|
||||
if (normalizedRegister(parts[0]).equals(tableBaseRegister) &&
|
||||
normalizedRegister(parts[1]).equals(tableBaseRegister)) {
|
||||
resolvedTableAddress = resolvedTableAddress.add(parseInteger(parts[2].trim().replaceFirst("^#", "")));
|
||||
}
|
||||
}
|
||||
}
|
||||
if (resolvedTableAddress == null || !currentProgram.getMemory().contains(resolvedTableAddress) ||
|
||||
(!dataAddresses.isEmpty() && !dataAddresses.contains(resolvedTableAddress))) continue;
|
||||
Address tableAddress = resolvedTableAddress;
|
||||
|
||||
Address branchBase = null;
|
||||
boolean addressChain = false;
|
||||
for (int index = loadIndex - 1; index < branchIndex; index += 1) {
|
||||
Instruction instruction = instructions.get(index);
|
||||
String mnemonic = instruction.getMnemonicString().toUpperCase(Locale.ROOT);
|
||||
String text = operands(instruction);
|
||||
if (index < loadIndex && mnemonic.equals("ADR") && text.matches("(?i)^[wx]\\d+,\\s*0x[0-9a-f]+$")) {
|
||||
int comma = text.indexOf(',');
|
||||
String destination = normalizedRegister(text.substring(0, comma));
|
||||
if (!destination.equals(normalizedRegister(operands(instructions.get(branchIndex))))) continue;
|
||||
branchBase = addressFromOperand(text.substring(comma + 1));
|
||||
}
|
||||
if (index > loadIndex && mnemonic.equals("ADD") && branchBase != null &&
|
||||
text.matches("(?i)^[wx]\\d+,\\s*[wx]\\d+,\\s*[wx]\\d+,\\s*LSL\\s*#(?:0x)?2$")) {
|
||||
String[] parts = text.split(",");
|
||||
String destination = normalizedRegister(parts[0]);
|
||||
String base = normalizedRegister(parts[1]);
|
||||
String offset = normalizedRegister(parts[2]);
|
||||
if (destination.equals(base) &&
|
||||
base.equals(normalizedRegister(operands(instructions.get(branchIndex)))) &&
|
||||
offset.equals(loadedRegister)) {
|
||||
addressChain = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (!addressChain || branchBase == null) continue;
|
||||
|
||||
int entryCount = -1;
|
||||
Address defaultTarget = null;
|
||||
for (int compareIndex = 0; compareIndex < loadIndex; compareIndex += 1) {
|
||||
Instruction compare = instructions.get(compareIndex);
|
||||
if (!compare.getMnemonicString().equalsIgnoreCase("CMP")) continue;
|
||||
java.util.regex.Matcher compareMatcher = java.util.regex.Pattern
|
||||
.compile("(?i)^([wx]\\d+),\\s*#?(0x[0-9a-f]+|[0-9]+)$")
|
||||
.matcher(operands(compare));
|
||||
if (!compareMatcher.matches()) continue;
|
||||
String comparedRegister = normalizedRegister(compareMatcher.group(1));
|
||||
long upperBound = parseInteger(compareMatcher.group(2));
|
||||
if (upperBound < 0 || upperBound >= 4096) continue;
|
||||
for (int guardIndex = compareIndex + 1; guardIndex < loadIndex; guardIndex += 1) {
|
||||
Instruction guard = instructions.get(guardIndex);
|
||||
String mnemonic = guard.getMnemonicString().toUpperCase(Locale.ROOT);
|
||||
if (!mnemonic.equals("B.HI")) continue;
|
||||
Address[] flows = guard.getFlows();
|
||||
if (flows.length != 1 || !recoveredTargetSet.contains(canonicalAddress(flows[0]))) continue;
|
||||
boolean supportedSequence = true;
|
||||
for (int sequenceIndex = compareIndex + 1; sequenceIndex < loadIndex; sequenceIndex += 1) {
|
||||
String sequenceMnemonic = instructions.get(sequenceIndex)
|
||||
.getMnemonicString().toUpperCase(Locale.ROOT);
|
||||
if (!Set.of("B.HI", "MOV", "ADRP", "ADD", "ADR").contains(sequenceMnemonic)) {
|
||||
supportedSequence = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
Set<String> selectorAliases = new HashSet<>();
|
||||
selectorAliases.add(comparedRegister);
|
||||
Set<String> boundedIndexes = new HashSet<>();
|
||||
if (compareMatcher.group(1).toLowerCase(Locale.ROOT).startsWith("x")) boundedIndexes.add(comparedRegister);
|
||||
for (int sequenceIndex = compareIndex + 1; sequenceIndex < loadIndex; sequenceIndex++) {
|
||||
Instruction instruction = instructions.get(sequenceIndex);
|
||||
String[] parts = operands(instruction).split(",");
|
||||
if (parts.length == 0) continue;
|
||||
if (instruction.getMnemonicString().equalsIgnoreCase("MOV")) {
|
||||
String destination = normalizedRegister(parts[0]);
|
||||
String source = parts.length == 2 ? normalizedRegister(parts[1]) : "";
|
||||
boolean aliasesSelector = selectorAliases.contains(source);
|
||||
boolean fullWidthBounded = boundedIndexes.contains(source) ||
|
||||
(parts[0].trim().toLowerCase(Locale.ROOT).startsWith("w") && aliasesSelector);
|
||||
if (aliasesSelector) selectorAliases.add(destination);
|
||||
else selectorAliases.remove(destination);
|
||||
if (fullWidthBounded) boundedIndexes.add(destination);
|
||||
else boundedIndexes.remove(destination);
|
||||
} else if (Set.of("ADRP", "ADD", "ADR").contains(instruction.getMnemonicString().toUpperCase(Locale.ROOT))) {
|
||||
selectorAliases.remove(normalizedRegister(parts[0]));
|
||||
boundedIndexes.remove(normalizedRegister(parts[0]));
|
||||
}
|
||||
}
|
||||
boolean selectorFeedsTable = boundedIndexes.contains(indexRegister);
|
||||
if (compareIndex < loadIndex - 3 && selectorFeedsTable && supportedSequence) {
|
||||
entryCount = (int) upperBound + 1;
|
||||
defaultTarget = flows[0];
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (entryCount > 0) break;
|
||||
}
|
||||
if (entryCount <= 0 || entryCount > 4096 || defaultTarget == null) continue;
|
||||
|
||||
List<RecoveredAArch64Mapping> mappings = new ArrayList<>();
|
||||
Set<String> mappedTargets = new HashSet<>();
|
||||
boolean valid = true;
|
||||
for (int caseValue = 0; caseValue < entryCount; caseValue += 1) {
|
||||
Address entry = tableAddress.add((long) caseValue * entrySize);
|
||||
int encodedOffset = entrySize == 1 ? currentProgram.getMemory().getByte(entry) & 0xff : currentProgram.getMemory().getShort(entry) & 0xffff;
|
||||
Address target = branchBase.add((long) encodedOffset * 4L);
|
||||
String canonicalTarget = canonicalAddress(target);
|
||||
if (!recoveredTargetSet.contains(canonicalTarget)) {
|
||||
valid = false;
|
||||
break;
|
||||
}
|
||||
mappedTargets.add(canonicalTarget);
|
||||
mappings.add(new RecoveredAArch64Mapping(caseValue, target));
|
||||
}
|
||||
if (!valid || mappedTargets.contains(canonicalAddress(defaultTarget))) continue;
|
||||
Set<String> accountedTargets = new HashSet<>(mappedTargets);
|
||||
accountedTargets.add(canonicalAddress(defaultTarget));
|
||||
if (!accountedTargets.equals(recoveredTargetSet)) continue;
|
||||
return new RecoveredAArch64RelativeTable(tableAddress, branchBase, entrySize, List.copyOf(mappings));
|
||||
}
|
||||
return -1;
|
||||
return null;
|
||||
}
|
||||
|
||||
private static String operands(Instruction instruction) {
|
||||
|
||||
@@ -24,29 +24,6 @@ REA can attach to a user-owned Chrome-family browser through the Chrome DevTools
|
||||
|
||||
Target discovery returns the complete in-scope target array in one result.
|
||||
|
||||
## Screenshot comparison memory budget
|
||||
|
||||
`compare_web_screenshots` admits both PNGs against a 256 MiB estimated
|
||||
working-memory budget before inflating either image. This is a processing
|
||||
buffer budget for this operation, not a PNG dimension limit, CDP transport
|
||||
limit, or process RSS guarantee. Header-valid dimensions that exceed the
|
||||
budget return `resource_constraint` with `details.resource: "memory"` and the
|
||||
estimated and maximum working-set bytes. Invalid PNG structure remains
|
||||
malformed input; encodings outside supported pixel-comparison coverage remain
|
||||
unsupported-target errors.
|
||||
|
||||
The conservative estimate includes the two decoder-owned decoded input
|
||||
buffers, the current image's concatenated IDAT buffer, Node zlib's default
|
||||
16 KiB output chunk backing rounded up for retained chunks, the concatenated
|
||||
inflate result while its output chunks are retained, one possible extra output
|
||||
chunk before zlib checks `maxOutputLength`, unfiltered scanline storage,
|
||||
RGB-to-RGBA expansion, and the first retained RGBA image while the second is
|
||||
decoded. RGBA output aliases its unfiltered buffer and is counted once.
|
||||
Admission uses checked integer arithmetic before image-sized allocation.
|
||||
Caller-owned JSON/base64 strings, input parsing and validation, transport
|
||||
buffers, garbage-collector behavior, and other runtime memory are outside this
|
||||
estimate; the budget does not promise a bound on total process memory.
|
||||
|
||||
Electron `file://` pages use a separate provider and target boundary; see [electron-observation.md](electron-observation.md).
|
||||
|
||||
## Provider authorities
|
||||
|
||||
@@ -880,7 +880,7 @@ For a client that requires manual configuration, use:
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents@6.4.0", "mcp"]
|
||||
"args": ["-y", "rea-agents@6.3.0", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -60,12 +60,6 @@ manifest and graph commitments, JavaScript Application Graph, static
|
||||
Electron summary, reconstruction statistics, and explicit limitations. It does
|
||||
not require a live Hopper, Ghidra, browser, or Electron process.
|
||||
|
||||
The application graph selects the shallowest `package.json` outside
|
||||
`node_modules` as its package root, breaking equal-depth ties by Unicode
|
||||
code-point order. Dependency manifests remain package observations but do not
|
||||
declare Electron main or renderer entries. If only dependency manifests are
|
||||
present, the inventory artifact remains the graph root.
|
||||
|
||||
## ASAR integrity
|
||||
|
||||
ASAR inventory checks Electron integrity metadata for embedded archive entries
|
||||
|
||||
@@ -308,23 +308,6 @@ that reference for a summary, one section/module, or a stable page, or call
|
||||
bundle exports stream canonical JSON into an atomically published file. A broad
|
||||
follow-up or `get_evidence_bundle` can also exceed the response budget; exporting
|
||||
preserves the complete session without sending it through a single MCP frame.
|
||||
To discover existing records without their large payloads, call
|
||||
`get_evidence_bundle` with `detail: "summary"`. Its `result.records` contain
|
||||
Evidence IDs, artifact/provider identities, analysis-profile digests and native
|
||||
procedure metadata. Optional `filters` match exact `evidence_id`, `operation`,
|
||||
`target_sha256`, `analysis_profile_digest` or `procedure_address`. Use a discovered
|
||||
ID with `inspect_analysis_view`; neither discovery nor retained selection starts
|
||||
a provider. `retention: "complete-record"` describes stored Evidence, while
|
||||
`native_dossier.value_flow` independently reports truncation, unavailable or
|
||||
unknown metadata. Missing omission counts remain null. Summary output carries
|
||||
every matching record; actual transport admission still applies. Omitted
|
||||
`detail` preserves complete canonical bundle delivery; filters require summary.
|
||||
|
||||
For a portable exported bundle, the CLI shares this projection:
|
||||
`rea inspect-evidence-bundle '{"path":"/tmp/evidence.json","detail":"summary","filters":{"operation":"analyze_function"}}' --format json`.
|
||||
The CLI validates and materializes that local bundle under its existing memory
|
||||
budget; summary delivery does not make arbitrary large files cheap to import.
|
||||
|
||||
Cancelling an export stops further serialization and removes its staging file.
|
||||
The destination changes only when a complete export is atomically published.
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
+1
-5
@@ -21,11 +21,7 @@ Use [the rea-changelog-update skill](https://github.com/morluto/rea/blob/main/.a
|
||||
to turn the bot draft into grouped Highlights, Changes, and Fixes with inline
|
||||
`Thanks @...` credit. `npm run release:notes -- inventory` collects the full
|
||||
selected Git range and paginated GitHub PR/author/closing-issue metadata;
|
||||
`record --summary` produces a compact provenance link for public notes, while
|
||||
`record` produces the full record for review. Credit a contribution once even
|
||||
when it appears in highlights, details and migrations; omit maintainer
|
||||
`@morluto` self-thanks. Keep complete authorship metadata in the inventory
|
||||
instead of duplicating a credit ledger in the release body. `check` validates credits
|
||||
`record` produces the complete contribution record, `check` validates credits
|
||||
and optional PR-body equality, and `render` writes the verified release section.
|
||||
Run a subcommand without its required options to see its usage. GitHub reads
|
||||
require an authenticated `gh`; record/check/render work offline from the saved
|
||||
|
||||
+9
-9
@@ -450,15 +450,15 @@ installation (`application.java.min` through `application.java.max`). Current
|
||||
12.1 releases require JDK 21 or newer and set no maximum. The lanes below still
|
||||
prove behavior on the verified Ghidra 12.1.4 and JDK 21 build.
|
||||
|
||||
| Ghidra lane | Supported runner/target | Additional local tools |
|
||||
| ------------------------------------------ | ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
|
||||
| `npm run verify:ghidra` | Linux x64/arm64 ELF or macOS x64/arm64 Mach-O | Host C compiler, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:seeds` | POSIX host; source-owned 32-bit x86 raw fixture; function order and BSS labels | Ghidra 12.1.4 and full JDK 21 |
|
||||
| `npm run verify:ghidra:swift` | macOS x64/arm64 Mach-O | Host Swift compiler, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:switch` | Linux x64 ELF; GCC/Clang optimized and stripped switch fixtures | GCC, Clang, GNU nm/objdump/strip, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:aarch64-jump-table` | Any supported Ghidra host; AArch64 ELF; signed/unsigned byte/halfword tables; host ARM64 Mach-O | Clang with AArch64 target support, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:cross-format` | Any supported Ghidra host; also analyzes AArch64 ELF, x86-64 PE, and x86-64 Mach-O | `clang`, LLD, and `lld-link` in addition to host-lane prerequisites |
|
||||
| `npm run verify:ghidra:windows` | Controlled Windows x64 with native x86-64 PE; `-- --x86` selects native x86 PE | Ghidra 12.1.4, full JDK 21, and the matching native artifact |
|
||||
| Ghidra lane | Supported runner/target | Additional local tools |
|
||||
| ------------------------------------------ | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
|
||||
| `npm run verify:ghidra` | Linux x64/arm64 ELF or macOS x64/arm64 Mach-O | Host C compiler, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:seeds` | POSIX host; source-owned 32-bit x86 raw fixture; function order and BSS labels | Ghidra 12.1.4 and full JDK 21 |
|
||||
| `npm run verify:ghidra:swift` | macOS x64/arm64 Mach-O | Host Swift compiler, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:switch` | Linux x64 ELF; GCC/Clang optimized and stripped switch fixtures | GCC, Clang, GNU nm/objdump/strip, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:aarch64-jump-table` | Any supported Ghidra host; AArch64 ELF; byte/halfword tables; host ARM64 Mach-O | Clang with AArch64 target support, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:cross-format` | Any supported Ghidra host; also analyzes AArch64 ELF, x86-64 PE, and x86-64 Mach-O | `clang`, LLD, and `lld-link` in addition to host-lane prerequisites |
|
||||
| `npm run verify:ghidra:windows` | Controlled Windows x64 with native x86-64 PE; `-- --x86` selects native x86 PE | Ghidra 12.1.4, full JDK 21, and the matching native artifact |
|
||||
|
||||
The optional Swift lane compiles a source-owned fixture and checks modern and
|
||||
provider-demangled procedure names through real stdio MCP, with and without the
|
||||
|
||||
@@ -1,7 +1,5 @@
|
||||
#include "authority.hpp"
|
||||
|
||||
#include <delayimp.h>
|
||||
|
||||
namespace rea {
|
||||
|
||||
static const napi_type_tag RESOURCE_TAG = {0x45fcbf0b49e84fe3ULL, 0x9e3e5074130de843ULL};
|
||||
@@ -123,13 +121,3 @@ static napi_value initialize(napi_env env, napi_value exports) {
|
||||
} // namespace rea
|
||||
|
||||
NAPI_MODULE(NODE_GYP_MODULE_NAME, rea::initialize)
|
||||
|
||||
// Node-API symbols are delay-imported from "node.exe". Resolve them against the
|
||||
// host executable so Node.js, Bun, and other Node-API hosts load the addon
|
||||
// regardless of their executable file name.
|
||||
static FARPROC WINAPI reaHostHook(unsigned event, PDelayLoadInfo info) {
|
||||
if (event != dliNotePreLoadLibrary || _stricmp(info->szDll, "node.exe") != 0) return nullptr;
|
||||
return reinterpret_cast<FARPROC>(GetModuleHandleW(nullptr));
|
||||
}
|
||||
|
||||
decltype(__pfnDliNotifyHook2) __pfnDliNotifyHook2 = reaHostHook;
|
||||
|
||||
@@ -219,10 +219,9 @@ std::unique_ptr<Runtime> createRuntime(const std::wstring& parentPath, const std
|
||||
require(BCryptGenRandom(nullptr, random.data(), random.size(), BCRYPT_USE_SYSTEM_PREFERRED_RNG) == 0,
|
||||
"Generate runtime identity failed", parentPath, ERROR_GEN_FAILURE);
|
||||
auto result = std::make_unique<Runtime>();
|
||||
const auto name = prefix + hex(random.data(), random.size());
|
||||
result->path = parent->path + (parent->path.back() == L'\\' ? L"" : L"\\") + name;
|
||||
result->path = parent->path + L"\\" + prefix + hex(random.data(), random.size());
|
||||
Security security;
|
||||
result->directory = createRelative(parent->get(), name,
|
||||
result->directory = createRelative(parent->get(), prefix + hex(random.data(), random.size()),
|
||||
DELETE | FILE_LIST_DIRECTORY | FILE_READ_ATTRIBUTES | READ_CONTROL,
|
||||
true, security, result->path);
|
||||
try {
|
||||
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "rea-agents",
|
||||
"version": "6.4.0",
|
||||
"version": "6.3.0",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "rea-agents",
|
||||
"version": "6.4.0",
|
||||
"version": "6.3.0",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@babel/parser": "8.0.4",
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "rea-agents",
|
||||
"mcpName": "io.github.morluto/rea",
|
||||
"version": "6.4.0",
|
||||
"version": "6.3.0",
|
||||
"description": "Reverse engineer anything from your terminal or agent with one CLI and MCP server.",
|
||||
"license": "MIT",
|
||||
"repository": {
|
||||
|
||||
@@ -84,8 +84,7 @@ if (!(await exists(join(headers, "node_api.h")))) {
|
||||
}
|
||||
const artifact = "rea-windows-x64.node";
|
||||
// MSVC short-import libraries do not produce a correct Node import table with
|
||||
// GNU ld. Generate a GNU delay-import library for the exact stable Node-API
|
||||
// symbols; addon.cc binds it to the host executable, whatever its file name.
|
||||
// GNU ld. Generate a GNU import library for the exact stable Node-API symbols.
|
||||
const sources = ["addon.cc", "filesystem.cc", "process.cc"];
|
||||
const sourceText = (
|
||||
await Promise.all(
|
||||
@@ -107,7 +106,7 @@ await writeFile(
|
||||
definition,
|
||||
`LIBRARY node.exe\nEXPORTS\n${symbols.join("\n")}\n`,
|
||||
);
|
||||
run("x86_64-w64-mingw32-dlltool", ["-d", definition, "-y", library]);
|
||||
run("x86_64-w64-mingw32-dlltool", ["-d", definition, "-l", library]);
|
||||
run(compiler, [
|
||||
"-std=c++17",
|
||||
"-O2",
|
||||
|
||||
@@ -1,61 +0,0 @@
|
||||
/* Keep the configured launcher identity alive while its resource-limited
|
||||
* Python child runs. Both remain in REA's owned process group. */
|
||||
#define _POSIX_C_SOURCE 200809L
|
||||
#include <errno.h>
|
||||
#include <signal.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <sys/resource.h>
|
||||
#include <sys/types.h>
|
||||
#include <sys/wait.h>
|
||||
#include <unistd.h>
|
||||
|
||||
int main(int argc, char **argv) {
|
||||
char *python = getenv("REA_VERIFY_LIMITED_PYTHON");
|
||||
char *limit = getenv("REA_VERIFY_LIMIT_OPTION");
|
||||
if (python == NULL || python[0] != '/' || limit == NULL) {
|
||||
fputs("limited Python fixture requires an absolute Python and limit\n", stderr);
|
||||
return 127;
|
||||
}
|
||||
struct rlimit core = {0, 0};
|
||||
if (setrlimit(RLIMIT_CORE, &core) != 0) {
|
||||
perror("setrlimit");
|
||||
return 127;
|
||||
}
|
||||
char **arguments = calloc((size_t)argc + 4, sizeof(char *));
|
||||
if (arguments == NULL) return 127;
|
||||
arguments[0] = "/usr/bin/prlimit";
|
||||
arguments[1] = limit;
|
||||
arguments[2] = "--";
|
||||
arguments[3] = python;
|
||||
for (int index = 1; index < argc; index++) arguments[index + 3] = argv[index];
|
||||
pid_t child = fork();
|
||||
if (child == 0) {
|
||||
execv(arguments[0], arguments);
|
||||
perror("execv");
|
||||
_exit(127);
|
||||
}
|
||||
free(arguments);
|
||||
if (child < 0) {
|
||||
perror("fork");
|
||||
return 127;
|
||||
}
|
||||
int status;
|
||||
while (waitpid(child, &status, 0) < 0) {
|
||||
if (errno == EINTR) continue;
|
||||
perror("waitpid");
|
||||
return 127;
|
||||
}
|
||||
if (WIFEXITED(status)) return WEXITSTATUS(status);
|
||||
if (WIFSIGNALED(status)) {
|
||||
int signum = WTERMSIG(status);
|
||||
signal(signum, SIG_DFL);
|
||||
sigset_t signals;
|
||||
sigemptyset(&signals);
|
||||
sigaddset(&signals, signum);
|
||||
sigprocmask(SIG_UNBLOCK, &signals, NULL);
|
||||
raise(signum);
|
||||
return 128 + signum;
|
||||
}
|
||||
return 127;
|
||||
}
|
||||
@@ -28,11 +28,6 @@ import {
|
||||
isolatedFixtureUrl,
|
||||
} from "./browser-runtime-world-e2e.mjs";
|
||||
|
||||
// Arming, progress delivery, isolated-world proof and the page action cross
|
||||
// separate transports. Leave time for their round trips on shared CI runners;
|
||||
// coverage can include a late action even after request collection has ended.
|
||||
const executionObservationMs = 2_000;
|
||||
|
||||
/** Exercise native browser runtime attribution through public CLI and stdio MCP, including arming. */
|
||||
export async function verifyBrowserRuntime(
|
||||
executable,
|
||||
@@ -215,7 +210,7 @@ async function executionMcp(
|
||||
const response = await client.callTool(
|
||||
{
|
||||
name: "observe_web_execution",
|
||||
arguments: { ...input, observation_ms: executionObservationMs },
|
||||
arguments: { ...input, observation_ms: 300 },
|
||||
},
|
||||
{
|
||||
timeout: 40_000,
|
||||
@@ -264,7 +259,7 @@ async function executionCli(entrypoint, input, env, action, isolatedContext) {
|
||||
input.cdp_endpoint,
|
||||
input.target_id,
|
||||
"--observation-ms",
|
||||
String(executionObservationMs),
|
||||
"300",
|
||||
"--json",
|
||||
],
|
||||
{ env, stdio: ["ignore", "pipe", "pipe"] },
|
||||
@@ -387,14 +382,7 @@ function assertExecution(evidence, site, requireBlock = true) {
|
||||
const request = result.requests.find((item) =>
|
||||
item.url.endsWith("/evidence?marker=chosen"),
|
||||
);
|
||||
assert.ok(
|
||||
request,
|
||||
`The selected request initiator must be retained: ${JSON.stringify({
|
||||
window: result.window,
|
||||
requests: result.requests.map((item) => item.url),
|
||||
excluded_requests: result.excluded_requests,
|
||||
})}`,
|
||||
);
|
||||
assert.ok(request, "The selected request initiator must be retained");
|
||||
assert.ok(
|
||||
request.callsites.some(
|
||||
(site) =>
|
||||
|
||||
@@ -176,9 +176,6 @@ const startStorageSite = async () => {
|
||||
origin: `http://127.0.0.1:${port}`,
|
||||
close: async () => {
|
||||
server.close();
|
||||
// Chrome can leave speculative TCP connections without an HTTP request;
|
||||
// server.close() alone waits indefinitely for those owned sockets.
|
||||
server.closeAllConnections();
|
||||
await once(server, "close");
|
||||
},
|
||||
};
|
||||
|
||||
@@ -101,13 +101,10 @@ export async function readReleaseHistory(run, base, target) {
|
||||
// REA history maps Anthropic and Cursor's agent trailers to ordinary User actors.
|
||||
// Keep those source identities in the inventory while excluding automation credit.
|
||||
const AGENT_ACCOUNTS = new Set(["claude", "cursoragent"]);
|
||||
// REA's maintainer requested external-contributor thanks without self-credit.
|
||||
const MAINTAINER_ACCOUNTS = new Set(["morluto"]);
|
||||
const human = (value) =>
|
||||
value?.__typename === "User" &&
|
||||
!value.login.endsWith("[bot]") &&
|
||||
!AGENT_ACCOUNTS.has(value.login.toLowerCase()) &&
|
||||
!MAINTAINER_ACCOUNTS.has(value.login.toLowerCase())
|
||||
!AGENT_ACCOUNTS.has(value.login.toLowerCase())
|
||||
? value.login
|
||||
: undefined;
|
||||
const unique = (values) =>
|
||||
@@ -166,12 +163,6 @@ export function contributionRecord(raw) {
|
||||
return `${lines.join("\n")}\n`;
|
||||
}
|
||||
|
||||
/** Keep the full authorship snapshot available without duplicating public thanks. */
|
||||
export function contributionSummary(raw) {
|
||||
const inventory = releaseInventorySchema.parse(raw);
|
||||
return `Full contribution record: [${inventory.commits.length} commits and ${inventory.pullRequests.length} merged PR${inventory.pullRequests.length === 1 ? "" : "s"}](https://github.com/${inventory.repository}/blob/rea-agents-${inventory.version}/docs/releases/${inventory.version}.contributions.json).\n`;
|
||||
}
|
||||
|
||||
export function selectedReleaseNotes(text, version) {
|
||||
const headings = [...text.matchAll(/^## \[([^\]]+)\].*$/gmu)];
|
||||
if (headings[0]?.[1] !== version)
|
||||
@@ -245,8 +236,7 @@ function requiredCredits(inventory, bullet, references) {
|
||||
if (matches[0].parents.length > 1) continue;
|
||||
for (const identity of matches[0].authors) {
|
||||
const handle = human(identity.user);
|
||||
if (handle)
|
||||
required.push({ handle, context: `commit ${matches[0].sha}` });
|
||||
if (handle) required.push({ handle, context: `commit ${match[2]}` });
|
||||
}
|
||||
}
|
||||
return required;
|
||||
@@ -274,8 +264,6 @@ function verifyEditorialCredits(inventory, prose) {
|
||||
]).map((handle) => handle.toLowerCase()),
|
||||
);
|
||||
const errors = [];
|
||||
const requiredByContext = new Map();
|
||||
const thankedByContext = new Map();
|
||||
for (const bullet of proseBullets(prose)) {
|
||||
const references = bulletPrs(bullet, inventory.repository);
|
||||
const start = bullet.indexOf("Thanks ");
|
||||
@@ -294,21 +282,13 @@ function verifyEditorialCredits(inventory, prose) {
|
||||
bullet,
|
||||
references,
|
||||
)) {
|
||||
const required = requiredByContext.get(context) ?? new Set();
|
||||
required.add(handle);
|
||||
requiredByContext.set(context, required);
|
||||
const credited = thankedByContext.get(context) ?? new Set();
|
||||
for (const login of thanked) credited.add(login);
|
||||
thankedByContext.set(context, credited);
|
||||
if (!thanked.has(handle.toLowerCase()))
|
||||
errors.push(`Missing Thanks @${handle} in bullet for ${context}`);
|
||||
}
|
||||
for (const handle of thanked)
|
||||
if (!eligible.has(handle))
|
||||
errors.push(`Unverified human credit @${handle}`);
|
||||
}
|
||||
for (const [context, required] of requiredByContext)
|
||||
for (const handle of required)
|
||||
if (!thankedByContext.get(context)?.has(handle.toLowerCase()))
|
||||
errors.push(`Missing Thanks @${handle} for ${context}`);
|
||||
return errors;
|
||||
}
|
||||
|
||||
@@ -325,15 +305,8 @@ export function verifyReleaseNotes(raw, text, body) {
|
||||
"Release notes exceed GitHub's body budget; shorten prose without dropping credit",
|
||||
);
|
||||
const record = contributionRecord(inventory).trim();
|
||||
const summary = contributionSummary(inventory).trim();
|
||||
const start = notes.endsWith(summary)
|
||||
? notes.length - summary.length
|
||||
: notes.indexOf("### Complete contribution record");
|
||||
if (
|
||||
start < 0 ||
|
||||
(notes.slice(start).trim() !== record &&
|
||||
notes.slice(start).trim() !== summary)
|
||||
)
|
||||
const start = notes.indexOf("### Complete contribution record");
|
||||
if (start < 0 || notes.slice(start).trim() !== record)
|
||||
throw new Error(
|
||||
"Contribution record differs from inventory; regenerate it with release-notes record",
|
||||
);
|
||||
|
||||
@@ -6,7 +6,6 @@ import { collectGitHubCredits } from "./lib/release-notes-github.mjs";
|
||||
import { verifyNotesPublication } from "./lib/release-notes-publication.mjs";
|
||||
import {
|
||||
contributionRecord,
|
||||
contributionSummary,
|
||||
readReleaseHistory,
|
||||
releaseInventorySchema,
|
||||
selectedReleaseNotes,
|
||||
@@ -74,7 +73,6 @@ async function main() {
|
||||
"pr-body": { type: "string" },
|
||||
source: { type: "string" },
|
||||
output: { type: "string" },
|
||||
summary: { type: "boolean", default: false },
|
||||
},
|
||||
});
|
||||
const [command] = positionals;
|
||||
@@ -94,7 +92,7 @@ async function main() {
|
||||
!values.manifest
|
||||
)
|
||||
throw new Error(
|
||||
"Usage: release-notes inventory|record|check|render --manifest FILE [--repository OWNER/REPO --base TAG --target SHA --version VERSION] [--notes FILE --pr-body FILE --source SHA --output FILE] [--summary for record]",
|
||||
"Usage: release-notes inventory|record|check|render --manifest FILE [--repository OWNER/REPO --base TAG --target SHA --version VERSION] [--notes FILE --pr-body FILE --source SHA --output FILE]",
|
||||
);
|
||||
if (command === "inventory") {
|
||||
await createInventory(values);
|
||||
@@ -105,14 +103,11 @@ async function main() {
|
||||
);
|
||||
await verifyInventoryHistory(run, inventory, values.source);
|
||||
if (command === "record") {
|
||||
const record = values.summary
|
||||
? contributionSummary(inventory)
|
||||
: contributionRecord(inventory);
|
||||
if (values.output)
|
||||
await writeFile(values.output, record, {
|
||||
await writeFile(values.output, contributionRecord(inventory), {
|
||||
flag: "wx",
|
||||
});
|
||||
else process.stdout.write(record);
|
||||
else process.stdout.write(contributionRecord(inventory));
|
||||
return;
|
||||
}
|
||||
const notes = await readFile(values.notes, "utf8");
|
||||
|
||||
@@ -15,10 +15,6 @@ import { promisify } from "node:util";
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { PrivateRuntimeRoot } from "../dist/process/PrivateRuntimeRoot.js";
|
||||
import {
|
||||
OwnedCommandFailure,
|
||||
runOwnedCommand,
|
||||
} from "../dist/process/OwnedCommand.js";
|
||||
import { parseEvidence } from "../dist/domain/evidence.js";
|
||||
import { mcpTextValue } from "./lib/mcp-verifier-results.mjs";
|
||||
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
|
||||
@@ -60,13 +56,6 @@ const root = await PrivateRuntimeRoot.create({
|
||||
const environment = Object.fromEntries(
|
||||
Object.entries(process.env).filter(([, value]) => typeof value === "string"),
|
||||
);
|
||||
const limitedPython = join(root.path, "limited-python");
|
||||
const limitedEnvironment = (option) => ({
|
||||
...environment,
|
||||
REA_PWNTOOLS_PYTHON: limitedPython,
|
||||
REA_VERIFY_LIMITED_PYTHON: python,
|
||||
REA_VERIFY_LIMIT_OPTION: option,
|
||||
});
|
||||
const client = new Client({ name: "binary-layout-verifier", version: "1" });
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
@@ -84,46 +73,6 @@ let cases = 0;
|
||||
let bootstrapCases = 0;
|
||||
const failures = [];
|
||||
try {
|
||||
await execute(
|
||||
"gcc",
|
||||
[
|
||||
"-std=c11",
|
||||
"-Wall",
|
||||
"-Wextra",
|
||||
"-Werror",
|
||||
fileURLToPath(
|
||||
new URL("./fixtures/binary-layout-limited-python.c", import.meta.url),
|
||||
),
|
||||
"-o",
|
||||
limitedPython,
|
||||
],
|
||||
{ timeout: 30_000, maxBuffer: 1024 * 1024 },
|
||||
);
|
||||
// A live limited child must retain its configured launcher's identity so
|
||||
// timeout cleanup can still verify and terminate the entire owned group.
|
||||
await assert.rejects(
|
||||
runOwnedCommand(
|
||||
{
|
||||
command: limitedPython,
|
||||
arguments: [
|
||||
"-I",
|
||||
"-c",
|
||||
"import time; print('limited child ready', flush=True); time.sleep(60)",
|
||||
],
|
||||
cwd: root.path,
|
||||
runId: `${run.run_id}-limited-python-timeout`,
|
||||
hostEnvironment: limitedEnvironment("--as=2147483648"),
|
||||
},
|
||||
{ timeoutMs: 2000, diagnosticBytes: 1024 },
|
||||
),
|
||||
(error) => {
|
||||
assert.ok(error instanceof OwnedCommandFailure);
|
||||
assert.equal(error.reason, "timeout");
|
||||
assert.equal(error.cleanupFailure, null);
|
||||
assert.ok(error.snapshot.stdout.text.includes("limited child ready"));
|
||||
return true;
|
||||
},
|
||||
);
|
||||
const bootstrapRoot = join(root.path, "bootstrap-boundary");
|
||||
await mkdir(bootstrapRoot);
|
||||
const bootstrapPath = join(bootstrapRoot, "layout.py");
|
||||
@@ -847,12 +796,19 @@ try {
|
||||
assert.deepEqual(await readFile(extendedPath), extendedObject);
|
||||
cases++;
|
||||
}
|
||||
for (const [option, expected] of [
|
||||
["--as=2147483648", '"address_space_bytes": 2147483648'],
|
||||
["--cpu=20", '"cpu_seconds": 20'],
|
||||
["--fsize=33554432", '"file_size_bytes": 33554432'],
|
||||
for (const [name, option, expected] of [
|
||||
["address-space", "--as=2147483648", '"address_space_bytes": 2147483648'],
|
||||
["cpu", "--cpu=20", '"cpu_seconds": 20'],
|
||||
["file-size", "--fsize=33554432", '"file_size_bytes": 33554432'],
|
||||
]) {
|
||||
const selectedEnvironment = limitedEnvironment(option);
|
||||
const wrapper = join(root.path, `python-limit-${name}`);
|
||||
const quotedPython = "'" + python.replaceAll("'", "'\"'\"'") + "'";
|
||||
await writeFile(
|
||||
wrapper,
|
||||
`#!/bin/sh\nexec /usr/bin/prlimit ${option} -- ${quotedPython} "$@"\n`,
|
||||
{ mode: 0o700 },
|
||||
);
|
||||
const limitedEnvironment = { ...environment, REA_PWNTOOLS_PYTHON: wrapper };
|
||||
const limitedClient = new Client({
|
||||
name: "limited-layout-verifier",
|
||||
version: "1",
|
||||
@@ -860,7 +816,7 @@ try {
|
||||
const limitedTransport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [entrypoint, "mcp"],
|
||||
env: selectedEnvironment,
|
||||
env: limitedEnvironment,
|
||||
stderr: "pipe",
|
||||
});
|
||||
try {
|
||||
@@ -870,7 +826,7 @@ try {
|
||||
mode,
|
||||
join(root.path, "protected"),
|
||||
undefined,
|
||||
selectedEnvironment,
|
||||
limitedEnvironment,
|
||||
limitedClient,
|
||||
);
|
||||
assert.ok(value.limitations.some((item) => item.includes(expected)));
|
||||
@@ -906,29 +862,17 @@ try {
|
||||
sectionHeavy.writeUInt16LE(sectionCount, 60);
|
||||
const sectionHeavyPath = join(root.path, "section-heavy.o");
|
||||
await writeFile(sectionHeavyPath, sectionHeavy);
|
||||
// pwntools materializes each PT_LOAD zero-filled tail during inspection.
|
||||
// One allocation larger than the whole address-space budget fails promptly;
|
||||
// a large section count instead races CPU work against the command deadline.
|
||||
const memoryHeavy = Buffer.from(await readFile(join(root.path, "protected")));
|
||||
const memoryPhOffset = Number(memoryHeavy.readBigUInt64LE(32));
|
||||
const memoryPhSize = memoryHeavy.readUInt16LE(54);
|
||||
const memoryPhCount = memoryHeavy.readUInt16LE(56);
|
||||
const lastLoadIndex = Array.from(
|
||||
{ length: memoryPhCount },
|
||||
(_, index) => index,
|
||||
).findLast(
|
||||
(index) =>
|
||||
memoryHeavy.readUInt32LE(memoryPhOffset + index * memoryPhSize) === 1,
|
||||
const memoryWrapper = join(root.path, "python-memory-constraint");
|
||||
const quotedPython = "'" + python.replaceAll("'", "'\"'\"'") + "'";
|
||||
await writeFile(
|
||||
memoryWrapper,
|
||||
`#!/bin/sh\nexec /usr/bin/prlimit --as=100663296 -- ${quotedPython} "$@"\n`,
|
||||
{ mode: 0o700 },
|
||||
);
|
||||
assert.notEqual(lastLoadIndex, undefined);
|
||||
const memorySizeOffset = memoryPhOffset + lastLoadIndex * memoryPhSize + 40;
|
||||
memoryHeavy.writeBigUInt64LE(
|
||||
memoryHeavy.readBigUInt64LE(memorySizeOffset) + 128n * 1024n * 1024n,
|
||||
memorySizeOffset,
|
||||
);
|
||||
const memoryHeavyPath = join(root.path, "memory-zero-tail");
|
||||
await writeFile(memoryHeavyPath, memoryHeavy);
|
||||
const memoryEnvironment = limitedEnvironment("--as=100663296");
|
||||
const memoryEnvironment = {
|
||||
...environment,
|
||||
REA_PWNTOOLS_PYTHON: memoryWrapper,
|
||||
};
|
||||
const memoryClient = new Client({
|
||||
name: "memory-layout-verifier",
|
||||
version: "1",
|
||||
@@ -944,7 +888,7 @@ try {
|
||||
for (const mode of ["cli", "mcp"]) {
|
||||
const error = await inspect(
|
||||
mode,
|
||||
memoryHeavyPath,
|
||||
sectionHeavyPath,
|
||||
"resource_constraint",
|
||||
memoryEnvironment,
|
||||
memoryClient,
|
||||
@@ -959,7 +903,7 @@ try {
|
||||
assert.ok(error.remediation.action.includes("memory"));
|
||||
cases++;
|
||||
}
|
||||
assert.deepEqual(await readFile(memoryHeavyPath), memoryHeavy);
|
||||
assert.deepEqual(await readFile(sectionHeavyPath), sectionHeavy);
|
||||
} finally {
|
||||
try {
|
||||
await memoryClient.close();
|
||||
@@ -980,7 +924,13 @@ try {
|
||||
cpuHeavy.writeBigUInt64LE(BigInt(cpuSectionCount), sectionOffset + 32);
|
||||
const cpuHeavyPath = join(root.path, "cpu-section-heavy.o");
|
||||
await writeFile(cpuHeavyPath, cpuHeavy);
|
||||
const cpuEnvironment = limitedEnvironment("--cpu=1:");
|
||||
const cpuWrapper = join(root.path, "python-cpu-constraint");
|
||||
await writeFile(
|
||||
cpuWrapper,
|
||||
`#!/bin/sh\nexec /usr/bin/prlimit --cpu=1: --core=0: -- ${quotedPython} "$@"\n`,
|
||||
{ mode: 0o700 },
|
||||
);
|
||||
const cpuEnvironment = { ...environment, REA_PWNTOOLS_PYTHON: cpuWrapper };
|
||||
const cpuClient = new Client({ name: "cpu-layout-verifier", version: "1" });
|
||||
const cpuTransport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
@@ -1013,7 +963,13 @@ try {
|
||||
}
|
||||
}
|
||||
for (const limit of [1024, 1]) {
|
||||
const fileEnvironment = limitedEnvironment(`--fsize=${limit}:`);
|
||||
const wrapper = join(root.path, `python-file-size-${limit}`);
|
||||
await writeFile(
|
||||
wrapper,
|
||||
`#!/bin/sh\nexec /usr/bin/prlimit --fsize=${limit}: --core=0: -- ${quotedPython} "$@"\n`,
|
||||
{ mode: 0o700 },
|
||||
);
|
||||
const fileEnvironment = { ...environment, REA_PWNTOOLS_PYTHON: wrapper };
|
||||
const fileClient = new Client({
|
||||
name: "file-size-layout-verifier",
|
||||
version: "1",
|
||||
|
||||
+97
-145
@@ -38,147 +38,104 @@ let update;
|
||||
let repositorySkill;
|
||||
let report;
|
||||
|
||||
const runStage = async (name, action) => {
|
||||
const started = performance.now();
|
||||
process.stderr.write(`[verify:package] ${name}: start\n`);
|
||||
const result = await action();
|
||||
process.stderr.write(
|
||||
`[verify:package] ${name}: complete (${Math.round(performance.now() - started)} ms)\n`,
|
||||
);
|
||||
return result;
|
||||
};
|
||||
|
||||
try {
|
||||
repositorySkill = await runStage("repository skill", () =>
|
||||
verifyRepositorySkill({ root, workspace }),
|
||||
);
|
||||
({ tarball } = await runStage("package archive", () =>
|
||||
verifyPackagePack({ root, workspace }),
|
||||
));
|
||||
const environmentData = await runStage("package environment", () =>
|
||||
verifyPackageEnvironment({
|
||||
root,
|
||||
workspace,
|
||||
evidenceRoot,
|
||||
referenceRoot,
|
||||
}),
|
||||
);
|
||||
const { cli, packageRunnerCli } = await runStage("package install", () =>
|
||||
verifyPackageInstall({
|
||||
tarball,
|
||||
prefix: environmentData.prefix,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
update = await runStage("package update", () =>
|
||||
verifyPackageUpdate({
|
||||
prefix: environmentData.prefix,
|
||||
packageRoot: join(
|
||||
environmentData.prefix,
|
||||
"lib",
|
||||
"node_modules",
|
||||
PRODUCT_IDENTITY.packageName,
|
||||
),
|
||||
tarball,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
mcpStartup = await runStage("MCP startup", () =>
|
||||
measureMcpStartup({
|
||||
command: cli,
|
||||
args: ["mcp"],
|
||||
environment: {
|
||||
...environmentData.environment,
|
||||
},
|
||||
policy: MCP_STARTUP_POLICY,
|
||||
}),
|
||||
);
|
||||
mcpModuleLoading = await runStage("MCP module loading", () =>
|
||||
profileMcpModuleLoading({
|
||||
command: cli,
|
||||
args: ["mcp"],
|
||||
environment: {
|
||||
...environmentData.environment,
|
||||
},
|
||||
policy: MCP_STARTUP_POLICY,
|
||||
packageName: PRODUCT_IDENTITY.packageName,
|
||||
}),
|
||||
);
|
||||
const { supportedSetupHost, hopperSetupSupported } = await runStage(
|
||||
"package discovery",
|
||||
() =>
|
||||
verifyPackageDiscovery({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
const { artifactArchive } = await runStage("artifact and Electron CLI", () =>
|
||||
verifyPackageArtifactAndElectron({
|
||||
cli,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
await runStage("managed analysis CLI", () =>
|
||||
verifyManaged({
|
||||
cli,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
await runStage("unknown provider CLI", () =>
|
||||
verifyUnknownProvider({
|
||||
repositorySkill = await verifyRepositorySkill({ root, workspace });
|
||||
({ tarball } = await verifyPackagePack({ root, workspace }));
|
||||
const environmentData = await verifyPackageEnvironment({
|
||||
root,
|
||||
workspace,
|
||||
evidenceRoot,
|
||||
referenceRoot,
|
||||
});
|
||||
const { cli, packageRunnerCli } = await verifyPackageInstall({
|
||||
tarball,
|
||||
prefix: environmentData.prefix,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
update = await verifyPackageUpdate({
|
||||
prefix: environmentData.prefix,
|
||||
packageRoot: join(
|
||||
environmentData.prefix,
|
||||
"lib",
|
||||
"node_modules",
|
||||
PRODUCT_IDENTITY.packageName,
|
||||
),
|
||||
tarball,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
mcpStartup = await measureMcpStartup({
|
||||
command: cli,
|
||||
args: ["mcp"],
|
||||
environment: {
|
||||
...environmentData.environment,
|
||||
},
|
||||
policy: MCP_STARTUP_POLICY,
|
||||
});
|
||||
mcpModuleLoading = await profileMcpModuleLoading({
|
||||
command: cli,
|
||||
args: ["mcp"],
|
||||
environment: {
|
||||
...environmentData.environment,
|
||||
},
|
||||
policy: MCP_STARTUP_POLICY,
|
||||
packageName: PRODUCT_IDENTITY.packageName,
|
||||
});
|
||||
const { supportedSetupHost, hopperSetupSupported } =
|
||||
await verifyPackageDiscovery({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
await runStage("platform CLI", () =>
|
||||
verifyPackagePlatform({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
await runStage("capabilities and search CLI", () =>
|
||||
verifyPackageCapabilitiesAndSearch({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
await runStage("evidence CLI", () =>
|
||||
verifyPackageEvidence({
|
||||
cli,
|
||||
evidenceRoot,
|
||||
referenceRoot,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
await runStage("setup and uninstall CLI", () =>
|
||||
verifyPackageSetup({
|
||||
cli,
|
||||
packageRunnerCli,
|
||||
environment: environmentData.environment,
|
||||
home: environmentData.home,
|
||||
npxLog: environmentData.npxLog,
|
||||
claudeConfig: environmentData.claudeConfig,
|
||||
codexConfig: environmentData.codexConfig,
|
||||
cursorConfig: environmentData.cursorConfig,
|
||||
codexTarget: environmentData.codexTarget,
|
||||
cursorTarget: environmentData.cursorTarget,
|
||||
supportedSetupHost,
|
||||
hopperSetupSupported,
|
||||
root,
|
||||
}),
|
||||
);
|
||||
await runStage("MCP tools and evidence", () =>
|
||||
verifyPackageMcp({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
evidenceRoot,
|
||||
artifactArchive,
|
||||
}),
|
||||
);
|
||||
});
|
||||
const { artifactArchive } = await verifyPackageArtifactAndElectron({
|
||||
cli,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
await verifyManaged({
|
||||
cli,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
await verifyUnknownProvider({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
await verifyPackagePlatform({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
await verifyPackageCapabilitiesAndSearch({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
await verifyPackageEvidence({
|
||||
cli,
|
||||
evidenceRoot,
|
||||
referenceRoot,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
await verifyPackageSetup({
|
||||
cli,
|
||||
packageRunnerCli,
|
||||
environment: environmentData.environment,
|
||||
home: environmentData.home,
|
||||
npxLog: environmentData.npxLog,
|
||||
claudeConfig: environmentData.claudeConfig,
|
||||
codexConfig: environmentData.codexConfig,
|
||||
cursorConfig: environmentData.cursorConfig,
|
||||
codexTarget: environmentData.codexTarget,
|
||||
cursorTarget: environmentData.cursorTarget,
|
||||
supportedSetupHost,
|
||||
hopperSetupSupported,
|
||||
root,
|
||||
});
|
||||
await verifyPackageMcp({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
evidenceRoot,
|
||||
artifactArchive,
|
||||
});
|
||||
report = {
|
||||
cli: true,
|
||||
analysisCli: true,
|
||||
@@ -216,13 +173,8 @@ try {
|
||||
mcpModuleLoading,
|
||||
};
|
||||
} finally {
|
||||
await runStage("workspace cleanup", () =>
|
||||
rm(workspace, { recursive: true, force: true }),
|
||||
);
|
||||
await rm(workspace, { recursive: true, force: true });
|
||||
}
|
||||
const verifierLineage = await runStage("verifier lineage", () =>
|
||||
completeVerifierRun(verifierRun),
|
||||
);
|
||||
process.stdout.write(
|
||||
`${JSON.stringify({ verifier_run: verifierLineage, ...report })}\n`,
|
||||
`${JSON.stringify({ verifier_run: await completeVerifierRun(verifierRun), ...report })}\n`,
|
||||
);
|
||||
|
||||
@@ -277,13 +277,11 @@ try {
|
||||
throw new Error("Scenario attachment terminated its external browser");
|
||||
|
||||
const profilesBefore = await scenarioProfiles();
|
||||
process.stderr.write("Browser verifier: scenario environment\n");
|
||||
const scenarioEnvironment = await verifyScenarioEnvironment(
|
||||
endpoint,
|
||||
target,
|
||||
site.origin,
|
||||
);
|
||||
process.stderr.write("Browser verifier: scenario storage\n");
|
||||
const scenarioStorage = await verifyScenarioStorage({
|
||||
executable,
|
||||
endpoint,
|
||||
@@ -341,9 +339,7 @@ try {
|
||||
`Browser scenario included unselected structured value: ${unselectedValue}`,
|
||||
);
|
||||
|
||||
process.stderr.write("Browser verifier: large screenshot\n");
|
||||
const largeScreenshot = await verifyLargeScreenshotE2e(endpoint, site.origin);
|
||||
process.stderr.write("Browser verifier: popup coverage\n");
|
||||
const popupEvents = await verifyPopupEventCoverage(executable);
|
||||
const networkEvidence = await verifyBrowserNetworkEvidence(
|
||||
executable,
|
||||
|
||||
@@ -674,113 +674,6 @@ export async function verifyRelativeSwitch(client, procedures, entrySize) {
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Oracle for relative-switch-signed.S: signed halfword byte offsets from a base
|
||||
* inside the case bodies, a CBZ route for the zero entry, and a B.CS bound.
|
||||
*/
|
||||
const SIGNED_RELATIVE_SWITCH_CASES = [
|
||||
{ value: 0, immediate: 200 },
|
||||
{ value: 1, immediate: 201 },
|
||||
{ value: 2, immediate: 202 },
|
||||
{ value: 3, immediate: 250 },
|
||||
{ value: 4, immediate: 204 },
|
||||
];
|
||||
|
||||
export async function verifyRelativeSignedSwitch(client, selectorBias = 0) {
|
||||
const names = await inventoryCall(client, "list_names", {
|
||||
document: null,
|
||||
address: null,
|
||||
});
|
||||
const procedure = requireProcedure(names, "rea_relative_switch_signed");
|
||||
const dossier = await functionCall(client, "analyze_function", {
|
||||
procedure: procedure.address,
|
||||
});
|
||||
const table = dossier.native_api?.jump_tables.find((item) =>
|
||||
item.data_sources.some(
|
||||
(source) =>
|
||||
source.entry_size_bytes === 2 &&
|
||||
source.entry_count === SIGNED_RELATIVE_SWITCH_CASES.length,
|
||||
),
|
||||
);
|
||||
if (table === undefined)
|
||||
throw new Error(
|
||||
`Exact signed halfword relative table unavailable: ${JSON.stringify(dossier.native_api)}`,
|
||||
);
|
||||
const expectedValues = SIGNED_RELATIVE_SWITCH_CASES.map(
|
||||
({ value }) => value + selectorBias,
|
||||
);
|
||||
if (
|
||||
table.mappings.some(
|
||||
(mapping) =>
|
||||
mapping.case_value !== null &&
|
||||
!expectedValues.includes(mapping.case_value),
|
||||
)
|
||||
)
|
||||
throw new Error(
|
||||
`Signed relative switch emitted an unsupported case label: ${JSON.stringify(table)}`,
|
||||
);
|
||||
if (table.mappings.length !== SIGNED_RELATIVE_SWITCH_CASES.length)
|
||||
throw new Error(
|
||||
`Signed relative switch has unexpected mappings: ${JSON.stringify(table)}`,
|
||||
);
|
||||
if (selectorBias !== 0) {
|
||||
if (
|
||||
!table.limitations.some((limitation) =>
|
||||
limitation.includes("table indices are not proven"),
|
||||
) ||
|
||||
!table.mappings.some((mapping) => mapping.case_value === null) ||
|
||||
table.mappings.some((mapping) => mapping.case_value === selectorBias + 3)
|
||||
)
|
||||
throw new Error(
|
||||
`Normalized selector must retain unknown case evidence: ${JSON.stringify(table)}`,
|
||||
);
|
||||
}
|
||||
for (const { value: tableIndex, immediate } of SIGNED_RELATIVE_SWITCH_CASES) {
|
||||
const value = tableIndex + selectorBias;
|
||||
// Without a proven relationship between the machine index and the typed
|
||||
// switch selector, the CBZ-only case must stay unknown rather than being
|
||||
// assigned the normalized index or Ghidra's unreachable base destination.
|
||||
if (selectorBias !== 0 && tableIndex === 3) continue;
|
||||
const mapping = table.mappings.find((item) => item.case_value === value);
|
||||
if (
|
||||
mapping === undefined ||
|
||||
mapping.confidence !== "high" ||
|
||||
(selectorBias !== 0 &&
|
||||
!mapping.evidence.some(
|
||||
(evidence) => evidence.source === "ghidra-clang-case-token",
|
||||
))
|
||||
)
|
||||
throw new Error(
|
||||
`Signed relative switch case ${value} was not verified: ${JSON.stringify(table)}`,
|
||||
);
|
||||
const instruction = await functionCall(
|
||||
client,
|
||||
"inspect_native_instruction",
|
||||
{ address: mapping.target_address },
|
||||
);
|
||||
if (
|
||||
instruction.status !== "decoded" ||
|
||||
!instruction.operands.some((operand) =>
|
||||
operand.components.some(
|
||||
(component) =>
|
||||
component.kind === "immediate" &&
|
||||
component.value === `0x${immediate.toString(16)}`,
|
||||
),
|
||||
)
|
||||
)
|
||||
throw new Error(
|
||||
`Signed relative switch target does not implement source case ${value}: ${JSON.stringify(instruction)}`,
|
||||
);
|
||||
}
|
||||
return {
|
||||
entry_size: 2,
|
||||
signed: true,
|
||||
selector_bias: selectorBias,
|
||||
dispatch_address: table.dispatch_address,
|
||||
mappings: table.mappings.length,
|
||||
};
|
||||
}
|
||||
|
||||
export async function verifyNativeValueTrace(client, procedures, target) {
|
||||
const entry = requireProcedure(procedures, "rea_ghidra_inventory_entry");
|
||||
const names = await inventoryCall(client, "list_names", {
|
||||
|
||||
@@ -38,7 +38,6 @@ import {
|
||||
verifyJumpTableDenseSwitch,
|
||||
verifyNativeTypeLayout,
|
||||
verifyRelativeSwitch,
|
||||
verifyRelativeSignedSwitch,
|
||||
verifyNativeValueTrace,
|
||||
} from "./verify-real-ghidra-function.mjs";
|
||||
const exec = promisify(execFile);
|
||||
@@ -174,36 +173,6 @@ try {
|
||||
]);
|
||||
}
|
||||
}
|
||||
const signedRelativeSource = fileURLToPath(
|
||||
new URL(
|
||||
"../tests/conformance/ghidra/relative-switch-signed.S",
|
||||
import.meta.url,
|
||||
),
|
||||
);
|
||||
for (const selectorBias of [0, 2, 10]) {
|
||||
for (const format of [
|
||||
"elf",
|
||||
...(expectedNativeTarget.architecture === "arm64" ? ["mach-o"] : []),
|
||||
]) {
|
||||
const targetPath = join(
|
||||
fixtureRoot,
|
||||
`relative-signed-${selectorBias}-${format}.o`,
|
||||
);
|
||||
await exec(clang, [
|
||||
...(format === "elf" ? ["--target=aarch64-linux-gnu"] : []),
|
||||
`-DREA_SELECTOR_BIAS=${selectorBias}`,
|
||||
"-c",
|
||||
signedRelativeSource,
|
||||
"-o",
|
||||
targetPath,
|
||||
]);
|
||||
crossTargets.push([
|
||||
targetPath,
|
||||
`relative-signed-${selectorBias}`,
|
||||
{ format, architecture: "arm64" },
|
||||
]);
|
||||
}
|
||||
}
|
||||
|
||||
await exec(clang, [
|
||||
"--target=aarch64-linux-gnu",
|
||||
@@ -553,30 +522,25 @@ async function verifyTarget(targetPath, variant, expectedTarget = null) {
|
||||
client,
|
||||
})
|
||||
: null;
|
||||
const probes = variant.startsWith("relative-signed-")
|
||||
? await verifyRelativeSignedSwitch(
|
||||
const probes = variant.startsWith("relative-")
|
||||
? await verifyRelativeSwitch(
|
||||
client,
|
||||
Number(variant.split("-").at(-1)),
|
||||
procedures,
|
||||
Number(variant.slice(-1)),
|
||||
)
|
||||
: variant.startsWith("relative-")
|
||||
? await verifyRelativeSwitch(
|
||||
client,
|
||||
procedures,
|
||||
Number(variant.slice(-1)),
|
||||
)
|
||||
: variant === "type-layout"
|
||||
? await verifyNativeTypeLayout(client, names)
|
||||
: variant === "jump-table"
|
||||
? await verifyJumpTableDenseSwitch(client, procedures)
|
||||
: variant === "custom" || variant === "aarch64-jump-table"
|
||||
? null
|
||||
: await verifyInventoryOperations({
|
||||
client,
|
||||
variant,
|
||||
procedures,
|
||||
names,
|
||||
strings,
|
||||
});
|
||||
: variant === "type-layout"
|
||||
? await verifyNativeTypeLayout(client, names)
|
||||
: variant === "jump-table"
|
||||
? await verifyJumpTableDenseSwitch(client, procedures)
|
||||
: variant === "custom" || variant === "aarch64-jump-table"
|
||||
? null
|
||||
: await verifyInventoryOperations({
|
||||
client,
|
||||
variant,
|
||||
procedures,
|
||||
names,
|
||||
strings,
|
||||
});
|
||||
const nativeValues =
|
||||
variant === "debug"
|
||||
? await verifyNativeValueTrace(client, procedures, parsedTarget.value)
|
||||
|
||||
@@ -6,7 +6,6 @@ import { createHash } from "node:crypto";
|
||||
import { once } from "node:events";
|
||||
import {
|
||||
access,
|
||||
copyFile,
|
||||
mkdir,
|
||||
mkdtemp,
|
||||
readFile,
|
||||
@@ -17,7 +16,7 @@ import {
|
||||
open,
|
||||
} from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { dirname, join, parse, resolve } from "node:path";
|
||||
import { join, resolve } from "node:path";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { verifyWindowsProtocolInput } from "./lib/windows-protocol-input.mjs";
|
||||
|
||||
@@ -33,9 +32,6 @@ const native = requireWindowsNativeAuthority();
|
||||
const { WindowsPrivateRuntime, windowsPrivateRuntime } = await import(
|
||||
pathToFileURL(join(packageRoot, "dist/windows/WindowsPrivateRuntime.js"))
|
||||
);
|
||||
const { ghidraSessionRoot } = await import(
|
||||
pathToFileURL(join(packageRoot, "dist/ghidra/GhidraSessionRoot.js"))
|
||||
);
|
||||
const workspace = await mkdtemp(join(tmpdir(), "rea-native-conformance-"));
|
||||
const environment = Object.entries(process.env)
|
||||
.filter(([, value]) => value !== undefined)
|
||||
@@ -81,31 +77,6 @@ const wait = async (child) => {
|
||||
};
|
||||
const report = { ok: false, artifact: "verified", controls: {}, processes: {} };
|
||||
try {
|
||||
// Exercise the loader in a process whose image is not named node.exe. The
|
||||
// ordinary Windows lane otherwise cannot detect eager Node-API imports.
|
||||
const renamedHost = join(workspace, "rea-renamed-node.exe");
|
||||
const renamedHostFixture = join(workspace, "renamed-host.mjs");
|
||||
const loaderUrl = pathToFileURL(
|
||||
join(packageRoot, "dist/windows/WindowsNativeLoader.js"),
|
||||
).href;
|
||||
await copyFile(process.execPath, renamedHost);
|
||||
await writeFile(
|
||||
renamedHostFixture,
|
||||
`import {basename} from 'node:path';
|
||||
import {requireWindowsNativeAuthority} from ${JSON.stringify(loaderUrl)};
|
||||
const {inspection}=requireWindowsNativeAuthority();
|
||||
process.stdout.write(JSON.stringify({executable:basename(process.execPath),abiVersion:inspection.abiVersion}));\n`,
|
||||
);
|
||||
const renamedHostResult = execFileSync(renamedHost, [renamedHostFixture], {
|
||||
encoding: "utf8",
|
||||
timeout: 20_000,
|
||||
windowsHide: true,
|
||||
});
|
||||
assert.deepEqual(JSON.parse(renamedHostResult), {
|
||||
executable: "rea-renamed-node.exe",
|
||||
abiVersion: 1,
|
||||
});
|
||||
report.controls.renamedHostNodeApi = true;
|
||||
const sourceDirectory = join(workspace, "source");
|
||||
await mkdir(sourceDirectory);
|
||||
const source = join(sourceDirectory, "target.bin");
|
||||
@@ -139,47 +110,6 @@ process.stdout.write(JSON.stringify({executable:basename(process.execPath),abiVe
|
||||
])
|
||||
assert.throws(() => native.call("open", [rejected]));
|
||||
report.controls.ordinaryDriveSeparatorsAndRequestedIdentity = true;
|
||||
const driveRoot = parse(workspace).root;
|
||||
const runtimeParent = ghidraSessionRoot({
|
||||
base: join(driveRoot, ".Tmp", "scratch"),
|
||||
});
|
||||
assert.equal(runtimeParent, driveRoot);
|
||||
const driveRootRuntime = native.call("runtime_create", [
|
||||
runtimeParent,
|
||||
"rea-drive-root-",
|
||||
]);
|
||||
try {
|
||||
assert.equal(driveRootRuntime.path.startsWith(`${driveRoot}\\`), false);
|
||||
assert.equal(dirname(driveRootRuntime.path), driveRoot);
|
||||
const snapshot = await native.call("runtime_snapshot", [
|
||||
driveRootRuntime.handle,
|
||||
source,
|
||||
"probe.bin",
|
||||
]);
|
||||
const sourceBytes = await readFile(source);
|
||||
assert.equal(
|
||||
snapshot.sha256,
|
||||
createHash("sha256").update(sourceBytes).digest("hex"),
|
||||
);
|
||||
assert.equal(snapshot.source.requestedPath, source);
|
||||
const readback = native.call("runtime_open", [
|
||||
driveRootRuntime.handle,
|
||||
"probe.bin",
|
||||
]);
|
||||
try {
|
||||
assert.equal(readback.size, sourceBytes.byteLength);
|
||||
assert.deepEqual(
|
||||
native.call("read", [readback.handle, 0, 65536]),
|
||||
sourceBytes.subarray(0, 65536),
|
||||
);
|
||||
} finally {
|
||||
native.call("close", [readback.handle]);
|
||||
}
|
||||
} finally {
|
||||
native.call("runtime_close", [driveRootRuntime.handle]);
|
||||
}
|
||||
assert.equal(await exists(driveRootRuntime.path), false);
|
||||
report.controls.driveRootRuntimeParent = true;
|
||||
const snapshotOwner = WindowsPrivateRuntime.create(
|
||||
workspace.replace(/\\/u, "/"),
|
||||
"single-flight-",
|
||||
|
||||
+2
-2
@@ -8,13 +8,13 @@
|
||||
"url": "https://github.com/morluto/rea",
|
||||
"source": "github"
|
||||
},
|
||||
"version": "6.4.0",
|
||||
"version": "6.3.0",
|
||||
"packages": [
|
||||
{
|
||||
"registryType": "npm",
|
||||
"registryBaseUrl": "https://registry.npmjs.org",
|
||||
"identifier": "rea-agents",
|
||||
"version": "6.4.0",
|
||||
"version": "6.3.0",
|
||||
"runtimeHint": "npx",
|
||||
"transport": {
|
||||
"type": "stdio"
|
||||
|
||||
@@ -1,21 +1,17 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { lstat, opendir, readFile, realpath, stat } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
import { parseBinary } from "plist";
|
||||
import { z } from "zod";
|
||||
|
||||
import { preflightPropertyListDecode } from "../artifacts/apple/InterfaceBuilderDecodeBudget.js";
|
||||
import { BinaryTargetError } from "../domain/configurationErrors.js";
|
||||
import { isPathWithinRoot } from "../domain/localPath.js";
|
||||
import {
|
||||
omitPrototypeKeys,
|
||||
parseXmlPropertyList,
|
||||
} from "../domain/propertyListKeys.js";
|
||||
import { parseXmlPropertyList } from "../domain/propertyListKeys.js";
|
||||
import { decodeXmlPlistText } from "../domain/propertyListXmlText.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
|
||||
/** Decoded representation bound for one binary Info.plist. */
|
||||
const BINARY_PLIST_DECODE_BYTES = 16 * 1024 * 1024;
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
/** Where one app bundle layout keeps its Info.plist and program file. */
|
||||
interface BundleLayout {
|
||||
@@ -197,8 +193,9 @@ const resolveLayoutExecutable = async (
|
||||
try {
|
||||
name =
|
||||
plist.subarray(0, 6).toString("ascii") === "bplist"
|
||||
? await (fileSystem.decodeBinaryPlist?.(plistPath) ??
|
||||
parseBinaryPlistExecutable(plist))
|
||||
? await (fileSystem.decodeBinaryPlist ?? readBinaryPlistExecutable)(
|
||||
plistPath,
|
||||
)
|
||||
: parseXmlPlistExecutable(decodeXmlPlistText(plist));
|
||||
} catch (cause: unknown) {
|
||||
return err(
|
||||
@@ -251,10 +248,6 @@ const resolveLayoutExecutable = async (
|
||||
const parseXmlPlistExecutable = (plist: string): string => {
|
||||
// An unrelated `__proto__` entry must not make the bundle unreadable.
|
||||
const { value } = parseXmlPropertyList(plist);
|
||||
return executableName(value);
|
||||
};
|
||||
|
||||
const executableName = (value: unknown): string => {
|
||||
const executable = executableEntrySchema.safeParse(value);
|
||||
if (!executable.success) throw new Error("CFBundleExecutable is missing");
|
||||
return executable.data.CFBundleExecutable;
|
||||
@@ -262,13 +255,19 @@ const executableName = (value: unknown): string => {
|
||||
|
||||
const executableEntrySchema = z.looseObject({ CFBundleExecutable: z.string() });
|
||||
|
||||
/** Decode a binary plist in-process, so no host plist tool is required. */
|
||||
const parseBinaryPlistExecutable = (plist: Buffer): string => {
|
||||
preflightPropertyListDecode(plist, BINARY_PLIST_DECODE_BYTES);
|
||||
// `plist` assigns a `__proto__` key as the prototype; copying drops it, as
|
||||
// the XML path does.
|
||||
return executableName(omitPrototypeKeys(parseBinary(plist)).value);
|
||||
};
|
||||
const readBinaryPlistExecutable = async (plistPath: string): Promise<string> =>
|
||||
// `-n` strips only the newline plutil appends; it requires macOS 12+.
|
||||
(
|
||||
await execFileAsync("/usr/bin/plutil", [
|
||||
"-extract",
|
||||
"CFBundleExecutable",
|
||||
"raw",
|
||||
"-n",
|
||||
"-o",
|
||||
"-",
|
||||
plistPath,
|
||||
])
|
||||
).stdout;
|
||||
|
||||
const isSafeExecutableName = (name: string): boolean =>
|
||||
name.length > 0 &&
|
||||
|
||||
@@ -138,7 +138,7 @@ export const runManagedProviderExecution = async (
|
||||
signal?: AbortSignal,
|
||||
): Promise<Result<AnalysisExecution, AnalysisError>> =>
|
||||
withProcessCancellation(signal, async (operationSignal) => {
|
||||
const session = await dependencies.createManagedBinarySession();
|
||||
const session = dependencies.createManagedBinarySession();
|
||||
return withSessionCleanup(
|
||||
session,
|
||||
async () => {
|
||||
@@ -194,7 +194,7 @@ const runAnalysis = async (
|
||||
const { logger, signal, snapshotPath } = options;
|
||||
const config = dependencies.readConfiguration();
|
||||
if (!config.ok) return cliError(config.error);
|
||||
const session = await dependencies.createBinarySession(config.value, logger);
|
||||
const session = dependencies.createBinarySession(config.value, logger);
|
||||
return withSessionCleanup(
|
||||
session,
|
||||
async () => {
|
||||
|
||||
@@ -10,8 +10,6 @@ export interface DirectAnalysisDependencies {
|
||||
readonly createBinarySession: (
|
||||
config: AppConfig,
|
||||
logger: Logger,
|
||||
) => BinarySession | Promise<BinarySession>;
|
||||
readonly createManagedBinarySession: () =>
|
||||
| BinarySession
|
||||
| Promise<BinarySession>;
|
||||
) => BinarySession;
|
||||
readonly createManagedBinarySession: () => BinarySession;
|
||||
}
|
||||
|
||||
@@ -15,7 +15,7 @@ export const runSessionStatus = async (
|
||||
): Promise<JsonValue> => {
|
||||
const config = dependencies.readConfiguration();
|
||||
if (!config.ok) return { error: projectAnalysisError(config.error) };
|
||||
const session = await dependencies.createBinarySession(config.value, logger);
|
||||
const session = dependencies.createBinarySession(config.value, logger);
|
||||
try {
|
||||
return {
|
||||
...jsonObjectSchema.parse(session.status()),
|
||||
|
||||
@@ -1,35 +0,0 @@
|
||||
import type { EvidenceBundle } from "../../domain/evidenceBundle.js";
|
||||
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
|
||||
import { err, ok, type Result } from "../../domain/result.js";
|
||||
import {
|
||||
summarizeEvidenceBundle,
|
||||
type EvidenceBundleSummary,
|
||||
type EvidenceRecordFilters,
|
||||
} from "../../domain/evidenceRecordSummary.js";
|
||||
|
||||
/** Select complete bundle delivery or discovery metadata without analysis payloads. */
|
||||
export const inspectEvidenceBundle = (
|
||||
bundle: EvidenceBundle,
|
||||
options: {
|
||||
readonly detail: "complete" | "summary";
|
||||
readonly filters?: EvidenceRecordFilters | undefined;
|
||||
},
|
||||
): Result<EvidenceBundle | EvidenceBundleSummary, AnalysisInputError> => {
|
||||
if (options.detail === "complete" && options.filters !== undefined)
|
||||
return err(
|
||||
new AnalysisInputError("get_evidence_bundle", undefined, [
|
||||
{
|
||||
path: ["filters"],
|
||||
reason: "invalid_value",
|
||||
message:
|
||||
"Record filters require detail: summary; complete returns the canonical bundle.",
|
||||
expected: "detail: summary",
|
||||
},
|
||||
]),
|
||||
);
|
||||
return ok(
|
||||
options.detail === "summary"
|
||||
? summarizeEvidenceBundle(bundle, options.filters)
|
||||
: bundle,
|
||||
);
|
||||
};
|
||||
@@ -26,7 +26,6 @@ import {
|
||||
staticInferenceEvidence,
|
||||
} from "./JavaScriptArtifactGraphEvidence.js";
|
||||
import { resolveArtifactPathByContext } from "./JavaScriptArtifactPathResolution.js";
|
||||
import { compareUnicodeCodePoints } from "../../domain/unicodeCodePointOrder.js";
|
||||
|
||||
interface PackageRoleInput {
|
||||
readonly packageNode: ApplicationNode;
|
||||
@@ -282,7 +281,6 @@ export const addJavaScriptPackageNodes = (
|
||||
context: JavaScriptArtifactGraphContext,
|
||||
): ApplicationNode[] => {
|
||||
const roots: ApplicationNode[] = [];
|
||||
const rootPath = applicationManifestPath(context);
|
||||
for (const packageValue of context.analysis.packages) {
|
||||
const file = context.filesByPath.get(packageValue.path);
|
||||
if (file === undefined) continue;
|
||||
@@ -311,7 +309,7 @@ export const addJavaScriptPackageNodes = (
|
||||
},
|
||||
],
|
||||
});
|
||||
if (file.path === rootPath) {
|
||||
if (roots.length === 0) {
|
||||
roots.push(node);
|
||||
context.accumulator.addEdge({
|
||||
source_node_id: node.node_id,
|
||||
@@ -332,7 +330,6 @@ export const addJavaScriptPackageNodes = (
|
||||
file,
|
||||
operation: "inventory-package",
|
||||
});
|
||||
if (dependencyManifest(file.path)) continue;
|
||||
addPackageRole(context, {
|
||||
packageNode: node,
|
||||
packageFile: file,
|
||||
@@ -349,24 +346,6 @@ export const addJavaScriptPackageNodes = (
|
||||
return roots;
|
||||
};
|
||||
|
||||
/** Dependency-only artifacts retain their inventory root. */
|
||||
const applicationManifestPath = (
|
||||
context: JavaScriptArtifactGraphContext,
|
||||
): string | undefined =>
|
||||
context.analysis.packages
|
||||
.map(({ path }) => path)
|
||||
.filter(
|
||||
(path) => context.filesByPath.has(path) && !dependencyManifest(path),
|
||||
)
|
||||
.sort(
|
||||
(left, right) =>
|
||||
left.split("/").length - right.split("/").length ||
|
||||
compareUnicodeCodePoints(left, right),
|
||||
)[0];
|
||||
|
||||
const dependencyManifest = (path: string): boolean =>
|
||||
path.split("/").slice(0, -1).includes("node_modules");
|
||||
|
||||
const createFileTarget = (
|
||||
context: JavaScriptArtifactGraphContext,
|
||||
file: JavaScriptArtifactFile,
|
||||
|
||||
@@ -43,7 +43,6 @@ import {
|
||||
type WebScreenshotDiff,
|
||||
} from "../domain/webScreenshot.js";
|
||||
import { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import { AnalysisInputError } from "../domain/analysisErrorCore.js";
|
||||
import { BrowserObservationError } from "../domain/browserObservationError.js";
|
||||
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
|
||||
import { type BrowserObservationOperation } from "../domain/browserObservationErrors.js";
|
||||
@@ -315,12 +314,6 @@ export class CdpBrowserProvider implements BrowserObservationPort {
|
||||
try {
|
||||
return ok(webScreenshotDiffSchema.parse(comparePngScreenshots(input)));
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof TypeError)
|
||||
return err(
|
||||
new AnalysisInputError("compare_web_screenshots", { cause }, [
|
||||
{ path: [], reason: "invalid_format", message: cause.message },
|
||||
]),
|
||||
);
|
||||
return err(providerError(cause, "compare_web_screenshots"));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -606,402 +606,3 @@ describe("SVG xlink:href context", () => {
|
||||
expect(dangling.urls).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
const SVG_NAMESPACE = "http://www.w3.org/2000/svg";
|
||||
const XLINK_NAMESPACE = "http://www.w3.org/1999/xlink";
|
||||
const XHTML_NAMESPACE = "http://www.w3.org/1999/xhtml";
|
||||
|
||||
const xhtmlRoot = (
|
||||
attributes: readonly (readonly [string, string])[] = [
|
||||
["xmlns", XHTML_NAMESPACE],
|
||||
["xmlns:svg", SVG_NAMESPACE],
|
||||
["xmlns:xlink", XLINK_NAMESPACE],
|
||||
],
|
||||
): SvgSnapshotNode => ({
|
||||
name: "html",
|
||||
parent: 0,
|
||||
attributes,
|
||||
});
|
||||
|
||||
const approvedHref = (
|
||||
nodeIndex: number,
|
||||
url: string,
|
||||
): {
|
||||
readonly node_index: number;
|
||||
readonly attribute: "href";
|
||||
readonly url: string;
|
||||
readonly destination_scope: "approved";
|
||||
} => ({
|
||||
node_index: nodeIndex,
|
||||
attribute: "href",
|
||||
url,
|
||||
destination_scope: "approved",
|
||||
});
|
||||
|
||||
describe("XML SVG roots and namespace scope restoration", () => {
|
||||
it.each(["svg", "document"])(
|
||||
"resolves namespaces under an unprefixed XML %s root",
|
||||
(name) => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
{
|
||||
name,
|
||||
parent: 0,
|
||||
attributes: [
|
||||
["xmlns", SVG_NAMESPACE],
|
||||
["xmlns:p", XLINK_NAMESPACE],
|
||||
],
|
||||
},
|
||||
{ name: "a", parent: 1, attributes: [["p:href", "/linked"]] },
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([approvedHref(2, `${origin}/linked`)]);
|
||||
},
|
||||
);
|
||||
|
||||
it.each([2, 3, 99])(
|
||||
"rejects self-parent, cyclic and dangling XML ancestry (%s)",
|
||||
(parent) => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot(),
|
||||
{
|
||||
name: "svg:a",
|
||||
parent,
|
||||
attributes: [
|
||||
["xmlns:svg", SVG_NAMESPACE],
|
||||
["xmlns:p", XLINK_NAMESPACE],
|
||||
["p:href", "/invalid"],
|
||||
],
|
||||
},
|
||||
{ name: "svg:g", parent: 2 },
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([]);
|
||||
},
|
||||
);
|
||||
|
||||
it("restores namespace declarations and undeclarations before visiting siblings", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot(),
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [
|
||||
["xmlns:xlink", ""],
|
||||
["xlink:href", "/undeclared"],
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [["xlink:href", "/inherited"]],
|
||||
},
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [
|
||||
["xmlns:svg", "other"],
|
||||
["xlink:href", "/rebound"],
|
||||
],
|
||||
},
|
||||
{ name: "svg:a", parent: 1, attributes: [["xlink:href", "/restored"]] },
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([
|
||||
approvedHref(3, `${origin}/inherited`),
|
||||
approvedHref(5, `${origin}/restored`),
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("XHTML and XML SVG namespace prefixes", () => {
|
||||
it("reports prefixed svg:a, svg:image, and an svg:a with no svg parent", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot(),
|
||||
{ name: "svg:svg", parent: 1 },
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 2,
|
||||
attributes: [
|
||||
["xlink:href", "/xl-pa1"],
|
||||
["xlink:href", "/later"],
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "svg:image",
|
||||
parent: 2,
|
||||
attributes: [["xlink:href", "guide"]],
|
||||
},
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [["xlink:href", "/xl-orphan"]],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(result.nodes[3]?.node_name).toBe("svg:a");
|
||||
expect(result.nodes[3]?.attribute_names).toEqual(["xlink:href"]);
|
||||
expect(result.urls).toEqual([
|
||||
approvedHref(3, `${origin}/xl-pa1`),
|
||||
approvedHref(4, `${origin}/screens/assets/guide`),
|
||||
approvedHref(5, `${origin}/xl-orphan`),
|
||||
]);
|
||||
});
|
||||
|
||||
it("reports svg:a under svg:foreignObject and skips XHTML anchors there", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot([
|
||||
["xmlns", XHTML_NAMESPACE],
|
||||
["xmlns:svg", SVG_NAMESPACE],
|
||||
["xmlns:xlink", XLINK_NAMESPACE],
|
||||
["xmlns:h", XHTML_NAMESPACE],
|
||||
]),
|
||||
{ name: "svg:svg", parent: 1 },
|
||||
{ name: "svg:foreignObject", parent: 2 },
|
||||
{
|
||||
name: "a",
|
||||
parent: 3,
|
||||
attributes: [["xlink:href", "/xl-html-fo"]],
|
||||
},
|
||||
{
|
||||
name: "h:a",
|
||||
parent: 3,
|
||||
attributes: [["xlink:href", "/xl-h"]],
|
||||
},
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 3,
|
||||
attributes: [["xlink:href", "/xl-fo-svg"]],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(urlFor(result, 4)).toBeUndefined();
|
||||
expect(urlFor(result, 5)).toBeUndefined();
|
||||
expect(result.urls).toEqual([approvedHref(6, `${origin}/xl-fo-svg`)]);
|
||||
});
|
||||
|
||||
it("reads XLink by namespace, including a prefix other than xlink", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot([["xmlns", XHTML_NAMESPACE]]),
|
||||
{
|
||||
name: "svg",
|
||||
parent: 1,
|
||||
attributes: [
|
||||
["xmlns", SVG_NAMESPACE],
|
||||
["xmlns:xlink", "http://example.com/not-xlink"],
|
||||
["xmlns:x", XLINK_NAMESPACE],
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "a",
|
||||
parent: 2,
|
||||
attributes: [
|
||||
["xlink:href", "/not-xlink"],
|
||||
["x:href", "/xl-custom"],
|
||||
["x:href", "/later"],
|
||||
],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(result.nodes[3]?.attribute_names).toEqual(["xlink:href", "x:href"]);
|
||||
expect(result.urls).toEqual([approvedHref(3, `${origin}/xl-custom`)]);
|
||||
});
|
||||
|
||||
it("keeps plain href ahead of a namespaced XLink href, including an empty href", () => {
|
||||
const baseUrl = `${origin}/screens/assets/`;
|
||||
const anchor = (
|
||||
parent: number,
|
||||
attributes: readonly (readonly [string, string])[],
|
||||
): SvgSnapshotNode => ({
|
||||
name: "svg:a",
|
||||
parent,
|
||||
attributes,
|
||||
});
|
||||
const result = captureSvg(
|
||||
svgDocument(
|
||||
[
|
||||
xhtmlRoot(),
|
||||
{ name: "svg:svg", parent: 1 },
|
||||
anchor(2, [
|
||||
["xlink:href", "/xlink-loses"],
|
||||
["href", "/href-wins"],
|
||||
]),
|
||||
anchor(2, [
|
||||
["href", "/href-wins"],
|
||||
["xlink:href", "/xlink-loses"],
|
||||
]),
|
||||
anchor(2, [
|
||||
["href", ""],
|
||||
["xlink:href", "/xlink-should-lose"],
|
||||
]),
|
||||
],
|
||||
baseUrl,
|
||||
),
|
||||
);
|
||||
expect(result.urls).toEqual([
|
||||
approvedHref(3, `${origin}/href-wins`),
|
||||
approvedHref(4, `${origin}/href-wins`),
|
||||
approvedHref(5, baseUrl),
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("XHTML SVG exclusions, HTML documents, and prefix scope", () => {
|
||||
it("classifies a prefixed SVG xlink:href outside the allowed origins as outside_policy", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot(),
|
||||
{ name: "svg:svg", parent: 1 },
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 2,
|
||||
attributes: [["xlink:href", "https://cdn.example.test/out"]],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([
|
||||
{
|
||||
node_index: 3,
|
||||
attribute: "href",
|
||||
url: null,
|
||||
destination_scope: "outside_policy",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("does not treat bogus:svg in another namespace as SVG", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot([
|
||||
["xmlns", XHTML_NAMESPACE],
|
||||
["xmlns:svg", SVG_NAMESPACE],
|
||||
["xmlns:xlink", XLINK_NAMESPACE],
|
||||
["xmlns:bogus", "http://example.com/not-svg"],
|
||||
]),
|
||||
{
|
||||
name: "bogus:svg",
|
||||
parent: 1,
|
||||
attributes: [["xlink:href", "/xl-bogus"]],
|
||||
},
|
||||
{
|
||||
name: "bogus:a",
|
||||
parent: 2,
|
||||
attributes: [["xlink:href", "/xl-bogus-a"]],
|
||||
},
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [["xlink:href", "/xl-kept"]],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(result.nodes[2]?.node_name).toBe("bogus:svg");
|
||||
expect(result.urls).toEqual([approvedHref(4, `${origin}/xl-kept`)]);
|
||||
});
|
||||
|
||||
it("leaves HTML documents on literal svg and xlink:href names", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
{ name: "HTML", parent: 0 },
|
||||
{
|
||||
name: "svg",
|
||||
parent: 1,
|
||||
attributes: [
|
||||
["xmlns", SVG_NAMESPACE],
|
||||
["xmlns:x", XLINK_NAMESPACE],
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "a",
|
||||
parent: 2,
|
||||
attributes: [["xlink:href", "/xl-literal"]],
|
||||
},
|
||||
{
|
||||
name: "a",
|
||||
parent: 2,
|
||||
attributes: [["x:href", "/xl-custom-html"]],
|
||||
},
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [
|
||||
["xmlns:svg", SVG_NAMESPACE],
|
||||
["xmlns:xlink", XLINK_NAMESPACE],
|
||||
["xlink:href", "/xl-prefixed-html"],
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "A",
|
||||
parent: 1,
|
||||
attributes: [["xlink:href", "/xl-html-anchor"]],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([approvedHref(3, `${origin}/xl-literal`)]);
|
||||
});
|
||||
|
||||
it("applies an element's own xmlns declarations to its name and href", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
{ name: "html", parent: 0, attributes: [["xmlns", XHTML_NAMESPACE]] },
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [
|
||||
["xmlns:svg", SVG_NAMESPACE],
|
||||
["xmlns:xlink", XLINK_NAMESPACE],
|
||||
["xlink:href", "/xl-self"],
|
||||
],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([approvedHref(2, `${origin}/xl-self`)]);
|
||||
});
|
||||
|
||||
it("resolves a prefix declared on a later ancestor", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 2,
|
||||
attributes: [["xlink:href", "/forward"]],
|
||||
},
|
||||
{ name: "svg:svg", parent: 3 },
|
||||
xhtmlRoot(),
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([approvedHref(1, `${origin}/forward`)]);
|
||||
});
|
||||
|
||||
it("resolves a deep chain with distinct namespace declarations without quadratic expansion", () => {
|
||||
const depth = 12_000;
|
||||
const elements: SvgSnapshotNode[] = [
|
||||
xhtmlRoot(),
|
||||
{ name: "svg:svg", parent: 1 },
|
||||
];
|
||||
let parent = 2;
|
||||
for (let index = 0; index < depth; index += 1) {
|
||||
elements.push({
|
||||
name: "svg:g",
|
||||
parent,
|
||||
attributes: [[`xmlns:p${index}`, `urn:prefix:${index}`]],
|
||||
});
|
||||
parent = elements.length;
|
||||
}
|
||||
elements.push({
|
||||
name: "svg:a",
|
||||
parent,
|
||||
attributes: [["xlink:href", "/deep"]],
|
||||
});
|
||||
const snapshot = svgDocument(elements);
|
||||
const result = captureSvg(snapshot);
|
||||
expect(result.urls).toEqual([
|
||||
approvedHref(elements.length, `${origin}/deep`),
|
||||
]);
|
||||
}, 30_000);
|
||||
});
|
||||
|
||||
@@ -11,7 +11,6 @@ import {
|
||||
} from "./CdpCaptureValues.js";
|
||||
import { exclusionReasonForUrl } from "./CdpCaptureEventHelpers.js";
|
||||
import type { CdpCaptureCompleteness } from "./CdpCaptureCompleteness.js";
|
||||
import { createDocumentSvgAncestry, domSvgLink } from "./CdpCaptureDomSvg.js";
|
||||
|
||||
export type CapturedResource = Omit<
|
||||
WebPageInspection["resources"][number],
|
||||
@@ -172,20 +171,24 @@ export const captureDom = (
|
||||
? documentNodes.attributes
|
||||
: [];
|
||||
const baseIndex = nodes.length;
|
||||
const ancestry = createDocumentSvgAncestry({
|
||||
nodeTypes,
|
||||
nodeNames,
|
||||
parents,
|
||||
strings,
|
||||
attributes,
|
||||
});
|
||||
const svgContexts = new Map<number, boolean>();
|
||||
total += nodeTypes.length;
|
||||
for (let index = 0; index < nodeTypes.length; index += 1) {
|
||||
const attributeIndexes = numberArray(attributes[index]);
|
||||
const parent = Math.trunc(parents[index] ?? -1);
|
||||
const nodeIndex = nodes.length;
|
||||
const nodeName = indexedString(strings, nodeNames[index]);
|
||||
const attributeNames = uniqueAttributeNames(attributeIndexes, strings);
|
||||
const rawAttributeNames = attributeIndexes
|
||||
.filter((_value, attributeIndex) => attributeIndex % 2 === 0)
|
||||
.map((value) => indexedString(strings, value));
|
||||
const seenAttributeNames = new Set<string>();
|
||||
const attributeNames: string[] = [];
|
||||
for (const name of rawAttributeNames) {
|
||||
if (!seenAttributeNames.has(name)) {
|
||||
seenAttributeNames.add(name);
|
||||
attributeNames.push(name);
|
||||
}
|
||||
}
|
||||
nodes.push({
|
||||
index: nodeIndex,
|
||||
parent_index: parent < 0 ? -1 : baseIndex + parent,
|
||||
@@ -203,7 +206,13 @@ export const captureDom = (
|
||||
nodeIndex,
|
||||
nodeName,
|
||||
allowedOrigins,
|
||||
...domSvgLink(ancestry, index),
|
||||
svgElement: elementInSvgContext(
|
||||
index,
|
||||
nodeNames,
|
||||
parents,
|
||||
strings,
|
||||
svgContexts,
|
||||
),
|
||||
});
|
||||
for (const url of metadata.urls) {
|
||||
urls.push(url);
|
||||
@@ -358,21 +367,6 @@ const indexedString = (strings: readonly string[], index: unknown): string => {
|
||||
return integer === undefined ? "" : (strings[Math.trunc(integer)] ?? "");
|
||||
};
|
||||
|
||||
const uniqueAttributeNames = (
|
||||
attributeIndexes: readonly number[],
|
||||
strings: readonly string[],
|
||||
): string[] => {
|
||||
const names: string[] = [];
|
||||
const seen = new Set<string>();
|
||||
for (let index = 0; index < attributeIndexes.length; index += 2) {
|
||||
const name = indexedString(strings, attributeIndexes[index]);
|
||||
if (seen.has(name)) continue;
|
||||
seen.add(name);
|
||||
names.push(name);
|
||||
}
|
||||
return names;
|
||||
};
|
||||
|
||||
const numberArray = (value: unknown): readonly number[] =>
|
||||
Array.isArray(value)
|
||||
? value.flatMap((item) => {
|
||||
@@ -390,22 +384,41 @@ interface DomMetadataOptions {
|
||||
readonly nodeName: string;
|
||||
readonly allowedOrigins: ReadonlySet<string>;
|
||||
readonly svgElement: boolean;
|
||||
readonly xmlDocument: boolean;
|
||||
readonly xlinkHref: string | undefined;
|
||||
}
|
||||
|
||||
// Chrome reads an SVG XLink href only when href itself is absent, and an
|
||||
// empty href still wins. HTML keeps the literal attribute name `xlink:href`.
|
||||
// XML uses any prefix bound to the XLink namespace. Both are stored as href.
|
||||
const hrefAttributeValue = (
|
||||
pairs: ReadonlyMap<string, string>,
|
||||
attribute: string,
|
||||
options: DomMetadataOptions,
|
||||
): string | undefined => {
|
||||
const declared = pairs.get(attribute);
|
||||
if (declared !== undefined) return declared;
|
||||
if (attribute !== "href" || !options.svgElement) return undefined;
|
||||
return options.xmlDocument ? options.xlinkHref : pairs.get("xlink:href");
|
||||
// DOMSnapshot carries no namespace URI. Chrome's HTML parser records SVG
|
||||
// local names (`svg`, `foreignObject`). Descendants of foreignObject are HTML
|
||||
// until a nested `svg`.
|
||||
const elementInSvgContext = (
|
||||
index: number,
|
||||
nodeNames: readonly number[],
|
||||
parents: readonly number[],
|
||||
strings: readonly string[],
|
||||
contexts: Map<number, boolean>,
|
||||
): boolean => {
|
||||
let current = index;
|
||||
const seen = new Set<number>();
|
||||
let inSvg = false;
|
||||
while (current >= 0 && current < nodeNames.length && !seen.has(current)) {
|
||||
const cached = contexts.get(current);
|
||||
if (cached !== undefined) {
|
||||
inSvg = cached;
|
||||
break;
|
||||
}
|
||||
seen.add(current);
|
||||
if (indexedString(strings, nodeNames[current]) === "svg") {
|
||||
inSvg = true;
|
||||
break;
|
||||
}
|
||||
const parent = Math.trunc(parents[current] ?? -1);
|
||||
if (parent < 0 || parent === current) break;
|
||||
if (indexedString(strings, nodeNames[parent]) === "foreignObject") break;
|
||||
current = parent;
|
||||
}
|
||||
// Cache the entire examined chain, including forward references and cycles,
|
||||
// so shared ancestors are examined once per document.
|
||||
for (const node of seen) contexts.set(node, inSvg);
|
||||
return inSvg;
|
||||
};
|
||||
|
||||
const domMetadata = (
|
||||
@@ -422,6 +435,7 @@ const domMetadata = (
|
||||
nodeIndex,
|
||||
nodeName,
|
||||
allowedOrigins,
|
||||
svgElement,
|
||||
} = options;
|
||||
const pairs = new Map<string, string>();
|
||||
for (let index = 0; index + 1 < attributes.length; index += 2) {
|
||||
@@ -433,7 +447,13 @@ const domMetadata = (
|
||||
}
|
||||
const urls: WebPageInspection["metadata"]["dom_urls"] = [];
|
||||
for (const attribute of domUrlAttributes) {
|
||||
const value = hrefAttributeValue(pairs, attribute, options);
|
||||
// Chrome reads SVG xlink:href only when href itself is absent, and an
|
||||
// empty href still wins. Report the fallback as href.
|
||||
const value =
|
||||
pairs.get(attribute) ??
|
||||
(attribute === "href" && svgElement
|
||||
? pairs.get("xlink:href")
|
||||
: undefined);
|
||||
if (value === undefined) continue;
|
||||
const tagName = nodeName.toLowerCase();
|
||||
// Chrome strips HTML whitespace for FORM.action, while submit controls
|
||||
|
||||
@@ -1,275 +0,0 @@
|
||||
import { numberValue } from "./CdpCaptureValues.js";
|
||||
|
||||
const SVG_NAMESPACE = "http://www.w3.org/2000/svg";
|
||||
const XLINK_NAMESPACE = "http://www.w3.org/1999/xlink";
|
||||
|
||||
interface DomSnapshotNodes {
|
||||
readonly nodeTypes: readonly number[];
|
||||
readonly nodeNames: readonly number[];
|
||||
readonly parents: readonly number[];
|
||||
readonly strings: readonly string[];
|
||||
readonly attributes: readonly unknown[];
|
||||
}
|
||||
|
||||
interface XmlSvgLink {
|
||||
readonly svgElement: boolean;
|
||||
readonly xlinkHref: string | undefined;
|
||||
}
|
||||
|
||||
interface DocumentSvgAncestry {
|
||||
readonly xml: boolean;
|
||||
readonly nodes: DomSnapshotNodes;
|
||||
readonly svgContexts: Map<number, boolean>;
|
||||
readonly xmlLinks: Map<number, XmlSvgLink>;
|
||||
}
|
||||
|
||||
interface DomSvgLink {
|
||||
readonly svgElement: boolean;
|
||||
readonly xmlDocument: boolean;
|
||||
readonly xlinkHref: string | undefined;
|
||||
}
|
||||
|
||||
const UNRESOLVED: XmlSvgLink = {
|
||||
svgElement: false,
|
||||
xlinkHref: undefined,
|
||||
};
|
||||
|
||||
const stringAt = (strings: readonly string[], index: unknown): string => {
|
||||
const integer = numberValue(index);
|
||||
return integer === undefined ? "" : (strings[Math.trunc(integer)] ?? "");
|
||||
};
|
||||
|
||||
const numberArray = (value: unknown): readonly number[] =>
|
||||
Array.isArray(value)
|
||||
? value.flatMap((item) => {
|
||||
const number = numberValue(item);
|
||||
return number === undefined ? [] : [number];
|
||||
})
|
||||
: [];
|
||||
|
||||
// DOMSnapshot has no namespace URI. HTML documents use uppercase `HTML` and
|
||||
// the parser's local names. XML documents preserve prefixed names and xmlns
|
||||
// declarations, including those on standalone SVG roots.
|
||||
const xmlDocument = (nodes: DomSnapshotNodes): boolean => {
|
||||
const documentIndex = nodes.nodeTypes.findIndex(
|
||||
(nodeType) => Math.trunc(nodeType) === 9,
|
||||
);
|
||||
if (documentIndex < 0) return false;
|
||||
for (let index = 0; index < nodes.nodeTypes.length; index += 1) {
|
||||
if (Math.trunc(nodes.nodeTypes[index] ?? 0) !== 1) continue;
|
||||
if (Math.trunc(nodes.parents[index] ?? -1) !== documentIndex) continue;
|
||||
const name = stringAt(nodes.strings, nodes.nodeNames[index]);
|
||||
if (name === "HTML") return false;
|
||||
if (name === "html" || name.includes(":")) return true;
|
||||
// Explicit namespaces also identify standalone SVG and other XML roots.
|
||||
// Namespace-free synthetic snapshots retain the legacy HTML behavior.
|
||||
const attributes = numberArray(nodes.attributes[index]);
|
||||
return attributes.some(
|
||||
(value, offset) =>
|
||||
offset % 2 === 0 &&
|
||||
xmlnsPrefix(stringAt(nodes.strings, value)) !== undefined,
|
||||
);
|
||||
}
|
||||
return false;
|
||||
};
|
||||
|
||||
export const createDocumentSvgAncestry = (
|
||||
nodes: DomSnapshotNodes,
|
||||
): DocumentSvgAncestry => {
|
||||
const xml = xmlDocument(nodes);
|
||||
return {
|
||||
xml,
|
||||
nodes,
|
||||
svgContexts: new Map(),
|
||||
xmlLinks: xml ? resolveXmlElements(nodes) : new Map(),
|
||||
};
|
||||
};
|
||||
|
||||
export const domSvgLink = (
|
||||
ancestry: DocumentSvgAncestry,
|
||||
index: number,
|
||||
): DomSvgLink => {
|
||||
if (!ancestry.xml) {
|
||||
return {
|
||||
svgElement: elementInSvgContext(
|
||||
index,
|
||||
ancestry.nodes,
|
||||
ancestry.svgContexts,
|
||||
),
|
||||
xmlDocument: false,
|
||||
xlinkHref: undefined,
|
||||
};
|
||||
}
|
||||
const resolved = ancestry.xmlLinks.get(index) ?? UNRESOLVED;
|
||||
return {
|
||||
svgElement: resolved.svgElement,
|
||||
xmlDocument: true,
|
||||
xlinkHref: resolved.xlinkHref,
|
||||
};
|
||||
};
|
||||
|
||||
// Chrome's HTML parser records SVG local names (`svg`, `foreignObject`).
|
||||
// Descendants of foreignObject are HTML until a nested `svg`.
|
||||
const elementInSvgContext = (
|
||||
index: number,
|
||||
nodes: DomSnapshotNodes,
|
||||
contexts: Map<number, boolean>,
|
||||
): boolean => {
|
||||
let current = index;
|
||||
const seen = new Set<number>();
|
||||
let inSvg = false;
|
||||
while (
|
||||
current >= 0 &&
|
||||
current < nodes.nodeNames.length &&
|
||||
!seen.has(current)
|
||||
) {
|
||||
const cached = contexts.get(current);
|
||||
if (cached !== undefined) {
|
||||
inSvg = cached;
|
||||
break;
|
||||
}
|
||||
seen.add(current);
|
||||
if (stringAt(nodes.strings, nodes.nodeNames[current]) === "svg") {
|
||||
inSvg = true;
|
||||
break;
|
||||
}
|
||||
const parent = Math.trunc(nodes.parents[current] ?? -1);
|
||||
if (parent < 0 || parent === current) break;
|
||||
if (stringAt(nodes.strings, nodes.nodeNames[parent]) === "foreignObject")
|
||||
break;
|
||||
current = parent;
|
||||
}
|
||||
// Cache the entire examined chain, including forward references and cycles,
|
||||
// so shared ancestors are examined once per document.
|
||||
for (const node of seen) contexts.set(node, inSvg);
|
||||
return inSvg;
|
||||
};
|
||||
|
||||
interface NamespaceRestore {
|
||||
readonly prefix: string;
|
||||
readonly previous: string | undefined;
|
||||
}
|
||||
|
||||
type XmlWalkStep =
|
||||
| { readonly node: number }
|
||||
| { readonly restore: readonly NamespaceRestore[] };
|
||||
|
||||
// Walk parent-before-child with one mutable namespace environment. Restore
|
||||
// declarations on exit so siblings retain their own scope. Only declarations
|
||||
// on the active path are retained, rather than cumulative maps at every node.
|
||||
const resolveXmlElements = (
|
||||
nodes: DomSnapshotNodes,
|
||||
): Map<number, XmlSvgLink> => {
|
||||
const children = new Map<number, number[]>();
|
||||
const pending: XmlWalkStep[] = [];
|
||||
for (let node = 0; node < nodes.nodeNames.length; node += 1) {
|
||||
const parent = Math.trunc(nodes.parents[node] ?? -1);
|
||||
if (parent < 0) pending.push({ node });
|
||||
else if (parent < nodes.nodeNames.length && parent !== node) {
|
||||
const siblings = children.get(parent);
|
||||
if (siblings === undefined) children.set(parent, [node]);
|
||||
else siblings.push(node);
|
||||
}
|
||||
}
|
||||
const bindings = new Map<string, string>();
|
||||
const resolved = new Map<number, XmlSvgLink>();
|
||||
while (pending.length > 0) {
|
||||
const step = pending.pop();
|
||||
if (step === undefined) break;
|
||||
if ("restore" in step) {
|
||||
for (const { prefix, previous } of step.restore) {
|
||||
if (previous === undefined) bindings.delete(prefix);
|
||||
else bindings.set(prefix, previous);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
const { node } = step;
|
||||
const attributes = numberArray(nodes.attributes[node]);
|
||||
const restore = applyDeclarations(bindings, attributes, nodes.strings);
|
||||
const svgElement =
|
||||
Math.trunc(nodes.nodeTypes[node] ?? 0) === 1 &&
|
||||
elementNamespace(
|
||||
stringAt(nodes.strings, nodes.nodeNames[node]),
|
||||
bindings,
|
||||
) === SVG_NAMESPACE;
|
||||
resolved.set(node, {
|
||||
svgElement,
|
||||
xlinkHref: svgElement
|
||||
? xlinkHrefValue(attributes, nodes.strings, bindings)
|
||||
: undefined,
|
||||
});
|
||||
pending.push({ restore });
|
||||
for (const child of children.get(node) ?? []) pending.push({ node: child });
|
||||
}
|
||||
// Cycles (including self-parents) and dangling chains cannot reach a root,
|
||||
// so remain unresolved instead of deriving URLs from an invalid ancestry.
|
||||
return resolved;
|
||||
};
|
||||
|
||||
const applyDeclarations = (
|
||||
bindings: Map<string, string>,
|
||||
attributes: readonly number[],
|
||||
strings: readonly string[],
|
||||
): readonly NamespaceRestore[] => {
|
||||
const restore: NamespaceRestore[] = [];
|
||||
const seen = new Set<string>();
|
||||
for (let index = 0; index + 1 < attributes.length; index += 2) {
|
||||
const prefix = xmlnsPrefix(stringAt(strings, attributes[index]));
|
||||
if (prefix === undefined || seen.has(prefix)) continue;
|
||||
seen.add(prefix);
|
||||
restore.push({ prefix, previous: bindings.get(prefix) });
|
||||
const value = stringAt(strings, attributes[index + 1]);
|
||||
if (value.length === 0) bindings.delete(prefix);
|
||||
else bindings.set(prefix, value);
|
||||
}
|
||||
return restore;
|
||||
};
|
||||
|
||||
const xmlnsPrefix = (name: string): string | undefined => {
|
||||
if (name === "xmlns") return "";
|
||||
if (!name.startsWith("xmlns:")) return undefined;
|
||||
const prefix = name.slice("xmlns:".length);
|
||||
if (prefix.length === 0 || prefix === "xmlns" || prefix.includes(":"))
|
||||
return undefined;
|
||||
return prefix;
|
||||
};
|
||||
|
||||
const elementNamespace = (
|
||||
nodeName: string,
|
||||
bindings: ReadonlyMap<string, string>,
|
||||
): string => {
|
||||
const colon = nodeName.indexOf(":");
|
||||
if (colon < 0) return bindings.get("") ?? "";
|
||||
const prefix = nodeName.slice(0, colon);
|
||||
const local = nodeName.slice(colon + 1);
|
||||
if (
|
||||
prefix.length === 0 ||
|
||||
local.length === 0 ||
|
||||
local.includes(":") ||
|
||||
prefix === "xmlns"
|
||||
)
|
||||
return "";
|
||||
return bindings.get(prefix) ?? "";
|
||||
};
|
||||
|
||||
// First prefixed href bound to the XLink namespace. Duplicate names keep the
|
||||
// earlier value. Plain href is applied by the caller and wins even when empty.
|
||||
const xlinkHrefValue = (
|
||||
attributes: readonly number[],
|
||||
strings: readonly string[],
|
||||
bindings: ReadonlyMap<string, string>,
|
||||
): string | undefined => {
|
||||
const seen = new Set<string>();
|
||||
for (let index = 0; index + 1 < attributes.length; index += 2) {
|
||||
const name = stringAt(strings, attributes[index]);
|
||||
if (seen.has(name)) continue;
|
||||
seen.add(name);
|
||||
const colon = name.indexOf(":");
|
||||
if (colon <= 0) continue;
|
||||
const prefix = name.slice(0, colon);
|
||||
const local = name.slice(colon + 1);
|
||||
if (local !== "href" || bindings.get(prefix) !== XLINK_NAMESPACE) continue;
|
||||
return stringAt(strings, attributes[index + 1]);
|
||||
}
|
||||
return undefined;
|
||||
};
|
||||
@@ -3,10 +3,6 @@ import { crc32, deflateSync } from "node:zlib";
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { comparePngScreenshots } from "./PngVisualDiff.js";
|
||||
import {
|
||||
AnalysisResourceConstraintError,
|
||||
AnalysisUnsupportedTargetError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import {
|
||||
compareWebScreenshotsInputSchema,
|
||||
createWebScreenshotArtifact,
|
||||
@@ -121,7 +117,7 @@ describe("PNG visual diff", () => {
|
||||
});
|
||||
|
||||
it("rejects malformed PNG dimensions after validating image data", () => {
|
||||
const image = artifact(2, 1, [0, 0, 0, 255]);
|
||||
const image = artifact(32_000_001, 1, [0, 0, 0, 255]);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
@@ -153,256 +149,6 @@ describe("PNG visual diff", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("PNG comparison memory budget", () => {
|
||||
it("classifies legal huge dimensions as a memory constraint before inflate", () => {
|
||||
const image = createWebScreenshotArtifact(headerOnlyPng(0x7fffffff, 1));
|
||||
const input = compareWebScreenshotsInputSchema.parse({
|
||||
before: image,
|
||||
after: image,
|
||||
});
|
||||
|
||||
try {
|
||||
comparePngScreenshots(input);
|
||||
throw new Error("expected PNG working-memory rejection");
|
||||
} catch (cause: unknown) {
|
||||
expect(cause).toBeInstanceOf(AnalysisResourceConstraintError);
|
||||
expect(cause).toMatchObject({
|
||||
resource: "memory",
|
||||
reportedLimits: {
|
||||
maximum_working_memory_bytes: 256 * 1024 * 1024,
|
||||
estimated_working_memory_bytes: "34359738499",
|
||||
before_dimensions: { width: 0x7fffffff, height: 1 },
|
||||
after_dimensions: { width: 0x7fffffff, height: 1 },
|
||||
},
|
||||
});
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("PNG chunk integrity", () => {
|
||||
it.each(["IHDR", "IDAT", "IEND"])(
|
||||
"rejects a corrupt %s chunk checksum",
|
||||
(type) => {
|
||||
const bytes = png(1, 1, [0, 0, 0, 255]);
|
||||
let offset = 8;
|
||||
while (offset < bytes.length) {
|
||||
const length = bytes.readUInt32BE(offset);
|
||||
if (bytes.subarray(offset + 4, offset + 8).toString("ascii") === type) {
|
||||
bytes[offset + 8 + length] = (bytes[offset + 8 + length] ?? 0) ^ 1;
|
||||
break;
|
||||
}
|
||||
offset += length + 12;
|
||||
}
|
||||
const image = createWebScreenshotArtifact(bytes);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: image,
|
||||
after: image,
|
||||
}),
|
||||
),
|
||||
).toThrow("Invalid PNG chunk checksum");
|
||||
},
|
||||
);
|
||||
|
||||
it("reports a CRC-valid but corrupt compressed stream as invalid PNG data", () => {
|
||||
const header = Buffer.from(png(1, 1, [0, 0, 0, 255]).subarray(16, 29));
|
||||
const image = createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]),
|
||||
chunk("IHDR", header),
|
||||
chunk("IDAT", Buffer.from("bad")),
|
||||
chunk("IEND", Buffer.alloc(0)),
|
||||
]),
|
||||
);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({ before: image, after: image }),
|
||||
),
|
||||
).toThrow("Invalid PNG image data");
|
||||
});
|
||||
|
||||
it("rejects high-bit bytes in a CRC-valid PNG chunk name", () => {
|
||||
const bytes = png(1, 1, [0, 0, 0, 255]);
|
||||
const headerChunk = Buffer.from(bytes.subarray(8, 33));
|
||||
headerChunk[4] = 0xc9;
|
||||
headerChunk.writeUInt32BE(crc32(headerChunk.subarray(4, 21)), 21);
|
||||
const malformed = createWebScreenshotArtifact(
|
||||
Buffer.concat([bytes.subarray(0, 8), headerChunk, bytes.subarray(33)]),
|
||||
);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: malformed,
|
||||
after: malformed,
|
||||
}),
|
||||
),
|
||||
).toThrow("Invalid PNG chunk type");
|
||||
});
|
||||
|
||||
it("rejects duplicate image headers and unknown critical chunks while preserving ancillary chunks", () => {
|
||||
const bytes = png(1, 1, [0, 0, 0, 255]);
|
||||
const duplicate = Buffer.concat([
|
||||
bytes.subarray(0, 33),
|
||||
bytes.subarray(8, 33),
|
||||
bytes.subarray(33),
|
||||
]);
|
||||
const unknown = Buffer.concat([
|
||||
bytes.subarray(0, 33),
|
||||
chunk("ABCD", Buffer.alloc(0)),
|
||||
bytes.subarray(33),
|
||||
]);
|
||||
for (const malformed of [duplicate, unknown]) {
|
||||
const image = createWebScreenshotArtifact(malformed);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: image,
|
||||
after: image,
|
||||
}),
|
||||
),
|
||||
).toThrow(/Invalid PNG header order|Unsupported critical PNG chunk/u);
|
||||
}
|
||||
const ancillary = createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
bytes.subarray(0, 33),
|
||||
chunk("tEXt", Buffer.from("Comment\0original capture")),
|
||||
bytes.subarray(33),
|
||||
]),
|
||||
);
|
||||
expect(
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: ancillary,
|
||||
after: createWebScreenshotArtifact(bytes),
|
||||
}),
|
||||
).status,
|
||||
).toBe("identical");
|
||||
});
|
||||
|
||||
it("rejects bytes after IEND and reports valid unsupported PNG encodings separately", () => {
|
||||
const bytes = png(1, 1, [0, 0, 0, 255]);
|
||||
const trailing = createWebScreenshotArtifact(
|
||||
Buffer.concat([bytes, Buffer.from([0])]),
|
||||
);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: trailing,
|
||||
after: trailing,
|
||||
}),
|
||||
),
|
||||
).toThrow("Invalid PNG end chunk");
|
||||
|
||||
const grayscaleHeader = Buffer.from(bytes.subarray(16, 29));
|
||||
grayscaleHeader[9] = 0;
|
||||
const grayscale = createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]),
|
||||
chunk("IHDR", grayscaleHeader),
|
||||
chunk("IDAT", deflateSync(Buffer.from([0, 30]))),
|
||||
chunk("IEND", Buffer.alloc(0)),
|
||||
]),
|
||||
);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: grayscale,
|
||||
after: grayscale,
|
||||
}),
|
||||
),
|
||||
).toThrow(AnalysisUnsupportedTargetError);
|
||||
});
|
||||
});
|
||||
|
||||
describe("PNG chunk ordering", () => {
|
||||
it("rejects malformed palettes, interrupted data and misplaced transparency while retaining valid chunks", () => {
|
||||
const header = Buffer.from(png(1, 1, [10, 20, 30, 255]).subarray(16, 29));
|
||||
header[9] = 2;
|
||||
const ihdr = chunk("IHDR", header);
|
||||
const compressed = deflateSync(Buffer.from([0, 10, 20, 30]));
|
||||
const idat = chunk("IDAT", compressed);
|
||||
const iend = chunk("IEND", Buffer.alloc(0));
|
||||
const palette = chunk("PLTE", Buffer.from([10, 20, 30]));
|
||||
const transparency = chunk("tRNS", Buffer.from([0, 10, 0, 20, 0, 30]));
|
||||
const ancillary = chunk("tEXt", Buffer.from("Comment\0retained"));
|
||||
const image = (...chunks: readonly Buffer[]) =>
|
||||
createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]),
|
||||
...chunks,
|
||||
]),
|
||||
);
|
||||
const original = image(ihdr, idat, iend);
|
||||
const cases = [
|
||||
[ihdr, chunk("PLTE", Buffer.alloc(1)), idat, iend],
|
||||
[ihdr, palette, palette, idat, iend],
|
||||
[ihdr, idat, palette, iend],
|
||||
[ihdr, transparency, palette, idat, iend],
|
||||
[ihdr, transparency, transparency, idat, iend],
|
||||
[ihdr, idat, transparency, iend],
|
||||
[
|
||||
ihdr,
|
||||
chunk("IDAT", compressed.subarray(0, 6)),
|
||||
ancillary,
|
||||
chunk("IDAT", compressed.subarray(6)),
|
||||
iend,
|
||||
],
|
||||
[ihdr, idat, chunk("IEND", Buffer.from([1]))],
|
||||
];
|
||||
for (const chunks of cases) {
|
||||
const malformed = image(...chunks);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: malformed,
|
||||
after: original,
|
||||
}),
|
||||
),
|
||||
).toThrow(
|
||||
/Invalid PNG palette|Invalid PNG transparency order|Nonconsecutive PNG image data|Invalid PNG end chunk/u,
|
||||
);
|
||||
}
|
||||
const indexedHeader = Buffer.from(header);
|
||||
indexedHeader[8] = 1;
|
||||
indexedHeader[9] = 3;
|
||||
const emptyTransparency = image(
|
||||
chunk("IHDR", indexedHeader),
|
||||
chunk("PLTE", Buffer.from([10, 20, 30])),
|
||||
chunk("tRNS", Buffer.alloc(0)),
|
||||
chunk("IDAT", deflateSync(Buffer.from([0, 0]))),
|
||||
iend,
|
||||
);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: emptyTransparency,
|
||||
after: original,
|
||||
}),
|
||||
),
|
||||
).toThrow("Unsupported PNG transparency");
|
||||
for (const valid of [
|
||||
image(ihdr, palette, idat, iend),
|
||||
image(ihdr, ancillary, idat, iend),
|
||||
image(
|
||||
ihdr,
|
||||
chunk("IDAT", compressed.subarray(0, 6)),
|
||||
chunk("IDAT", compressed.subarray(6)),
|
||||
iend,
|
||||
),
|
||||
]) {
|
||||
expect(
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: valid,
|
||||
after: original,
|
||||
}),
|
||||
).status,
|
||||
).toBe("identical");
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
const artifact = (width: number, height: number, pixels: readonly number[]) =>
|
||||
createWebScreenshotArtifact(png(width, height, pixels));
|
||||
|
||||
@@ -427,19 +173,6 @@ const png = (
|
||||
]);
|
||||
};
|
||||
|
||||
const headerOnlyPng = (width: number, height: number): Buffer => {
|
||||
const header = Buffer.alloc(13);
|
||||
header.writeUInt32BE(width, 0);
|
||||
header.writeUInt32BE(height, 4);
|
||||
header.set([8, 6, 0, 0, 0], 8);
|
||||
return Buffer.concat([
|
||||
Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]),
|
||||
chunk("IHDR", header),
|
||||
chunk("IDAT", deflateSync(Buffer.from([0]))),
|
||||
chunk("IEND", Buffer.alloc(0)),
|
||||
]);
|
||||
};
|
||||
|
||||
const chunk = (type: string, data: Buffer): Buffer => {
|
||||
const result = Buffer.alloc(12 + data.byteLength);
|
||||
result.writeUInt32BE(data.byteLength, 0);
|
||||
|
||||
+32
-268
@@ -1,14 +1,10 @@
|
||||
import { constants as zlibConstants, crc32, inflateSync } from "node:zlib";
|
||||
import { inflateSync } from "node:zlib";
|
||||
|
||||
import type {
|
||||
CompareWebScreenshotsInput,
|
||||
WebScreenshotArtifact,
|
||||
WebScreenshotDiff,
|
||||
} from "../domain/webScreenshot.js";
|
||||
import {
|
||||
AnalysisResourceConstraintError,
|
||||
AnalysisUnsupportedTargetError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import { decodeCanonicalBase64 } from "../domain/webScreenshot.js";
|
||||
|
||||
interface DecodedPng {
|
||||
@@ -17,32 +13,12 @@ interface DecodedPng {
|
||||
readonly rgba: Buffer;
|
||||
}
|
||||
|
||||
interface ParsedPng {
|
||||
readonly bytes: Buffer;
|
||||
readonly width: number;
|
||||
readonly height: number;
|
||||
readonly channels: number;
|
||||
readonly compressed: readonly Buffer[];
|
||||
readonly transparentColor: readonly number[] | undefined;
|
||||
readonly expectedRawBytes: bigint;
|
||||
readonly unfilteredBytes: bigint;
|
||||
readonly rgbaBytes: bigint;
|
||||
}
|
||||
|
||||
/** Conservative allowance for the simultaneously live PNG comparison buffers. */
|
||||
const PNG_COMPARISON_WORKING_MEMORY_BYTES = 256 * 1024 * 1024;
|
||||
const PNG_INFLATE_CHUNK_BYTES = zlibConstants.Z_DEFAULT_CHUNK;
|
||||
|
||||
/** Compare two validated screenshots using deterministic RGBA metrics. */
|
||||
export const comparePngScreenshots = (
|
||||
input: CompareWebScreenshotsInput,
|
||||
): WebScreenshotDiff => {
|
||||
admitEncodedInputPair(input.before, input.after);
|
||||
const beforePlan = parsePng(input.before, "before");
|
||||
const afterPlan = parsePng(input.after, "after");
|
||||
admitPngPair(beforePlan, afterPlan);
|
||||
const before = decodePng(beforePlan);
|
||||
const after = decodePng(afterPlan);
|
||||
const before = decodePng(input.before);
|
||||
const after = decodePng(input.after);
|
||||
if (before.width !== after.width || before.height !== after.height)
|
||||
return {
|
||||
status: "dimension_mismatch",
|
||||
@@ -97,10 +73,7 @@ export const comparePngScreenshots = (
|
||||
};
|
||||
};
|
||||
|
||||
const parsePng = (
|
||||
artifact: WebScreenshotArtifact,
|
||||
field: "before" | "after",
|
||||
): ParsedPng => {
|
||||
const decodePng = (artifact: WebScreenshotArtifact): DecodedPng => {
|
||||
const bytes = decodeCanonicalBase64(artifact.data_base64);
|
||||
if (bytes === undefined || !bytes.subarray(0, 8).equals(PNG_SIGNATURE))
|
||||
throw new TypeError("Invalid PNG signature");
|
||||
@@ -108,277 +81,68 @@ const parsePng = (
|
||||
let header: ReturnType<typeof parseHeader> | undefined;
|
||||
const compressed: Buffer[] = [];
|
||||
let sawEnd = false;
|
||||
let sawPalette = false;
|
||||
let imageDataEnded = false;
|
||||
let paletteEntries = 0;
|
||||
let transparentColor: readonly number[] | undefined;
|
||||
let sawTransparency = false;
|
||||
let unsupportedReason: string | undefined;
|
||||
while (offset < bytes.length) {
|
||||
if (offset + 12 > bytes.length) throw new TypeError("Truncated PNG chunk");
|
||||
const length = bytes.readUInt32BE(offset);
|
||||
if (length > 0x7fffffff) throw new TypeError("Invalid PNG chunk length");
|
||||
const type = bytes.subarray(offset + 4, offset + 8).toString("latin1");
|
||||
const type = bytes.subarray(offset + 4, offset + 8).toString("ascii");
|
||||
const dataStart = offset + 8;
|
||||
const dataEnd = dataStart + length;
|
||||
if (dataEnd + 4 > bytes.length) throw new TypeError("Truncated PNG data");
|
||||
const data = bytes.subarray(dataStart, dataEnd);
|
||||
if (
|
||||
crc32(bytes.subarray(offset + 4, dataEnd)) !== bytes.readUInt32BE(dataEnd)
|
||||
)
|
||||
throw new TypeError("Invalid PNG chunk checksum");
|
||||
if (!/^[A-Za-z]{4}$/u.test(type) || type[2] !== type[2]?.toUpperCase())
|
||||
throw new TypeError("Invalid PNG chunk type");
|
||||
if (type !== "IDAT" && compressed.length > 0) imageDataEnded = true;
|
||||
if (type === "IHDR") {
|
||||
if (offset !== 8 || header !== undefined)
|
||||
throw new TypeError("Invalid PNG header order");
|
||||
header = parseHeader(data);
|
||||
} else if (header === undefined)
|
||||
throw new TypeError("Invalid PNG header order");
|
||||
if (type === "IHDR") header = parseHeader(data);
|
||||
else if (type === "tRNS") {
|
||||
if (sawTransparency || compressed.length > 0)
|
||||
throw new TypeError("Invalid PNG transparency order");
|
||||
sawTransparency = true;
|
||||
if (header.colorType === 2 && data.length === 6) {
|
||||
transparentColor = [
|
||||
data.readUInt16BE(0),
|
||||
data.readUInt16BE(2),
|
||||
data.readUInt16BE(4),
|
||||
];
|
||||
} else if (
|
||||
(header.colorType === 0 && data.length === 2) ||
|
||||
(header.colorType === 3 &&
|
||||
sawPalette &&
|
||||
data.length > 0 &&
|
||||
data.length <= paletteEntries)
|
||||
) {
|
||||
unsupportedReason ??=
|
||||
"PNG transparency for this color type is not supported.";
|
||||
} else {
|
||||
if (header?.channels !== 3 || data.length !== 6)
|
||||
throw new TypeError("Unsupported PNG transparency");
|
||||
}
|
||||
} else if (type === "PLTE") {
|
||||
if (
|
||||
sawPalette ||
|
||||
sawTransparency ||
|
||||
compressed.length > 0 ||
|
||||
length === 0 ||
|
||||
length > 768 ||
|
||||
length % 3 !== 0 ||
|
||||
(header.colorType === 3 && length / 3 > 2 ** header.bitDepth) ||
|
||||
(header.colorType !== 2 &&
|
||||
header.colorType !== 3 &&
|
||||
header.colorType !== 6)
|
||||
)
|
||||
throw new TypeError("Invalid PNG palette");
|
||||
sawPalette = true;
|
||||
paletteEntries = length / 3;
|
||||
} else if (type === "IDAT") {
|
||||
if (imageDataEnded) throw new TypeError("Nonconsecutive PNG image data");
|
||||
compressed.push(data);
|
||||
} else if (type === "IEND") {
|
||||
if (length !== 0 || dataEnd + 4 !== bytes.length)
|
||||
throw new TypeError("Invalid PNG end chunk");
|
||||
transparentColor = [
|
||||
data.readUInt16BE(0),
|
||||
data.readUInt16BE(2),
|
||||
data.readUInt16BE(4),
|
||||
];
|
||||
} else if (type === "IDAT") compressed.push(data);
|
||||
else if (type === "IEND") {
|
||||
sawEnd = true;
|
||||
break;
|
||||
} else if (/^[A-Z]/u.test(type)) {
|
||||
unsupportedReason ??= `Unsupported critical PNG chunk ${type}.`;
|
||||
}
|
||||
offset = dataEnd + 4;
|
||||
}
|
||||
if (header === undefined || !sawEnd || compressed.length === 0)
|
||||
throw new TypeError("Incomplete PNG");
|
||||
if (header.colorType === 3 && !sawPalette)
|
||||
throw new TypeError("Missing PNG palette");
|
||||
if (unsupportedReason !== undefined)
|
||||
throw new AnalysisUnsupportedTargetError(
|
||||
"compare_web_screenshots",
|
||||
field,
|
||||
unsupportedReason,
|
||||
);
|
||||
if (!header.supported)
|
||||
throw new AnalysisUnsupportedTargetError(
|
||||
"compare_web_screenshots",
|
||||
field,
|
||||
"PNG encoding is not supported for pixel comparison.",
|
||||
);
|
||||
const pixels = BigInt(header.width) * BigInt(header.height);
|
||||
const expectedRawBytes =
|
||||
(BigInt(header.width) * BigInt(header.channels) + 1n) *
|
||||
BigInt(header.height);
|
||||
return {
|
||||
bytes,
|
||||
width: header.width,
|
||||
height: header.height,
|
||||
channels: header.channels,
|
||||
compressed,
|
||||
transparentColor,
|
||||
expectedRawBytes,
|
||||
unfilteredBytes: pixels * BigInt(header.channels),
|
||||
rgbaBytes: pixels * 4n,
|
||||
};
|
||||
};
|
||||
|
||||
const decodePng = (plan: ParsedPng): DecodedPng => {
|
||||
const expected = Number(plan.expectedRawBytes);
|
||||
const raw = inflatePng(Buffer.concat(plan.compressed), expected + 1);
|
||||
const rowBytes = header.width * header.channels;
|
||||
const expected = (rowBytes + 1) * header.height;
|
||||
const raw = inflateSync(Buffer.concat(compressed), {
|
||||
maxOutputLength: expected + 1,
|
||||
});
|
||||
if (raw.byteLength !== expected) throw new TypeError("Unexpected PNG size");
|
||||
return {
|
||||
width: plan.width,
|
||||
height: plan.height,
|
||||
width: header.width,
|
||||
height: header.height,
|
||||
rgba: expandRgba(
|
||||
unfilter(raw, plan.width, plan.height, plan.channels),
|
||||
plan.channels,
|
||||
plan.transparentColor,
|
||||
unfilter(raw, header.width, header.height, header.channels),
|
||||
header.channels,
|
||||
transparentColor,
|
||||
),
|
||||
};
|
||||
};
|
||||
|
||||
const admitEncodedInputPair = (
|
||||
before: WebScreenshotArtifact,
|
||||
after: WebScreenshotArtifact,
|
||||
): void => {
|
||||
const sourceBytes = BigInt(before.bytes) + BigInt(after.bytes);
|
||||
if (sourceBytes > BigInt(PNG_COMPARISON_WORKING_MEMORY_BYTES))
|
||||
throw pngWorkingMemoryError(sourceBytes, null, null);
|
||||
};
|
||||
|
||||
const admitPngPair = (before: ParsedPng, after: ParsedPng): void => {
|
||||
// Input buffers remain live for both plans. For each decode, count the
|
||||
// concatenated IDAT, rounded zlib output chunks, any concatenated output and
|
||||
// unfiltered rows, plus RGB-to-RGBA output. The first completed RGBA image
|
||||
// remains live while the second is decoded.
|
||||
const inputBytes = BigInt(before.bytes.byteLength + after.bytes.byteLength);
|
||||
const estimate = (current: ParsedPng, retained: ParsedPng | undefined) => {
|
||||
const compressedBytes = current.compressed.reduce(
|
||||
(total, chunk) => total + BigInt(chunk.byteLength),
|
||||
0n,
|
||||
);
|
||||
const rgbaExpansion = current.channels === 3 ? current.rgbaBytes : 0n;
|
||||
return (
|
||||
inputBytes +
|
||||
(retained?.rgbaBytes ?? 0n) +
|
||||
compressedBytes +
|
||||
inflateBufferAllowance(current.expectedRawBytes) +
|
||||
current.unfilteredBytes +
|
||||
rgbaExpansion
|
||||
);
|
||||
};
|
||||
const estimateBytes = [
|
||||
estimate(before, undefined),
|
||||
estimate(after, before),
|
||||
].reduce((maximum, value) => (value > maximum ? value : maximum), 0n);
|
||||
if (estimateBytes > BigInt(PNG_COMPARISON_WORKING_MEMORY_BYTES))
|
||||
throw pngWorkingMemoryError(estimateBytes, before, after);
|
||||
};
|
||||
|
||||
const inflateBufferAllowance = (expectedRawBytes: bigint): bigint => {
|
||||
const chunkBytes = BigInt(PNG_INFLATE_CHUNK_BYTES);
|
||||
const maximumOutputBytes = expectedRawBytes + 1n;
|
||||
const roundUpToChunk = (bytes: bigint): bigint =>
|
||||
((bytes + chunkBytes - 1n) / chunkBytes) * chunkBytes;
|
||||
|
||||
// Node checks maxOutputLength after writing a chunk, so a rejected stream can
|
||||
// retain one chunk's worth of output past the limit before it throws.
|
||||
const rejectedOutputChunks = roundUpToChunk(
|
||||
maximumOutputBytes + chunkBytes - 1n,
|
||||
);
|
||||
|
||||
// Sync zlib retains each output chunk while Buffer.concat creates the final
|
||||
// result. A full final chunk also allocates the next scratch chunk first.
|
||||
const returnedOutputChunks = roundUpToChunk(maximumOutputBytes);
|
||||
const scratchChunk =
|
||||
expectedRawBytes % chunkBytes === 0n ||
|
||||
maximumOutputBytes % chunkBytes === 0n
|
||||
? chunkBytes
|
||||
: 0n;
|
||||
const concatenatedOutput =
|
||||
maximumOutputBytes > chunkBytes ? maximumOutputBytes : 0n;
|
||||
const returnedOutput =
|
||||
returnedOutputChunks + scratchChunk + concatenatedOutput;
|
||||
|
||||
return rejectedOutputChunks > returnedOutput
|
||||
? rejectedOutputChunks
|
||||
: returnedOutput;
|
||||
};
|
||||
|
||||
const pngWorkingMemoryError = (
|
||||
estimateBytes: bigint,
|
||||
before: Pick<ParsedPng, "width" | "height"> | null,
|
||||
after: Pick<ParsedPng, "width" | "height"> | null,
|
||||
) =>
|
||||
new AnalysisResourceConstraintError(
|
||||
"compare_web_screenshots",
|
||||
"memory",
|
||||
"Screenshot comparison exceeds REA's 256 MiB estimated working-memory budget.",
|
||||
{
|
||||
boundary: "png-screenshot-comparison",
|
||||
estimated_working_memory_bytes: estimateBytes.toString(),
|
||||
maximum_working_memory_bytes: PNG_COMPARISON_WORKING_MEMORY_BYTES,
|
||||
before_dimensions:
|
||||
before === null ? null : { width: before.width, height: before.height },
|
||||
after_dimensions:
|
||||
after === null ? null : { width: after.width, height: after.height },
|
||||
},
|
||||
{
|
||||
remediationAction:
|
||||
"Compare smaller screenshots or reduce their viewport dimensions before capture.",
|
||||
},
|
||||
);
|
||||
|
||||
const inflatePng = (compressed: Buffer, maxOutputLength: number): Buffer => {
|
||||
try {
|
||||
return inflateSync(compressed, {
|
||||
maxOutputLength,
|
||||
chunkSize: PNG_INFLATE_CHUNK_BYTES,
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
if (
|
||||
cause instanceof Error &&
|
||||
"code" in cause &&
|
||||
(cause.code === "Z_DATA_ERROR" ||
|
||||
cause.code === "Z_BUF_ERROR" ||
|
||||
cause.code === "ERR_BUFFER_TOO_LARGE")
|
||||
)
|
||||
throw new TypeError("Invalid PNG image data", { cause });
|
||||
throw cause;
|
||||
}
|
||||
};
|
||||
|
||||
const parseHeader = (data: Buffer) => {
|
||||
if (data.byteLength !== 13) throw new TypeError("Invalid PNG header");
|
||||
const width = data.readUInt32BE(0);
|
||||
const height = data.readUInt32BE(4);
|
||||
const bitDepth = data[8] ?? 0;
|
||||
const colorType = data[9] ?? -1;
|
||||
const validDepth =
|
||||
(colorType === 0 && [1, 2, 4, 8, 16].includes(bitDepth)) ||
|
||||
(colorType === 2 && [8, 16].includes(bitDepth)) ||
|
||||
(colorType === 3 && [1, 2, 4, 8].includes(bitDepth)) ||
|
||||
(colorType === 4 && [8, 16].includes(bitDepth)) ||
|
||||
(colorType === 6 && [8, 16].includes(bitDepth));
|
||||
const bitDepth = data[8];
|
||||
const colorType = data[9];
|
||||
if (
|
||||
width === 0 ||
|
||||
height === 0 ||
|
||||
width > 0x7fffffff ||
|
||||
height > 0x7fffffff ||
|
||||
!validDepth ||
|
||||
!Number.isSafeInteger(width * height) ||
|
||||
bitDepth !== 8 ||
|
||||
(colorType !== 2 && colorType !== 6) ||
|
||||
data[10] !== 0 ||
|
||||
data[11] !== 0 ||
|
||||
(data[12] !== 0 && data[12] !== 1)
|
||||
data[12] !== 0
|
||||
)
|
||||
throw new TypeError("Invalid PNG header");
|
||||
return {
|
||||
width,
|
||||
height,
|
||||
colorType,
|
||||
bitDepth,
|
||||
interlace: data[12],
|
||||
channels: colorType === 6 ? 4 : 3,
|
||||
supported:
|
||||
bitDepth === 8 && (colorType === 2 || colorType === 6) && data[12] === 0,
|
||||
};
|
||||
throw new TypeError("Unsupported PNG");
|
||||
return { width, height, channels: colorType === 6 ? 4 : 3 };
|
||||
};
|
||||
|
||||
const unfilter = (
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { access, stat } from "node:fs/promises";
|
||||
import { access } from "node:fs/promises";
|
||||
import { constants } from "node:fs";
|
||||
import { isAbsolute } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
@@ -32,8 +32,6 @@ export class MitmproxyCaptureAdapter implements HistoricalCaptureFormatAdapter {
|
||||
);
|
||||
try {
|
||||
await access(command, constants.R_OK | constants.X_OK);
|
||||
if (!(await stat(command)).isFile())
|
||||
throw new TypeError("Configured path is not a regular file");
|
||||
} catch (cause: unknown) {
|
||||
const reason = `Configured mitmdump executable is unavailable: ${command}.`;
|
||||
throw new AnalysisCapabilityUnavailableError(
|
||||
|
||||
@@ -1,12 +1,6 @@
|
||||
import { z } from "incur";
|
||||
|
||||
import { inspectAnalysisViewValidated } from "../application/analysisView/AnalysisViewService.js";
|
||||
import { readEvidenceBundle } from "../application/EvidenceBundleFiles.js";
|
||||
import { inspectEvidenceBundle } from "../application/investigation/InspectEvidenceBundle.js";
|
||||
import {
|
||||
getEvidenceBundleInputSchema,
|
||||
importEvidenceBundleInputSchema,
|
||||
} from "../contracts/sessionToolSchemas.js";
|
||||
import { CLI_COMMANDS } from "../cliCommandNames.js";
|
||||
import { parseCliJsonInput } from "../cliJsonInput.js";
|
||||
import { logCliCommand } from "../cliLogging.js";
|
||||
@@ -22,53 +16,6 @@ export const registerAnalysisViewCommands = (
|
||||
cli: CliInstance,
|
||||
logger: Logger,
|
||||
): void => {
|
||||
cli.command(CLI_COMMANDS.inspectEvidenceBundle, {
|
||||
description:
|
||||
"Inspect complete or selected discovery metadata from a local Evidence bundle",
|
||||
args: z.object({
|
||||
inputJson: z
|
||||
.string()
|
||||
.describe(
|
||||
"JSON or JSON file with absolute path, detail and optional record filters",
|
||||
),
|
||||
}),
|
||||
run: ({ args }) =>
|
||||
logCliCommand(logger, CLI_COMMANDS.inspectEvidenceBundle, async () => {
|
||||
const input = await parseCliJsonInput(
|
||||
args.inputJson,
|
||||
CLI_COMMANDS.inspectEvidenceBundle,
|
||||
);
|
||||
if (!input.ok) return input.error;
|
||||
const schema = getEvidenceBundleInputSchema.safeExtend({
|
||||
path: importEvidenceBundleInputSchema.shape.path,
|
||||
});
|
||||
const parsed = schema.safeParse(input.value);
|
||||
if (!parsed.success)
|
||||
return {
|
||||
error: "Evidence bundle inspection failed",
|
||||
...projectAnalysisError(
|
||||
analysisInputErrorFromIssues(
|
||||
CLI_COMMANDS.inspectEvidenceBundle,
|
||||
parsed.error.issues,
|
||||
input.value,
|
||||
),
|
||||
),
|
||||
};
|
||||
const loaded = await readEvidenceBundle(parsed.data.path);
|
||||
if (!loaded.ok)
|
||||
return {
|
||||
error: "Evidence bundle inspection failed",
|
||||
...projectAnalysisError(loaded.error),
|
||||
};
|
||||
const result = inspectEvidenceBundle(loaded.value, parsed.data);
|
||||
return result.ok
|
||||
? result.value
|
||||
: {
|
||||
error: "Evidence bundle inspection failed",
|
||||
...projectAnalysisError(result.error),
|
||||
};
|
||||
}),
|
||||
});
|
||||
cli.command(CLI_COMMANDS.inspectAnalysisView, {
|
||||
description:
|
||||
"Project a selected view of completed layout, JavaScript application or native function Evidence JSON",
|
||||
|
||||
@@ -43,7 +43,6 @@ export const CLI_COMMANDS = Object.freeze({
|
||||
inspectBinaryLayout: "inspect-binary-layout",
|
||||
inspectPeResources: "inspect-pe-resources",
|
||||
inspectAnalysisView: "inspect-analysis-view",
|
||||
inspectEvidenceBundle: "inspect-evidence-bundle",
|
||||
inspectFirmwareRegions: "inspect-firmware-regions",
|
||||
extractFirmware: "extract-firmware",
|
||||
inspectAndroidPackage: "inspect-android-package",
|
||||
|
||||
@@ -7,6 +7,7 @@ import {
|
||||
} from "../application/DirectAnalysis.js";
|
||||
import { runSessionStatus as executeSessionStatus } from "../application/DirectAnalysisStatus.js";
|
||||
import type { DirectAnalysisDependencies } from "../application/DirectAnalysisDependencies.js";
|
||||
import { createBinarySession, createManagedBinarySession } from "./binary.js";
|
||||
|
||||
/** Bind one-shot analysis and status commands to one caller-selected environment. */
|
||||
export const createDirectAnalysis = (
|
||||
@@ -15,14 +16,9 @@ export const createDirectAnalysis = (
|
||||
const environment = snapshotEnvironment(selectedEnvironment);
|
||||
const dependencies: DirectAnalysisDependencies = {
|
||||
readConfiguration: () => parseConfig(environment),
|
||||
createBinarySession: async (config, logger) => {
|
||||
const { createBinarySession } = await import("./binary.js");
|
||||
return createBinarySession(config, logger, environment);
|
||||
},
|
||||
createManagedBinarySession: async () => {
|
||||
const { createManagedBinarySession } = await import("./binary.js");
|
||||
return createManagedBinarySession();
|
||||
},
|
||||
createBinarySession: (config, logger) =>
|
||||
createBinarySession(config, logger, environment),
|
||||
createManagedBinarySession,
|
||||
};
|
||||
return {
|
||||
runDirectAnalysis: executeDirectAnalysis.bind(undefined, dependencies),
|
||||
@@ -38,9 +34,6 @@ export type DirectAnalysis = ReturnType<typeof createDirectAnalysis>;
|
||||
export const runManagedProviderExecution = executeManagedProvider.bind(
|
||||
undefined,
|
||||
{
|
||||
createManagedBinarySession: async () => {
|
||||
const { createManagedBinarySession } = await import("./binary.js");
|
||||
return createManagedBinarySession();
|
||||
},
|
||||
createManagedBinarySession,
|
||||
},
|
||||
);
|
||||
|
||||
@@ -159,7 +159,7 @@ export const SESSION_TOOL_CONTRACTS = [
|
||||
),
|
||||
session(
|
||||
"get_evidence_bundle",
|
||||
"Return every retained Evidence record and residual unknown as a complete canonical bundle by default. detail=summary discovers retained records without copying or transferring their payloads; optionally filter by exact Evidence ID, operation, target SHA-256, analysis-profile digest or canonical native procedure address. Metadata distinguishes complete record retention from native value-flow truncation or unavailable/unknown coverage. Use a discovered evidence_id with inspect_analysis_view without restarting a provider. Filters apply only to summary delivery.",
|
||||
"Return every Evidence record and residual unknown currently retained by this session as one inline bundle for direct inspection or follow-up workflows.",
|
||||
getEvidenceBundleInputSchema,
|
||||
),
|
||||
session(
|
||||
|
||||
@@ -6,13 +6,9 @@ import { localPathStringSchema } from "../domain/localPath.js";
|
||||
import { processTraceSpecificationSchema } from "../domain/process/processTraceSpecification.js";
|
||||
import { evidenceSchema } from "../domain/evidence.js";
|
||||
import { prefixedDigestSchema } from "./../domain/digests.js";
|
||||
import { evidenceRecordFiltersSchema } from "../domain/evidenceRecordSummary.js";
|
||||
|
||||
/** Complete retained Evidence, or selected discovery metadata without payloads. */
|
||||
export const getEvidenceBundleInputSchema = z.strictObject({
|
||||
detail: z.enum(["complete", "summary"]).default("complete"),
|
||||
filters: evidenceRecordFiltersSchema.optional(),
|
||||
});
|
||||
/** Return the current canonical Evidence bundle inline. */
|
||||
export const getEvidenceBundleInputSchema = z.strictObject({});
|
||||
|
||||
/** Optional document selection for volatile navigation context. */
|
||||
export const navigationContextInputSchema = z.strictObject({
|
||||
|
||||
@@ -53,10 +53,6 @@ export const TOOL_EXAMPLE_OVERRIDES: Readonly<
|
||||
trace_native_ui_action: { action: "buildTapped:" },
|
||||
open_binary: { path: "/tmp/fixture" },
|
||||
export_evidence_bundle: { path: "/tmp/evidence.json" },
|
||||
get_evidence_bundle: {
|
||||
detail: "summary",
|
||||
filters: { operation: "analyze_function", procedure_address: "0x1000" },
|
||||
},
|
||||
inspect_address_context: { address: "0x1000" },
|
||||
import_evidence_bundle: { path: "/tmp/evidence.json" },
|
||||
capture_process_scenario: {
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { z } from "zod";
|
||||
import { evidenceBundleSummarySchema } from "../domain/evidenceRecordSummary.js";
|
||||
import { BINARY_ARCHITECTURES } from "../domain/binaryTargetTypes.js";
|
||||
import { nativeFunctionAnnotationsSchema } from "../domain/native/nativeFunctionAnnotations.js";
|
||||
import { nativeLoadImageSchema } from "../domain/native/nativeLoadImage.js";
|
||||
@@ -471,9 +470,7 @@ export const sessionOutputSchemas = {
|
||||
unknowns: z.number().int().min(0),
|
||||
}),
|
||||
),
|
||||
get_evidence_bundle: lifecycleResultOf(
|
||||
z.union([evidenceBundleSchema, evidenceBundleSummarySchema]),
|
||||
),
|
||||
get_evidence_bundle: lifecycleResultOf(evidenceBundleSchema),
|
||||
get_navigation_context: lifecycleResultOf(
|
||||
z.object({
|
||||
document: z.string(),
|
||||
|
||||
@@ -47,7 +47,7 @@ export const analysisErrorRemediationAction = (
|
||||
return (
|
||||
error.remediationAction ??
|
||||
(error.resource === "transport"
|
||||
? "Discover retained records with get_evidence_bundle and detail: summary, then inspect compatible records through inspect_analysis_view using an exact Evidence reference. Export complete retained session evidence through export_evidence_bundle to a caller-selected path, or use complete CLI JSON output. The connection remains usable."
|
||||
? "Export retained session evidence through export_evidence_bundle to a caller-selected path, or use complete CLI JSON output. Tools that accept retained-evidence references can inspect compatible records. The connection remains usable."
|
||||
: error.resource === "cpu"
|
||||
? "Review the reported worker CPU limits and observed signal. Retry with sufficient CPU time or a smaller artifact; REA retains tighter inherited limits."
|
||||
: error.resource === "file-size"
|
||||
|
||||
@@ -21,12 +21,6 @@ describe("cooperative canonical JSON digest", () => {
|
||||
),
|
||||
{ payload: `${"x".repeat(8191)}𐀀${"\n\ud800".repeat(20_000)}` },
|
||||
{ [`${"x".repeat(8191)}𐀀${"\n".repeat(20_000)}`]: "value" },
|
||||
Object.fromEntries(
|
||||
Array.from({ length: 512 }, (_, index) => [
|
||||
`escaped\n\"\\𐀀${"x".repeat(32)}${index}`,
|
||||
{ repeated: index },
|
||||
]),
|
||||
),
|
||||
];
|
||||
it.each(values.map((value) => ({ value })))(
|
||||
"preserves complete canonical meaning (%#)",
|
||||
@@ -61,18 +55,6 @@ describe("cooperative canonical JSON digest", () => {
|
||||
expect(steps.next().done).toBe(true);
|
||||
});
|
||||
|
||||
it("continues object hashing when a pending property is omitted", () => {
|
||||
const value: { [key: string]: JsonValue } = {
|
||||
a: "x".repeat(32_768),
|
||||
b: 1,
|
||||
c: { retained: true },
|
||||
};
|
||||
const steps = canonicalJsonDigestSteps(value);
|
||||
expect(steps.next().done).toBe(false);
|
||||
delete value.b;
|
||||
expect(complete(steps)).toBe(digestCanonicalValue(value));
|
||||
});
|
||||
|
||||
it("rejects sparse arrays, nonfinite numbers and cycles", () => {
|
||||
const sparse: JsonValue[] = [];
|
||||
sparse.length = 2;
|
||||
|
||||
@@ -26,9 +26,6 @@ export function* canonicalJsonDigestSteps(
|
||||
const hash = createHash("sha256");
|
||||
const ancestors = new Set<object>();
|
||||
const stack: DigestFrame[] = [{ kind: "value", value }];
|
||||
const encodedKeys = new Map<string, string>();
|
||||
// Keep reusable key encodings within one existing digest-buffer quantum.
|
||||
let remainingKeyCodeUnits = 8192;
|
||||
let buffered = "";
|
||||
let flushed = false;
|
||||
const emit = (part: string): void => {
|
||||
@@ -72,22 +69,22 @@ export function* canonicalJsonDigestSteps(
|
||||
stack.push(frame);
|
||||
continue;
|
||||
}
|
||||
let item: JsonValue;
|
||||
if (frame.kind === "array") {
|
||||
if (frame.index === frame.length) {
|
||||
emit("]");
|
||||
ancestors.delete(frame.value);
|
||||
continue;
|
||||
}
|
||||
const element = frame.value[frame.index];
|
||||
if (element === undefined)
|
||||
const item = frame.value[frame.index];
|
||||
if (item === undefined)
|
||||
throw new TypeError(
|
||||
"Canonical JSON digest requires dense JSON arrays",
|
||||
);
|
||||
if (frame.index++ > 0) emit(",");
|
||||
stack.push(frame);
|
||||
item = element;
|
||||
} else if (frame.kind === "object") {
|
||||
stack.push(frame, { kind: "value", value: item });
|
||||
continue;
|
||||
}
|
||||
if (frame.kind === "object") {
|
||||
if (frame.index === frame.keys.length) {
|
||||
emit("}");
|
||||
ancestors.delete(frame.value);
|
||||
@@ -96,42 +93,25 @@ export function* canonicalJsonDigestSteps(
|
||||
const key = frame.keys[frame.index++];
|
||||
if (key === undefined)
|
||||
throw new TypeError("Canonical JSON object changed during hashing");
|
||||
const property = frame.value[key];
|
||||
if (property === undefined) {
|
||||
stack.push(frame);
|
||||
continue;
|
||||
}
|
||||
const item = frame.value[key];
|
||||
if (item === undefined) continue;
|
||||
if (frame.written++ > 0) emit(",");
|
||||
if (key.length > 8192) {
|
||||
emit('"');
|
||||
stack.push(
|
||||
frame,
|
||||
{ kind: "value", value: property },
|
||||
{ kind: "value", value: item },
|
||||
{ kind: "separator" },
|
||||
{ kind: "string", value: key, index: 0 },
|
||||
);
|
||||
continue;
|
||||
}
|
||||
let encodedKey = encodedKeys.get(key);
|
||||
if (encodedKey === undefined) {
|
||||
encodedKey = JSON.stringify(key);
|
||||
if (encodedKey.length <= remainingKeyCodeUnits) {
|
||||
encodedKeys.set(key, encodedKey);
|
||||
remainingKeyCodeUnits -= encodedKey.length;
|
||||
}
|
||||
}
|
||||
emit(encodedKey);
|
||||
emit(JSON.stringify(key));
|
||||
emit(":");
|
||||
stack.push(frame);
|
||||
item = property;
|
||||
} else {
|
||||
item = frame.value;
|
||||
}
|
||||
// Resume containers directly; scalar members need no temporary value frame.
|
||||
if (flushed) {
|
||||
flushed = false;
|
||||
yield;
|
||||
stack.push(frame, { kind: "value", value: item });
|
||||
continue;
|
||||
}
|
||||
const item = frame.value;
|
||||
if (typeof item === "string" && item.length > 8192) {
|
||||
emit('"');
|
||||
stack.push({ kind: "string", value: item, index: 0 });
|
||||
@@ -171,7 +151,6 @@ export function* canonicalJsonDigestSteps(
|
||||
} finally {
|
||||
stack.length = 0;
|
||||
ancestors.clear();
|
||||
encodedKeys.clear();
|
||||
buffered = "";
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,55 +0,0 @@
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import { createEvidence } from "./evidence.js";
|
||||
import { createEvidenceBundle } from "./evidenceBundle.js";
|
||||
import { summarizeEvidenceBundle } from "./evidenceRecordSummary.js";
|
||||
|
||||
it("preserves native identity independently of missing or incompatible value-flow metadata", () => {
|
||||
const records = [
|
||||
undefined,
|
||||
{ available: true, truncated: true },
|
||||
{ unexpected: "legacy" },
|
||||
{ available: false, reason: "Decompiler unavailable" },
|
||||
].map((flow) =>
|
||||
createEvidence(
|
||||
{ path: "/fixtures/native.exe", format: "pe", sha256: "a".repeat(64) },
|
||||
{ id: "provider", name: "Provider", version: "1" },
|
||||
{
|
||||
operation: "analyze_function",
|
||||
parameters: {},
|
||||
result: {
|
||||
procedure: { address: "space:0x10", name: "entry" },
|
||||
...(flow === undefined ? {} : { native_value_flow: flow }),
|
||||
},
|
||||
},
|
||||
),
|
||||
);
|
||||
const summary = summarizeEvidenceBundle(createEvidenceBundle(records), {
|
||||
procedure_address: "space:0x10",
|
||||
});
|
||||
expect(summary.matching_records).toBe(4);
|
||||
const native = summary.records.map((r) => r.native_dossier);
|
||||
expect(
|
||||
native.every((r) => r.available && r.procedure_address === "space:0x10"),
|
||||
).toBe(true);
|
||||
expect(native).toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.objectContaining({ value_flow: { status: "not-recorded" } }),
|
||||
expect.objectContaining({
|
||||
value_flow: {
|
||||
status: "observed",
|
||||
truncated: true,
|
||||
omitted_operations_lower_bound: null,
|
||||
known_omitted_inputs: null,
|
||||
known_omitted_edges: null,
|
||||
},
|
||||
}),
|
||||
expect.objectContaining({
|
||||
value_flow: { status: "unknown", reason: expect.any(String) },
|
||||
}),
|
||||
expect.objectContaining({
|
||||
value_flow: { status: "unavailable", reason: "Decompiler unavailable" },
|
||||
}),
|
||||
]),
|
||||
);
|
||||
});
|
||||
@@ -1,166 +0,0 @@
|
||||
import { z } from "zod";
|
||||
|
||||
import { digestSchema, prefixedDigestSchema } from "./digests.js";
|
||||
import { evidenceSchema, type Evidence } from "./evidence.js";
|
||||
import type { EvidenceBundle } from "./evidenceBundle.js";
|
||||
|
||||
/** Exact retained-record selectors; selection never resolves a provider address. */
|
||||
export const evidenceRecordFiltersSchema = z.strictObject({
|
||||
evidence_id: prefixedDigestSchema("ev").optional(),
|
||||
operation: z.string().min(1).optional(),
|
||||
target_sha256: digestSchema.optional(),
|
||||
analysis_profile_digest: digestSchema.optional(),
|
||||
procedure_address: z.string().min(1).optional(),
|
||||
});
|
||||
export type EvidenceRecordFilters = z.infer<typeof evidenceRecordFiltersSchema>;
|
||||
|
||||
const countSchema = z.number().int().nonnegative();
|
||||
const nativeMetadataSchema = z.object({
|
||||
procedure: z.object({ address: z.string(), name: z.string().nullish() }),
|
||||
native_value_flow: z.unknown().optional(),
|
||||
});
|
||||
const valueFlowMetadataSchema = z.union([
|
||||
z.object({
|
||||
available: z.literal(true),
|
||||
truncated: z.boolean(),
|
||||
omitted_operations_lower_bound: countSchema.optional(),
|
||||
known_omitted_inputs: countSchema.optional(),
|
||||
known_omitted_edges: countSchema.optional(),
|
||||
}),
|
||||
z.object({ available: z.literal(false), reason: z.string() }),
|
||||
]);
|
||||
|
||||
const nativeSummarySchema = z.discriminatedUnion("available", [
|
||||
z.strictObject({
|
||||
available: z.literal(true),
|
||||
procedure_address: z.string(),
|
||||
procedure_name: z.string().nullable(),
|
||||
value_flow: z.discriminatedUnion("status", [
|
||||
z.strictObject({
|
||||
status: z.literal("observed"),
|
||||
truncated: z.boolean(),
|
||||
omitted_operations_lower_bound: countSchema.nullable(),
|
||||
known_omitted_inputs: countSchema.nullable(),
|
||||
known_omitted_edges: countSchema.nullable(),
|
||||
}),
|
||||
z.strictObject({ status: z.literal("unavailable"), reason: z.string() }),
|
||||
z.strictObject({ status: z.literal("not-recorded") }),
|
||||
z.strictObject({ status: z.literal("unknown"), reason: z.string() }),
|
||||
]),
|
||||
}),
|
||||
z.strictObject({
|
||||
available: z.literal(false),
|
||||
reason: z.enum(["different-operation", "not-recorded-or-incompatible"]),
|
||||
}),
|
||||
]);
|
||||
|
||||
/** Metadata for complete retained records, independent of analysis completeness. */
|
||||
export const evidenceBundleSummarySchema = z.strictObject({
|
||||
kind: z.literal("evidence-bundle-summary"),
|
||||
filters: z.record(z.string(), z.string()),
|
||||
total_retained_records: countSchema,
|
||||
matching_records: countSchema,
|
||||
unknown_revisions: countSchema,
|
||||
records: z.array(
|
||||
z.strictObject({
|
||||
evidence_id: evidenceSchema.shape.evidence_id,
|
||||
subject: evidenceSchema.shape.subject,
|
||||
provider: evidenceSchema.shape.provider,
|
||||
operation: evidenceSchema.shape.operation,
|
||||
confidence: evidenceSchema.shape.confidence,
|
||||
authority: evidenceSchema.shape.authority,
|
||||
analysis_profile_digest: digestSchema.nullable(),
|
||||
retention: z.literal("complete-record"),
|
||||
limitation_count: countSchema,
|
||||
native_dossier: nativeSummarySchema,
|
||||
}),
|
||||
),
|
||||
});
|
||||
export type EvidenceBundleSummary = z.infer<typeof evidenceBundleSummarySchema>;
|
||||
|
||||
const nativeSummary = (
|
||||
evidence: Evidence,
|
||||
): z.infer<typeof nativeSummarySchema> => {
|
||||
if (evidence.operation !== "analyze_function")
|
||||
return { available: false, reason: "different-operation" };
|
||||
// Read only producer metadata. Parsing the full dossier would allocate every
|
||||
// large collection merely to discover its already authenticated parent ID.
|
||||
const parsed = nativeMetadataSchema.safeParse(evidence.normalized_result);
|
||||
if (!parsed.success)
|
||||
return { available: false, reason: "not-recorded-or-incompatible" };
|
||||
const flow = valueFlowMetadataSchema.safeParse(parsed.data.native_value_flow);
|
||||
return {
|
||||
available: true,
|
||||
procedure_address: parsed.data.procedure.address,
|
||||
procedure_name: parsed.data.procedure.name ?? null,
|
||||
value_flow:
|
||||
parsed.data.native_value_flow == null
|
||||
? { status: "not-recorded" }
|
||||
: !flow.success
|
||||
? {
|
||||
status: "unknown",
|
||||
reason: "Value-flow metadata has an incompatible representation.",
|
||||
}
|
||||
: flow.data.available
|
||||
? {
|
||||
status: "observed",
|
||||
truncated: flow.data.truncated,
|
||||
omitted_operations_lower_bound:
|
||||
flow.data.omitted_operations_lower_bound ?? null,
|
||||
known_omitted_inputs: flow.data.known_omitted_inputs ?? null,
|
||||
known_omitted_edges: flow.data.known_omitted_edges ?? null,
|
||||
}
|
||||
: { status: "unavailable", reason: flow.data.reason },
|
||||
};
|
||||
};
|
||||
|
||||
/** Discover retained Evidence without reading or copying its complete payloads. */
|
||||
export const summarizeEvidenceBundle = (
|
||||
bundle: EvidenceBundle,
|
||||
filters: EvidenceRecordFilters = {},
|
||||
): EvidenceBundleSummary => {
|
||||
const selectedFilters: Record<string, string> = {};
|
||||
for (const [field, value] of Object.entries(filters))
|
||||
if (value !== undefined) selectedFilters[field] = value;
|
||||
const records: EvidenceBundleSummary["records"] = [];
|
||||
for (const evidence of bundle.records) {
|
||||
if (
|
||||
(filters.evidence_id !== undefined &&
|
||||
filters.evidence_id !== evidence.evidence_id) ||
|
||||
(filters.operation !== undefined &&
|
||||
filters.operation !== evidence.operation) ||
|
||||
(filters.target_sha256 !== undefined &&
|
||||
filters.target_sha256 !== evidence.subject?.digest.sha256) ||
|
||||
(filters.analysis_profile_digest !== undefined &&
|
||||
filters.analysis_profile_digest !== evidence.analysis_profile?.digest)
|
||||
)
|
||||
continue;
|
||||
const native = nativeSummary(evidence);
|
||||
if (
|
||||
filters.procedure_address !== undefined &&
|
||||
(!native.available ||
|
||||
filters.procedure_address !== native.procedure_address)
|
||||
)
|
||||
continue;
|
||||
records.push({
|
||||
evidence_id: evidence.evidence_id,
|
||||
subject: evidence.subject,
|
||||
provider: evidence.provider,
|
||||
operation: evidence.operation,
|
||||
confidence: evidence.confidence,
|
||||
authority: evidence.authority,
|
||||
analysis_profile_digest: evidence.analysis_profile?.digest ?? null,
|
||||
retention: "complete-record",
|
||||
limitation_count: evidence.limitations.length,
|
||||
native_dossier: native,
|
||||
});
|
||||
}
|
||||
return {
|
||||
kind: "evidence-bundle-summary",
|
||||
filters: selectedFilters,
|
||||
total_retained_records: bundle.records.length,
|
||||
matching_records: records.length,
|
||||
unknown_revisions: bundle.unknowns.length,
|
||||
records,
|
||||
};
|
||||
};
|
||||
@@ -72,62 +72,6 @@ describe("read-only NativeAOT PE parser", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("does not discover an RTR directory in raw padding beyond VirtualSize", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const sectionTable = 0x98 + 0xf0;
|
||||
const dataSection = sectionTable + 40;
|
||||
bytes.writeUInt32LE(0x20, dataSection + 8);
|
||||
const parsed = parseNativeAotPe(
|
||||
bytes,
|
||||
nativeAotPeDigest(bytes),
|
||||
SUMMARY_BUDGET,
|
||||
);
|
||||
expect(parsed.summary).toMatchObject({
|
||||
status: "not_applicable",
|
||||
reason:
|
||||
"No supported ReadyToRun directory signature was found in initialized non-executable PE sections.",
|
||||
method_tables: 0,
|
||||
});
|
||||
});
|
||||
|
||||
it("uses raw size as the virtual extent when VirtualSize is zero", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const sectionTable = 0x98 + 0xf0;
|
||||
bytes.writeUInt32LE(0, sectionTable + 40 + 8);
|
||||
const parsed = parseNativeAotPe(
|
||||
bytes,
|
||||
nativeAotPeDigest(bytes),
|
||||
SUMMARY_BUDGET,
|
||||
);
|
||||
expect(parsed.summary).toMatchObject({
|
||||
status: "complete",
|
||||
method_tables: 3,
|
||||
derived_overlay: { address: "0x140002380", size_bytes: 1 },
|
||||
});
|
||||
});
|
||||
|
||||
it("accepts executable-section targets in the zero-filled virtual tail", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const sectionTable = 0x98 + 0xf0;
|
||||
bytes.writeUInt32LE(0x300, sectionTable + 8);
|
||||
const slotOffset = 0x600 + (0x2280 - 0x2000) + 24;
|
||||
// RVA 0x1200 is executable-section zero fill: virtual size 0x300, raw size 0x200.
|
||||
bytes.writeBigUInt64LE(0x140001200n, slotOffset);
|
||||
const parsed = parseNativeAotPe(
|
||||
bytes,
|
||||
nativeAotPeDigest(bytes),
|
||||
SUMMARY_BUDGET,
|
||||
"0x140000000",
|
||||
);
|
||||
expect(parsed.summary).toMatchObject({
|
||||
status: "complete",
|
||||
method_tables: 3,
|
||||
});
|
||||
expect(parsed.readTypeDetail("0x140002280", SUMMARY_BUDGET)).toMatchObject({
|
||||
virtual_slots: [{ slot: 0, target_address: "0x140001200" }],
|
||||
});
|
||||
});
|
||||
|
||||
it("does not claim an image mapping when Ghidra rebased the image", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
expect(
|
||||
|
||||
@@ -13,7 +13,7 @@ export const NATIVE_AOT_MAX_WORK_UNITS = 64 * 1024 * 1024;
|
||||
export const NATIVE_AOT_MAX_WORKING_MEMORY_BYTES = 64 * 1024 * 1024;
|
||||
/** Version of the built-in read-only PE metadata interpretation in profiles. */
|
||||
export const NATIVE_AOT_PROFILE_CONTRACT_REVISION =
|
||||
"rtr-9.1-x64-pe-read-only-v3";
|
||||
"rtr-9.1-x64-pe-read-only-v1";
|
||||
|
||||
export type NativeAotParserLimits = {
|
||||
readonly sourceBytes: number;
|
||||
@@ -81,12 +81,6 @@ type Section = {
|
||||
readonly rawSize: number;
|
||||
readonly characteristics: number;
|
||||
};
|
||||
// Compatible PE loaders use raw size when VirtualSize is zero. Otherwise,
|
||||
// RVA containment follows the declared VirtualSize, independent of file size.
|
||||
const virtualExtent = (section: Section): number =>
|
||||
section.virtualSize === 0 ? section.rawSize : section.virtualSize;
|
||||
const fileBackedVirtualExtent = (section: Section): number =>
|
||||
Math.min(virtualExtent(section), section.rawSize);
|
||||
type RtrSection = {
|
||||
readonly type: number;
|
||||
readonly flags: number;
|
||||
@@ -423,7 +417,10 @@ class PeImage {
|
||||
});
|
||||
for (const section of this.sections) {
|
||||
this.#range(section.rawOffset, section.rawSize);
|
||||
if (section.rva + virtualExtent(section) > this.sizeOfImage)
|
||||
if (
|
||||
section.rva + Math.max(section.virtualSize, section.rawSize) >
|
||||
this.sizeOfImage
|
||||
)
|
||||
throw new Error("PE section exceeds the declared image extent");
|
||||
}
|
||||
}
|
||||
@@ -459,7 +456,7 @@ class PeImage {
|
||||
for (const section of this.sections) {
|
||||
if (this.#budget !== undefined && !this.#budget.consumeWork(1))
|
||||
return null;
|
||||
const extent = virtualExtent(section);
|
||||
const extent = Math.max(section.virtualSize, section.rawSize);
|
||||
if (relative < section.rva || relative - section.rva > extent - size)
|
||||
continue;
|
||||
const delta = relative - section.rva;
|
||||
@@ -469,7 +466,6 @@ class PeImage {
|
||||
}
|
||||
return null;
|
||||
}
|
||||
// This checks executable section membership, not file-backed instruction bytes.
|
||||
isExecutable(address: bigint): boolean {
|
||||
if (address < this.imageBase) return false;
|
||||
const rva = address - this.imageBase;
|
||||
@@ -479,7 +475,7 @@ class PeImage {
|
||||
if (
|
||||
(section.characteristics & 0x20000000) !== 0 &&
|
||||
rva >= BigInt(section.rva) &&
|
||||
rva < BigInt(section.rva + virtualExtent(section))
|
||||
rva < BigInt(section.rva + section.rawSize)
|
||||
)
|
||||
return true;
|
||||
}
|
||||
@@ -496,7 +492,10 @@ class PeImage {
|
||||
if (this.#budget !== undefined && !this.#budget.consumeWork(1)) return 0;
|
||||
if (
|
||||
rva >= BigInt(candidate.rva) &&
|
||||
rva < BigInt(candidate.rva + virtualExtent(candidate))
|
||||
rva <
|
||||
BigInt(
|
||||
candidate.rva + Math.max(candidate.virtualSize, candidate.rawSize),
|
||||
)
|
||||
) {
|
||||
section = candidate;
|
||||
break;
|
||||
@@ -504,7 +503,9 @@ class PeImage {
|
||||
}
|
||||
return section === undefined
|
||||
? 0
|
||||
: section.rva + virtualExtent(section) - Number(rva);
|
||||
: section.rva +
|
||||
Math.max(section.virtualSize, section.rawSize) -
|
||||
Number(rva);
|
||||
}
|
||||
#range(offset: number, size: number): void {
|
||||
if (
|
||||
@@ -601,7 +602,7 @@ const findDirectory = (
|
||||
)
|
||||
continue;
|
||||
const begin = section.rawOffset;
|
||||
const end = begin + fileBackedVirtualExtent(section);
|
||||
const end = begin + section.rawSize;
|
||||
const first = (begin + 7) & ~7;
|
||||
const candidateBytes =
|
||||
first <= end - 16 ? (Math.floor((end - 16 - first) / 8) + 1) * 8 : 0;
|
||||
@@ -969,11 +970,10 @@ const discoverTables = (
|
||||
(section.characteristics & 0x20000000) !== 0
|
||||
)
|
||||
continue;
|
||||
const extent = fileBackedVirtualExtent(section);
|
||||
if (extent > 0) {
|
||||
if (section.rawSize > 0) {
|
||||
ranges.push({
|
||||
start: image.addressForRva(section.rva),
|
||||
end: image.addressForRva(section.rva + extent),
|
||||
end: image.addressForRva(section.rva + section.rawSize),
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -984,7 +984,9 @@ const discoverTables = (
|
||||
(candidate.characteristics & 0x20000000) === 0 &&
|
||||
hydrated.base >= image.addressForRva(candidate.rva) &&
|
||||
hydrated.base + BigInt(hydrated.sizeBytes) <=
|
||||
image.addressForRva(candidate.rva + virtualExtent(candidate)),
|
||||
image.addressForRva(
|
||||
candidate.rva + Math.max(candidate.virtualSize, candidate.rawSize),
|
||||
),
|
||||
);
|
||||
if (section !== undefined)
|
||||
ranges.push({
|
||||
@@ -1590,10 +1592,11 @@ export const parseNativeAotPe = (
|
||||
? "raw-image-pointer-scan"
|
||||
: "raw-image-and-rehydrated-pointer-scan",
|
||||
},
|
||||
diagnostics:
|
||||
hydrationError === null
|
||||
diagnostics: [
|
||||
...(hydrationError === null
|
||||
? []
|
||||
: [`Read-only hydration stopped: ${hydrationError}`],
|
||||
: [`Read-only hydration stopped: ${hydrationError}`]),
|
||||
],
|
||||
limitations: reportLimitations,
|
||||
};
|
||||
if (Buffer.byteLength(JSON.stringify(summary)) > maximumSummaryBytes)
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
/** Generated from package.json and package-lock.json; do not edit. */
|
||||
export const PACKAGE_METADATA = {
|
||||
name: "rea-agents",
|
||||
version: "6.4.0", // x-release-please-version
|
||||
version: "6.3.0", // x-release-please-version
|
||||
serverSdkVersion: "2.3.1",
|
||||
clientSdkVersion: "2.3.1",
|
||||
coreSdkVersion: "2.3.1",
|
||||
|
||||
@@ -226,7 +226,7 @@ describe("Ghidra platform support", () => {
|
||||
executable_role: target.executableRole,
|
||||
managed: target.managed,
|
||||
native_aot_metadata: {
|
||||
contract_revision: "rtr-9.1-x64-pe-read-only-v3",
|
||||
contract_revision: "rtr-9.1-x64-pe-read-only-v1",
|
||||
source_bytes: 128 * 1024 * 1024,
|
||||
work_units: 64 * 1024 * 1024,
|
||||
working_memory_bytes: 64 * 1024 * 1024,
|
||||
|
||||
@@ -44,15 +44,6 @@ describe("ghidraSessionRoot", () => {
|
||||
).toBe(posixBase("Users", "operator"));
|
||||
});
|
||||
|
||||
it("can select a filesystem root when the inherited temp starts with a dot-prefixed element", () => {
|
||||
expect(
|
||||
ghidraSessionRoot({
|
||||
base: posixBase(".Tmp", "scratch"),
|
||||
platform: "win32",
|
||||
}),
|
||||
).toBe(resolve(sep));
|
||||
});
|
||||
|
||||
it.skipIf(!POSIX)("prefers the platform temp directory by default", () => {
|
||||
expect(
|
||||
ghidraSessionRoot({ base: posixBase("home", "operator", ".cache") }),
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { constants } from "node:fs";
|
||||
import { access, realpath, stat } from "node:fs/promises";
|
||||
import { access, realpath } from "node:fs/promises";
|
||||
import { isAbsolute } from "node:path";
|
||||
import {
|
||||
AnalysisCancelledError,
|
||||
@@ -87,8 +87,6 @@ export const resolveWakaruCommand = async (
|
||||
if (!isAbsolute(limiter))
|
||||
throw new TypeError("REA_JAVASCRIPT_PRLIMIT_COMMAND must be absolute");
|
||||
await access(limiter, constants.X_OK);
|
||||
if (!(await stat(limiter)).isFile())
|
||||
throw new TypeError(`Resource limiter is not a regular file: ${limiter}`);
|
||||
return { command, limiter, ...fingerprint };
|
||||
} catch (cause: unknown) {
|
||||
const reason = `Configured Wakaru or util-linux prlimit is unavailable: ${recoveryFailureMessage(cause)}`;
|
||||
|
||||
@@ -28,12 +28,6 @@ interface Section {
|
||||
readonly size: number;
|
||||
}
|
||||
|
||||
// RVA ownership follows the declared virtual extent. Raw size describes
|
||||
// initialized file bytes and can include file-alignment padding; loaders may
|
||||
// map page-rounded tails, but those bytes are outside the declared section.
|
||||
const virtualExtent = (section: Section): number =>
|
||||
section.virtualSize === 0 ? section.size : section.virtualSize;
|
||||
|
||||
/** Header admission and unambiguous file-backed RVA mapping, independent of section names. */
|
||||
export const readPeResourceLayout = (bytes: Buffer) => {
|
||||
requirePeRange(bytes, 0, 64);
|
||||
@@ -70,7 +64,7 @@ export const readPeResourceLayout = (bytes: Buffer) => {
|
||||
offset: bytes.readUInt32LE(at + 20),
|
||||
size: bytes.readUInt32LE(at + 16),
|
||||
};
|
||||
if (section.rva + virtualExtent(section) > 0x100000000)
|
||||
if (section.rva + Math.max(section.virtualSize, section.size) > 0x100000000)
|
||||
peFailure("PE section RVA range overflows.");
|
||||
if (section.size > 0) {
|
||||
requirePeRange(bytes, section.offset, section.size);
|
||||
@@ -96,14 +90,15 @@ export const readPeResourceLayout = (bytes: Buffer) => {
|
||||
if (rva + size > 0x100000000) peFailure("Resource RVA range overflows.");
|
||||
const candidates = sections.filter(
|
||||
(section) =>
|
||||
rva >= section.rva && rva - section.rva < virtualExtent(section),
|
||||
rva >= section.rva &&
|
||||
rva - section.rva < Math.max(section.size, section.virtualSize),
|
||||
);
|
||||
if (rva < headerSize) {
|
||||
if (
|
||||
sections.some(
|
||||
(section) =>
|
||||
section.rva < rva + size &&
|
||||
section.rva + virtualExtent(section) > rva,
|
||||
section.rva + Math.max(section.size, section.virtualSize) > rva,
|
||||
) ||
|
||||
size > headerSize - rva
|
||||
)
|
||||
@@ -116,8 +111,6 @@ export const readPeResourceLayout = (bytes: Buffer) => {
|
||||
const section = candidates[0];
|
||||
if (section === undefined) return peFailure("Missing resource section.");
|
||||
const within = rva - section.rva;
|
||||
if (size > virtualExtent(section) - within)
|
||||
peFailure("Resource range leaves its section's virtual size.");
|
||||
if (within > section.size || size > section.size - within)
|
||||
peFailure("Resource range is not entirely file-backed in one section.");
|
||||
// A second section starting inside the requested interval is ambiguous too.
|
||||
@@ -126,7 +119,7 @@ export const readPeResourceLayout = (bytes: Buffer) => {
|
||||
(other) =>
|
||||
other !== section &&
|
||||
other.rva < rva + size &&
|
||||
other.rva + virtualExtent(other) > rva,
|
||||
other.rva + Math.max(other.size, other.virtualSize) > rva,
|
||||
)
|
||||
)
|
||||
peFailure("Resource interval intersects overlapping virtual sections.");
|
||||
|
||||
@@ -259,62 +259,3 @@ describe("PE header admission and bounded traversal", () => {
|
||||
await expect(parsing).rejects.toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe("PE resource section virtual extent", () => {
|
||||
const sectionSizes = (fixture: ReturnType<typeof peResourceFixture>) => {
|
||||
const virtualSize = fixture.bytes.readUInt32LE(fixture.sectionAt + 8);
|
||||
const rawSize = fixture.bytes.readUInt32LE(fixture.sectionAt + 16);
|
||||
expect(rawSize - virtualSize).toBeGreaterThanOrEqual(8);
|
||||
return { virtualSize, rawSize };
|
||||
};
|
||||
|
||||
it.each([
|
||||
[0, "Resource RVA has missing or overlapping section mappings."],
|
||||
[-2, "Resource range leaves its section's virtual size."],
|
||||
])(
|
||||
"rejects a payload reaching raw alignment padding past VirtualSize (start %i)",
|
||||
async (start, message) => {
|
||||
const fixture = peResourceFixture();
|
||||
const { virtualSize } = sectionSizes(fixture);
|
||||
const rva = 0x2000 + virtualSize + start;
|
||||
fixture.bytes.writeUInt32LE(rva, fixture.dataOffsets[0]);
|
||||
fixture.bytes.writeUInt32LE(4, (fixture.dataOffsets[0] ?? 0) + 4);
|
||||
await expect(parse(fixture.bytes)).rejects.toMatchObject({
|
||||
reason: "format",
|
||||
message,
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("maps raw bytes when VirtualSize is zero, as the image loader does", async () => {
|
||||
const fixture = peResourceFixture();
|
||||
fixture.bytes.writeUInt32LE(0, fixture.sectionAt + 8);
|
||||
expect(await parse(fixture.bytes)).toMatchObject({
|
||||
coverage: { status: "complete", resources: 5 },
|
||||
});
|
||||
});
|
||||
|
||||
it("selects a later section that starts inside earlier raw padding", async () => {
|
||||
const fixture = peResourceFixture();
|
||||
const { virtualSize } = sectionSizes(fixture);
|
||||
const laterRva = 0x2000 + Math.ceil(virtualSize / 4) * 4;
|
||||
const rawOffset = fixture.bytes.length;
|
||||
const payload = Buffer.from("LATE");
|
||||
const bytes = Buffer.concat([fixture.bytes, payload]);
|
||||
bytes.writeUInt16LE(2, 0x86);
|
||||
const later = fixture.sectionAt + 40;
|
||||
bytes.writeUInt32LE(payload.length, later + 8);
|
||||
bytes.writeUInt32LE(laterRva, later + 12);
|
||||
bytes.writeUInt32LE(payload.length, later + 16);
|
||||
bytes.writeUInt32LE(rawOffset, later + 20);
|
||||
bytes.writeUInt32LE(laterRva, fixture.dataOffsets[0]);
|
||||
bytes.writeUInt32LE(payload.length, (fixture.dataOffsets[0] ?? 0) + 4);
|
||||
|
||||
const result = peResourcesSchema.parse(await parse(bytes));
|
||||
expect(result.resources[0]?.payload).toMatchObject({
|
||||
rva: laterRva,
|
||||
location: { offset: rawOffset, bytes: payload.length },
|
||||
sha256: createHash("sha256").update(payload).digest("hex"),
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -44,6 +44,11 @@ export class PeResourcesProvider implements PeResourcesPort {
|
||||
}
|
||||
}
|
||||
|
||||
const errorCode = (cause: unknown): unknown =>
|
||||
cause instanceof Error && "code" in cause ? cause.code : undefined;
|
||||
|
||||
// A missing or non-regular caller selection is an input error, as for the
|
||||
// EVM bytecode carrier read; only other I/O remains an artifact fault.
|
||||
const peResourcesFailure = (
|
||||
cause: unknown,
|
||||
input: InspectPeResourcesInput,
|
||||
@@ -51,56 +56,43 @@ const peResourcesFailure = (
|
||||
) => {
|
||||
const operation = "inspect_pe_resources";
|
||||
if (signal?.aborted) return new AnalysisCancelledError(operation, { cause });
|
||||
|
||||
if (cause instanceof ArtifactReaderFailure) {
|
||||
if (cause.reason === "limit")
|
||||
return new AnalysisResourceConstraintError(
|
||||
operation,
|
||||
"memory",
|
||||
cause.message,
|
||||
{
|
||||
max_file_bytes: input.max_file_bytes,
|
||||
max_entries: input.max_entries,
|
||||
},
|
||||
{ cause },
|
||||
);
|
||||
if (cause.reason === "integrity")
|
||||
return new AnalysisArtifactChangedError(
|
||||
operation,
|
||||
input.path,
|
||||
cause.message,
|
||||
{ cause },
|
||||
);
|
||||
if (cause.reason === "path" || cause.reason === "format")
|
||||
return new AnalysisInputError(operation, { cause }, [
|
||||
{
|
||||
path: ["path"],
|
||||
reason: "invalid_format",
|
||||
message: cause.message,
|
||||
},
|
||||
]);
|
||||
if (cause.reason === "cancelled")
|
||||
return new AnalysisCancelledError(operation, { cause });
|
||||
}
|
||||
|
||||
if (cause instanceof Error && "code" in cause) {
|
||||
if (cause.code === "EACCES" || cause.code === "EPERM")
|
||||
return new AnalysisAccessDeniedError(operation, input.path, cause.code, {
|
||||
cause,
|
||||
});
|
||||
if (cause.code === "ENOENT" || cause.code === "ENOTDIR")
|
||||
return new AnalysisInputError(operation, { cause }, [
|
||||
{
|
||||
path: ["path"],
|
||||
reason: "invalid_value",
|
||||
message: `Selected PE image could not be read (${cause.code}): ${input.path}.`,
|
||||
},
|
||||
]);
|
||||
}
|
||||
|
||||
const reason = cause instanceof ArtifactReaderFailure ? cause.reason : "io";
|
||||
const message = cause instanceof Error ? cause.message : String(cause);
|
||||
if (reason === "limit")
|
||||
return new AnalysisResourceConstraintError(
|
||||
operation,
|
||||
"memory",
|
||||
message,
|
||||
{
|
||||
max_file_bytes: input.max_file_bytes,
|
||||
max_entries: input.max_entries,
|
||||
},
|
||||
{ cause },
|
||||
);
|
||||
if (reason === "path")
|
||||
return new AnalysisInputError(operation, { cause }, [
|
||||
{ path: ["path"], reason: "invalid_format", message },
|
||||
]);
|
||||
if (reason === "integrity")
|
||||
return new AnalysisArtifactChangedError(operation, input.path, message, {
|
||||
cause,
|
||||
});
|
||||
const code = errorCode(cause);
|
||||
if (code === "ENOENT" || code === "ENOTDIR")
|
||||
return new AnalysisInputError(operation, { cause }, [
|
||||
{
|
||||
path: ["path"],
|
||||
reason: "invalid_value",
|
||||
message: `Selected PE image could not be read (${code}): ${input.path}.`,
|
||||
},
|
||||
]);
|
||||
if (code === "EACCES" || code === "EPERM")
|
||||
return new AnalysisAccessDeniedError(operation, input.path, code, {
|
||||
cause,
|
||||
});
|
||||
return new ArtifactOperationError(
|
||||
operation,
|
||||
cause instanceof ArtifactReaderFailure ? cause.reason : "io",
|
||||
reason,
|
||||
undefined,
|
||||
cause instanceof Error ? cause.message : "PE artifact inspection failed.",
|
||||
{ cause },
|
||||
|
||||
@@ -339,61 +339,11 @@ interface OwnedCleanupPlan {
|
||||
readonly unverified: readonly OwnershipSweepUnverifiedProcess[];
|
||||
}
|
||||
|
||||
// A process-table read can race exit and retain the old state with an altered
|
||||
// command. Only a fresh table proving that PID is no longer live can settle
|
||||
// that mismatch; surviving group members still require normal token checks.
|
||||
const settleLauncherExit = async (
|
||||
const createOwnedCleanupPlan = async (
|
||||
ownership: OwnedProcessGroup,
|
||||
processes: readonly ProcessTableEntry[],
|
||||
host: ProcessOwnershipHost,
|
||||
): Promise<
|
||||
| readonly ProcessTableEntry[]
|
||||
| Extract<ProcessCleanupResult, { readonly cleaned: false }>
|
||||
> => {
|
||||
const launcher = processes.find(({ pid }) => pid === ownership.leaderPid);
|
||||
if (launcher === undefined) return processes;
|
||||
const reason = launcherIdentityFailure(launcher, ownership);
|
||||
if (reason === null) return processes;
|
||||
const refreshed = await readLiveProcessTable(host);
|
||||
if (
|
||||
refreshed.available &&
|
||||
!refreshed.processes.some(({ pid }) => pid === launcher.pid)
|
||||
) {
|
||||
// Refreshing must not discard a surviving detached descendant's lineage,
|
||||
// including a child that changed groups while the launcher exited.
|
||||
const descendants = descendantsOf([launcher.pid], processes);
|
||||
const descendantPids = new Set(descendants.map(({ pid }) => pid));
|
||||
const descendantGroupIds = new Set(
|
||||
descendants.map(({ processGroupId }) => processGroupId),
|
||||
);
|
||||
const refreshedDescendantPids = new Set(
|
||||
descendantsOf(
|
||||
descendants.map(({ pid }) => pid),
|
||||
refreshed.processes,
|
||||
).map(({ pid }) => pid),
|
||||
);
|
||||
if (
|
||||
refreshed.processes.some(
|
||||
(process) =>
|
||||
process.processGroupId !== ownership.processGroupId &&
|
||||
(refreshedDescendantPids.has(process.pid) ||
|
||||
descendantPids.has(process.pid) ||
|
||||
descendantGroupIds.has(process.processGroupId)),
|
||||
)
|
||||
)
|
||||
return { cleaned: false, reason };
|
||||
return refreshed.processes;
|
||||
}
|
||||
return { cleaned: false, reason };
|
||||
};
|
||||
|
||||
const createOwnedCleanupPlan = async (
|
||||
ownership: OwnedProcessGroup,
|
||||
initialProcesses: readonly ProcessTableEntry[],
|
||||
host: ProcessOwnershipHost,
|
||||
): Promise<OwnedCleanupPlan | ProcessCleanupResult> => {
|
||||
const processes = await settleLauncherExit(ownership, initialProcesses, host);
|
||||
if ("cleaned" in processes) return processes;
|
||||
const sampledProcessGroupIds = new Set(
|
||||
(ownership.sampledProcessGroupIds ?? []).filter(
|
||||
(processGroupId) =>
|
||||
@@ -447,7 +397,10 @@ const createOwnedCleanupPlan = async (
|
||||
};
|
||||
let descendants: readonly ProcessTableEntry[] = [];
|
||||
if (launcher !== undefined) {
|
||||
descendants = descendantsOf([launcher.pid], processes);
|
||||
const identityFailure = launcherIdentityFailure(launcher, ownership);
|
||||
if (identityFailure !== null)
|
||||
return { cleaned: false, reason: identityFailure };
|
||||
descendants = descendantsOf(launcher.pid, processes);
|
||||
}
|
||||
const descendantPids = new Set(descendants.map(({ pid }) => pid));
|
||||
const processGroupIds = new Set<number>([ownership.processGroupId]);
|
||||
@@ -543,13 +496,7 @@ const revalidateOwnedProcessGroup = async (
|
||||
): Promise<{ readonly empty: boolean } | ProcessCleanupResult> => {
|
||||
let members: readonly ProcessTableEntry[];
|
||||
try {
|
||||
const processes = await settleLauncherExit(
|
||||
ownership,
|
||||
liveProcesses(await host.listProcesses()),
|
||||
host,
|
||||
);
|
||||
if ("cleaned" in processes) return processes;
|
||||
members = processes.filter(
|
||||
members = liveProcesses(await host.listProcesses()).filter(
|
||||
({ processGroupId: observedGroupId }) =>
|
||||
observedGroupId === processGroupId,
|
||||
);
|
||||
@@ -566,6 +513,14 @@ const revalidateOwnedProcessGroup = async (
|
||||
expectedIdentities,
|
||||
);
|
||||
if (failures.length > 0) return cleanupValidationFailure(failures);
|
||||
if (processGroupId === ownership.processGroupId) {
|
||||
const launcher = members.find(({ pid }) => pid === ownership.leaderPid);
|
||||
if (launcher !== undefined) {
|
||||
const identityFailure = launcherIdentityFailure(launcher, ownership);
|
||||
if (identityFailure !== null)
|
||||
return { cleaned: false, reason: identityFailure };
|
||||
}
|
||||
}
|
||||
if (
|
||||
processGroupId !== ownership.processGroupId &&
|
||||
members.length > 0 &&
|
||||
@@ -895,7 +850,7 @@ const scanTokenOwnedProcesses = async (
|
||||
const relatedGroupIds = new Set<number>();
|
||||
if (relation !== undefined) {
|
||||
relatedPids.add(relation.leaderPid);
|
||||
for (const descendant of descendantsOf([relation.leaderPid], processes))
|
||||
for (const descendant of descendantsOf(relation.leaderPid, processes))
|
||||
relatedPids.add(descendant.pid);
|
||||
relatedGroupIds.add(relation.processGroupId);
|
||||
for (const processGroupId of relation.sampledProcessGroupIds ?? [])
|
||||
|
||||
@@ -3,7 +3,6 @@ import {
|
||||
cleanupOwnedProcessGroup,
|
||||
cleanupWindowsProcessTree,
|
||||
type ProcessOwnershipHost,
|
||||
type ProcessTableEntry,
|
||||
type WindowsProcessTreeHost,
|
||||
} from "./ProcessOwnership.js";
|
||||
import {
|
||||
@@ -12,20 +11,18 @@ import {
|
||||
} from "./ProcessOwnershipObservation.js";
|
||||
import { host, ownership } from "./ProcessOwnership.fixture.js";
|
||||
|
||||
const launcher: ProcessTableEntry = {
|
||||
pid: 100,
|
||||
parentPid: 1,
|
||||
processGroupId: 100,
|
||||
state: "S",
|
||||
command: "fixture",
|
||||
};
|
||||
|
||||
describe("owned process-group cleanup validation: ownership and lineage", () => {
|
||||
it("fails closed when a descendant in another process group lacks the token", async () => {
|
||||
const adapter: ProcessOwnershipHost = {
|
||||
listProcesses: () =>
|
||||
Promise.resolve([
|
||||
launcher,
|
||||
{
|
||||
pid: 100,
|
||||
parentPid: 1,
|
||||
processGroupId: 100,
|
||||
state: "S",
|
||||
command: "fixture",
|
||||
},
|
||||
{
|
||||
pid: 101,
|
||||
parentPid: 100,
|
||||
@@ -214,9 +211,16 @@ describe("owned process-group cleanup liveness rechecks", () => {
|
||||
);
|
||||
|
||||
it("accepts a member that exits during ownership revalidation", async () => {
|
||||
const liveLauncher = {
|
||||
pid: 100,
|
||||
parentPid: 1,
|
||||
processGroupId: 100,
|
||||
state: "S",
|
||||
command: "fixture",
|
||||
};
|
||||
const listProcesses = vi
|
||||
.fn<ProcessOwnershipHost["listProcesses"]>()
|
||||
.mockResolvedValueOnce([launcher])
|
||||
.mockResolvedValueOnce([liveLauncher])
|
||||
.mockResolvedValue([]);
|
||||
const signalGroup = vi.fn();
|
||||
const adapter: ProcessOwnershipHost = {
|
||||
@@ -235,181 +239,6 @@ describe("owned process-group cleanup liveness rechecks", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("owned process-group cleanup validation: refreshed descendant lineage", () => {
|
||||
it.each(["planning", "signaling"])(
|
||||
"retains cleanup uncertainty for newly detached descendants during %s",
|
||||
async (phase) => {
|
||||
const child = { ...launcher, pid: 101, parentPid: 100 };
|
||||
const stale = { ...launcher, command: "[MainThread]" };
|
||||
const signalGroup = vi.fn();
|
||||
let reads = 0;
|
||||
const adapter: ProcessOwnershipHost = {
|
||||
listProcesses: () => {
|
||||
reads += 1;
|
||||
if (phase === "signaling" && reads === 1)
|
||||
return Promise.resolve([launcher, child]);
|
||||
if (reads === (phase === "planning" ? 1 : 2))
|
||||
return Promise.resolve([stale, child]);
|
||||
return Promise.resolve([
|
||||
{ ...child, parentPid: 1 },
|
||||
{ ...child, pid: 102, parentPid: 101 },
|
||||
{ ...child, pid: 103, parentPid: 102, processGroupId: 103 },
|
||||
]);
|
||||
},
|
||||
environment: () =>
|
||||
Promise.resolve({ REA_PROCESS_RUN_ID: ownership.runId }),
|
||||
signalGroup,
|
||||
};
|
||||
await expect(
|
||||
cleanupOwnedProcessGroup(
|
||||
{ ...ownership, expectedCommand: "fixture" },
|
||||
adapter,
|
||||
),
|
||||
).resolves.toMatchObject({ cleaned: false });
|
||||
expect(signalGroup).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("owned process-group cleanup validation: launcher exit races", () => {
|
||||
it.each([101, 999])(
|
||||
"retains cleanup uncertainty for a detached descendant in group %s after launcher exit",
|
||||
async (processGroupId) => {
|
||||
const child = {
|
||||
...launcher,
|
||||
pid: 101,
|
||||
parentPid: 100,
|
||||
processGroupId: 101,
|
||||
};
|
||||
const signalGroup = vi.fn();
|
||||
const listProcesses = vi
|
||||
.fn<ProcessOwnershipHost["listProcesses"]>()
|
||||
.mockResolvedValueOnce([
|
||||
{ ...launcher, command: "[MainThread]" },
|
||||
child,
|
||||
])
|
||||
.mockResolvedValue([{ ...child, parentPid: 1, processGroupId }]);
|
||||
await expect(
|
||||
cleanupOwnedProcessGroup(
|
||||
{
|
||||
...ownership,
|
||||
expectedCommand: "fixture",
|
||||
},
|
||||
{
|
||||
listProcesses,
|
||||
environment: () =>
|
||||
Promise.resolve({ REA_PROCESS_RUN_ID: "run-token" }),
|
||||
signalGroup,
|
||||
},
|
||||
),
|
||||
).resolves.toMatchObject({ cleaned: false });
|
||||
expect(signalGroup).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
it.each(["planning", "signaling"])(
|
||||
"settles a stale launcher command during %s without skipping surviving group ownership",
|
||||
async (phase) => {
|
||||
for (const remaining of ["absent", "zombie", "owned", "foreign"]) {
|
||||
const stale = { ...launcher, command: "[MainThread]" };
|
||||
const child = { ...launcher, pid: 101, command: "child" };
|
||||
let reads = 0;
|
||||
const signalGroup = vi.fn();
|
||||
const adapter: ProcessOwnershipHost = {
|
||||
listProcesses: () => {
|
||||
reads += 1;
|
||||
if (phase === "signaling" && reads === 1)
|
||||
return Promise.resolve([launcher]);
|
||||
if (reads === (phase === "planning" ? 1 : 2))
|
||||
return Promise.resolve([stale]);
|
||||
return Promise.resolve(
|
||||
remaining === "absent"
|
||||
? []
|
||||
: remaining === "zombie"
|
||||
? [{ ...stale, state: "Z" }]
|
||||
: [child],
|
||||
);
|
||||
},
|
||||
environment: () =>
|
||||
Promise.resolve({
|
||||
REA_PROCESS_RUN_ID:
|
||||
remaining === "foreign" && reads > 1
|
||||
? "other-run"
|
||||
: "run-token",
|
||||
}),
|
||||
signalGroup,
|
||||
};
|
||||
const result = await cleanupOwnedProcessGroup(
|
||||
{ ...ownership, expectedCommand: "fixture" },
|
||||
adapter,
|
||||
);
|
||||
if (remaining === "foreign")
|
||||
expect(result).toMatchObject({
|
||||
cleaned: false,
|
||||
reason: "process tree contains an unowned or PID-reused process",
|
||||
});
|
||||
else
|
||||
expect(result).toEqual({
|
||||
cleaned: true,
|
||||
signaled: remaining === "owned",
|
||||
});
|
||||
if (remaining === "owned")
|
||||
expect(signalGroup).toHaveBeenCalledWith(100, "SIGKILL");
|
||||
else expect(signalGroup).not.toHaveBeenCalled();
|
||||
}
|
||||
},
|
||||
);
|
||||
it("fails closed when a mismatched launcher's exit cannot be inspected", async () => {
|
||||
const { adapter, signalGroup } = host({
|
||||
100: { REA_PROCESS_RUN_ID: "run-token" },
|
||||
});
|
||||
const listProcesses = vi
|
||||
.fn<ProcessOwnershipHost["listProcesses"]>()
|
||||
.mockResolvedValueOnce(await adapter.listProcesses())
|
||||
.mockRejectedValue(new Error("process table unavailable"));
|
||||
await expect(
|
||||
cleanupOwnedProcessGroup(
|
||||
{ ...ownership, expectedCommand: "/owned/hopper" },
|
||||
{ ...adapter, listProcesses },
|
||||
),
|
||||
).resolves.toMatchObject({
|
||||
cleaned: false,
|
||||
reason: expect.stringContaining("command identity did not match"),
|
||||
});
|
||||
expect(signalGroup).not.toHaveBeenCalled();
|
||||
});
|
||||
it.each(["planning", "signaling"])(
|
||||
"rejects a surviving launcher PID in another group during %s",
|
||||
async (phase) => {
|
||||
let reads = 0;
|
||||
const signalGroup = vi.fn();
|
||||
const adapter: ProcessOwnershipHost = {
|
||||
listProcesses: () => {
|
||||
reads += 1;
|
||||
return Promise.resolve([
|
||||
phase === "signaling" && reads === 1
|
||||
? launcher
|
||||
: reads === (phase === "planning" ? 1 : 2)
|
||||
? { ...launcher, command: "[MainThread]" }
|
||||
: { ...launcher, processGroupId: 999 },
|
||||
]);
|
||||
},
|
||||
environment: () => Promise.resolve({ REA_PROCESS_RUN_ID: "run-token" }),
|
||||
signalGroup,
|
||||
};
|
||||
await expect(
|
||||
cleanupOwnedProcessGroup(
|
||||
{ ...ownership, expectedCommand: "fixture" },
|
||||
adapter,
|
||||
),
|
||||
).resolves.toMatchObject({
|
||||
cleaned: false,
|
||||
reason: expect.stringContaining("command identity did not match"),
|
||||
});
|
||||
expect(signalGroup).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("owned process-group cleanup validation: exited members", () => {
|
||||
it("preserves cancellation during process-table observation instead of reporting unverifiable ownership", async () => {
|
||||
const controller = new AbortController();
|
||||
@@ -464,7 +293,13 @@ describe("owned process-group cleanup validation: exited members", () => {
|
||||
const adapter: ProcessOwnershipHost = {
|
||||
listProcesses: () =>
|
||||
Promise.resolve([
|
||||
launcher,
|
||||
{
|
||||
pid: 100,
|
||||
parentPid: 1,
|
||||
processGroupId: 100,
|
||||
state: "S",
|
||||
command: "fixture",
|
||||
},
|
||||
{
|
||||
pid: 101,
|
||||
parentPid: 100,
|
||||
|
||||
@@ -116,7 +116,7 @@ export const observeOwnedProcessLineage = async (
|
||||
const identityFailure = launcherIdentityFailure(launcher, ownership);
|
||||
if (identityFailure !== null)
|
||||
return unavailableLineage(ownership, identityFailure);
|
||||
const descendants = descendantsOf([launcher.pid], processes);
|
||||
const descendants = descendantsOf(launcher.pid, processes);
|
||||
const verifiedDescendants: ProcessTableEntry[] = [];
|
||||
for await (const { process: member, observation } of readProcessRunTokens(
|
||||
host,
|
||||
|
||||
@@ -6,9 +6,9 @@ export const liveProcesses = (
|
||||
): readonly ProcessTableEntry[] =>
|
||||
members.filter(({ state }) => !state.startsWith("Z"));
|
||||
|
||||
/** Return every live-table descendant of the selected roots in breadth-first order. */
|
||||
/** Return every live-table descendant of a launcher in breadth-first order. */
|
||||
export const descendantsOf = (
|
||||
rootPids: readonly number[],
|
||||
launcherPid: number,
|
||||
processes: readonly ProcessTableEntry[],
|
||||
): ProcessTableEntry[] => {
|
||||
const childrenByParent = new Map<number, ProcessTableEntry[]>();
|
||||
@@ -18,8 +18,8 @@ export const descendantsOf = (
|
||||
childrenByParent.set(process.parentPid, siblings);
|
||||
}
|
||||
const descendants: ProcessTableEntry[] = [];
|
||||
const pending = rootPids.flatMap((pid) => childrenByParent.get(pid) ?? []);
|
||||
const visited = new Set<number>(rootPids);
|
||||
const pending = [...(childrenByParent.get(launcherPid) ?? [])];
|
||||
const visited = new Set<number>([launcherPid]);
|
||||
for (const process of pending) {
|
||||
if (visited.has(process.pid)) continue;
|
||||
visited.add(process.pid);
|
||||
|
||||
@@ -8,7 +8,6 @@ import {
|
||||
import { toolContract } from "../contracts/toolContracts.js";
|
||||
import type { EvidenceBundle } from "../domain/evidenceBundle.js";
|
||||
import { ok } from "../domain/result.js";
|
||||
import { inspectEvidenceBundle } from "../application/investigation/InspectEvidenceBundle.js";
|
||||
import { toolRegistrationOptions } from "./toolRegistrationOptions.js";
|
||||
|
||||
interface EvidenceToolRegistration {
|
||||
@@ -73,9 +72,9 @@ const registerSnapshotEvidenceTool = ({
|
||||
server.registerTool(
|
||||
snapshotContract.name,
|
||||
toolRegistrationOptions(snapshotContract),
|
||||
(input) =>
|
||||
() =>
|
||||
server.delivery.toCallToolResult(
|
||||
inspectEvidenceBundle(bundleForSerialization(session), input),
|
||||
ok(bundleForSerialization(session)),
|
||||
snapshotContract,
|
||||
),
|
||||
);
|
||||
|
||||
@@ -1,120 +0,0 @@
|
||||
import { mkdir, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { expect } from "vitest";
|
||||
import { z } from "zod";
|
||||
|
||||
import { javascriptApplicationAnalysisResultSchema } from "../../../src/domain/javascript/javascriptApplicationAnalysis.js";
|
||||
import { connectLocalToolsMcp } from "../../fixtures/localToolsMcp.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
import { cliTest } from "../../support/cli/cliFixture.js";
|
||||
|
||||
cliTest.for(["CLI", "stdio MCP"] as const)(
|
||||
"keeps dependency manifests out of application roots and Electron entries through %s",
|
||||
async (surface, { cli }) => {
|
||||
const root = await createTestTempDirectory("rea-application-manifest-");
|
||||
const dependency = join(root, "node_modules", "dep");
|
||||
const nested = join(root, "a");
|
||||
await Promise.all([mkdir(dependency, { recursive: true }), mkdir(nested)]);
|
||||
await Promise.all([
|
||||
writeFile(
|
||||
join(nested, "package.json"),
|
||||
JSON.stringify({ name: "nested" }),
|
||||
),
|
||||
writeFile(
|
||||
join(root, "package.json"),
|
||||
JSON.stringify({ name: "app", main: "main.js" }),
|
||||
),
|
||||
writeFile(
|
||||
join(root, "main.js"),
|
||||
"throw new Error('source must remain inert');",
|
||||
),
|
||||
writeFile(
|
||||
join(dependency, "package.json"),
|
||||
JSON.stringify({
|
||||
name: "dep",
|
||||
main: "index.js",
|
||||
browser: "browser.js",
|
||||
}),
|
||||
),
|
||||
writeFile(join(dependency, "index.js"), "module.exports = {};"),
|
||||
writeFile(join(dependency, "browser.js"), "export const browser = true;"),
|
||||
]);
|
||||
|
||||
let document: unknown;
|
||||
if (surface === "CLI") {
|
||||
const response = await cli.run({
|
||||
arguments: ["analyze-javascript-application", root, "--json"],
|
||||
});
|
||||
expect(response.exitCode).toBe(0);
|
||||
document = response.json;
|
||||
} else {
|
||||
const { call } = await connectLocalToolsMcp();
|
||||
const response = await call("analyze_javascript_application", {
|
||||
input_path: root,
|
||||
});
|
||||
expect(response.isError, JSON.stringify(response)).not.toBe(true);
|
||||
document = response.structuredContent;
|
||||
}
|
||||
const { graph } = z
|
||||
.object({ normalized_result: javascriptApplicationAnalysisResultSchema })
|
||||
.parse(document).normalized_result;
|
||||
expect(
|
||||
graph.nodes
|
||||
.filter(({ node_id }) => graph.root_node_ids.includes(node_id))
|
||||
.flatMap(({ observations }) => observations.map(({ label }) => label)),
|
||||
).toEqual(["app"]);
|
||||
expect(
|
||||
graph.nodes
|
||||
.filter(({ kind }) => kind === "electron-main")
|
||||
.flatMap(({ observations }) =>
|
||||
observations.map(({ properties }) => properties.declared_path),
|
||||
),
|
||||
).toEqual(["main.js"]);
|
||||
expect(
|
||||
graph.nodes.filter(({ kind }) => kind === "electron-renderer"),
|
||||
).toEqual([]);
|
||||
expect(
|
||||
graph.nodes
|
||||
.filter(({ kind }) => kind === "package")
|
||||
.flatMap(({ observations }) => observations.map(({ label }) => label))
|
||||
.sort(),
|
||||
).toEqual(["app", "dep", "nested"]);
|
||||
},
|
||||
);
|
||||
|
||||
cliTest(
|
||||
"keeps the artifact root when only dependency manifests are present",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-dependency-only-");
|
||||
const dependency = join(root, "node_modules", "dep");
|
||||
await mkdir(dependency, { recursive: true });
|
||||
await writeFile(
|
||||
join(dependency, "package.json"),
|
||||
JSON.stringify({ name: "dep", main: "index.js" }),
|
||||
);
|
||||
await writeFile(join(dependency, "index.js"), "module.exports = {};");
|
||||
const response = await cli.run({
|
||||
arguments: ["analyze-javascript-application", root, "--json"],
|
||||
});
|
||||
expect(response.exitCode, response.stderr).toBe(0);
|
||||
const { graph } = z
|
||||
.object({ normalized_result: javascriptApplicationAnalysisResultSchema })
|
||||
.parse(response.json).normalized_result;
|
||||
expect(
|
||||
graph.nodes
|
||||
.filter(({ node_id }) => graph.root_node_ids.includes(node_id))
|
||||
.map(({ kind }) => kind),
|
||||
).toEqual(["artifact"]);
|
||||
expect(
|
||||
graph.nodes
|
||||
.filter(({ kind }) => kind === "package")
|
||||
.flatMap(({ observations }) => observations.map(({ label }) => label)),
|
||||
).toEqual(["dep"]);
|
||||
expect(
|
||||
graph.nodes.some(
|
||||
({ kind }) => kind === "electron-main" || kind === "electron-renderer",
|
||||
),
|
||||
).toBe(false);
|
||||
},
|
||||
);
|
||||
@@ -6,7 +6,7 @@ import { promisify } from "node:util";
|
||||
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { afterAll, describe, expect, it } from "vitest";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { z } from "zod";
|
||||
|
||||
import { javascriptApplicationAnalysisResultSchema } from "../../../src/domain/javascript/javascriptApplicationAnalysis.js";
|
||||
@@ -18,27 +18,6 @@ const requireFixture = createRequire(import.meta.url);
|
||||
const comparisonEnvelope = z.object({
|
||||
normalized_result: javaScriptExportShapeComparisonResultSchema,
|
||||
});
|
||||
let mcpComparisonConnection:
|
||||
| {
|
||||
readonly client: Client;
|
||||
readonly transport: StdioClientTransport;
|
||||
readonly ready: Promise<void>;
|
||||
}
|
||||
| undefined;
|
||||
|
||||
afterAll(async () => {
|
||||
if (mcpComparisonConnection === undefined) return;
|
||||
try {
|
||||
await mcpComparisonConnection.ready;
|
||||
} catch {
|
||||
// Keep the failed connection owned until both resources are closed below.
|
||||
}
|
||||
try {
|
||||
await mcpComparisonConnection.client.close();
|
||||
} finally {
|
||||
await mcpComparisonConnection.transport.close();
|
||||
}
|
||||
});
|
||||
|
||||
describe("exported function binding resolution through the CLI", () => {
|
||||
it.each([
|
||||
@@ -310,113 +289,6 @@ describe("CommonJS assignment-chain receiver identity through the CLI", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("CommonJS export identity and scope regressions", () => {
|
||||
it("does not export a named write through the original exports alias after replacement", async () => {
|
||||
const fixture = await analyzeVersions(
|
||||
(count) =>
|
||||
`const VERSION = ${String(count)};\nexports.parse = (module.exports = () => ({ count: VERSION }));`,
|
||||
"parse",
|
||||
"app.cjs",
|
||||
);
|
||||
for (const path of [fixture.leftPath, fixture.rightPath]) {
|
||||
const value = requireFixture(path);
|
||||
expect(typeof value).toBe("function");
|
||||
expect(Reflect.ownKeys(value)).toEqual(["length", "name"]);
|
||||
}
|
||||
const cli = await compareThroughCli(fixture);
|
||||
expect(cli).toMatchObject({
|
||||
left: { status: "missing" },
|
||||
right: { status: "missing" },
|
||||
summary: { added: 0, removed: 0, changed: 0 },
|
||||
});
|
||||
expect(await compareThroughMcp(fixture.input)).toEqual(cli);
|
||||
});
|
||||
|
||||
it("keeps a dead var shadow of the alias-chain RHS unavailable", async () => {
|
||||
const fixture = await analyzeVersions(
|
||||
(count) =>
|
||||
`function parse() { return { count: ${String(count)} }; }\nfunction configure() { if (false) { var parse = () => ({ count: 7 }); } module.exports = exports = parse; }\nconfigure();`,
|
||||
"default",
|
||||
"app.cjs",
|
||||
);
|
||||
expect(requireFixture(fixture.leftPath)).toBeUndefined();
|
||||
expect(requireFixture(fixture.rightPath)).toBeUndefined();
|
||||
const cli = await compareThroughCli(fixture);
|
||||
expectUncertainComparison(cli);
|
||||
expect(await compareThroughMcp(fixture.input)).toEqual(cli);
|
||||
});
|
||||
|
||||
it.each(["+=", "||=", "??="] as const)(
|
||||
"does not treat module.exports %s as a definite function export",
|
||||
async (operator) => {
|
||||
const fixture = await analyzeVersions(
|
||||
(count) =>
|
||||
`module.exports ${operator} (() => ({ kind: "result", count: ${String(count)} }));`,
|
||||
"default",
|
||||
"app.cjs",
|
||||
);
|
||||
const left = requireFixture(fixture.leftPath);
|
||||
const right = requireFixture(fixture.rightPath);
|
||||
if (operator === "+=") {
|
||||
expect(typeof left).toBe("string");
|
||||
expect(typeof right).toBe("string");
|
||||
} else {
|
||||
expect(left).toEqual({});
|
||||
expect(right).toEqual({});
|
||||
}
|
||||
const cli = await compareThroughCli(fixture);
|
||||
expectUncertainComparison(cli);
|
||||
expect(await compareThroughMcp(fixture.input)).toEqual(cli);
|
||||
},
|
||||
);
|
||||
|
||||
it("keeps a parameter-default mutation unknown in a CommonJS return shape", async () => {
|
||||
const fixture = await analyzeVersions(
|
||||
(count) =>
|
||||
`const source = { value: 0 }; module.exports = exports = function current(arg = null, set = (arg = source)) { var arg; arg.value = ${String(count)}; return { kind: "result", count: source.value }; };`,
|
||||
"default",
|
||||
"app.cjs",
|
||||
);
|
||||
expect(requireFixture(fixture.leftPath)()).toEqual({
|
||||
kind: "result",
|
||||
count: 1,
|
||||
});
|
||||
expect(requireFixture(fixture.rightPath)()).toEqual({
|
||||
kind: "result",
|
||||
count: 2,
|
||||
});
|
||||
for (const evidence of [fixture.input.left, fixture.input.right]) {
|
||||
const analysis = z
|
||||
.object({
|
||||
normalized_result: javascriptApplicationAnalysisResultSchema,
|
||||
})
|
||||
.parse(evidence).normalized_result;
|
||||
const shapes = analysis.graph.nodes
|
||||
.flatMap(({ observations }) => observations)
|
||||
.find(
|
||||
({ properties }) =>
|
||||
properties.semantic_role === "export-return-shapes" &&
|
||||
properties.exported_name === "default",
|
||||
)?.properties.static_return_shapes;
|
||||
expect(shapes).toEqual([
|
||||
expect.objectContaining({
|
||||
fields: expect.arrayContaining([
|
||||
expect.objectContaining({ path: "/count", state: "unknown" }),
|
||||
]),
|
||||
}),
|
||||
]);
|
||||
}
|
||||
const cli = await compareThroughCli(fixture);
|
||||
expect(cli).toMatchObject({
|
||||
left: { status: "selected" },
|
||||
right: { status: "selected" },
|
||||
summary: { added: 0, removed: 0, changed: 0 },
|
||||
changes: [],
|
||||
});
|
||||
expect(await compareThroughMcp(fixture.input)).toEqual(cli);
|
||||
});
|
||||
});
|
||||
|
||||
describe("stable exported function bindings through the CLI", () => {
|
||||
it.each([
|
||||
{
|
||||
@@ -926,37 +798,34 @@ const compareThroughCli = async (
|
||||
};
|
||||
|
||||
const compareThroughMcp = async (input: Record<string, unknown>) => {
|
||||
const client = await getMcpComparisonClient();
|
||||
const response = await client.callTool({
|
||||
name: "compare_javascript_export_shapes",
|
||||
arguments: input,
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [resolve("scripts/rea.mjs"), "mcp"],
|
||||
cwd: process.cwd(),
|
||||
env: { PATH: process.env.PATH ?? "", REA_LOG_LEVEL: "silent" },
|
||||
stderr: "pipe",
|
||||
});
|
||||
expect(response.isError).not.toBe(true);
|
||||
await client.ping();
|
||||
return comparisonEnvelope.parse(response.structuredContent).normalized_result;
|
||||
};
|
||||
|
||||
const getMcpComparisonClient = async (): Promise<Client> => {
|
||||
if (mcpComparisonConnection === undefined) {
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [resolve("scripts/rea.mjs"), "mcp"],
|
||||
cwd: process.cwd(),
|
||||
env: { PATH: process.env.PATH ?? "", REA_LOG_LEVEL: "silent" },
|
||||
stderr: "pipe",
|
||||
const client = new Client({
|
||||
name: "exported-function-bindings",
|
||||
version: "1",
|
||||
});
|
||||
try {
|
||||
await client.connect(transport);
|
||||
const response = await client.callTool({
|
||||
name: "compare_javascript_export_shapes",
|
||||
arguments: input,
|
||||
});
|
||||
const client = new Client({
|
||||
name: "exported-function-bindings",
|
||||
version: "1",
|
||||
});
|
||||
mcpComparisonConnection = {
|
||||
client,
|
||||
transport,
|
||||
ready: client.connect(transport),
|
||||
};
|
||||
expect(response.isError).not.toBe(true);
|
||||
await client.ping();
|
||||
return comparisonEnvelope.parse(response.structuredContent)
|
||||
.normalized_result;
|
||||
} finally {
|
||||
try {
|
||||
await client.close();
|
||||
} finally {
|
||||
await transport.close();
|
||||
}
|
||||
}
|
||||
await mcpComparisonConnection.ready;
|
||||
return mcpComparisonConnection.client;
|
||||
};
|
||||
|
||||
const runtimeExport = async (
|
||||
|
||||
@@ -8,87 +8,7 @@ import { toolContract } from "../../../src/contracts/toolContracts.js";
|
||||
import { peResourcesSchema } from "../../../src/domain/native/peResources.js";
|
||||
import { peResourceFixture } from "../../../src/native/pe/PeResources.fixture.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
|
||||
import { cliTest, type TestCli } from "../../support/cli/cliFixture.js";
|
||||
|
||||
const expectPeInputFailureAcrossAdapters = async (input: {
|
||||
readonly cli: TestCli;
|
||||
readonly client: Client;
|
||||
readonly toolName: string;
|
||||
readonly path: string;
|
||||
readonly reason: "invalid_format" | "invalid_value";
|
||||
}) => {
|
||||
const { cli, client, toolName, path, reason } = input;
|
||||
const result = await cli.run({
|
||||
arguments: ["inspect-pe-resources", path, "--json"],
|
||||
});
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.json).toMatchObject({
|
||||
code: "invalid_request",
|
||||
category: "invalid_input",
|
||||
details: {
|
||||
operation: "inspect_pe_resources",
|
||||
issues: [{ path: ["path"], reason }],
|
||||
},
|
||||
});
|
||||
const response = await client.callTool({
|
||||
name: toolName,
|
||||
arguments: { path },
|
||||
});
|
||||
expect(response.isError).toBe(true);
|
||||
expect(parseMcpToolError(response)).toEqual({ error: result.json });
|
||||
};
|
||||
|
||||
const expectPePaddingFailureAcrossAdapters = async (input: {
|
||||
readonly cli: TestCli;
|
||||
readonly client: Client;
|
||||
readonly toolName: string;
|
||||
readonly path: string;
|
||||
}) => {
|
||||
const { cli, client, toolName, path } = input;
|
||||
const fixture = peResourceFixture();
|
||||
const virtualSize = fixture.bytes.readUInt32LE(fixture.sectionAt + 8);
|
||||
fixture.bytes.writeUInt32LE(
|
||||
0x2000 + virtualSize,
|
||||
fixture.dataOffsets[0] ?? 0,
|
||||
);
|
||||
fixture.bytes.writeUInt32LE(4, (fixture.dataOffsets[0] ?? 0) + 4);
|
||||
await writeFile(path, fixture.bytes);
|
||||
const cliResult = await cli.run({
|
||||
arguments: ["inspect-pe-resources", path, "--json"],
|
||||
});
|
||||
expect(cliResult.exitCode).not.toBe(0);
|
||||
const mcpResult = await client.callTool({
|
||||
name: toolName,
|
||||
arguments: { path },
|
||||
});
|
||||
expect(mcpResult.isError).toBe(true);
|
||||
expect(peResourcesSchema.safeParse(mcpResult.structuredContent).success).toBe(
|
||||
false,
|
||||
);
|
||||
};
|
||||
|
||||
const expectPeInputFailuresAcrossAdapters = async (input: {
|
||||
readonly cli: TestCli;
|
||||
readonly client: Client;
|
||||
readonly toolName: string;
|
||||
readonly root: string;
|
||||
}) => {
|
||||
const { cli, client, toolName, root } = input;
|
||||
const malformedPath = join(root, "malformed.exe");
|
||||
await writeFile(malformedPath, Buffer.from("not a PE image"));
|
||||
for (const [path, reason] of [
|
||||
[join(root, "missing.exe"), "invalid_value"],
|
||||
[malformedPath, "invalid_format"],
|
||||
] as const)
|
||||
await expectPeInputFailureAcrossAdapters({
|
||||
cli,
|
||||
client,
|
||||
toolName,
|
||||
path,
|
||||
reason,
|
||||
});
|
||||
};
|
||||
import { cliTest } from "../../support/cli/cliFixture.js";
|
||||
|
||||
cliTest(
|
||||
"inspects PE32/PE32+ resources through compiled CLI and real stdio MCP without an active target",
|
||||
@@ -172,18 +92,6 @@ cliTest(
|
||||
).toBe(evidence.evidence_id);
|
||||
expect(await readFile(path)).toEqual(fixture.bytes);
|
||||
}
|
||||
await expectPePaddingFailureAcrossAdapters({
|
||||
cli,
|
||||
client,
|
||||
toolName: contract.name,
|
||||
path: join(root, "raw-padding.exe"),
|
||||
});
|
||||
await expectPeInputFailuresAcrossAdapters({
|
||||
cli,
|
||||
client,
|
||||
toolName: contract.name,
|
||||
root,
|
||||
});
|
||||
const absent = peResourceFixture();
|
||||
absent.bytes.fill(0, absent.directoryAt, absent.directoryAt + 8);
|
||||
const path = join(root, "absent.exe");
|
||||
|
||||
@@ -30,9 +30,8 @@ const chromeExecutable = [
|
||||
);
|
||||
|
||||
// Issue 1586's page, plus the HTML, image, use, and foreign-origin cases the
|
||||
// real snapshot must distinguish. The XHTML page is the prefixed-svg gap
|
||||
// reported on PR 1602. The tests are skipped when no Chrome-family executable
|
||||
// is installed.
|
||||
// real snapshot must distinguish. The test is skipped when no Chrome-family
|
||||
// executable is installed.
|
||||
const pageHtml = `<!doctype html><title>x</title>
|
||||
<svg width="10" height="10">
|
||||
<a id="s2" xlink:href="/xl-s2h"><text>s2</text></a>
|
||||
@@ -160,161 +159,6 @@ it.skipIf(chromeExecutable === undefined)(
|
||||
},
|
||||
);
|
||||
|
||||
const xhtmlPage = `<?xml version="1.0" encoding="UTF-8"?>
|
||||
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:svg="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
|
||||
<head><title>x</title></head>
|
||||
<body>
|
||||
<svg:svg width="10" height="10">
|
||||
<svg:a id="pa1" xlink:href="/xl-pa1"><svg:text>p</svg:text></svg:a>
|
||||
<svg:image id="img" xlink:href="/xl-image"/>
|
||||
<svg:foreignObject width="10" height="10">
|
||||
<a id="fo-a" xlink:href="/xl-html-fo">no</a>
|
||||
<h:a xmlns:h="http://www.w3.org/1999/xhtml" id="fo-h" xlink:href="/xl-h">no</h:a>
|
||||
<svg:a id="fo-svg" xlink:href="/xl-fo-svg"><svg:text>y</svg:text></svg:a>
|
||||
</svg:foreignObject>
|
||||
</svg:svg>
|
||||
<svg:a id="orphan" xlink:href="/xl-orphan"><svg:text>o</svg:text></svg:a>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" xmlns:x="http://www.w3.org/1999/xlink" width="10" height="10">
|
||||
<a id="custom" x:href="/xl-custom"><text>c</text></a>
|
||||
<a id="both" href="/href-wins" x:href="/xlink-loses"><text>b</text></a>
|
||||
<a id="empty" href="" x:href="/xlink-empty"><text>e</text></a>
|
||||
</svg>
|
||||
<svg:a id="out" xlink:href="https://cdn.example.test/out"><svg:text>out</svg:text></svg:a>
|
||||
<bogus:svg xmlns:bogus="http://example.com/not-svg" id="bogus" xlink:href="/xl-bogus"/>
|
||||
<a id="xhtml-a" xlink:href="/xl-xhtml-a">no</a>
|
||||
</body>
|
||||
</html>
|
||||
`;
|
||||
|
||||
for (const { format, page, contentType, nodeNames, link, destinations } of [
|
||||
{
|
||||
format: "XHTML",
|
||||
page: xhtmlPage,
|
||||
contentType: "application/xhtml+xml",
|
||||
nodeNames: ["html", "svg:a", "bogus:svg"],
|
||||
link: { name: "svg:a", attribute: "xlink:href" },
|
||||
destinations: [
|
||||
"/xl-pa1",
|
||||
"/xl-image",
|
||||
"/xl-fo-svg",
|
||||
"/xl-orphan",
|
||||
"/xl-custom",
|
||||
"/href-wins",
|
||||
"",
|
||||
null,
|
||||
],
|
||||
},
|
||||
{
|
||||
format: "SVG",
|
||||
page: `<svg xmlns="http://www.w3.org/2000/svg" xmlns:p="http://www.w3.org/1999/xlink"><a p:href="/linked"><text>link</text></a></svg>`,
|
||||
contentType: "image/svg+xml",
|
||||
nodeNames: ["svg", "a"],
|
||||
link: { name: "a", attribute: "p:href" },
|
||||
destinations: ["/linked"],
|
||||
},
|
||||
]) {
|
||||
it.skipIf(chromeExecutable === undefined)(
|
||||
`reports namespaced SVG links from a standalone ${format} document through headless Chrome`,
|
||||
async ({ processes }) => {
|
||||
const executable = chromeExecutable;
|
||||
if (executable === undefined) return;
|
||||
const profile = await createTestTempDirectory("rea-svg-xhtml-");
|
||||
const server = createServer((_request, response) => {
|
||||
response.writeHead(200, {
|
||||
"content-type": `${contentType}; charset=utf-8`,
|
||||
});
|
||||
response.end(page);
|
||||
});
|
||||
await new Promise<void>((resolve) => {
|
||||
server.listen(0, "127.0.0.1", resolve);
|
||||
});
|
||||
const address = server.address();
|
||||
if (address === null || typeof address === "string")
|
||||
throw new Error("XHTML fixture server did not bind a TCP port");
|
||||
const origin = `http://127.0.0.1:${String(address.port)}`;
|
||||
const pageUrl = `${origin}/page.xhtml`;
|
||||
let stderr = "";
|
||||
const child = processes.spawn(executable, [
|
||||
"--headless=new",
|
||||
...(process.env.REA_BROWSER_NO_SANDBOX === "true"
|
||||
? ["--no-sandbox"]
|
||||
: []),
|
||||
"--remote-debugging-address=127.0.0.1",
|
||||
"--remote-debugging-port=0",
|
||||
`--user-data-dir=${profile}`,
|
||||
"--no-first-run",
|
||||
"--no-default-browser-check",
|
||||
"--disable-background-networking",
|
||||
"--disable-component-update",
|
||||
"--disable-dev-shm-usage",
|
||||
"--disable-sync",
|
||||
"--metrics-recording-only",
|
||||
pageUrl,
|
||||
]);
|
||||
child.stdout.on("data", () => undefined);
|
||||
child.stderr.on("data", (chunk: Buffer) => {
|
||||
if (stderr.length < 8_192) stderr += chunk.toString("utf8");
|
||||
});
|
||||
try {
|
||||
const port = await waitForBrowserDevtoolsPort({
|
||||
child,
|
||||
executable,
|
||||
activePortPath: join(profile, "DevToolsActivePort"),
|
||||
stderr: () => stderr,
|
||||
timeoutMs: 20_000,
|
||||
});
|
||||
const endpoint = `http://127.0.0.1:${String(port)}`;
|
||||
const provider = new CdpBrowserProvider();
|
||||
const targetId = await pageTarget(provider, endpoint, origin, pageUrl);
|
||||
const result = await provider.inspectPage(
|
||||
inspectWebPageInputSchema.parse({
|
||||
cdp_endpoint: endpoint,
|
||||
allowed_origins: [origin],
|
||||
target_id: targetId,
|
||||
observation_ms: 0,
|
||||
}),
|
||||
);
|
||||
if (!result.ok) throw result.error;
|
||||
const urls = result.value.metadata.dom_urls;
|
||||
for (const name of nodeNames) {
|
||||
expect(
|
||||
result.value.dom.nodes.some((node) => node.node_name === name),
|
||||
).toBe(true);
|
||||
}
|
||||
expect(
|
||||
result.value.dom.nodes.some(
|
||||
(node) =>
|
||||
node.node_name === link.name &&
|
||||
node.attribute_names.includes(link.attribute),
|
||||
),
|
||||
).toBe(true);
|
||||
expect(urls).toEqual(
|
||||
destinations.map((destination) =>
|
||||
expect.objectContaining({
|
||||
attribute: "href",
|
||||
url:
|
||||
destination === null
|
||||
? null
|
||||
: destination === ""
|
||||
? pageUrl
|
||||
: `${origin}${destination}`,
|
||||
destination_scope:
|
||||
destination === null ? "outside_policy" : "approved",
|
||||
}),
|
||||
),
|
||||
);
|
||||
} finally {
|
||||
child.kill("SIGTERM");
|
||||
if (!(await waitForExit(child, 2_000))) child.kill("SIGKILL");
|
||||
await waitForExit(child, 2_000);
|
||||
await new Promise<void>((resolve) => {
|
||||
server.close(() => resolve());
|
||||
});
|
||||
}
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
const pageTarget = async (
|
||||
provider: CdpBrowserProvider,
|
||||
endpoint: string,
|
||||
|
||||
@@ -2,10 +2,8 @@ import { PrivateRuntimeRoot } from "../../../src/process/PrivateRuntimeRoot.js";
|
||||
import {
|
||||
access,
|
||||
chmod,
|
||||
mkdir,
|
||||
mkdtemp,
|
||||
rm,
|
||||
symlink,
|
||||
truncate,
|
||||
writeFile,
|
||||
} from "node:fs/promises";
|
||||
@@ -13,7 +11,6 @@ import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { expect, it, onTestFinished } from "vitest";
|
||||
import { HistoricalCaptureDecoder } from "../../../src/browser/history/HistoricalCaptureDecoder.js";
|
||||
import { MitmproxyCaptureAdapter } from "../../../src/browser/history/MitmproxyCaptureAdapter.js";
|
||||
import { HAR_CAPTURE_PROVIDER_IDENTITY } from "../../../src/browser/history/CaptureRelease.js";
|
||||
import {
|
||||
inspectWebNetworkCaptureInputSchema,
|
||||
@@ -23,41 +20,6 @@ import { historicalHar } from "../../fixtures/historicalHar.js";
|
||||
import { decodeHarCapture } from "../../../src/browser/history/HarCapture.js";
|
||||
import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js";
|
||||
|
||||
const linux = it.skipIf(process.platform !== "linux");
|
||||
|
||||
linux("refuses a directory configured as the mitmdump executable", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "rea-mitmdump-command-"));
|
||||
onTestFinished(() => rm(root, { recursive: true, force: true }));
|
||||
const command = join(root, "mitmdump");
|
||||
await mkdir(command);
|
||||
|
||||
await expect(
|
||||
new MitmproxyCaptureAdapter({ REA_MITMDUMP_COMMAND: command }).command(
|
||||
join(root, "request.json"),
|
||||
join(root, "runtime"),
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
_tag: "AnalysisCapabilityUnavailableError",
|
||||
reason: expect.stringContaining(command),
|
||||
});
|
||||
});
|
||||
|
||||
linux("allows a symlink to a regular mitmdump executable", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "rea-mitmdump-command-symlink-"));
|
||||
onTestFinished(() => rm(root, { recursive: true, force: true }));
|
||||
const command = join(root, "mitmdump");
|
||||
const target = join(root, "mitmdump-bin");
|
||||
await writeFile(target, "#!/bin/sh\n", { mode: 0o700 });
|
||||
await symlink(target, command);
|
||||
|
||||
await expect(
|
||||
new MitmproxyCaptureAdapter({ REA_MITMDUMP_COMMAND: command }).command(
|
||||
join(root, "request.json"),
|
||||
join(root, "runtime"),
|
||||
),
|
||||
).resolves.toMatchObject({ command });
|
||||
});
|
||||
|
||||
it.each([true, false])(
|
||||
"binds decoder replies to the complete selected provider identity (matching name: %s)",
|
||||
async (matching) => {
|
||||
|
||||
@@ -9,53 +9,6 @@ import { cliTest } from "../../support/cli/cliFixture.js";
|
||||
import { analysisViewLayoutEvidence } from "../../fixtures/analysisView.js";
|
||||
import { createEvidence } from "../../../src/domain/evidence.js";
|
||||
import { ghidraFunctionDossier } from "../../../src/domain/ghidraValues.fixture.js";
|
||||
import { createEvidenceBundle } from "../../../src/domain/evidenceBundle.js";
|
||||
import { inspectEvidenceBundle } from "../../../src/application/investigation/InspectEvidenceBundle.js";
|
||||
|
||||
cliTest(
|
||||
"discovers native records from a complete portable bundle using the shared workflow",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-evidence-discovery-cli-");
|
||||
const parent = createEvidence(
|
||||
{ path: "/fixtures/native.exe", format: "pe", sha256: "a".repeat(64) },
|
||||
{ id: "ghidra", name: "Ghidra", version: "12.1.4" },
|
||||
{
|
||||
operation: "analyze_function",
|
||||
parameters: {},
|
||||
result: ghidraFunctionDossier(),
|
||||
},
|
||||
);
|
||||
const bundle = createEvidenceBundle([parent]);
|
||||
const path = join(root, "evidence.json");
|
||||
await writeFile(path, JSON.stringify(bundle));
|
||||
const options = {
|
||||
detail: "summary" as const,
|
||||
filters: { evidence_id: parent.evidence_id },
|
||||
};
|
||||
const result = await cli.run({
|
||||
arguments: [
|
||||
"inspect-evidence-bundle",
|
||||
JSON.stringify({ path, ...options }),
|
||||
"--json",
|
||||
],
|
||||
environment: { HOME: root, XDG_CONFIG_HOME: root, XDG_CACHE_HOME: root },
|
||||
});
|
||||
expect(result.exitCode).toBe(0);
|
||||
const expected = inspectEvidenceBundle(bundle, options);
|
||||
if (!expected.ok) throw expected.error;
|
||||
expect(result.json).toEqual(expected.value);
|
||||
const complete = await cli.run({
|
||||
arguments: [
|
||||
"inspect-evidence-bundle",
|
||||
JSON.stringify({ path }),
|
||||
"--json",
|
||||
],
|
||||
environment: { HOME: root, XDG_CONFIG_HOME: root, XDG_CACHE_HOME: root },
|
||||
});
|
||||
expect(complete.exitCode).toBe(0);
|
||||
expect(complete.json).toEqual(bundle);
|
||||
},
|
||||
);
|
||||
cliTest(
|
||||
"projects one section from a JSON file of inline layout Evidence",
|
||||
async ({ cli }) => {
|
||||
|
||||
@@ -97,14 +97,6 @@ describe("browser comparison JSON file inputs", () => {
|
||||
INTEGRATION_TEST_TIMEOUT_MS,
|
||||
);
|
||||
|
||||
it(
|
||||
"reports malformed PNG chunks as invalid input and valid unsupported PNGs as unsupported targets",
|
||||
async () => {
|
||||
await assertPngInputErrorProjections();
|
||||
},
|
||||
INTEGRATION_TEST_TIMEOUT_MS,
|
||||
);
|
||||
|
||||
it(
|
||||
"identifies a missing capture file instead of reporting malformed JSON",
|
||||
async () => {
|
||||
@@ -131,105 +123,3 @@ describe("browser comparison JSON file inputs", () => {
|
||||
INTEGRATION_TEST_TIMEOUT_MS,
|
||||
);
|
||||
});
|
||||
|
||||
const assertPngInputErrorProjections = async (): Promise<void> => {
|
||||
const damagedBytes = noisePng(1, 1);
|
||||
damagedBytes[29] = (damagedBytes[29] ?? 0) ^ 1;
|
||||
const damaged = JSON.stringify(createWebScreenshotArtifact(damagedBytes));
|
||||
expect(
|
||||
await runCli(["compare-web-screenshots", damaged, damaged, "--json"]),
|
||||
).toMatchObject({
|
||||
code: "invalid_request",
|
||||
details: {
|
||||
operation: "compare_web_screenshots",
|
||||
issues: [
|
||||
{ path: [], reason: "invalid_format", message: expect.any(String) },
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const header = Buffer.alloc(13);
|
||||
header.writeUInt32BE(1, 0);
|
||||
header.writeUInt32BE(1, 4);
|
||||
header.set([8, 2, 0, 0, 0], 8);
|
||||
const malformedDeflate = JSON.stringify(
|
||||
createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
|
||||
pngChunk("IHDR", header),
|
||||
pngChunk("IDAT", Buffer.from("bad")),
|
||||
pngChunk("IEND", Buffer.alloc(0)),
|
||||
]),
|
||||
),
|
||||
);
|
||||
expect(
|
||||
await runCli([
|
||||
"compare-web-screenshots",
|
||||
malformedDeflate,
|
||||
malformedDeflate,
|
||||
"--json",
|
||||
]),
|
||||
).toMatchObject({
|
||||
code: "invalid_request",
|
||||
details: {
|
||||
operation: "compare_web_screenshots",
|
||||
issues: [{ path: [], reason: "invalid_format" }],
|
||||
},
|
||||
});
|
||||
|
||||
const grayscaleHeader = Buffer.alloc(13);
|
||||
grayscaleHeader.writeUInt32BE(1, 0);
|
||||
grayscaleHeader.writeUInt32BE(1, 4);
|
||||
grayscaleHeader.set([8, 0, 0, 0, 0], 8);
|
||||
const grayscale = JSON.stringify(
|
||||
createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
|
||||
pngChunk("IHDR", grayscaleHeader),
|
||||
pngChunk("IDAT", deflateSync(Buffer.from([0, 30]))),
|
||||
pngChunk("IEND", Buffer.alloc(0)),
|
||||
]),
|
||||
),
|
||||
);
|
||||
expect(
|
||||
await runCli(["compare-web-screenshots", grayscale, grayscale, "--json"]),
|
||||
).toMatchObject({
|
||||
code: "unsupported_target",
|
||||
details: { operation: "compare_web_screenshots" },
|
||||
});
|
||||
|
||||
const oversizedHeader = Buffer.alloc(13);
|
||||
oversizedHeader.writeUInt32BE(0x7fffffff, 0);
|
||||
oversizedHeader.writeUInt32BE(1, 4);
|
||||
oversizedHeader.set([8, 6, 0, 0, 0], 8);
|
||||
const hugeDimensions = JSON.stringify(
|
||||
createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
|
||||
pngChunk("IHDR", oversizedHeader),
|
||||
pngChunk("IDAT", deflateSync(Buffer.from([0]))),
|
||||
pngChunk("IEND", Buffer.alloc(0)),
|
||||
]),
|
||||
),
|
||||
);
|
||||
expect(
|
||||
await runCli([
|
||||
"compare-web-screenshots",
|
||||
hugeDimensions,
|
||||
hugeDimensions,
|
||||
"--json",
|
||||
]),
|
||||
).toMatchObject({
|
||||
code: "resource_constraint",
|
||||
details: {
|
||||
operation: "compare_web_screenshots",
|
||||
resource: "memory",
|
||||
reported_limits: {
|
||||
maximum_working_memory_bytes: 256 * 1024 * 1024,
|
||||
estimated_working_memory_bytes: "34359738499",
|
||||
before_dimensions: { width: 0x7fffffff, height: 1 },
|
||||
after_dimensions: { width: 0x7fffffff, height: 1 },
|
||||
},
|
||||
},
|
||||
});
|
||||
};
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { mkdir, writeFile } from "node:fs/promises";
|
||||
import { mkdir } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
@@ -9,34 +9,20 @@ import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
|
||||
const execute = promisify(execFile);
|
||||
const MARKER = "REA_TEST_RESOLVED ";
|
||||
// Dependencies and composition modules used by one command family; startup
|
||||
// must not resolve them just to register the complete command catalog.
|
||||
const DEFERRED_IMPORTS = [
|
||||
{
|
||||
name: "playwright-core",
|
||||
path: "/node_modules/playwright-core/",
|
||||
},
|
||||
{
|
||||
name: "isomorphic-git",
|
||||
path: "/node_modules/isomorphic-git/",
|
||||
},
|
||||
{
|
||||
name: "binary-composition",
|
||||
path: "/dist/composition/binary.js",
|
||||
},
|
||||
] as const;
|
||||
// Dependencies that only one command uses; startup must not resolve them.
|
||||
const DEFERRED_PACKAGES = ["playwright-core", "isomorphic-git"] as const;
|
||||
// Report each resolved deferred package on stderr without changing resolution.
|
||||
const RESOLUTION_HOOK = `data:text/javascript,${encodeURIComponent(`
|
||||
import { registerHooks } from "node:module";
|
||||
const imports = ${JSON.stringify(DEFERRED_IMPORTS)};
|
||||
const packages = ${JSON.stringify(DEFERRED_PACKAGES)};
|
||||
registerHooks({
|
||||
resolve(specifier, context, next) {
|
||||
const resolved = next(specifier, context);
|
||||
const name = imports.find((item) =>
|
||||
resolved.url.includes(item.path),
|
||||
const name = packages.find((item) =>
|
||||
resolved.url.includes("/node_modules/" + item + "/"),
|
||||
);
|
||||
if (name !== undefined)
|
||||
process.stderr.write(${JSON.stringify(MARKER)} + name.name + "\\n");
|
||||
process.stderr.write(${JSON.stringify(MARKER)} + name + "\\n");
|
||||
return resolved;
|
||||
},
|
||||
});
|
||||
@@ -82,45 +68,9 @@ describe("CLI startup imports", () => {
|
||||
).resolves.toContain("isomorphic-git");
|
||||
});
|
||||
|
||||
it("does not load command-specific code to register the command catalog", async () => {
|
||||
it("does not load command-specific packages to register commands", async () => {
|
||||
await expect(
|
||||
resolvedPackages(["scripts/rea.mjs", "--help"]),
|
||||
).resolves.toEqual(new Set());
|
||||
});
|
||||
|
||||
it("does not load native provider composition for JavaScript application analysis", async () => {
|
||||
const source = await createTestTempDirectory("rea-cli-js-startup-");
|
||||
await mkdir(join(source, "src"));
|
||||
await writeFile(
|
||||
join(source, "package.json"),
|
||||
JSON.stringify({
|
||||
name: "startup-fixture",
|
||||
version: "1.0.0",
|
||||
main: "src/index.js",
|
||||
}),
|
||||
);
|
||||
await writeFile(
|
||||
join(source, "src/index.js"),
|
||||
"export const answer = 42;\n",
|
||||
);
|
||||
|
||||
await expect(
|
||||
resolvedPackages([
|
||||
"scripts/rea.mjs",
|
||||
"analyze-javascript-application",
|
||||
source,
|
||||
"--json",
|
||||
]),
|
||||
).resolves.toEqual(new Set());
|
||||
});
|
||||
|
||||
it("loads native provider composition when a provider operation is called", async () => {
|
||||
await expect(
|
||||
resolvedPackages(
|
||||
evaluate(
|
||||
'const { createDirectAnalysis } = await import("./dist/composition/directAnalysis.js"); await createDirectAnalysis({}).runSessionStatus();',
|
||||
),
|
||||
),
|
||||
).resolves.toContain("binary-composition");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -80,11 +80,7 @@ describe("compiled reference-source import preflight failures", () => {
|
||||
const directory = await createTestTempDirectory(
|
||||
"rea-reference-cli-status-",
|
||||
);
|
||||
// Exercise JSON escaping on POSIX too; Windows paths already need it.
|
||||
const root = join(
|
||||
directory,
|
||||
process.platform === "win32" ? "input" : "input\\escaped",
|
||||
);
|
||||
const root = join(directory, "input");
|
||||
if (scenario.rootKind === "regular-file")
|
||||
await writeFile(root, "inert input\n");
|
||||
const result = await cli.run({
|
||||
|
||||
@@ -10,7 +10,6 @@ import {
|
||||
import { join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
import { buildBinary } from "plist";
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { parseBinaryTarget } from "../../../src/application/BinaryTargetResolver.js";
|
||||
@@ -692,71 +691,6 @@ describe("iOS-style app bundle targets", () => {
|
||||
);
|
||||
});
|
||||
|
||||
describe("binary Info.plist app bundle targets", () => {
|
||||
it.each([
|
||||
["Mac.app", ["Contents", "MacOS"], ["Contents", "Info.plist"]],
|
||||
["Phone.app", [], ["Info.plist"]],
|
||||
])(
|
||||
"resolves %s from a binary Info.plist on every host",
|
||||
async (bundle, programs, plist) => {
|
||||
const directory = await createTestTempDirectory("rea-app-bplist-");
|
||||
// Windows file names cannot hold a tab or end in a space.
|
||||
const names =
|
||||
process.platform === "win32"
|
||||
? ["Ordinary"]
|
||||
: ["Ordinary", " App ", "Tab\t"];
|
||||
for (const name of names) {
|
||||
const app = join(directory, name, bundle);
|
||||
const executable = join(app, ...programs, name);
|
||||
await mkdir(join(app, ...programs), { recursive: true });
|
||||
await mkdir(join(app, ...plist.slice(0, -1)), { recursive: true });
|
||||
await writeFile(
|
||||
join(app, ...plist),
|
||||
buildBinary({ CFBundleExecutable: name, CFBundleVersion: "1" }),
|
||||
);
|
||||
await writeFile(executable, thinMach(0xfeedfacf, 0x0100000c));
|
||||
const result = await parseBinaryTarget(app, {
|
||||
cwd: directory,
|
||||
hostArchitecture: "arm64",
|
||||
});
|
||||
expect(result.ok && result.value).toMatchObject({
|
||||
path: await realpath(executable),
|
||||
format: "mach-o",
|
||||
});
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it.each([
|
||||
[
|
||||
"XML",
|
||||
"<plist><dict><key>__proto__</key><dict><key>CFBundleExecutable</key><string>Forged</string></dict></dict></plist>",
|
||||
],
|
||||
[
|
||||
"binary",
|
||||
buildBinary(JSON.parse('{"__proto__":{"CFBundleExecutable":"Forged"}}')),
|
||||
],
|
||||
])(
|
||||
"ignores CFBundleExecutable nested under __proto__ in %s Info.plist files",
|
||||
async (_encoding, contents) => {
|
||||
const directory = await createTestTempDirectory("rea-app-proto-only-");
|
||||
const app = join(directory, "Proto.app");
|
||||
await mkdir(join(app, "Contents", "MacOS"), { recursive: true });
|
||||
await writeFile(join(app, "Contents", "Info.plist"), contents);
|
||||
await writeFile(
|
||||
join(app, "Contents", "MacOS", "Forged"),
|
||||
thinMach(0xfeedfacf, 0x0100000c),
|
||||
);
|
||||
expect(await resolveAppBundleExecutable(app)).toMatchObject({
|
||||
ok: false,
|
||||
error: {
|
||||
reason: expect.stringContaining("CFBundleExecutable is missing"),
|
||||
},
|
||||
});
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("DOS binary target I/O", () => {
|
||||
it("validates a complete DOS file beyond the initial metadata probe", async () => {
|
||||
const directory = await createTestTempDirectory("rea-dos-target-");
|
||||
|
||||
@@ -6,19 +6,16 @@ import { peResourceFixture } from "../../../src/native/pe/PeResources.fixture.js
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
|
||||
describe("PE resource stable filesystem boundary", () => {
|
||||
it("reports non-file and malformed PE selections as caller input errors", async () => {
|
||||
it("rejects symlinks, missing files and directories as caller input without inventing empty inventory", async () => {
|
||||
const root = await createTestTempDirectory("rea-pe-resource-files-");
|
||||
const path = join(root, "image.exe");
|
||||
const alias = join(root, "alias.exe");
|
||||
const malformed = join(root, "malformed.exe");
|
||||
await writeFile(path, peResourceFixture().bytes);
|
||||
await writeFile(malformed, Buffer.from("not a PE image"));
|
||||
await symlink(path, alias);
|
||||
const service = createPeResourcesService();
|
||||
for (const [candidate, reason] of [
|
||||
[root, "invalid_format"],
|
||||
[alias, "invalid_format"],
|
||||
[malformed, "invalid_format"],
|
||||
[join(root, "missing.exe"), "invalid_value"],
|
||||
[join(path, "child.exe"), "invalid_value"],
|
||||
] as const) {
|
||||
@@ -31,7 +28,7 @@ describe("PE resource stable filesystem boundary", () => {
|
||||
{
|
||||
path: ["path"],
|
||||
reason,
|
||||
message: expect.any(String),
|
||||
message: expect.stringContaining(candidate),
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
@@ -83,7 +83,6 @@ async function fixture() {
|
||||
author("platformbot", "Bot"),
|
||||
author("claude"),
|
||||
author("cursoragent"),
|
||||
author("morluto"),
|
||||
]),
|
||||
associatedPullRequests: page([]),
|
||||
},
|
||||
@@ -187,7 +186,7 @@ it("inventories side branches and paginated credits, then checks and renders syn
|
||||
);
|
||||
expect(f.record).toContain("Thanks @dana.");
|
||||
expect(f.record).not.toMatch(
|
||||
/@(?:merger|automation|platformbot|claude|cursoragent|morluto)/u,
|
||||
/@(?:merger|automation|platformbot|claude|cursoragent)/u,
|
||||
);
|
||||
const body = join(f.directory, "body.md");
|
||||
await writeFile(
|
||||
@@ -212,37 +211,6 @@ it("inventories side branches and paginated credits, then checks and renders syn
|
||||
await expect(
|
||||
f.invoke(["render", "--manifest", f.manifest, "--output", output]),
|
||||
).rejects.toMatchObject({ stderr: expect.stringContaining("EEXIST") });
|
||||
const { stdout: summary } = await f.invoke([
|
||||
"record",
|
||||
"--manifest",
|
||||
f.manifest,
|
||||
"--summary",
|
||||
]);
|
||||
expect(summary).toContain("3 commits and 1 merged PR]");
|
||||
expect(summary).toContain(
|
||||
"/blob/rea-agents-6.4.0/docs/releases/6.4.0.contributions.json",
|
||||
);
|
||||
expect(summary).not.toContain("Thanks");
|
||||
const conciseNotes = f.notes
|
||||
.replace(f.record, summary)
|
||||
.replace(
|
||||
"### Changes\n",
|
||||
"### Changes\n\n- Capability details (#101).\n- Related migration (#51).\n",
|
||||
);
|
||||
await writeFile(join(f.directory, "CHANGELOG.md"), conciseNotes);
|
||||
await f.invoke(["check", "--manifest", f.manifest]);
|
||||
await writeFile(
|
||||
join(f.directory, "CHANGELOG.md"),
|
||||
conciseNotes.replace(
|
||||
"6.4.0.contributions.json",
|
||||
"6.3.0.contributions.json",
|
||||
),
|
||||
);
|
||||
await expect(
|
||||
f.invoke(["check", "--manifest", f.manifest]),
|
||||
).rejects.toMatchObject({
|
||||
stderr: expect.stringContaining("Contribution record differs"),
|
||||
});
|
||||
await expect(
|
||||
f.invoke([
|
||||
"inventory",
|
||||
@@ -270,10 +238,6 @@ it("rejects lost grouped credits, fabricated references, bots, and stale PR bodi
|
||||
throw new Error("Fixture is missing its direct contribution");
|
||||
for (const [replacement, error] of [
|
||||
[f.bullet.replace(", @bob", ""), "Missing Thanks @bob"],
|
||||
[
|
||||
f.bullet.replace(", @bob", "") + "\n- Thanks @bob for other work.",
|
||||
"Missing Thanks @bob",
|
||||
],
|
||||
[f.bullet + " (#999)", "Unknown release reference #999"],
|
||||
[f.bullet + "a".repeat(65_536), "inline PR body limit"],
|
||||
[f.bullet + "汉".repeat(43_000), "GitHub's body budget"],
|
||||
@@ -281,6 +245,10 @@ it("rejects lost grouped credits, fabricated references, bots, and stale PR bodi
|
||||
f.bullet + `\n- Fixed direct behavior (${directLink}).`,
|
||||
"Missing Thanks @dana",
|
||||
],
|
||||
[
|
||||
f.bullet + "\n- Fixed the reported problem (#51).",
|
||||
"Missing Thanks @reporter",
|
||||
],
|
||||
[
|
||||
f.bullet + " Thanks @automation[bot].",
|
||||
"Unverified human credit @automation[bot]",
|
||||
|
||||
@@ -1,47 +0,0 @@
|
||||
import { mkdir, symlink, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { expect, it } from "vitest";
|
||||
import { resolveWakaruCommand } from "../../../src/javascript/recovery/WakaruCommand.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
|
||||
const linuxX64 = it.skipIf(
|
||||
process.platform !== "linux" || process.arch !== "x64",
|
||||
);
|
||||
|
||||
linuxX64(
|
||||
"refuses a directory configured as the Wakaru resource limiter",
|
||||
async () => {
|
||||
const root = await createTestTempDirectory("rea-wakaru-limiter-");
|
||||
const command = join(root, "wakaru");
|
||||
const limiter = join(root, "prlimit");
|
||||
await writeFile(command, "#!/bin/sh\n", { mode: 0o700 });
|
||||
await mkdir(limiter);
|
||||
|
||||
await expect(
|
||||
resolveWakaruCommand({
|
||||
REA_WAKARU_COMMAND: command,
|
||||
REA_JAVASCRIPT_PRLIMIT_COMMAND: limiter,
|
||||
}),
|
||||
).rejects.toMatchObject({
|
||||
_tag: "AnalysisCapabilityUnavailableError",
|
||||
reason: expect.stringContaining(limiter),
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
linuxX64("allows a symlink to a regular Wakaru resource limiter", async () => {
|
||||
const root = await createTestTempDirectory("rea-wakaru-limiter-symlink-");
|
||||
const command = join(root, "wakaru");
|
||||
const limiter = join(root, "prlimit");
|
||||
const target = join(root, "prlimit-bin");
|
||||
await writeFile(command, "#!/bin/sh\n", { mode: 0o700 });
|
||||
await writeFile(target, "#!/bin/sh\n", { mode: 0o700 });
|
||||
await symlink(target, limiter);
|
||||
|
||||
await expect(
|
||||
resolveWakaruCommand({
|
||||
REA_WAKARU_COMMAND: command,
|
||||
REA_JAVASCRIPT_PRLIMIT_COMMAND: limiter,
|
||||
}),
|
||||
).resolves.toMatchObject({ limiter });
|
||||
});
|
||||
@@ -7,8 +7,6 @@ import { z } from "zod";
|
||||
import { BinaryLayoutService } from "../../../src/application/binaryDiagnostics/BinaryLayoutService.js";
|
||||
import { toolContract } from "../../../src/contracts/toolContracts.js";
|
||||
import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js";
|
||||
import { createAnalysisProfile } from "../../../src/domain/analysisProfile.js";
|
||||
import { jsonObjectSchema } from "../../../src/domain/jsonValue.js";
|
||||
import { functionDossierSchema } from "../../../src/domain/hopperValues.js";
|
||||
import { ghidraFunctionDossier } from "../../../src/domain/ghidraValues.fixture.js";
|
||||
import { ok } from "../../../src/domain/result.js";
|
||||
@@ -48,115 +46,6 @@ const connect = async (binaryLayout?: BinaryLayoutService) => {
|
||||
return { client, session };
|
||||
};
|
||||
|
||||
it("discovers an oversized native dossier by exact metadata and reads its retained view", async () => {
|
||||
const { client, session } = await connect();
|
||||
const provider = { id: "ghidra", name: "Ghidra", version: "12.1.4" };
|
||||
const profile = createAnalysisProfile(provider, { language: "fixture" });
|
||||
const parent = createEvidence(
|
||||
{ path: "/fixtures/large.exe", format: "pe", sha256: "a".repeat(64) },
|
||||
provider,
|
||||
{
|
||||
operation: "analyze_function",
|
||||
parameters: { procedure: "0x401000" },
|
||||
analysisProfile: profile,
|
||||
result: {
|
||||
...jsonObjectSchema.parse(ghidraFunctionDossier()),
|
||||
pseudocode: "x".repeat(6 * 1024 * 1024),
|
||||
},
|
||||
rawResult: null,
|
||||
},
|
||||
);
|
||||
const other = createEvidence(
|
||||
{ path: "/fixtures/other.exe", format: "pe", sha256: "b".repeat(64) },
|
||||
provider,
|
||||
{
|
||||
operation: "analyze_function",
|
||||
parameters: {},
|
||||
result: ghidraFunctionDossier(),
|
||||
},
|
||||
);
|
||||
for (const record of [parent, other]) {
|
||||
const retained = session.recordEvidence(record);
|
||||
if (!retained.ok) throw retained.error;
|
||||
}
|
||||
const broad = await client.callTool({
|
||||
name: "get_evidence_bundle",
|
||||
arguments: {},
|
||||
});
|
||||
const broadError = parseMcpToolError(broad).error;
|
||||
expect(broadError.code).toBe("resource_constraint");
|
||||
expect(broadError.remediation?.action).toContain(
|
||||
"get_evidence_bundle and detail: summary",
|
||||
);
|
||||
const response = await client.callTool({
|
||||
name: "get_evidence_bundle",
|
||||
arguments: {
|
||||
detail: "summary",
|
||||
filters: {
|
||||
operation: "analyze_function",
|
||||
target_sha256: "a".repeat(64),
|
||||
analysis_profile_digest: profile.digest,
|
||||
procedure_address: "0x401000",
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(response.isError).not.toBe(true);
|
||||
const result = toolContract("get_evidence_bundle").outputSchema.parse(
|
||||
response.structuredContent,
|
||||
).result;
|
||||
expect(result).toMatchObject({
|
||||
kind: "evidence-bundle-summary",
|
||||
total_retained_records: 2,
|
||||
matching_records: 1,
|
||||
records: [
|
||||
{
|
||||
evidence_id: parent.evidence_id,
|
||||
analysis_profile_digest: profile.digest,
|
||||
retention: "complete-record",
|
||||
native_dossier: { available: true, procedure_address: "0x401000" },
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(JSON.stringify(response)).not.toContain("pseudocode");
|
||||
expect(
|
||||
JSON.parse(response.content.find((c) => c.type === "text")?.text ?? "null"),
|
||||
).toEqual(response.structuredContent);
|
||||
const page = await client.callTool({
|
||||
name: "inspect_analysis_view",
|
||||
arguments: {
|
||||
source: { kind: "retained-evidence", evidence_id: parent.evidence_id },
|
||||
view: { kind: "native", facet: "pseudocode", offset: 0, limit: 32 },
|
||||
},
|
||||
});
|
||||
expect(page.isError).not.toBe(true);
|
||||
expect(
|
||||
toolContract("inspect_analysis_view").outputSchema.parse(
|
||||
page.structuredContent,
|
||||
).normalized_result,
|
||||
).toMatchObject({
|
||||
item: { text: "x".repeat(32) },
|
||||
coverage: { total: 6 * 1024 * 1024, exhausted: false },
|
||||
});
|
||||
const empty = await client.callTool({
|
||||
name: "get_evidence_bundle",
|
||||
arguments: {
|
||||
detail: "summary",
|
||||
filters: { evidence_id: `ev_${"e".repeat(64)}` },
|
||||
},
|
||||
});
|
||||
expect(
|
||||
toolContract("get_evidence_bundle").outputSchema.parse(
|
||||
empty.structuredContent,
|
||||
).result,
|
||||
).toMatchObject({ matching_records: 0, records: [] });
|
||||
await client.ping();
|
||||
const invalid = await client.callTool({
|
||||
name: "get_evidence_bundle",
|
||||
arguments: { filters: { operation: "analyze_function" } },
|
||||
});
|
||||
expect(parseMcpToolError(invalid).error.code).toBe("invalid_request");
|
||||
});
|
||||
|
||||
it("advertises exact schemas and projects one layout section from retained Evidence", async () => {
|
||||
const layout = analysisViewLayoutFixture();
|
||||
const service = new BinaryLayoutService({
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
|
||||
import { crc32, deflateSync } from "node:zlib";
|
||||
import { parseEvidence } from "../../../src/domain/evidence.js";
|
||||
import { ok as resultOk } from "../../../src/domain/result.js";
|
||||
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
|
||||
@@ -8,7 +7,6 @@ import { afterEach, expect, it } from "vitest";
|
||||
import { createTestBinarySession } from "../../fixtures/binarySession.js";
|
||||
import { CdpBrowserProvider } from "../../../src/browser/CdpBrowserProvider.js";
|
||||
import { webPageInspectionSchema } from "../../../src/domain/browserObservationSchemas.js";
|
||||
import { createWebScreenshotArtifact } from "../../../src/domain/webScreenshot.js";
|
||||
import { JAVASCRIPT_RUNTIME_RECONCILIATION_EXAMPLE } from "../../../src/contracts/javascript/javascriptRuntimeReconciliationExample.js";
|
||||
import { TOOL_CONTRACTS } from "../../../src/contracts/toolContracts.js";
|
||||
import { createServer } from "../../../src/server/createServer.js";
|
||||
@@ -264,8 +262,6 @@ const verifySessionAndComparisonTools = async (
|
||||
expect(visual.structuredContent).toMatchObject({
|
||||
normalized_result: { status: "identical", changed_pixels: 0 },
|
||||
});
|
||||
|
||||
await assertScreenshotPngFailures(connected.client);
|
||||
expect(inspected.structuredContent).toMatchObject({
|
||||
normalized_result: expect.any(Object),
|
||||
operation: "inspect_web_page",
|
||||
@@ -274,68 +270,6 @@ const verifySessionAndComparisonTools = async (
|
||||
});
|
||||
};
|
||||
|
||||
const pngChunk = (type: string, data: Buffer): Buffer => {
|
||||
const chunk = Buffer.alloc(12 + data.length);
|
||||
chunk.writeUInt32BE(data.length, 0);
|
||||
chunk.write(type, 4, 4, "ascii");
|
||||
data.copy(chunk, 8);
|
||||
chunk.writeUInt32BE(
|
||||
crc32(chunk.subarray(4, 8 + data.length)),
|
||||
8 + data.length,
|
||||
);
|
||||
return chunk;
|
||||
};
|
||||
|
||||
const assertScreenshotPngFailures = async (client: Client): Promise<void> => {
|
||||
const header = Buffer.alloc(13);
|
||||
header.writeUInt32BE(1, 0);
|
||||
header.writeUInt32BE(1, 4);
|
||||
header.set([8, 2, 0, 0, 0], 8);
|
||||
const malformedPng = Buffer.concat([
|
||||
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
|
||||
pngChunk("IHDR", header),
|
||||
pngChunk("IDAT", Buffer.from("bad")),
|
||||
pngChunk("IEND", Buffer.alloc(0)),
|
||||
]);
|
||||
const malformedArtifact = createWebScreenshotArtifact(malformedPng);
|
||||
const malformed = await client.callTool({
|
||||
name: "compare_web_screenshots",
|
||||
arguments: {
|
||||
before: malformedArtifact,
|
||||
after: malformedArtifact,
|
||||
},
|
||||
});
|
||||
expect(parseMcpToolError(malformed)).toMatchObject({
|
||||
error: {
|
||||
code: "invalid_request",
|
||||
category: "invalid_input",
|
||||
details: {
|
||||
operation: "compare_web_screenshots",
|
||||
issues: [{ path: [], reason: "invalid_format" }],
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
const grayscaleHeader = Buffer.alloc(13);
|
||||
grayscaleHeader.writeUInt32BE(1, 0);
|
||||
grayscaleHeader.writeUInt32BE(1, 4);
|
||||
grayscaleHeader.set([8, 0, 0, 0, 0], 8);
|
||||
const unsupportedPng = Buffer.concat([
|
||||
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
|
||||
pngChunk("IHDR", grayscaleHeader),
|
||||
pngChunk("IDAT", deflateSync(Buffer.from([0, 30]))),
|
||||
pngChunk("IEND", Buffer.alloc(0)),
|
||||
]);
|
||||
const unsupported = createWebScreenshotArtifact(unsupportedPng);
|
||||
const rejectedFormat = await client.callTool({
|
||||
name: "compare_web_screenshots",
|
||||
arguments: { before: unsupported, after: unsupported },
|
||||
});
|
||||
expect(parseMcpToolError(rejectedFormat)).toMatchObject({
|
||||
error: { code: "unsupported_target", category: "unsupported_target" },
|
||||
});
|
||||
};
|
||||
|
||||
it("does not attach to a target outside the request's allowed origin scope", async () => {
|
||||
const browser = await startFakeCdpBrowser();
|
||||
browsers.push(browser);
|
||||
@@ -361,43 +295,6 @@ it("does not attach to a target outside the request's allowed origin scope", asy
|
||||
expect(browser.commands).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("projects legal oversized PNG dimensions as a memory constraint", async () => {
|
||||
const header = Buffer.alloc(13);
|
||||
header.writeUInt32BE(0x7fffffff, 0);
|
||||
header.writeUInt32BE(1, 4);
|
||||
header.set([8, 6, 0, 0, 0], 8);
|
||||
const artifact = createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
|
||||
pngChunk("IHDR", header),
|
||||
pngChunk("IDAT", deflateSync(Buffer.from([0]))),
|
||||
pngChunk("IEND", Buffer.alloc(0)),
|
||||
]),
|
||||
);
|
||||
const connected = await connectBrowser();
|
||||
const result = await connected.client.callTool({
|
||||
name: "compare_web_screenshots",
|
||||
arguments: { before: artifact, after: artifact },
|
||||
});
|
||||
|
||||
expect(parseMcpToolError(result)).toMatchObject({
|
||||
error: {
|
||||
code: "resource_constraint",
|
||||
category: "resource_constraint",
|
||||
details: {
|
||||
operation: "compare_web_screenshots",
|
||||
resource: "memory",
|
||||
reported_limits: {
|
||||
maximum_working_memory_bytes: 256 * 1024 * 1024,
|
||||
estimated_working_memory_bytes: "34359738499",
|
||||
before_dimensions: { width: 0x7fffffff, height: 1 },
|
||||
after_dimensions: { width: 0x7fffffff, height: 1 },
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
const connectBrowser = async () => {
|
||||
const session = createTestBinarySession(() => ({
|
||||
execute: () => Promise.resolve(observed(null)),
|
||||
|
||||
@@ -39,11 +39,3 @@ bounded, and address-bearing; it never compares their text with Hopper. The
|
||||
callback fixture also proves that an unresolved targetless flow is not silently
|
||||
promoted to a direct callee. Every target runs in a separate owned project and
|
||||
must leave no process, socket, project, or runtime root after close.
|
||||
|
||||
`relative-switch-signed.S` covers signed halfword offsets, the CBZ zero-entry
|
||||
route, and selector biases of 0, 2 and 10 in the AArch64 jump-table lane. The
|
||||
nonzero biases exercise overlapping and disjoint typed label/index ranges. REA
|
||||
retains Ghidra's typed labels when the compared selector has an earlier register
|
||||
definition; additional labels remain unknown until their relationship to the
|
||||
typed switch selector can be proved. The oracle rejects extra numeric labels
|
||||
and verifies each retained target against the source instruction immediate.
|
||||
|
||||
@@ -1,83 +0,0 @@
|
||||
// Hand-written signed halfword relative dispatch, modelled on the AOSP
|
||||
// RenderScript blend kernel shape reported in issue #487:
|
||||
// adrp/add table; cmp; b.cs default; ldrsh; cbz null; adr base; add; br.
|
||||
// Entries are signed byte offsets from Lsigned_base (no scaling), so cases
|
||||
// before the base encode negative offsets. A zero entry means "no case body"
|
||||
// and is routed through the CBZ before the branch. The source labels and the
|
||||
// immediates below are the oracle; no REA or Ghidra result defines them.
|
||||
// case 0 -> 200, 1 -> 201, 2 -> 202, 3 -> null path (250), 4 -> 204,
|
||||
// selector >= 5 -> default (-1).
|
||||
#ifndef REA_SELECTOR_BIAS
|
||||
#define REA_SELECTOR_BIAS 0
|
||||
#endif
|
||||
// Nonzero variants switch on the original selector (bias .. bias+4),
|
||||
// while the guarded table index is selector - bias.
|
||||
#if defined(__APPLE__)
|
||||
.section __TEXT,__text,regular,pure_instructions
|
||||
.globl _rea_relative_switch_signed
|
||||
.p2align 2
|
||||
_rea_relative_switch_signed:
|
||||
#else
|
||||
.text
|
||||
.globl rea_relative_switch_signed
|
||||
.type rea_relative_switch_signed,%function
|
||||
.p2align 2
|
||||
rea_relative_switch_signed:
|
||||
#endif
|
||||
#if defined(__APPLE__)
|
||||
adrp x5, Lsigned_table@PAGE
|
||||
add x5, x5, Lsigned_table@PAGEOFF
|
||||
#else
|
||||
adrp x5, Lsigned_table
|
||||
add x5, x5, :lo12:Lsigned_table
|
||||
#endif
|
||||
#if REA_SELECTOR_BIAS != 0
|
||||
sub w8, w0, #REA_SELECTOR_BIAS
|
||||
cmp w8, #5
|
||||
b.cs Lsigned_default
|
||||
ldrsh x6, [x5, w8, uxtw #1]
|
||||
#else
|
||||
cmp w0, #5
|
||||
b.cs Lsigned_default
|
||||
ldrsh x6, [x5, w0, uxtw #1]
|
||||
#endif
|
||||
cbz x6, Lsigned_null
|
||||
adr x5, Lsigned_base
|
||||
add x6, x5, x6
|
||||
br x6
|
||||
Lsigned_case0:
|
||||
mov w0, #200
|
||||
ret
|
||||
Lsigned_case1:
|
||||
mov w0, #201
|
||||
// The base sits inside case 1 so that no real case has a zero offset.
|
||||
Lsigned_base:
|
||||
ret
|
||||
Lsigned_case2:
|
||||
mov w0, #202
|
||||
ret
|
||||
Lsigned_case4:
|
||||
mov w0, #204
|
||||
ret
|
||||
Lsigned_null:
|
||||
mov w0, #250
|
||||
ret
|
||||
Lsigned_default:
|
||||
mov w0, #-1
|
||||
ret
|
||||
#if !defined(__APPLE__)
|
||||
.size rea_relative_switch_signed, .-rea_relative_switch_signed
|
||||
#endif
|
||||
|
||||
#if defined(__APPLE__)
|
||||
.section __TEXT,__const
|
||||
#else
|
||||
.section .rodata
|
||||
#endif
|
||||
.p2align 1
|
||||
Lsigned_table:
|
||||
.short Lsigned_case0-Lsigned_base
|
||||
.short Lsigned_case1-Lsigned_base
|
||||
.short Lsigned_case2-Lsigned_base
|
||||
.short 0
|
||||
.short Lsigned_case4-Lsigned_base
|
||||
+1
-1
@@ -118,7 +118,7 @@ const responseBodyResult = (options: FakeOptions) =>
|
||||
};
|
||||
|
||||
const screenshotResult = () => ({
|
||||
data: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg==",
|
||||
data: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PzWvWQAAAABJRU5ErkJggg==",
|
||||
});
|
||||
|
||||
const storageUsageResult = () => ({
|
||||
|
||||
@@ -142,161 +142,8 @@
|
||||
terminal.
|
||||
</p>
|
||||
<p class="requirements">
|
||||
You'll need Node.js 22 (22.19+), 24 (24.11+) or 26+. npm comes with
|
||||
Node.js.
|
||||
You'll need Node.js 22 (22.19+), 24 (24.11+) or 26+, and npm.
|
||||
</p>
|
||||
<details class="evidence-details" id="install-node">
|
||||
<summary>Don't have Node.js, or not sure?</summary>
|
||||
<p>Open a terminal and check:</p>
|
||||
<div class="code-block">
|
||||
<div class="code-header">
|
||||
<span>Terminal</span
|
||||
><button
|
||||
class="copy-button"
|
||||
type="button"
|
||||
data-copy="node-check"
|
||||
aria-label="Copy Node.js version check"
|
||||
>
|
||||
Copy
|
||||
</button>
|
||||
</div>
|
||||
<pre><code id="node-check">node -v
|
||||
npm -v</code></pre>
|
||||
</div>
|
||||
<p>
|
||||
If <code>node -v</code> prints v22.19+, v24.11+ or v26+, you're
|
||||
ready. If it prints an older version or "not recognized" / "command
|
||||
not found", install Node.js 24 LTS:
|
||||
</p>
|
||||
<p><strong>Windows (PowerShell)</strong></p>
|
||||
<div class="code-block">
|
||||
<div class="code-header">
|
||||
<span>Windows PowerShell</span
|
||||
><button
|
||||
class="copy-button"
|
||||
type="button"
|
||||
data-copy="node-windows"
|
||||
aria-label="Copy Node.js installation command for Windows"
|
||||
>
|
||||
Copy
|
||||
</button>
|
||||
</div>
|
||||
<pre><code id="node-windows">winget install OpenJS.NodeJS.LTS</code></pre>
|
||||
</div>
|
||||
<p>
|
||||
Type Y if asked to accept the terms. Close PowerShell, open it
|
||||
again, then run <code>node -v</code>.
|
||||
</p>
|
||||
<p>
|
||||
If PowerShell later says
|
||||
<em>"running scripts is disabled on this system"</em> when you run
|
||||
<code>npx</code>, run this once and try again:
|
||||
</p>
|
||||
<div class="code-block">
|
||||
<div class="code-header">
|
||||
<span>Windows PowerShell</span
|
||||
><button
|
||||
class="copy-button"
|
||||
type="button"
|
||||
data-copy="node-windows-policy"
|
||||
aria-label="Copy PowerShell execution policy command"
|
||||
>
|
||||
Copy
|
||||
</button>
|
||||
</div>
|
||||
<pre><code id="node-windows-policy">Set-ExecutionPolicy -Scope CurrentUser RemoteSigned</code></pre>
|
||||
</div>
|
||||
<p><strong>macOS</strong></p>
|
||||
<p>If you use Homebrew:</p>
|
||||
<div class="code-block">
|
||||
<div class="code-header">
|
||||
<span>macOS · Homebrew</span
|
||||
><button
|
||||
class="copy-button"
|
||||
type="button"
|
||||
data-copy="node-macos-brew"
|
||||
aria-label="Copy Homebrew Node.js installation command"
|
||||
>
|
||||
Copy
|
||||
</button>
|
||||
</div>
|
||||
<pre><code id="node-macos-brew">brew install node</code></pre>
|
||||
</div>
|
||||
<p>Otherwise:</p>
|
||||
<div class="code-block">
|
||||
<div class="code-header">
|
||||
<span>macOS</span
|
||||
><button
|
||||
class="copy-button"
|
||||
type="button"
|
||||
data-copy="node-macos-fnm"
|
||||
aria-label="Copy fnm installation command for macOS"
|
||||
>
|
||||
Copy
|
||||
</button>
|
||||
</div>
|
||||
<pre><code id="node-macos-fnm">curl -fsSL https://fnm.vercel.app/install | bash</code></pre>
|
||||
</div>
|
||||
<p>Close Terminal, open it again, then:</p>
|
||||
<div class="code-block">
|
||||
<div class="code-header">
|
||||
<span>macOS</span
|
||||
><button
|
||||
class="copy-button"
|
||||
type="button"
|
||||
data-copy="node-macos-install"
|
||||
aria-label="Copy Node.js 24 commands for macOS"
|
||||
>
|
||||
Copy
|
||||
</button>
|
||||
</div>
|
||||
<pre><code id="node-macos-install">fnm install 24
|
||||
fnm default 24</code></pre>
|
||||
</div>
|
||||
<p><strong>Linux</strong></p>
|
||||
<div class="code-block">
|
||||
<div class="code-header">
|
||||
<span>Linux</span
|
||||
><button
|
||||
class="copy-button"
|
||||
type="button"
|
||||
data-copy="node-linux-fnm"
|
||||
aria-label="Copy fnm installation command for Linux"
|
||||
>
|
||||
Copy
|
||||
</button>
|
||||
</div>
|
||||
<pre><code id="node-linux-fnm">curl -fsSL https://fnm.vercel.app/install | bash</code></pre>
|
||||
</div>
|
||||
<p>
|
||||
The installer needs <code>curl</code> and <code>unzip</code> (on
|
||||
Debian/Ubuntu: <code>sudo apt install curl unzip</code>). Close the
|
||||
terminal, open it again, then:
|
||||
</p>
|
||||
<div class="code-block">
|
||||
<div class="code-header">
|
||||
<span>Linux</span
|
||||
><button
|
||||
class="copy-button"
|
||||
type="button"
|
||||
data-copy="node-linux-install"
|
||||
aria-label="Copy Node.js 24 commands for Linux"
|
||||
>
|
||||
Copy
|
||||
</button>
|
||||
</div>
|
||||
<pre><code id="node-linux-install">fnm install 24
|
||||
fnm default 24</code></pre>
|
||||
</div>
|
||||
<p>
|
||||
Your distribution's own <code>nodejs</code> package is often older
|
||||
than 22.19, so check <code>node -v</code> if you use it instead.
|
||||
</p>
|
||||
<p>
|
||||
Prefer a regular installer? Download the LTS version from
|
||||
<a href="https://nodejs.org/en/download">nodejs.org</a>.
|
||||
</p>
|
||||
</details>
|
||||
</div>
|
||||
<div class="article-layout">
|
||||
<nav class="contents" aria-label="On this page">
|
||||
@@ -317,44 +164,6 @@ fnm default 24</code></pre>
|
||||
Build, and other supported MCP clients. Grok Bot's connector is
|
||||
added from its chat.
|
||||
</p>
|
||||
<details class="evidence-details">
|
||||
<summary>New to coding agents?</summary>
|
||||
<p>
|
||||
A coding agent is an AI assistant that works on your computer
|
||||
instead of in a browser tab. It can read files and run commands,
|
||||
and it asks before making changes. REA gives it
|
||||
reverse-engineering tools.
|
||||
</p>
|
||||
<p>
|
||||
If you don't have one yet, pick one, follow its install page and
|
||||
sign in:
|
||||
</p>
|
||||
<ul>
|
||||
<li>
|
||||
<strong
|
||||
><a href="https://code.claude.com/docs/en/setup"
|
||||
>Claude Code</a
|
||||
></strong
|
||||
>
|
||||
by Anthropic. Needs a paid Claude plan.
|
||||
</li>
|
||||
<li>
|
||||
<strong
|
||||
><a href="https://github.com/openai/codex">Codex</a></strong
|
||||
>
|
||||
by OpenAI. Sign in with your ChatGPT account.
|
||||
</li>
|
||||
<li>
|
||||
<strong
|
||||
><a href="https://opencode.ai/docs">OpenCode</a></strong
|
||||
>, open source. Connect the model provider of your choice.
|
||||
</li>
|
||||
</ul>
|
||||
<p>
|
||||
Then open it and copy the prompt below into it. The agent runs
|
||||
REA's setup and shows you the plan before changing anything.
|
||||
</p>
|
||||
</details>
|
||||
<div class="agent-terminal">
|
||||
<div class="terminal-header">
|
||||
<span class="terminal-title"
|
||||
@@ -382,28 +191,6 @@ fnm default 24</code></pre>
|
||||
</div>
|
||||
<details class="evidence-details">
|
||||
<summary>Or run setup yourself</summary>
|
||||
<details class="evidence-details">
|
||||
<summary>How do I open a terminal?</summary>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Windows:</strong> open the Start menu, type
|
||||
<em>PowerShell</em> and press Enter.
|
||||
</li>
|
||||
<li>
|
||||
<strong>macOS:</strong> press <kbd>⌘</kbd> <kbd>Space</kbd>,
|
||||
type <em>Terminal</em> and press Enter.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Linux:</strong> press <kbd>Ctrl</kbd> <kbd>Alt</kbd>
|
||||
<kbd>T</kbd> on most desktops.
|
||||
</li>
|
||||
</ul>
|
||||
<p>
|
||||
Paste a command with <kbd>Ctrl</kbd> <kbd>V</kbd> (macOS:
|
||||
<kbd>⌘</kbd> <kbd>V</kbd>; Linux terminals: <kbd>Ctrl</kbd>
|
||||
<kbd>Shift</kbd> <kbd>V</kbd>), then press Enter.
|
||||
</p>
|
||||
</details>
|
||||
<div class="code-block">
|
||||
<div class="code-header">
|
||||
<span>Terminal</span
|
||||
@@ -424,24 +211,10 @@ fnm default 24</code></pre>
|
||||
connection and the matching investigation skill, keeping a backup
|
||||
of existing configuration. It asks before applying the changes.
|
||||
</p>
|
||||
<p>Restart your client when setup finishes.</p>
|
||||
<details class="evidence-details">
|
||||
<summary>How do I restart my client?</summary>
|
||||
<p>
|
||||
<strong>Desktop apps</strong> (Claude Desktop, Cursor, Windsurf,
|
||||
VS Code): quit the app completely, then open it again. Closing
|
||||
the window isn't enough. On macOS press <kbd>⌘</kbd>
|
||||
<kbd>Q</kbd>; on Windows, also quit from the tray icon if there
|
||||
is one.
|
||||
</p>
|
||||
<p>
|
||||
<strong>Terminal agents</strong> (Claude Code, Codex, Gemini
|
||||
CLI): exit the session and start it again.
|
||||
</p>
|
||||
</details>
|
||||
<p>
|
||||
Give your agent a target path and a specific question. For a
|
||||
JavaScript or Electron app, you might start with:
|
||||
Restart your client when setup finishes. Give your agent a target
|
||||
path and a specific question. For a JavaScript or Electron app,
|
||||
you might start with:
|
||||
</p>
|
||||
<blockquote class="prompt">
|
||||
<p>
|
||||
@@ -451,27 +224,6 @@ fnm default 24</code></pre>
|
||||
</p>
|
||||
</blockquote>
|
||||
<p>Replace the path with your local target.</p>
|
||||
<details class="evidence-details">
|
||||
<summary>How do I get a folder's full path?</summary>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Windows:</strong> in File Explorer, right-click the
|
||||
folder and choose <em>Copy as path</em>. On Windows 10, hold
|
||||
Shift while right-clicking.
|
||||
</li>
|
||||
<li>
|
||||
<strong>macOS:</strong> in Finder, right-click the folder,
|
||||
hold Option and choose <em>Copy "…" as Pathname</em>.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Any system:</strong> drag the folder into a terminal
|
||||
window and its path appears.
|
||||
</li>
|
||||
</ul>
|
||||
<p>
|
||||
In a terminal, put quotes around a path that contains spaces.
|
||||
</p>
|
||||
</details>
|
||||
</section>
|
||||
|
||||
<section class="article-section" id="terminal">
|
||||
@@ -500,37 +252,6 @@ fnm default 24</code></pre>
|
||||
<code>.asar</code> file. On Windows, a path such as
|
||||
<code>"D:/apps/example"</code> works here.
|
||||
</p>
|
||||
<details class="evidence-details">
|
||||
<summary>Where do I find an app's .asar file?</summary>
|
||||
<p>
|
||||
Electron apps usually keep it in a
|
||||
<code>resources</code> folder:
|
||||
</p>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Windows:</strong>
|
||||
<code
|
||||
>C:\Users\<you>\AppData\Local\Programs\<App>\resources\app.asar</code
|
||||
>
|
||||
</li>
|
||||
<li>
|
||||
<strong>macOS:</strong>
|
||||
<code
|
||||
>/Applications/<App>.app/Contents/Resources/app.asar</code
|
||||
>
|
||||
(right-click the app and choose
|
||||
<em>Show Package Contents</em>)
|
||||
</li>
|
||||
<li>
|
||||
<strong>Linux:</strong> often
|
||||
<code>/opt/<App>/resources/app.asar</code>
|
||||
</li>
|
||||
</ul>
|
||||
<p>
|
||||
Some apps install elsewhere. Search the app's folder for
|
||||
<code>app.asar</code>.
|
||||
</p>
|
||||
</details>
|
||||
<p>
|
||||
The result identifies the app, shows connections between modules
|
||||
and gives their source locations. Connections REA cannot resolve
|
||||
|
||||
@@ -143,14 +143,6 @@
|
||||
</p>
|
||||
</div>
|
||||
<div class="guide-index">
|
||||
<section class="guide-entry">
|
||||
<p class="eyebrow">First investigation</p>
|
||||
<div>
|
||||
<h2>Set up REA</h2>
|
||||
<p>Connect your coding agent or use REA from the terminal.</p>
|
||||
<a class="text-link" href="../get-started/">Get started with REA</a>
|
||||
</div>
|
||||
</section>
|
||||
<section class="guide-entry">
|
||||
<p class="eyebrow">Native binaries</p>
|
||||
<div>
|
||||
@@ -207,6 +199,14 @@
|
||||
<a class="text-link" href="browser/">Browser observation guide</a>
|
||||
</div>
|
||||
</section>
|
||||
<section class="guide-entry">
|
||||
<p class="eyebrow">First investigation</p>
|
||||
<div>
|
||||
<h2>Set up REA</h2>
|
||||
<p>Connect your coding agent or use REA from the terminal.</p>
|
||||
<a class="text-link" href="../get-started/">Get started with REA</a>
|
||||
</div>
|
||||
</section>
|
||||
</div>
|
||||
</main>
|
||||
<!-- website:footer:start -->
|
||||
|
||||
@@ -168,33 +168,6 @@
|
||||
returns its instructions and callers, which give the agent a way
|
||||
to investigate the missing arithmetic.
|
||||
</p>
|
||||
<details class="evidence-details">
|
||||
<summary>New to these terms?</summary>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Instructions:</strong> the steps the processor runs,
|
||||
shown in readable form such as <code>MOV</code> or
|
||||
<code>ADD</code>.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Decompiler result / pseudocode:</strong> an analysis
|
||||
tool's attempt to turn instructions back into C-like code. It
|
||||
can miss parts, as in this example.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Caller:</strong> a function that calls this one. It
|
||||
shows which values are passed in.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Address:</strong> a location in the program, written
|
||||
in hexadecimal, such as <code>0x00406400</code>.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Constants:</strong> fixed numbers stored in the
|
||||
program, read as raw bytes and decoded.
|
||||
</li>
|
||||
</ul>
|
||||
</details>
|
||||
<figure class="workflow-figure">
|
||||
<p class="figure-title">
|
||||
Three pieces of evidence, one calculation
|
||||
@@ -238,40 +211,9 @@
|
||||
<h2>Ask about the behavior</h2>
|
||||
<p>
|
||||
After
|
||||
<a href="../../get-started/#agent"
|
||||
>connecting REA to your agent</a
|
||||
>
|
||||
and
|
||||
<a href="../../get-started/#native"
|
||||
>setting up a native analysis tool</a
|
||||
<a href="../../get-started/#agent">connecting REA to your agent</a
|
||||
>, provide the local executable and a question:
|
||||
</p>
|
||||
<details class="evidence-details">
|
||||
<summary>Which analysis tool should I use?</summary>
|
||||
<p>REA needs one of these to read native programs:</p>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Hopper</strong> (macOS or Linux): REA's setup detects
|
||||
it or offers to install it. On macOS, open Hopper once and
|
||||
choose demo mode or activate a license before your first
|
||||
analysis.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Ghidra</strong> (macOS or Linux; experimental on
|
||||
Windows): free, but needs an existing Ghidra 12.1.x
|
||||
installation and a 64-bit JDK 21 or newer.
|
||||
</li>
|
||||
<li>
|
||||
<strong>IDA</strong> (Windows): for an existing IDA setup with
|
||||
an MCP registration.
|
||||
</li>
|
||||
</ul>
|
||||
<p>
|
||||
See
|
||||
<a href="../../get-started/#native">Set up native analysis</a>
|
||||
for the steps.
|
||||
</p>
|
||||
</details>
|
||||
<div class="agent-terminal">
|
||||
<div class="terminal-header">
|
||||
<span class="terminal-title"
|
||||
@@ -297,28 +239,8 @@
|
||||
</div>
|
||||
<p class="caption">
|
||||
Example prompt. Give your agent the path to your local DX-Ball
|
||||
1.07 executable. REA doesn't include DX-Ball, so use your own
|
||||
copy, or ask the same kind of question about another small Windows
|
||||
program you have.
|
||||
1.07 executable.
|
||||
</p>
|
||||
<details class="evidence-details">
|
||||
<summary>How do I get a file's full path?</summary>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Windows:</strong> in File Explorer, right-click the
|
||||
file and choose <em>Copy as path</em>. On Windows 10, hold
|
||||
Shift while right-clicking.
|
||||
</li>
|
||||
<li>
|
||||
<strong>macOS:</strong> in Finder, right-click the file, hold
|
||||
Option and choose <em>Copy "…" as Pathname</em>.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Any system:</strong> drag the file into a terminal
|
||||
window and its path appears.
|
||||
</li>
|
||||
</ul>
|
||||
</details>
|
||||
</section>
|
||||
<section class="article-section" id="follow">
|
||||
<h2>Read the REA results</h2>
|
||||
|
||||
Reference in New Issue
Block a user