Compare commits
34
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a56f7bb668 | ||
|
|
ee46323b49 | ||
|
|
a99857df23 | ||
|
|
6b043dc3de | ||
|
|
4003151ec1 | ||
|
|
92dac97102 | ||
|
|
89ea2cad19 | ||
|
|
ea06264a56 | ||
|
|
59b4a3c319 | ||
|
|
ea1c164e5e | ||
|
|
dc676b4d85 | ||
|
|
ad3e41d46e | ||
|
|
17f704050f | ||
|
|
c8820e03db | ||
|
|
ab80ec640a | ||
|
|
9ce6ef895e | ||
|
|
b3e977e6d0 | ||
|
|
a99b2fd3b5 | ||
|
|
4e4b15b825 | ||
|
|
21dbcf7a14 | ||
|
|
76296c5a38 | ||
|
|
e38550462e | ||
|
|
80896a1d64 | ||
|
|
e5bc4e85ec | ||
|
|
38221cb12e | ||
|
|
3749ccec47 | ||
|
|
a67dbf6a48 | ||
|
|
bd3a9bbbaf | ||
|
|
74050978be | ||
|
|
b212e92f9b | ||
|
|
990e1ad8c1 | ||
|
|
76d8f68cb9 | ||
|
|
fe574c0c9f | ||
|
|
3a7a3074e3 |
@@ -25,6 +25,7 @@ node scripts/release-notes.mjs inventory \
|
||||
--manifest /tmp/rea-VERSION.contributions.json
|
||||
node scripts/release-notes.mjs record \
|
||||
--manifest /tmp/rea-VERSION.contributions.json \
|
||||
--summary \
|
||||
--output /tmp/rea-VERSION.record.md
|
||||
```
|
||||
|
||||
@@ -56,25 +57,31 @@ version/date/link heading. Use single-line Markdown list items under:
|
||||
PR/commit references required by the existing checkpoint validator.
|
||||
|
||||
Group related work by analyst workflow or user impact. Explain the resulting
|
||||
behavior; internal cleanup, tests, CI, and routine documentation generally
|
||||
belong only in the contribution record. Read ambiguous PRs/diffs before making
|
||||
behavior in short, concrete entries. Highlights summarize the release; avoid
|
||||
repeating their full descriptions in Changes. Internal cleanup, tests, CI, and
|
||||
routine documentation generally belong only in the provenance file. Read ambiguous PRs/diffs before making
|
||||
claims. Distinguish simulated-provider checks from real-provider verification.
|
||||
|
||||
Each bullet covering a PR carries its link and `Thanks @...` for its verified
|
||||
human author, known co-authors, and confirmed closing-issue reporters. Grouping
|
||||
must preserve **all** those credits in the same bullet. Include the issue
|
||||
links. Direct-commit bullets carry the commit link and known human credit.
|
||||
Credit each represented contribution once in the public notes with its link
|
||||
and `Thanks @...` for its verified human author, known co-authors, and confirmed
|
||||
closing-issue reporters. Grouping preserves all those credits in one line.
|
||||
When Highlights, detailed entries or migrations reference the same work,
|
||||
put its thanks in one of those entries, not all of them. Include issue links.
|
||||
Direct-commit bullets carry the commit link and known human credit.
|
||||
Exclude GitHub bots and known agent accounts. REA's observed Anthropic and
|
||||
Cursor co-author trailers resolve to `@claude` and `@cursoragent` with GitHub
|
||||
type `User`; the credit policy excludes them while retaining their metadata.
|
||||
Do not copy OpenClaw's maintainer-specific exclusions.
|
||||
At the REA maintainer's request, omit `@morluto` self-thanks while preserving
|
||||
authorship in the inventory. Other human contributors remain eligible.
|
||||
Use explicit GitHub links for cross-repository references to avoid ambiguous
|
||||
`#NNN` numbers. Unknown handles stay unknown; they are not fabricated credit.
|
||||
|
||||
Append the generated `### Complete contribution record` verbatim as the last
|
||||
section of this release. It accounts for every discovered in-range PR and
|
||||
direct contribution, including internal work omitted from the prose. Do not
|
||||
hand-edit generated credits to make validation pass. Refresh the inventory
|
||||
Append the generated `record --summary` provenance link verbatim at the end
|
||||
of this release. The linked inventory accounts for every discovered in-range
|
||||
PR and direct contribution, including internal work omitted from the prose.
|
||||
Do not paste the full `record` output into public notes: it duplicates thanks
|
||||
and obscures the user-facing changes. Full records remain available for review.
|
||||
Do not hand-edit generated credits to make validation pass. Refresh the inventory
|
||||
when product history changes; a prose-only correction can reuse the snapshot.
|
||||
|
||||
## Finalize with Release Please
|
||||
|
||||
@@ -135,6 +135,16 @@ session to a selected path. The analysis remains complete in the ledger; avoid
|
||||
repeating it merely to request the same oversized response. Complete CLI JSON
|
||||
output also streams without a single MCP frame.
|
||||
|
||||
When the retained parent ID is unknown, use `get_evidence_bundle` with
|
||||
`detail: "summary"` and exact filters for operation, target SHA-256,
|
||||
analysis-profile digest or canonical procedure address. Select its returned
|
||||
Evidence ID for the next view instead of requesting the complete bundle.
|
||||
`retention: "complete-record"` means the whole record is stored; native
|
||||
value-flow truncation and unknown metadata remain separate. For an exported
|
||||
portable bundle, use `rea inspect-evidence-bundle` with JSON containing `path`,
|
||||
`detail: "summary"` and the same `filters`. Reading that file still requires
|
||||
validating its full contents within the local memory budget.
|
||||
|
||||
A client can also truncate a successful tool result without a REA transport
|
||||
error. Treat that preview as incomplete; use the retained parent ID for smaller
|
||||
views or export when the full record is needed. Do not parse the preview as
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
{
|
||||
".": "6.3.0"
|
||||
".": "6.4.0"
|
||||
}
|
||||
|
||||
@@ -1,5 +1,69 @@
|
||||
# Changelog
|
||||
|
||||
## [6.4.0](https://github.com/morluto/rea/compare/rea-agents-6.3.0...rea-agents-6.4.0) (2026-10-11)
|
||||
|
||||
### Highlights
|
||||
|
||||
- Android: added device inspection, split-APK acquisition, logs, screenshots, file transfers and explicit app lifecycle actions through ADB, plus manifest and localized-string decoding with Apktool. ([#1552](https://github.com/morluto/rea/pull/1552), [#1556](https://github.com/morluto/rea/pull/1556)) Thanks @3QL.
|
||||
- JEB: browse project units and decompile selected items through a caller-started local JEB MCP server. ([#1536](https://github.com/morluto/rea/pull/1536)) Thanks @3QL.
|
||||
- Large analyses: discover retained Evidence without loading every payload, and inspect JavaScript graphs, binary layouts and function dossiers through summaries and selected views. ([#1682](https://github.com/morluto/rea/pull/1682), [#1568](https://github.com/morluto/rea/pull/1568), [#1570](https://github.com/morluto/rea/pull/1570), [#1345](https://github.com/morluto/rea/pull/1345), [#1507](https://github.com/morluto/rea/pull/1507), [#1247](https://github.com/morluto/rea/issues/1247)) Thanks @jhuang-tw, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- JavaScript: isolate source analysis in an owned process with configurable heap and timeout limits; retain completed facts and explicit unknowns when analysis runs out of resources. ([#1651](https://github.com/morluto/rea/pull/1651), [#1563](https://github.com/morluto/rea/issues/1563), [#1565](https://github.com/morluto/rea/issues/1565)) Thanks @N0zoM1z0, @peter-research.
|
||||
- Ghidra: configure import language/compiler specifications and seed functions, code and labels before auto-analysis. ([#1667](https://github.com/morluto/rea/pull/1667)) Thanks @sqmagellan.
|
||||
- RPC discovery: recognize supported tRPC calls, HTTP-link URLs and named GraphQL operations in JavaScript applications and web bundles. ([#1670](https://github.com/morluto/rea/pull/1670)) Thanks @pompiliodonatoapp.
|
||||
|
||||
### Changes
|
||||
|
||||
- Added Ghidra support for Windows PE DLLs and bounded MIPS ELF analysis, including carved MIPS inputs with incomplete section tables. Windows P0 remains read-only. ([#1303](https://github.com/morluto/rea/pull/1303), [#1362](https://github.com/morluto/rea/pull/1362), [#1629](https://github.com/morluto/rea/pull/1629), [#1290](https://github.com/morluto/rea/issues/1290), [#718](https://github.com/morluto/rea/issues/718), [#1619](https://github.com/morluto/rea/issues/1619)) Thanks @bqw371, @jaelliot, @kvalls, @N0zoM1z0.
|
||||
- Enabled `set_address_name` and `set_addresses_names` for Ghidra sessions with mutation authority, including namespace-aware data labels and per-address batch results. ([#1635](https://github.com/morluto/rea/pull/1635), [#1634](https://github.com/morluto/rea/issues/1634)) Thanks @giri256.
|
||||
- Recovered additional signed AArch64 relative jump-table patterns. ([#1592](https://github.com/morluto/rea/pull/1592)) Thanks @RishTheFish21.
|
||||
- Added PE resource inspection and read-only NativeAOT metadata parsing for supported x64 PE32+/.NET 8 RTR 9.1 inputs. The NativeAOT parser’s Windows-host Ghidra workflow remains unverified on a real Windows host. ([#1482](https://github.com/morluto/rea/pull/1482), [76d8f68](https://github.com/morluto/rea/commit/76d8f68cb918dfb43869f2b649a0b6c0c1cf266d), [#1254](https://github.com/morluto/rea/issues/1254)) Thanks @N0zoM1z0, @thangldw.
|
||||
- Added Pi, Hermes and Qwen Code setup, including Hermes active-profile support. ([#1376](https://github.com/morluto/rea/pull/1376), [#1403](https://github.com/morluto/rea/pull/1403), [#1415](https://github.com/morluto/rea/pull/1415), [#1409](https://github.com/morluto/rea/issues/1409)) Thanks @rasatpetabit, @yiliang114, @zaydiscold.
|
||||
- Added `REA_MCP_INPUT_SCHEMA_PROFILE=compact` for size-capped MCP clients such as Kimi/Moonshot, while retaining canonical runtime validation. ([#1514](https://github.com/morluto/rea/pull/1514), [#1580](https://github.com/morluto/rea/pull/1580), [#1484](https://github.com/morluto/rea/issues/1484), [#1581](https://github.com/morluto/rea/issues/1581)) Thanks @haythemsellami, @N0zoM1z0.
|
||||
- Added live process-capture progress and optional `finalization_ms` to capture shutdown output before timeout escalation; cancellation remains immediate. ([#1483](https://github.com/morluto/rea/pull/1483), [#1583](https://github.com/morluto/rea/pull/1583), [#1252](https://github.com/morluto/rea/issues/1252), [#1253](https://github.com/morluto/rea/issues/1253), [modelcontextprotocol/typescript-sdk#2997](https://github.com/modelcontextprotocol/typescript-sdk/issues/2997)) Thanks @N0zoM1z0, @nikolay-dementiev, @sathariels.
|
||||
- Improved Codex onboarding and recovery from observed client, PATH and provider blockers. ([#1613](https://github.com/morluto/rea/pull/1613)) Thanks @N0zoM1z0.
|
||||
- Reduced redirected JSON and terminal output size, and deferred heavy imports until their commands need them. ([#1413](https://github.com/morluto/rea/pull/1413), [#1450](https://github.com/morluto/rea/pull/1450), [#1439](https://github.com/morluto/rea/pull/1439), [21dbcf7](https://github.com/morluto/rea/commit/21dbcf7a1458219989ad291e44ce7d017378b727)) Thanks @kaoru0822-kitauji.
|
||||
|
||||
### Fixes
|
||||
|
||||
- Electron analysis: follow renamed and default-imported APIs and optional calls, ignore overwritten identities and media preload attributes, and resolve packaged `app.asar` pages and unpacked companions. ([#1391](https://github.com/morluto/rea/pull/1391), [#1393](https://github.com/morluto/rea/pull/1393), [#1400](https://github.com/morluto/rea/pull/1400), [#1664](https://github.com/morluto/rea/pull/1664), [#1673](https://github.com/morluto/rea/pull/1673)) Thanks @kaoru0822-kitauji, @marnixpluim, @N0zoM1z0, @suninrain086.
|
||||
- JavaScript data flow: preserve returned/yielded reference uncertainty, invalidate property-stored aliases correctly, and retain bundled-module runtime scope. ([#1437](https://github.com/morluto/rea/pull/1437), [#1445](https://github.com/morluto/rea/pull/1445), [#1471](https://github.com/morluto/rea/pull/1471), [#1577](https://github.com/morluto/rea/pull/1577), [#1428](https://github.com/morluto/rea/issues/1428), [#1442](https://github.com/morluto/rea/issues/1442), [#1449](https://github.com/morluto/rea/issues/1449), [#1576](https://github.com/morluto/rea/issues/1576)) Thanks @CassieFox23, @dakjdakd, @N0zoM1z0.
|
||||
- JavaScript packages: resolve exact exports subpaths, preserve trailing-slash directory requests and exact CommonJS keys, and respect shadowed `require` bindings. ([#1424](https://github.com/morluto/rea/pull/1424), [#1615](https://github.com/morluto/rea/pull/1615), [5485483](https://github.com/morluto/rea/commit/5485483f7da2f6cb66441bb784aaf2a1a5f8731a), [8b2f851](https://github.com/morluto/rea/commit/8b2f8514c50e469c3add8ce9ba7ebebef888b14f), [f998fcb](https://github.com/morluto/rea/commit/f998fcb1cca9994c3589fb70d6e0127fc215f76f), [1e51c73](https://github.com/morluto/rea/commit/1e51c732bbf08191688d6526bf58ab2650b601e3), [#1365](https://github.com/morluto/rea/issues/1365)) Thanks @akshitkhokhani, @AWUQAQ, @immanuwell, @rudycelekli.
|
||||
- Large JavaScript inputs: handle oversized inline source maps, deep returns and reassignment loops; avoid argument expansion limits and yield during large analysis passes. ([#1386](https://github.com/morluto/rea/pull/1386), [#1368](https://github.com/morluto/rea/pull/1368), [#1390](https://github.com/morluto/rea/pull/1390), [#1539](https://github.com/morluto/rea/pull/1539), [#1525](https://github.com/morluto/rea/pull/1525), [#1542](https://github.com/morluto/rea/pull/1542), [#1545](https://github.com/morluto/rea/pull/1545), [#1557](https://github.com/morluto/rea/pull/1557), [#1380](https://github.com/morluto/rea/issues/1380), [#1401](https://github.com/morluto/rea/issues/1401), [#1538](https://github.com/morluto/rea/issues/1538), [#1462](https://github.com/morluto/rea/issues/1462)) Thanks @cold-atom, @KadirJeager, @kaoru0822-kitauji, @marnixpluim, @N0zoM1z0, @rudycelekli.
|
||||
- JavaScript receivers: expand escaped methods once, index returned-object targets, and avoid replaying earlier writes or duplicating retained graph allocations. ([#1517](https://github.com/morluto/rea/pull/1517), [#1537](https://github.com/morluto/rea/pull/1537), [#1444](https://github.com/morluto/rea/pull/1444), [#1601](https://github.com/morluto/rea/pull/1601), [#1495](https://github.com/morluto/rea/issues/1495)) Thanks @CassieFox23, @geekwolf, @kaoru0822-kitauji, @N0zoM1z0, @zeynelacikgoez.
|
||||
- Browser captures: retain WebSocket URLs, empty connections and frame order; compare only common known payload coverage and preserve incomplete WebMCP replay coverage. ([db4e360](https://github.com/morluto/rea/commit/db4e3608465486466591b594cf77037ac150c988), [aa257ea](https://github.com/morluto/rea/commit/aa257ea4b16f2d2a35c12df63359d650b48b1154), [e52c0a7](https://github.com/morluto/rea/commit/e52c0a72c59c740315e89149881a27ce4d89a11c))
|
||||
- Browser scenarios: seed storage in the correct partition, normalize loopback attach identities, retain declared-secret redaction and require verified lineage before Windows Electron signals. ([#1604](https://github.com/morluto/rea/pull/1604), [#1590](https://github.com/morluto/rea/pull/1590), [#1588](https://github.com/morluto/rea/pull/1588), [8b4c3de](https://github.com/morluto/rea/commit/8b4c3de3b3a3b580adec266f94f29ccedf1da228), [#1497](https://github.com/morluto/rea/issues/1497), [#1591](https://github.com/morluto/rea/issues/1591), [#1589](https://github.com/morluto/rea/issues/1589)) Thanks @Adolanium, @akushonkamen, @dakjdakd.
|
||||
- Added Termux Android browser-scenario support. ([#1655](https://github.com/morluto/rea/pull/1655)) Thanks @tyloryang.
|
||||
- Web inputs: accept BOM-prefixed HAR, preserve the first duplicate DOM attribute and resolve SVG/XLink URLs in HTML and XHTML. ([#1509](https://github.com/morluto/rea/pull/1509), [#1516](https://github.com/morluto/rea/pull/1516), [#1602](https://github.com/morluto/rea/pull/1602), [#1683](https://github.com/morluto/rea/pull/1683), [#1496](https://github.com/morluto/rea/issues/1496), [#1586](https://github.com/morluto/rea/issues/1586)) Thanks @CassieFox23, @kaoru0822-kitauji, @sathariels, @taljeri.
|
||||
- Screenshot comparisons: validate PNG chunks, distinguish corruption from unsupported encodings and bound comparison memory. ([4e4b15b](https://github.com/morluto/rea/commit/4e4b15b8250b29e176d4191c9610a3931b693474), [b3e977e](https://github.com/morluto/rea/commit/b3e977e6d073ab1783cece8581279f87fc360411)) Thanks @rudycelekli.
|
||||
- Archives: preserve case-distinct names, report collisions at their diverging segment, detect nested formats from bytes and reject Windows alternate-stream extraction paths. ([#1480](https://github.com/morluto/rea/pull/1480), [#1505](https://github.com/morluto/rea/pull/1505), [#1553](https://github.com/morluto/rea/pull/1553), [573598c](https://github.com/morluto/rea/commit/573598c6324698cc48ceafd24ed3864670e08d00), [#1470](https://github.com/morluto/rea/issues/1470), [#1491](https://github.com/morluto/rea/issues/1491)) Thanks @CassieFox23, @kaoru0822-kitauji.
|
||||
- Cleanup: retain failed readers, ASAR handles, output trees and provider workers for retry without discarding completed output or partial observations. ([ffe0574](https://github.com/morluto/rea/commit/ffe0574e17b9d973860371083cb1a351be912131), [de0d341](https://github.com/morluto/rea/commit/de0d341524dc68250a45b41eb7e49fcecc1e32a9), [3f8c51e](https://github.com/morluto/rea/commit/3f8c51e9b506d88fc41aded7b1a22b9a09037c26), [d1235fa](https://github.com/morluto/rea/commit/d1235fadc901e1fb9ab3149d53063306c8545414), [026126e](https://github.com/morluto/rea/commit/026126eafe741e5a8b857433587fd615cdd44c19))
|
||||
- Providers: canonicalize IDA workspace roots before identity checks, respect IDA cancellation, serialize JEB acquisition and correct Ghidra no-return/imported-symbol recovery. ([#1481](https://github.com/morluto/rea/pull/1481), [#1419](https://github.com/morluto/rea/pull/1419), [5f30ea6](https://github.com/morluto/rea/commit/5f30ea6ff2dd89aab7af6da6e8dfa5e508ed459b), [f88c990](https://github.com/morluto/rea/commit/f88c99021b88ce67e513025901f57478d8107d0c), [49e6f00](https://github.com/morluto/rea/commit/49e6f00ed08b225abf9ddb7c762553607fdab115)) Thanks @AS9929, @JonahSimon, @Y0UW1SH.
|
||||
- Binary metadata: respect PE/.NET section extents, bound binary-plist expansion, scope keyed-archive traversal and decode app `Info.plist` files without `plutil`. ([#1678](https://github.com/morluto/rea/pull/1678), [#1515](https://github.com/morluto/rea/pull/1515), [#1378](https://github.com/morluto/rea/pull/1378), [#1476](https://github.com/morluto/rea/pull/1476), [975dff2](https://github.com/morluto/rea/commit/975dff2e25b14b634e14dd5e0db1537d0ca9815b), [76296c5](https://github.com/morluto/rea/commit/76296c5a3868ba3b54be6ed3896719283d6de5f8)) Thanks @jayhemnani9910, @kaoru0822-kitauji, @rudycelekli.
|
||||
- Windows runtimes: preserve drive-root paths when creating private provider directories. ([ee46323](https://github.com/morluto/rea/commit/ee46323b493994902b04824a7dc8aae2a8117903))
|
||||
- Compatibility: load Windows native controls under Node-API hosts with renamed executables, preserve dylib directory requirements, support Wakaru 1.x and reject directory-valued Hopper launchers. ([#1531](https://github.com/morluto/rea/pull/1531), [#1623](https://github.com/morluto/rea/pull/1623), [#1540](https://github.com/morluto/rea/pull/1540), [#1426](https://github.com/morluto/rea/pull/1426)) Thanks @evenafterallisaid, @Fazmin, @pionxzh, @xeropresence.
|
||||
- Reference imports: normalize internal symlink lookup consistently, preserve exact paths and exclusion provenance, report root access failures and reject FIFO replacements without blocking. ([#1535](https://github.com/morluto/rea/pull/1535), [2ca3ad2](https://github.com/morluto/rea/commit/2ca3ad238608adfca80a7ff1aaadea6e59a58225), [6636045](https://github.com/morluto/rea/commit/6636045c9c5b472bba51afee9d434394861e93a4), [3967cfa](https://github.com/morluto/rea/commit/3967cfa43e786c65a57e89c265343983d19ef9d0), [f1c58cc](https://github.com/morluto/rea/commit/f1c58cc53fbd83b968e11b3e2eccebca51299c54)) Thanks @anujsingh-cse.
|
||||
- Setup: honor active configuration and skill roots, preserve configured MCP environments, validate Hermes/OMP profiles before writing, recognize Omarchy and retain recovery backups. ([#1504](https://github.com/morluto/rea/pull/1504), [#1607](https://github.com/morluto/rea/pull/1607), [#1528](https://github.com/morluto/rea/pull/1528), [#1562](https://github.com/morluto/rea/pull/1562), [#1662](https://github.com/morluto/rea/pull/1662), [a011485](https://github.com/morluto/rea/commit/a01148553e2a37c72ae78405418ab648876a96a6), [#1606](https://github.com/morluto/rea/issues/1606), [#1521](https://github.com/morluto/rea/issues/1521), [#1561](https://github.com/morluto/rea/issues/1561)) Thanks @Adkid-Zephyr, @CassieFox23, @LuisTaffelli, @N0zoM1z0, @peter-research.
|
||||
- Process capture: retain descendant lineage after launcher exit, preserve identity uncertainty before signaling and release terminals after observation failures. ([80896a1](https://github.com/morluto/rea/commit/80896a1d64b216ef893907249cd4252f86215d49), [c916850](https://github.com/morluto/rea/commit/c9168508527302ac57bc86385ccf0d824697b5dc)) Thanks @Y0UW1SH.
|
||||
- CLI/MCP errors: accept JSON files for web comparisons, preserve native option help and union correction details, remove invalid aliases and validate doctor against its discovered output contract. ([#1397](https://github.com/morluto/rea/pull/1397), [#1395](https://github.com/morluto/rea/pull/1395), [#1425](https://github.com/morluto/rea/pull/1425), [#1461](https://github.com/morluto/rea/pull/1461), [8b4c96e](https://github.com/morluto/rea/commit/8b4c96e6c6b3ed6806e4cd72b556b5db6114304a), [#1421](https://github.com/morluto/rea/issues/1421), [#1454](https://github.com/morluto/rea/issues/1454)) Thanks @Fazmin, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
- WebSocket results: use `network.websocket_connections` instead of `network.websocket_events`. Connections contain `request_id`, `url` and `events`; allow `payload_bytes: null` when size is unknown. No legacy alias remains. ([db4e360](https://github.com/morluto/rea/commit/db4e3608465486466591b594cf77037ac150c988))
|
||||
- Direct resource APIs: pass a caller-owned `ArtifactResourceScope` to inventory/extraction, JavaScript analysis, reference imports and web exports; retain it through cleanup/shutdown. For `readJavaScriptArtifactFiles`, scope is argument three and abort signal is argument four. Keep each root inventory source’s single cleanup owner. Use `OwnedFileHandle` owners, injected ASAR `fd`/`read` handles instead of `createReadStream`, VCS `Result` values, and `WebScriptExportError`’s options object. CLI/MCP inputs for the scoped operations are unchanged. ([ffe0574](https://github.com/morluto/rea/commit/ffe0574e17b9d973860371083cb1a351be912131), [53e7fad](https://github.com/morluto/rea/commit/53e7fadbd078422ba3d55b5499a70d237c5f4908), [684199b](https://github.com/morluto/rea/commit/684199ba0be7822266d57ddb8a40e95f087fdc42), [be75ed5](https://github.com/morluto/rea/commit/be75ed5e4b4bfb146af497f8fa5d172a1ea19462), [be6272f](https://github.com/morluto/rea/commit/be6272f26115bd8e94f2ab0df4cf092004587a18), [e0e49e9](https://github.com/morluto/rea/commit/e0e49e9dd53d9d1f1d2330b23f7f880dc1c81774), [95a77b6](https://github.com/morluto/rea/commit/95a77b6e25420b3879bd0e32bdbb14d452b38776))
|
||||
- Saved reference graphs: exclusions require their matching `pattern` and distinguish `project-ignored` from `default-ignored`. Re-import graphs missing that provenance. ([6585293](https://github.com/morluto/rea/commit/6585293c562853b95c3a70df02c12ad8cc17afc4))
|
||||
- Terminal captures: keep initial and resized buffers within 1,000,000 cells using `columns × (rows + scrollback)`; oversized scenarios are rejected before launch. ([f9ea9a9](https://github.com/morluto/rea/commit/f9ea9a9268c854d31048da9f8a50cfa37703fe5a))
|
||||
- Optional agent-verification reports: accept schema version 4, nullable `expectedFirstTool` and `targetClarificationPassed`. Production CLI/MCP contracts are unchanged. ([#1613](https://github.com/morluto/rea/pull/1613))
|
||||
- Capture comparisons: recapture both sides with matching `literal-replacements-v1` semantics when custom normalization patterns are used. Legacy unversioned pattern captures remain readable but cannot be compared with new pattern captures; captures without patterns are unaffected. ([#1456](https://github.com/morluto/rea/pull/1456), [2b6f14f](https://github.com/morluto/rea/commit/2b6f14f528297cf12c776c960af0859f5cfbefbf)) Thanks @Madhavan006, @N0zoM1z0.
|
||||
- Artifact graph edges: read child artifact IDs and paths from the occurrence selected by `occurrence_id`, not edge fields `child_artifact_id`/`logical_path`. Earlier inventory Evidence no longer validates against the new commitment. ([#1493](https://github.com/morluto/rea/pull/1493)) Thanks @kaoru0822-kitauji.
|
||||
- Semantic graph schema: nodes and relations no longer include `identifier_strategy`, and relation IDs no longer hash it. Earlier Evidence no longer validates against this schema. ([#1474](https://github.com/morluto/rea/pull/1474)) Thanks @kaoru0822-kitauji.
|
||||
- Semantic identities: select IDs from the parent graph; read exact event names from `event_name`, endpoints from `endpoint` and literal values from `properties.value`. New analyses use digest-based IDs for long/reserved event and endpoint names and `value-sha256` literal role keys, with shorter labels; string literal labels now read `string literal`. Previously stored graphs retain their original IDs and remain valid workflow inputs. ([#1427](https://github.com/morluto/rea/pull/1427), [#1423](https://github.com/morluto/rea/pull/1423), [#1407](https://github.com/morluto/rea/pull/1407)) Thanks @N0zoM1z0.
|
||||
|
||||
### Contributors
|
||||
|
||||
- Thanks also to @BaaRai, @commit-regrets, @coygeek, @Franky100-pig, @inii-sayaa, @jopemachine, @JosephDemarest, @keemsisi, @Totopo27 for contributions included in this release.
|
||||
|
||||
Full contribution record: [336 commits and 163 merged PRs](https://github.com/morluto/rea/blob/rea-agents-6.4.0/docs/releases/6.4.0.contributions.json).
|
||||
|
||||
## [6.3.0](https://github.com/morluto/rea/compare/rea-agents-6.2.0...rea-agents-6.3.0) (2026-10-09)
|
||||
|
||||
|
||||
|
||||
+389
-184
@@ -2004,9 +2004,20 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
);
|
||||
JumpTable.LoadTable[] loadTables = jumpTable.getLoadTables();
|
||||
RecoveredAArch64RelativeTable recoveredAArch64 =
|
||||
loadTables.length == 0
|
||||
? recoverAArch64RelativeTable(jumpTable)
|
||||
: null;
|
||||
recoverAArch64RelativeTable(jumpTable);
|
||||
if (recoveredAArch64 != null && loadTables.length > 0) {
|
||||
// A decoded table only adds evidence when Ghidra's own load table
|
||||
// describes the same bytes; any disagreement keeps Ghidra's view.
|
||||
boolean agrees = false;
|
||||
for (JumpTable.LoadTable loadTable : loadTables) {
|
||||
if (loadTable.getAddress().equals(recoveredAArch64.tableAddress) &&
|
||||
loadTable.getSize() == recoveredAArch64.entrySize &&
|
||||
loadTable.getNum() == recoveredAArch64.mappings.size()) {
|
||||
agrees = true;
|
||||
}
|
||||
}
|
||||
if (!agrees) recoveredAArch64 = null;
|
||||
}
|
||||
List<Address> referencedDataAddresses = loadTables.length == 0
|
||||
? dispatchPathDataReferences(jumpTable)
|
||||
: List.of();
|
||||
@@ -2040,7 +2051,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
source.add("evidence", evidence);
|
||||
dataSources.add(source);
|
||||
}
|
||||
if (recoveredAArch64 != null) {
|
||||
if (recoveredAArch64 != null && loadTables.length == 0) {
|
||||
JsonObject source = new JsonObject();
|
||||
source.addProperty(
|
||||
"address",
|
||||
@@ -2055,9 +2066,16 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
inferenceEvidence(
|
||||
"jump-table",
|
||||
"ghidra-instruction-and-memory",
|
||||
"Decoded " + recoveredAArch64.mappings.size() +
|
||||
"Decoded " + recoveredAArch64.mappings.size() + " " +
|
||||
(recoveredAArch64.signedEntries ? "signed" : "unsigned") +
|
||||
" entries of size " + recoveredAArch64.entrySize + " bytes from " + canonicalAddress(recoveredAArch64.tableAddress) +
|
||||
"; the AArch64 LDRB-or-LDRH/ADR/ADD/BR chain uses a selector guarded by an unsigned upper-bound branch."
|
||||
"; the AArch64 ADRP/ADD table, indexed load, ADR base, ADD" +
|
||||
(recoveredAArch64.offsetShift == 0 ? "" : " LSL #" + recoveredAArch64.offsetShift) +
|
||||
"/BR chain uses a selector bounded by CMP #" + recoveredAArch64.compareImmediate +
|
||||
" and " + recoveredAArch64.guardMnemonic + " on one straight-line path" +
|
||||
(recoveredAArch64.nullTarget == null
|
||||
? "."
|
||||
: "; zero entries are routed by CBZ to " + canonicalAddress(recoveredAArch64.nullTarget) + ".")
|
||||
)
|
||||
);
|
||||
source.add("evidence", evidence);
|
||||
@@ -2096,6 +2114,33 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
result.add("data_sources", dataSources);
|
||||
Address[] targets = jumpTable.getCases();
|
||||
TypedJumpTableCases typedCases = typedJumpTableCases(jumpTable, caseTokens);
|
||||
// Table indices identify the register bounded by CMP. Typed labels may
|
||||
// instead identify a selector before normalization (for example, SUB
|
||||
// index, selector, #10). Do not merge those distinct value identities.
|
||||
if (recoveredAArch64 != null && recoveredAArch64.priorSelectorDefinition != null) {
|
||||
typedCases.limitations.add(
|
||||
"REA decoded the relative table, but the compared selector has an earlier register definition at " +
|
||||
canonicalAddress(recoveredAArch64.priorSelectorDefinition) + "; its table indices are not proven to use the typed switch selector's case values. Ghidra's typed labels are retained and additional case labels remain unknown."
|
||||
);
|
||||
recoveredAArch64 = null;
|
||||
}
|
||||
if (recoveredAArch64 != null) {
|
||||
for (Map.Entry<Long, Address> typed : typedCases.cases.entrySet()) {
|
||||
long value = typed.getKey();
|
||||
if (value < 0 || value >= recoveredAArch64.mappings.size()) {
|
||||
typedCases.limitations.add("Typed case labels and decoded table indices do not use a verified common selector; Ghidra's typed labels are retained.");
|
||||
recoveredAArch64 = null;
|
||||
break;
|
||||
}
|
||||
RecoveredAArch64Mapping decoded = recoveredAArch64.mappings.get((int) value);
|
||||
boolean divertedZero = decoded.nullEntry && typed.getValue().equals(recoveredAArch64.unreachableZeroTarget);
|
||||
if (!typed.getValue().equals(decoded.targetAddress) && !divertedZero) {
|
||||
typedCases.limitations.add("Typed case evidence and the decoded relative table disagree on a destination; Ghidra's typed labels are retained.");
|
||||
recoveredAArch64 = null;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
JsonArray mappings = new JsonArray();
|
||||
JsonArray defaultTargets = new JsonArray();
|
||||
for (Address target : typedCases.defaults) {
|
||||
@@ -2103,27 +2148,30 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
}
|
||||
Set<Address> coveredTargets = new HashSet<>(typedCases.cases.values());
|
||||
coveredTargets.addAll(typedCases.defaults);
|
||||
if (recoveredAArch64 != null && recoveredAArch64.unreachableZeroTarget != null) {
|
||||
Address unreachable = recoveredAArch64.unreachableZeroTarget;
|
||||
coveredTargets.add(unreachable);
|
||||
for (RecoveredAArch64Mapping recovered : recoveredAArch64.mappings) {
|
||||
if (recovered.nullEntry && unreachable.equals(typedCases.cases.get((long) recovered.tableIndex))) {
|
||||
typedCases.cases.remove((long) recovered.tableIndex);
|
||||
}
|
||||
}
|
||||
typedCases.limitations.add(
|
||||
"Ghidra recovered destination " + canonicalAddress(unreachable) +
|
||||
" as the zero-offset decode of the relative table, but the CBZ before the dispatch diverts zero entries to " +
|
||||
canonicalAddress(recoveredAArch64.nullTarget) + ", so the indirect branch never reaches it for an in-range case."
|
||||
);
|
||||
}
|
||||
if (recoveredAArch64 != null) {
|
||||
for (int index = 0; index < recoveredAArch64.mappings.size(); index += 1) {
|
||||
monitor.checkCancelled();
|
||||
RecoveredAArch64Mapping recovered = recoveredAArch64.mappings.get(index);
|
||||
if (
|
||||
typedCases.unknownTargets.contains(recovered.targetAddress) ||
|
||||
typedCases.ambiguousValues.contains((long) recovered.caseValue)
|
||||
typedCases.ambiguousValues.contains((long) recovered.tableIndex)
|
||||
) continue;
|
||||
Address typedTarget = typedCases.cases.get((long) recovered.caseValue);
|
||||
if (typedTarget != null && !typedTarget.equals(recovered.targetAddress)) {
|
||||
typedCases.cases.remove((long) recovered.caseValue);
|
||||
typedCases.ambiguousValues.add((long) recovered.caseValue);
|
||||
typedCases.unknownTargets.add(typedTarget);
|
||||
typedCases.unknownTargets.add(recovered.targetAddress);
|
||||
typedCases.limitations.add(
|
||||
"Typed case evidence and the verified relative table disagree on a destination; both destinations remain unknown for that case."
|
||||
);
|
||||
continue;
|
||||
}
|
||||
JsonObject mapping = new JsonObject();
|
||||
mapping.addProperty("case_value", recovered.caseValue);
|
||||
mapping.addProperty("case_value", recovered.tableIndex);
|
||||
String targetAddress = canonicalAddress(recovered.targetAddress);
|
||||
mapping.addProperty("target_address", targetAddress);
|
||||
mapping.addProperty("confidence", "high");
|
||||
@@ -2132,14 +2180,18 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
inferenceEvidence(
|
||||
"jump-table",
|
||||
"ghidra-instruction-and-memory",
|
||||
"Case value " + recovered.caseValue + (recoveredAArch64.entrySize == 1 ? " indexes byte " : " indexes halfword at byte offset ") +
|
||||
(recovered.caseValue * recoveredAArch64.entrySize) + " at " + canonicalAddress(recoveredAArch64.tableAddress) +
|
||||
"; the loaded unsigned entry selects a four-byte instruction offset from " +
|
||||
canonicalAddress(recoveredAArch64.branchBase) + ", reaching " + targetAddress + "."
|
||||
"Case value " + recovered.tableIndex + (recoveredAArch64.entrySize == 1 ? " indexes byte " : " indexes halfword at byte offset ") +
|
||||
(recovered.tableIndex * recoveredAArch64.entrySize) + " at " + canonicalAddress(recoveredAArch64.tableAddress) +
|
||||
(recovered.nullEntry
|
||||
? "; the entry is zero, so the CBZ before the dispatch branches to " + targetAddress + "."
|
||||
: "; the loaded " + (recoveredAArch64.signedEntries ? "signed" : "unsigned") +
|
||||
" entry " + recovered.encodedEntry + " selects a " +
|
||||
(recoveredAArch64.offsetShift == 2 ? "four-byte instruction" : "byte") +
|
||||
" offset from " + canonicalAddress(recoveredAArch64.branchBase) + ", reaching " + targetAddress + ".")
|
||||
)
|
||||
);
|
||||
mapping.add("evidence", evidence);
|
||||
if (!typedCases.cases.containsKey((long) recovered.caseValue)) {
|
||||
if (!typedCases.cases.containsKey((long) recovered.tableIndex)) {
|
||||
mappings.add(mapping);
|
||||
}
|
||||
coveredTargets.add(recovered.targetAddress);
|
||||
@@ -2213,7 +2265,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
);
|
||||
}
|
||||
if (recoveredAArch64 == null && loadTables.length == 0 && currentProgram.getLanguageID().getIdAsString().startsWith("AARCH64:")) {
|
||||
limitations.add("REA did not verify a supported byte/halfword relative dispatch chain with exact unsigned bounds and register definitions; unsupported encodings and ambiguous bounds remain unknown.");
|
||||
limitations.add("REA did not verify a supported signed or unsigned byte/halfword relative dispatch chain with exact unsigned bounds and register definitions; unsupported encodings and ambiguous bounds remain unknown.");
|
||||
}
|
||||
if (targets.length == 0) {
|
||||
limitations.add(
|
||||
@@ -2343,12 +2395,21 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
}
|
||||
|
||||
private static final class RecoveredAArch64Mapping {
|
||||
private final int caseValue;
|
||||
private final int tableIndex;
|
||||
private final Address targetAddress;
|
||||
private final long encodedEntry;
|
||||
private final boolean nullEntry;
|
||||
|
||||
private RecoveredAArch64Mapping(int caseValue, Address targetAddress) {
|
||||
this.caseValue = caseValue;
|
||||
private RecoveredAArch64Mapping(
|
||||
int tableIndex,
|
||||
Address targetAddress,
|
||||
long encodedEntry,
|
||||
boolean nullEntry
|
||||
) {
|
||||
this.tableIndex = tableIndex;
|
||||
this.targetAddress = targetAddress;
|
||||
this.encodedEntry = encodedEntry;
|
||||
this.nullEntry = nullEntry;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2356,24 +2417,70 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
private final Address tableAddress;
|
||||
private final Address branchBase;
|
||||
private final int entrySize;
|
||||
private final Address priorSelectorDefinition;
|
||||
private final boolean signedEntries;
|
||||
private final int offsetShift;
|
||||
private final String guardMnemonic;
|
||||
private final long compareImmediate;
|
||||
private final Address nullTarget;
|
||||
private final Address unreachableZeroTarget;
|
||||
private final List<RecoveredAArch64Mapping> mappings;
|
||||
|
||||
private RecoveredAArch64RelativeTable(
|
||||
Address tableAddress,
|
||||
Address branchBase,
|
||||
int entrySize,
|
||||
Address priorSelectorDefinition,
|
||||
boolean signedEntries,
|
||||
int offsetShift,
|
||||
String guardMnemonic,
|
||||
long compareImmediate,
|
||||
Address nullTarget,
|
||||
Address unreachableZeroTarget,
|
||||
List<RecoveredAArch64Mapping> mappings
|
||||
) {
|
||||
this.tableAddress = tableAddress;
|
||||
this.branchBase = branchBase;
|
||||
this.entrySize = entrySize;
|
||||
this.priorSelectorDefinition = priorSelectorDefinition;
|
||||
this.signedEntries = signedEntries;
|
||||
this.offsetShift = offsetShift;
|
||||
this.guardMnemonic = guardMnemonic;
|
||||
this.compareImmediate = compareImmediate;
|
||||
this.nullTarget = nullTarget;
|
||||
this.unreachableZeroTarget = unreachableZeroTarget;
|
||||
this.mappings = mappings;
|
||||
}
|
||||
}
|
||||
|
||||
private static final Pattern AARCH64_DISPATCH_ADD = Pattern.compile(
|
||||
"(?i)^([wx]\\d+),\\s*([wx]\\d+),\\s*([wx]\\d+)(?:,\\s*LSL\\s*#(?:0x)?([0-9a-f]+))?$"
|
||||
);
|
||||
private static final Pattern AARCH64_TABLE_LOAD = Pattern.compile(
|
||||
"(?i)^([wx]\\d+),\\s*\\[([wx]\\d+),\\s*([wx]\\d+)(?:,\\s*(LSL|UXTW)\\s*(?:#(?:0x)?([0-9a-f]+))?)?\\s*\\]$"
|
||||
);
|
||||
private static final Pattern AARCH64_REGISTER_AND_ADDRESS = Pattern.compile(
|
||||
"(?i)^([wx]\\d+),\\s*(0x[0-9a-f]+)$"
|
||||
);
|
||||
private static final Pattern AARCH64_ADD_IMMEDIATE = Pattern.compile(
|
||||
"(?i)^([wx]\\d+),\\s*([wx]\\d+),\\s*#?(0x[0-9a-f]+|[0-9]+)$"
|
||||
);
|
||||
private static final Pattern AARCH64_COMPARE_IMMEDIATE = Pattern.compile(
|
||||
"(?i)^([wx]\\d+),\\s*#?(0x[0-9a-f]+|[0-9]+)$"
|
||||
);
|
||||
private static final Pattern AARCH64_MOVE_REGISTER = Pattern.compile(
|
||||
"(?i)^([wx]\\d+),\\s*([wx]\\d+)$"
|
||||
);
|
||||
private static final Set<String> AARCH64_FLAG_PRESERVING =
|
||||
Set.of("MOV", "ADRP", "ADR", "ADD");
|
||||
|
||||
/**
|
||||
* Recover only the directly verified AArch64 byte/halfword-indexed relative branch
|
||||
* table form. Other instruction sequences keep their case labels unknown.
|
||||
* Recover only directly verified AArch64 relative branch tables:
|
||||
* ADRP/ADD table base, LDRB/LDRH/LDRSB/LDRSH entry load with an exact
|
||||
* index scale, ADR branch base, optional CBZ on the loaded entry, and
|
||||
* ADD (byte or LSL #2 scale)/BR, with a selector bounded by CMP #imm and an
|
||||
* unsigned B.HI or B.CS/B.HS guard on one straight-line path. Any other
|
||||
* instruction sequence keeps its case labels unknown.
|
||||
*/
|
||||
private RecoveredAArch64RelativeTable recoverAArch64RelativeTable(
|
||||
JumpTable jumpTable
|
||||
@@ -2395,167 +2502,265 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
if (instruction.getAddress().equals(jumpTable.getSwitchAddress())) break;
|
||||
}
|
||||
int branchIndex = instructions.size() - 1;
|
||||
if (branchIndex < 1 ||
|
||||
!instructions.get(branchIndex).getMnemonicString().equalsIgnoreCase("BR")) {
|
||||
if (branchIndex < 1 || !mnemonic(instructions.get(branchIndex)).equals("BR")) {
|
||||
return null;
|
||||
}
|
||||
Set<String> recoveredTargetSet = new HashSet<>();
|
||||
for (Address target : jumpTable.getCases()) recoveredTargetSet.add(canonicalAddress(target));
|
||||
if (recoveredTargetSet.isEmpty()) return null;
|
||||
return matchAArch64RelativeDispatch(
|
||||
instructions,
|
||||
branchIndex,
|
||||
recoveredTargetSet,
|
||||
dispatchPathDataReferences(jumpTable)
|
||||
);
|
||||
}
|
||||
|
||||
private RecoveredAArch64RelativeTable matchAArch64RelativeDispatch(
|
||||
List<Instruction> instructions,
|
||||
int branchIndex,
|
||||
Set<String> recoveredTargetSet,
|
||||
List<Address> dataAddresses
|
||||
) throws Exception {
|
||||
String branchRegister = normalizedRegister(operands(instructions.get(branchIndex)));
|
||||
|
||||
// ADD Xd, Xn, Xm{, LSL #s}; BR Xd. LSL applies to Xm only.
|
||||
Instruction dispatchAdd = instructions.get(branchIndex - 1);
|
||||
if (!mnemonic(dispatchAdd).equals("ADD")) return null;
|
||||
java.util.regex.Matcher addMatcher = AARCH64_DISPATCH_ADD.matcher(operands(dispatchAdd));
|
||||
if (!addMatcher.matches() || !addMatcher.group(1).toLowerCase(Locale.ROOT).startsWith("x")) return null;
|
||||
if (!normalizedRegister(addMatcher.group(1)).equals(branchRegister)) return null;
|
||||
int offsetShift = addMatcher.group(4) == null ? 0 : Integer.parseInt(addMatcher.group(4), 16);
|
||||
if (offsetShift != 0 && offsetShift != 2) return null;
|
||||
String firstSource = normalizedRegister(addMatcher.group(2));
|
||||
String secondSource = normalizedRegister(addMatcher.group(3));
|
||||
if (firstSource.equals(secondSource)) return null;
|
||||
|
||||
// The entry register is the source whose latest definition is the
|
||||
// table load; the other source must be defined by ADR.
|
||||
int loadIndex = -1;
|
||||
String entryRegister = null;
|
||||
String baseRegister = null;
|
||||
for (String candidate : offsetShift == 0 ? List.of(secondSource, firstSource) : List.of(secondSource)) {
|
||||
int writer = latestWriter(instructions, branchIndex - 1, candidate);
|
||||
if (writer >= 0 && Set.of("LDRB", "LDRH", "LDRSB", "LDRSH").contains(mnemonic(instructions.get(writer)))) {
|
||||
loadIndex = writer;
|
||||
entryRegister = candidate;
|
||||
baseRegister = candidate.equals(firstSource) ? secondSource : firstSource;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (loadIndex < 0) return null;
|
||||
|
||||
Instruction load = instructions.get(loadIndex);
|
||||
String loadMnemonic = mnemonic(load);
|
||||
int entrySize = loadMnemonic.endsWith("B") ? 1 : 2;
|
||||
boolean signedEntries = loadMnemonic.startsWith("LDRS");
|
||||
java.util.regex.Matcher loadMatcher = AARCH64_TABLE_LOAD.matcher(operands(load));
|
||||
if (!loadMatcher.matches()) return null;
|
||||
// A signed load into a W register sign-extends only to 32 bits; the
|
||||
// 64-bit dispatch ADD would then read a zero-extended value.
|
||||
if (signedEntries && !loadMatcher.group(1).toLowerCase(Locale.ROOT).startsWith("x")) return null;
|
||||
String tableBaseRegister = normalizedRegister(loadMatcher.group(2));
|
||||
String indexRegister = normalizedRegister(loadMatcher.group(3));
|
||||
boolean wordIndex = loadMatcher.group(3).toLowerCase(Locale.ROOT).startsWith("w");
|
||||
String extend = loadMatcher.group(4) == null ? null : loadMatcher.group(4).toUpperCase(Locale.ROOT);
|
||||
int indexShift = loadMatcher.group(5) == null ? 0 : Integer.parseInt(loadMatcher.group(5), 16);
|
||||
if (wordIndex ? !"UXTW".equals(extend) : "UXTW".equals(extend)) return null;
|
||||
if (indexShift != (entrySize == 1 ? 0 : 1)) return null;
|
||||
|
||||
// Only ADR and one CBZ on the entry may sit between load and ADD.
|
||||
Address nullTarget = null;
|
||||
for (int index = loadIndex + 1; index < branchIndex - 1; index += 1) {
|
||||
Instruction instruction = instructions.get(index);
|
||||
String name = mnemonic(instruction);
|
||||
if (name.equals("ADR")) continue;
|
||||
if (name.equals("CBZ") && nullTarget == null) {
|
||||
String[] parts = operands(instruction).split(",");
|
||||
Address[] flows = instruction.getFlows();
|
||||
if (parts.length != 2 || flows.length != 1 ||
|
||||
!normalizedRegister(parts[0]).equals(entryRegister)) return null;
|
||||
nullTarget = flows[0];
|
||||
continue;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
Address[] recoveredTargets = jumpTable.getCases();
|
||||
Set<String> recoveredTargetSet = new HashSet<>();
|
||||
for (Address target : recoveredTargets) recoveredTargetSet.add(canonicalAddress(target));
|
||||
List<Address> dataAddresses = dispatchPathDataReferences(jumpTable);
|
||||
int baseIndex = latestWriter(instructions, branchIndex - 1, baseRegister);
|
||||
if (baseIndex < 0 || !mnemonic(instructions.get(baseIndex)).equals("ADR")) return null;
|
||||
java.util.regex.Matcher baseMatcher = AARCH64_REGISTER_AND_ADDRESS.matcher(operands(instructions.get(baseIndex)));
|
||||
if (!baseMatcher.matches()) return null;
|
||||
Address branchBase = addressFromOperand(baseMatcher.group(2));
|
||||
|
||||
for (int loadIndex = 0; loadIndex < branchIndex; loadIndex += 1) {
|
||||
Instruction load = instructions.get(loadIndex);
|
||||
int entrySize = load.getMnemonicString().equalsIgnoreCase("LDRB") ? 1 : load.getMnemonicString().equalsIgnoreCase("LDRH") ? 2 : 0;
|
||||
if (entrySize == 0 || branchIndex != loadIndex + 2 || loadIndex < 3 ||
|
||||
!instructions.get(loadIndex - 1).getMnemonicString().equalsIgnoreCase("ADR") ||
|
||||
!instructions.get(loadIndex - 2).getMnemonicString().equalsIgnoreCase("ADD") ||
|
||||
!instructions.get(loadIndex - 3).getMnemonicString().equalsIgnoreCase("ADRP")) continue;
|
||||
String loadOperands = operands(load);
|
||||
java.util.regex.Matcher loadMatcher = java.util.regex.Pattern
|
||||
.compile(entrySize == 1 ? "(?i)^[wx]\\d+,\\s*\\[([wx]\\d+),\\s*([wx]\\d+)\\]$" : "(?i)^[wx]\\d+,\\s*\\[([wx]\\d+),\\s*([wx]\\d+),\\s*LSL\\s*#(?:0x)?1\\]$")
|
||||
.matcher(loadOperands);
|
||||
if (!loadMatcher.matches()) continue;
|
||||
String loadedRegister = normalizedRegister(loadOperands.substring(0, loadOperands.indexOf(',')));
|
||||
String tableBaseRegister = normalizedRegister(loadMatcher.group(1));
|
||||
String indexRegister = normalizedRegister(loadMatcher.group(2));
|
||||
int tableAddIndex = latestWriter(instructions, loadIndex, tableBaseRegister);
|
||||
if (tableAddIndex < 0 || !mnemonic(instructions.get(tableAddIndex)).equals("ADD")) return null;
|
||||
java.util.regex.Matcher tableAddMatcher = AARCH64_ADD_IMMEDIATE.matcher(operands(instructions.get(tableAddIndex)));
|
||||
if (!tableAddMatcher.matches() ||
|
||||
!normalizedRegister(tableAddMatcher.group(1)).equals(tableBaseRegister) ||
|
||||
!normalizedRegister(tableAddMatcher.group(2)).equals(tableBaseRegister)) return null;
|
||||
int tablePageIndex = latestWriter(instructions, tableAddIndex, tableBaseRegister);
|
||||
if (tablePageIndex < 0 || !mnemonic(instructions.get(tablePageIndex)).equals("ADRP")) return null;
|
||||
java.util.regex.Matcher pageMatcher = AARCH64_REGISTER_AND_ADDRESS.matcher(operands(instructions.get(tablePageIndex)));
|
||||
if (!pageMatcher.matches() || !normalizedRegister(pageMatcher.group(1)).equals(tableBaseRegister)) return null;
|
||||
Address tableAddress = addressFromOperand(pageMatcher.group(2)).add(parseInteger(tableAddMatcher.group(3)));
|
||||
if (!currentProgram.getMemory().contains(tableAddress) ||
|
||||
(!dataAddresses.isEmpty() && !dataAddresses.contains(tableAddress))) return null;
|
||||
|
||||
Address resolvedTableAddress = null;
|
||||
for (int index = loadIndex - 3; index < loadIndex - 1; index += 1) {
|
||||
Instruction instruction = instructions.get(index);
|
||||
String mnemonic = instruction.getMnemonicString().toUpperCase(Locale.ROOT);
|
||||
String text = operands(instruction);
|
||||
if (mnemonic.equals("ADRP") &&
|
||||
text.matches("(?i)^[wx]\\d+,\\s*0x[0-9a-f]+$")) {
|
||||
String[] parts = text.split(",");
|
||||
if (normalizedRegister(parts[0]).equals(tableBaseRegister)) {
|
||||
resolvedTableAddress = addressFromOperand(parts[1]);
|
||||
}
|
||||
}
|
||||
else if (mnemonic.equals("ADD") && resolvedTableAddress != null &&
|
||||
text.matches("(?i)^[wx]\\d+,\\s*[wx]\\d+,\\s*#?(?:0x[0-9a-f]+|[0-9]+)$")) {
|
||||
String[] parts = text.split(",");
|
||||
if (normalizedRegister(parts[0]).equals(tableBaseRegister) &&
|
||||
normalizedRegister(parts[1]).equals(tableBaseRegister)) {
|
||||
resolvedTableAddress = resolvedTableAddress.add(parseInteger(parts[2].trim().replaceFirst("^#", "")));
|
||||
}
|
||||
}
|
||||
// Unsigned guard: CMP Rs, #imm, then B.HI (imm + 1 entries) or
|
||||
// B.CS/B.HS (imm entries), with only flag-preserving moves and
|
||||
// address arithmetic between the compare and the load.
|
||||
int guardIndex = -1;
|
||||
for (int index = loadIndex - 1; index >= 0; index -= 1) {
|
||||
String name = mnemonic(instructions.get(index));
|
||||
if (name.startsWith("B.")) {
|
||||
guardIndex = index;
|
||||
break;
|
||||
}
|
||||
if (resolvedTableAddress == null || !currentProgram.getMemory().contains(resolvedTableAddress) ||
|
||||
(!dataAddresses.isEmpty() && !dataAddresses.contains(resolvedTableAddress))) continue;
|
||||
Address tableAddress = resolvedTableAddress;
|
||||
|
||||
Address branchBase = null;
|
||||
boolean addressChain = false;
|
||||
for (int index = loadIndex - 1; index < branchIndex; index += 1) {
|
||||
Instruction instruction = instructions.get(index);
|
||||
String mnemonic = instruction.getMnemonicString().toUpperCase(Locale.ROOT);
|
||||
String text = operands(instruction);
|
||||
if (index < loadIndex && mnemonic.equals("ADR") && text.matches("(?i)^[wx]\\d+,\\s*0x[0-9a-f]+$")) {
|
||||
int comma = text.indexOf(',');
|
||||
String destination = normalizedRegister(text.substring(0, comma));
|
||||
if (!destination.equals(normalizedRegister(operands(instructions.get(branchIndex))))) continue;
|
||||
branchBase = addressFromOperand(text.substring(comma + 1));
|
||||
}
|
||||
if (index > loadIndex && mnemonic.equals("ADD") && branchBase != null &&
|
||||
text.matches("(?i)^[wx]\\d+,\\s*[wx]\\d+,\\s*[wx]\\d+,\\s*LSL\\s*#(?:0x)?2$")) {
|
||||
String[] parts = text.split(",");
|
||||
String destination = normalizedRegister(parts[0]);
|
||||
String base = normalizedRegister(parts[1]);
|
||||
String offset = normalizedRegister(parts[2]);
|
||||
if (destination.equals(base) &&
|
||||
base.equals(normalizedRegister(operands(instructions.get(branchIndex)))) &&
|
||||
offset.equals(loadedRegister)) {
|
||||
addressChain = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (!addressChain || branchBase == null) continue;
|
||||
|
||||
int entryCount = -1;
|
||||
Address defaultTarget = null;
|
||||
for (int compareIndex = 0; compareIndex < loadIndex; compareIndex += 1) {
|
||||
Instruction compare = instructions.get(compareIndex);
|
||||
if (!compare.getMnemonicString().equalsIgnoreCase("CMP")) continue;
|
||||
java.util.regex.Matcher compareMatcher = java.util.regex.Pattern
|
||||
.compile("(?i)^([wx]\\d+),\\s*#?(0x[0-9a-f]+|[0-9]+)$")
|
||||
.matcher(operands(compare));
|
||||
if (!compareMatcher.matches()) continue;
|
||||
String comparedRegister = normalizedRegister(compareMatcher.group(1));
|
||||
long upperBound = parseInteger(compareMatcher.group(2));
|
||||
if (upperBound < 0 || upperBound >= 4096) continue;
|
||||
for (int guardIndex = compareIndex + 1; guardIndex < loadIndex; guardIndex += 1) {
|
||||
Instruction guard = instructions.get(guardIndex);
|
||||
String mnemonic = guard.getMnemonicString().toUpperCase(Locale.ROOT);
|
||||
if (!mnemonic.equals("B.HI")) continue;
|
||||
Address[] flows = guard.getFlows();
|
||||
if (flows.length != 1 || !recoveredTargetSet.contains(canonicalAddress(flows[0]))) continue;
|
||||
boolean supportedSequence = true;
|
||||
for (int sequenceIndex = compareIndex + 1; sequenceIndex < loadIndex; sequenceIndex += 1) {
|
||||
String sequenceMnemonic = instructions.get(sequenceIndex)
|
||||
.getMnemonicString().toUpperCase(Locale.ROOT);
|
||||
if (!Set.of("B.HI", "MOV", "ADRP", "ADD", "ADR").contains(sequenceMnemonic)) {
|
||||
supportedSequence = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
Set<String> selectorAliases = new HashSet<>();
|
||||
selectorAliases.add(comparedRegister);
|
||||
Set<String> boundedIndexes = new HashSet<>();
|
||||
if (compareMatcher.group(1).toLowerCase(Locale.ROOT).startsWith("x")) boundedIndexes.add(comparedRegister);
|
||||
for (int sequenceIndex = compareIndex + 1; sequenceIndex < loadIndex; sequenceIndex++) {
|
||||
Instruction instruction = instructions.get(sequenceIndex);
|
||||
String[] parts = operands(instruction).split(",");
|
||||
if (parts.length == 0) continue;
|
||||
if (instruction.getMnemonicString().equalsIgnoreCase("MOV")) {
|
||||
String destination = normalizedRegister(parts[0]);
|
||||
String source = parts.length == 2 ? normalizedRegister(parts[1]) : "";
|
||||
boolean aliasesSelector = selectorAliases.contains(source);
|
||||
boolean fullWidthBounded = boundedIndexes.contains(source) ||
|
||||
(parts[0].trim().toLowerCase(Locale.ROOT).startsWith("w") && aliasesSelector);
|
||||
if (aliasesSelector) selectorAliases.add(destination);
|
||||
else selectorAliases.remove(destination);
|
||||
if (fullWidthBounded) boundedIndexes.add(destination);
|
||||
else boundedIndexes.remove(destination);
|
||||
} else if (Set.of("ADRP", "ADD", "ADR").contains(instruction.getMnemonicString().toUpperCase(Locale.ROOT))) {
|
||||
selectorAliases.remove(normalizedRegister(parts[0]));
|
||||
boundedIndexes.remove(normalizedRegister(parts[0]));
|
||||
}
|
||||
}
|
||||
boolean selectorFeedsTable = boundedIndexes.contains(indexRegister);
|
||||
if (compareIndex < loadIndex - 3 && selectorFeedsTable && supportedSequence) {
|
||||
entryCount = (int) upperBound + 1;
|
||||
defaultTarget = flows[0];
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (entryCount > 0) break;
|
||||
}
|
||||
if (entryCount <= 0 || entryCount > 4096 || defaultTarget == null) continue;
|
||||
|
||||
List<RecoveredAArch64Mapping> mappings = new ArrayList<>();
|
||||
Set<String> mappedTargets = new HashSet<>();
|
||||
boolean valid = true;
|
||||
for (int caseValue = 0; caseValue < entryCount; caseValue += 1) {
|
||||
Address entry = tableAddress.add((long) caseValue * entrySize);
|
||||
int encodedOffset = entrySize == 1 ? currentProgram.getMemory().getByte(entry) & 0xff : currentProgram.getMemory().getShort(entry) & 0xffff;
|
||||
Address target = branchBase.add((long) encodedOffset * 4L);
|
||||
String canonicalTarget = canonicalAddress(target);
|
||||
if (!recoveredTargetSet.contains(canonicalTarget)) {
|
||||
valid = false;
|
||||
break;
|
||||
}
|
||||
mappedTargets.add(canonicalTarget);
|
||||
mappings.add(new RecoveredAArch64Mapping(caseValue, target));
|
||||
}
|
||||
if (!valid || mappedTargets.contains(canonicalAddress(defaultTarget))) continue;
|
||||
Set<String> accountedTargets = new HashSet<>(mappedTargets);
|
||||
accountedTargets.add(canonicalAddress(defaultTarget));
|
||||
if (!accountedTargets.equals(recoveredTargetSet)) continue;
|
||||
return new RecoveredAArch64RelativeTable(tableAddress, branchBase, entrySize, List.copyOf(mappings));
|
||||
if (!AARCH64_FLAG_PRESERVING.contains(name)) return null;
|
||||
}
|
||||
return null;
|
||||
if (guardIndex < 1) return null;
|
||||
Instruction guard = instructions.get(guardIndex);
|
||||
String guardMnemonic = mnemonic(guard);
|
||||
if (!Set.of("B.HI", "B.CS", "B.HS").contains(guardMnemonic)) return null;
|
||||
Address[] guardFlows = guard.getFlows();
|
||||
if (guardFlows.length != 1) return null;
|
||||
Address defaultTarget = guardFlows[0];
|
||||
int compareIndex = -1;
|
||||
for (int index = guardIndex - 1; index >= 0; index -= 1) {
|
||||
String name = mnemonic(instructions.get(index));
|
||||
if (name.equals("CMP")) {
|
||||
compareIndex = index;
|
||||
break;
|
||||
}
|
||||
if (!AARCH64_FLAG_PRESERVING.contains(name)) return null;
|
||||
}
|
||||
if (compareIndex < 0) return null;
|
||||
java.util.regex.Matcher compareMatcher = AARCH64_COMPARE_IMMEDIATE.matcher(operands(instructions.get(compareIndex)));
|
||||
if (!compareMatcher.matches()) return null;
|
||||
String comparedRegister = normalizedRegister(compareMatcher.group(1));
|
||||
boolean comparedFullWidth = compareMatcher.group(1).toLowerCase(Locale.ROOT).startsWith("x");
|
||||
long compareImmediate = parseInteger(compareMatcher.group(2));
|
||||
long entryCountLong = guardMnemonic.equals("B.HI") ? compareImmediate + 1 : compareImmediate;
|
||||
if (entryCountLong <= 0 || entryCountLong > 4096) return null;
|
||||
int entryCount = (int) entryCountLong;
|
||||
|
||||
// The loaded index must carry the compared value: either the compared
|
||||
// register itself, unchanged since CMP, or a MOV copy made after CMP.
|
||||
// A W compare bounds only the low 32 bits, so an X index needs a
|
||||
// zero-extending W copy; a W index is zero-extended by UXTW.
|
||||
int indexWriter = latestWriter(instructions, loadIndex, indexRegister);
|
||||
boolean bounded = false;
|
||||
if (indexWriter < compareIndex) {
|
||||
bounded = indexRegister.equals(comparedRegister) && (wordIndex || comparedFullWidth);
|
||||
}
|
||||
else if (mnemonic(instructions.get(indexWriter)).equals("MOV")) {
|
||||
java.util.regex.Matcher moveMatcher = AARCH64_MOVE_REGISTER.matcher(operands(instructions.get(indexWriter)));
|
||||
if (moveMatcher.matches() &&
|
||||
normalizedRegister(moveMatcher.group(2)).equals(comparedRegister) &&
|
||||
latestWriter(instructions, indexWriter, comparedRegister) < compareIndex) {
|
||||
boolean wordCopy = moveMatcher.group(1).toLowerCase(Locale.ROOT).startsWith("w");
|
||||
bounded = wordCopy || comparedFullWidth;
|
||||
}
|
||||
}
|
||||
if (!bounded) return null;
|
||||
|
||||
// Every fact above holds only on one straight-line fall-through path
|
||||
// with no other entry into the window.
|
||||
int windowStart = Math.min(Math.min(tablePageIndex, baseIndex), compareIndex);
|
||||
for (int index = windowStart + 1; index <= branchIndex; index += 1) {
|
||||
monitor.checkCancelled();
|
||||
Instruction previous = instructions.get(index - 1);
|
||||
Instruction current = instructions.get(index);
|
||||
Address fallThrough = previous.getFallThrough();
|
||||
if (fallThrough == null || !fallThrough.equals(current.getAddress())) return null;
|
||||
ReferenceIterator incoming = currentProgram.getReferenceManager().getReferencesTo(current.getAddress());
|
||||
while (incoming.hasNext()) {
|
||||
if (incoming.next().getReferenceType().isFlow()) return null;
|
||||
}
|
||||
}
|
||||
|
||||
List<RecoveredAArch64Mapping> mappings = new ArrayList<>();
|
||||
Set<String> mappedTargets = new HashSet<>();
|
||||
for (int tableIndex = 0; tableIndex < entryCount; tableIndex += 1) {
|
||||
monitor.checkCancelled();
|
||||
Address entry = tableAddress.add((long) tableIndex * entrySize);
|
||||
if (!currentProgram.getMemory().contains(entry)) return null;
|
||||
long encoded;
|
||||
if (entrySize == 1) {
|
||||
byte value = currentProgram.getMemory().getByte(entry);
|
||||
encoded = signedEntries ? value : value & 0xffL;
|
||||
}
|
||||
else {
|
||||
short value = currentProgram.getMemory().getShort(entry);
|
||||
encoded = signedEntries ? value : value & 0xffffL;
|
||||
}
|
||||
if (encoded == 0 && nullTarget != null) {
|
||||
mappings.add(new RecoveredAArch64Mapping(tableIndex, nullTarget, encoded, true));
|
||||
continue;
|
||||
}
|
||||
Address target;
|
||||
try {
|
||||
target = branchBase.addNoWrap(encoded << offsetShift);
|
||||
}
|
||||
catch (ghidra.program.model.address.AddressOverflowException overflow) {
|
||||
return null;
|
||||
}
|
||||
String canonicalTarget = canonicalAddress(target);
|
||||
if (!recoveredTargetSet.contains(canonicalTarget)) return null;
|
||||
mappedTargets.add(canonicalTarget);
|
||||
mappings.add(new RecoveredAArch64Mapping(tableIndex, target, encoded, false));
|
||||
}
|
||||
if (mappedTargets.contains(canonicalAddress(defaultTarget))) return null;
|
||||
// Ghidra's recovered destinations must be fully accounted for by the
|
||||
// decoded entries, the guard default, and the CBZ null path.
|
||||
Set<String> accountedTargets = new HashSet<>(mappedTargets);
|
||||
accountedTargets.add(canonicalAddress(defaultTarget));
|
||||
if (nullTarget != null) accountedTargets.add(canonicalAddress(nullTarget));
|
||||
// Ghidra does not model the CBZ, so it may decode a zero entry as a
|
||||
// branch to the base itself. Only a zero entry encodes offset 0, and
|
||||
// the CBZ diverts every zero entry, so the BR never reaches the base.
|
||||
Address unreachableZeroTarget = null;
|
||||
boolean divertedZeroEntry = mappings.stream().anyMatch(mapping -> mapping.nullEntry);
|
||||
if (divertedZeroEntry &&
|
||||
recoveredTargetSet.contains(canonicalAddress(branchBase)) &&
|
||||
!mappedTargets.contains(canonicalAddress(branchBase)) &&
|
||||
!branchBase.equals(defaultTarget) &&
|
||||
!branchBase.equals(nullTarget)) {
|
||||
unreachableZeroTarget = branchBase;
|
||||
accountedTargets.add(canonicalAddress(branchBase));
|
||||
}
|
||||
if (!accountedTargets.containsAll(recoveredTargetSet)) return null;
|
||||
int selectorWriter = latestWriter(instructions, compareIndex, comparedRegister);
|
||||
return new RecoveredAArch64RelativeTable(
|
||||
tableAddress,
|
||||
branchBase,
|
||||
entrySize,
|
||||
selectorWriter < 0 ? null : instructions.get(selectorWriter).getAddress(),
|
||||
signedEntries,
|
||||
offsetShift,
|
||||
guardMnemonic,
|
||||
compareImmediate,
|
||||
nullTarget,
|
||||
unreachableZeroTarget,
|
||||
List.copyOf(mappings)
|
||||
);
|
||||
}
|
||||
|
||||
private static String mnemonic(Instruction instruction) {
|
||||
return instruction.getMnemonicString().toUpperCase(Locale.ROOT);
|
||||
}
|
||||
|
||||
/** Index of the last instruction before {@code before} that writes {@code register}, or -1. */
|
||||
private static int latestWriter(List<Instruction> instructions, int before, String register) {
|
||||
for (int index = before - 1; index >= 0; index -= 1) {
|
||||
for (Object result : instructions.get(index).getResultObjects()) {
|
||||
if (result instanceof Register written &&
|
||||
normalizedRegister(written.getBaseRegister().getName()).equals(register)) {
|
||||
return index;
|
||||
}
|
||||
}
|
||||
}
|
||||
return -1;
|
||||
}
|
||||
|
||||
private static String operands(Instruction instruction) {
|
||||
|
||||
@@ -24,6 +24,29 @@ REA can attach to a user-owned Chrome-family browser through the Chrome DevTools
|
||||
|
||||
Target discovery returns the complete in-scope target array in one result.
|
||||
|
||||
## Screenshot comparison memory budget
|
||||
|
||||
`compare_web_screenshots` admits both PNGs against a 256 MiB estimated
|
||||
working-memory budget before inflating either image. This is a processing
|
||||
buffer budget for this operation, not a PNG dimension limit, CDP transport
|
||||
limit, or process RSS guarantee. Header-valid dimensions that exceed the
|
||||
budget return `resource_constraint` with `details.resource: "memory"` and the
|
||||
estimated and maximum working-set bytes. Invalid PNG structure remains
|
||||
malformed input; encodings outside supported pixel-comparison coverage remain
|
||||
unsupported-target errors.
|
||||
|
||||
The conservative estimate includes the two decoder-owned decoded input
|
||||
buffers, the current image's concatenated IDAT buffer, Node zlib's default
|
||||
16 KiB output chunk backing rounded up for retained chunks, the concatenated
|
||||
inflate result while its output chunks are retained, one possible extra output
|
||||
chunk before zlib checks `maxOutputLength`, unfiltered scanline storage,
|
||||
RGB-to-RGBA expansion, and the first retained RGBA image while the second is
|
||||
decoded. RGBA output aliases its unfiltered buffer and is counted once.
|
||||
Admission uses checked integer arithmetic before image-sized allocation.
|
||||
Caller-owned JSON/base64 strings, input parsing and validation, transport
|
||||
buffers, garbage-collector behavior, and other runtime memory are outside this
|
||||
estimate; the budget does not promise a bound on total process memory.
|
||||
|
||||
Electron `file://` pages use a separate provider and target boundary; see [electron-observation.md](electron-observation.md).
|
||||
|
||||
## Provider authorities
|
||||
|
||||
+64
-35
@@ -1,16 +1,25 @@
|
||||
# Optional NativeAOT metadata recovery
|
||||
|
||||
REA integrates [Washi1337/ghidra-nativeaot](https://github.com/Washi1337/ghidra-nativeaot)
|
||||
through an optional headless adapter. The original source is an opt-in submodule,
|
||||
pinned to `effeb734fc570c32650f88b159608979dc7b423e`; its MIT license and existing
|
||||
source notices are retained. No upstream source or algorithms are rewritten.
|
||||
REA has a built-in read-only snapshot parser for x64 PE targets with .NET 8 RTR
|
||||
9.1 metadata. It parses the exact admitted Ghidra session snapshot and uses an
|
||||
in-memory overlay for rehydrated metadata; it does not define Ghidra DataTypes,
|
||||
labels, functions, or memory bytes. The hydration semantics are adapted from
|
||||
[Washi1337/ghidra-nativeaot](https://github.com/Washi1337/ghidra-nativeaot) at
|
||||
`effeb734fc570c32650f88b159608979dc7b423e` (MIT), cross-checked against
|
||||
.NET's `MethodTable.h` at runtime commit
|
||||
`a3b2d40328be0be3f8dd43f0e5cc925b8827b044` (MIT). The required notice ships in
|
||||
[`licenses/nativeaot-parser-MIT.txt`](../licenses/nativeaot-parser-MIT.txt).
|
||||
|
||||
The initial tested layout is .NET runtime **8.0.22**, NativeAOT RTR **9.1**,
|
||||
x86-64 ELF and native Windows PE **targets analyzed on Linux x64**, using
|
||||
Ghidra **12.1.4** and JDK **21**. The build accepts any Ghidra 12.1.x release and
|
||||
javac 21 or newer, and it compiles with `--release 21`. Real verification of
|
||||
this layout remains the Ghidra 12.1.4 and JDK 21 pair. Other NativeAOT layouts,
|
||||
architectures, and hosts are unsupported.
|
||||
The optional headless Ghidra adapter remains available for database annotations
|
||||
and broader recovery. It uses the same pinned upstream commit as an opt-in
|
||||
submodule and is a separate, more capable workflow.
|
||||
|
||||
The parser currently handles x64 PE32+ and RTR 9.1, including bounded
|
||||
DEHYDRATED_DATA processing, MethodTable relationships, virtual slots, and
|
||||
frozen strings. Other RTR versions and PE architectures are reported as
|
||||
unsupported or partial. The matching .NET 8.0.22 fixture is source-built and
|
||||
the parser's Windows-host Ghidra workflow has not yet had a real Windows-host
|
||||
run. This does not establish Windows Ghidra import or decompiler behavior.
|
||||
Windows Ghidra P0 has no database mutation authority. A PE/CLI or ReadyToRun
|
||||
assembly should use `inspect_managed_artifact`; it is not NativeAOT.
|
||||
|
||||
@@ -43,42 +52,62 @@ arbitrary caller-supplied bytes. The bounded loader accepts regular JARs up to
|
||||
|
||||
## Use existing tools
|
||||
|
||||
Open the executable with provider `ghidra`. `inspect_native_load_image` returns
|
||||
optional `observations.metadata_recovery` inline, including the detected header,
|
||||
layout, recovery status, type category paths/addresses, diagnostics, annotation
|
||||
coverage and derived-memory SHA. The enclosing `unsupported` load-image status
|
||||
for PE/ELF concerns independent DOS loader verification; metadata recovery has
|
||||
its own status. It is not a failed import.
|
||||
Open the executable with provider `ghidra`. For an x64 PE, `inspect_native_load_image`
|
||||
returns `observations.metadata_recovery` inline from the captured source snapshot,
|
||||
including the RTR header, MethodTable addresses, frozen strings, derived overlay
|
||||
identity, and examined/recovered counts. Its enclosing `unsupported` load-image
|
||||
status means independent PE load-image verification was not performed; it does
|
||||
not mean the metadata parser failed or the executable failed to load.
|
||||
|
||||
Pass one returned type path or its typed address to `inspect_native_data_type`.
|
||||
Its optional `metadata_recovery` includes related/base type, implemented
|
||||
interfaces and virtual slots with targets and known procedure names. Follow
|
||||
these with `analyze_function`, string searches and ordinary cross-references.
|
||||
For CLI use:
|
||||
Pass a returned MethodTable address to `inspect_native_data_type`. A direct PE
|
||||
query also derives the snapshot metadata from the authenticated Ghidra session
|
||||
handshake, so it does not require a preceding load-image call. In a persistent
|
||||
MCP session, load-image followed by type inspection reuses that same derivation.
|
||||
The database result remains `unavailable` when Ghidra has no DataType at that
|
||||
address; its `metadata_recovery.source` is `read-only-derived-overlay` and
|
||||
carries related type, interface, slot, and target observations. Generated
|
||||
address identities do not claim original managed names. The parser does not
|
||||
merge frozen literals into Ghidra's database string inventory. Follow slot
|
||||
targets with `analyze_function` and ordinary cross-references.
|
||||
|
||||
The built-in CLI path uses a MethodTable address returned by the parser or
|
||||
another observation:
|
||||
|
||||
```sh
|
||||
rea inspect-native-load-image ./NativeAotFixture --provider ghidra --json
|
||||
rea inspect-native-data-type ./NativeAotFixture --type /NativeAOT/MethodTables/Class_ADDRESS_MT --provider ghidra --json
|
||||
rea inspect-native-data-type ./NativeAotFixture --address 0xMETHODTABLE_ADDRESS --provider ghidra --json
|
||||
rea function ./NativeAotFixture 0xADDRESS --provider ghidra --json
|
||||
```
|
||||
|
||||
No new MCP workflow or approval flag is required. Configuration opts into the
|
||||
adapter; without it ordinary Ghidra profiles/results remain unchanged.
|
||||
The generated `/NativeAOT/...` category path belongs to the optional headless
|
||||
adapter. Use `--type /NativeAOT/MethodTables/Class_ADDRESS_MT` only when that
|
||||
adapter is configured and has created the corresponding Ghidra DataType.
|
||||
Built-in PE analysis includes its parser contract and resource policy in the
|
||||
Ghidra analysis profile; non-PE profiles do not include that contract.
|
||||
|
||||
## Evidence boundaries
|
||||
|
||||
Discovery prefers an imported RTR symbol, otherwise the upstream initialized,
|
||||
non-executable-memory signature heuristic. REA rejects ambiguous candidates,
|
||||
unknown layout versions, malformed directory rows, unsupported flags and
|
||||
unmapped ranges before recovery. It requires the verified dehydrated layout.
|
||||
The built-in parser uses an initialized non-executable PE signature scan and
|
||||
requires a unique RTR 9.1 header and exactly one DEHYDRATED_DATA row. Rehydrated
|
||||
bytes live only in a parser-owned overlay, identified by virtual address, size,
|
||||
and SHA-256; the caller's source Buffer remains unchanged. Limits derive from
|
||||
captured section extents and format field bounds. Separate resource policies
|
||||
admit up to 128 MiB of captured PE bytes, 64 MiB-equivalent parser work, 64 MiB
|
||||
of exact parser-owned overlay/index Buffers, and 4 MiB of serialized NativeAOT
|
||||
report JSON. These are local parser limits, independent of MCP delivery; the
|
||||
buffer counter does not claim to cap V8 heap or total process memory. Rows that
|
||||
do not fit the report budget are omitted with explicit partial status and
|
||||
coverage. Coverage counts describe the pointer scan, recovered MethodTables,
|
||||
and frozen-string candidates examined, not every possible runtime type or
|
||||
object. Cancellation is honored during snapshot I/O and checked immediately
|
||||
before and after parsing; the synchronous parser call itself is bounded but
|
||||
cannot be interrupted mid-call. Parser output never claims that a Ghidra
|
||||
database type was defined.
|
||||
|
||||
The adapter performs rehydration, method-table recovery and frozen-object
|
||||
annotation in one ephemeral Program transaction. Fatal failure/cancellation
|
||||
rolls it back. Recovery may change analysis-memory bytes, never the original
|
||||
executable file. Derived ranges have a digest and **no original file offset**;
|
||||
they are not captured runtime state. Candidate/committed instance annotation
|
||||
counts distinguish partial frozen-object coverage. Coverage concerns the
|
||||
recovered candidate set, not every possible object or every runtime type.
|
||||
The optional adapter performs its annotations in an ephemeral Program
|
||||
transaction. That workflow may change analysis-memory bytes, never the original
|
||||
executable file. It is unavailable on Windows P0, which has no database
|
||||
mutation authority.
|
||||
|
||||
REA preserves pre-recovery calling conventions and uses the loaded compiler
|
||||
specification default for newly created methods. The upstream universal
|
||||
|
||||
@@ -8,9 +8,9 @@ Executable bytes remain unchanged.
|
||||
Use [installation](installation.md#ghidra) for supported hosts, prerequisites,
|
||||
configuration, startup deadlines, resource controls, and cleanup recovery.
|
||||
[Windows P0](windows-ghidra-p0.md) has its own admission and lifecycle boundary;
|
||||
[NativeAOT recovery](ghidra-nativeaot.md) is an optional, separately verified
|
||||
extension. Check the [release boundary](installation.md#released-package-and-main)
|
||||
when using npm.
|
||||
[NativeAOT observations](ghidra-nativeaot.md) include a built-in read-only PE
|
||||
snapshot parser and a separate optional annotation extension. Check the
|
||||
[release boundary](installation.md#released-package-and-main) when using npm.
|
||||
|
||||
See [import language and analysis seeds](installation.md#ghidra-import-language-and-analysis-seeds)
|
||||
to override the header-detected language or seed analysis.
|
||||
|
||||
@@ -880,7 +880,7 @@ For a client that requires manual configuration, use:
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents@6.3.0", "mcp"]
|
||||
"args": ["-y", "rea-agents@6.4.0", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -60,6 +60,12 @@ manifest and graph commitments, JavaScript Application Graph, static
|
||||
Electron summary, reconstruction statistics, and explicit limitations. It does
|
||||
not require a live Hopper, Ghidra, browser, or Electron process.
|
||||
|
||||
The application graph selects the shallowest `package.json` outside
|
||||
`node_modules` as its package root, breaking equal-depth ties by Unicode
|
||||
code-point order. Dependency manifests remain package observations but do not
|
||||
declare Electron main or renderer entries. If only dependency manifests are
|
||||
present, the inventory artifact remains the graph root.
|
||||
|
||||
## ASAR integrity
|
||||
|
||||
ASAR inventory checks Electron integrity metadata for embedded archive entries
|
||||
@@ -247,6 +253,46 @@ while locally shadowed receivers can still contribute network endpoint
|
||||
candidates. A template recovered with a parser error and no cooked value stays
|
||||
dynamic; its raw spelling is not treated as a valid JavaScript string.
|
||||
|
||||
RPC client syntax also contributes network endpoint candidates, in both
|
||||
`analyze_javascript_application` and `analyze_web_bundle`, distinguished by
|
||||
`mechanism`:
|
||||
|
||||
- `trpc:procedure:<method>` records the static member path between the root
|
||||
expression and the method, such as `invoice.list` in
|
||||
`root.invoice.list.useQuery(...)`. tRPC proxies keep these property names
|
||||
through identifier minification. React hooks and TanStack option factories
|
||||
(`useQuery`, `useMutation`, `useInfiniteQuery`, `useSubscription`, suspense
|
||||
and prefetch variants, `queryOptions`, `infiniteQueryOptions`,
|
||||
`mutationOptions`, `subscriptionOptions`) are recognized directly. The
|
||||
vanilla `query` and `mutate` methods are recognized on a root bound to
|
||||
`createTRPCClient` or `createTRPCProxyClient` in the same lexical binding, or on a
|
||||
path of at least two segments in a source containing `TRPCClientError`,
|
||||
`trpc-accept`, or `/trpc`. Factory bindings must have one unconditional
|
||||
initializer and no binding assignments; shadowed locals do not inherit client
|
||||
identity. `subscribe` additionally requires a tRPC observer
|
||||
argument (`onData`, `onError`, `onStarted`, `onStopped`, `onComplete`, or
|
||||
`onConnectionStateChange`). `this` roots, browser globals, and any path
|
||||
containing an RTK Query `endpoints` segment are never procedures; cache-key
|
||||
helpers such as `queryKey` issue no request and are excluded.
|
||||
- `trpc:link:<link>` records the literal `url` option of `httpLink`,
|
||||
`httpBatchLink`, `httpBatchStreamLink`, and `httpSubscriptionLink` when the
|
||||
link's callee name survives bundling, including `(0, module.httpBatchLink)`
|
||||
import-call wrappers.
|
||||
- `graphql:query`, `graphql:mutation`, and `graphql:subscription` record a
|
||||
named operation found at the start of a string or cooked template literal —
|
||||
the name must be followed by variable definitions (`($`), a directive, or a
|
||||
selection set starting with a field name — or in a precompiled
|
||||
`OperationDefinition` document object. A graphql-tag `loc.source.body` copy is
|
||||
not reported twice. Anonymous operations are not reported.
|
||||
|
||||
A procedure path is the server-side procedure name, not a resolved URL; the
|
||||
transport URL depends on the link configuration and batching. When the proxy is
|
||||
reached through a member expression, such as a webpack module binding
|
||||
(`l.S.invoice.list`) or a property holding the proxy (`o.api.post.byId`), the
|
||||
recorded path keeps those leading carrier segments; the procedure path is its
|
||||
suffix. Utility calls such as `useUtils().invoice.list.fetch()`, `useQueries`,
|
||||
and links whose callee was renamed by a minifier are not recognized.
|
||||
|
||||
Each recovered bundle module retains the exact factory-source digest. A complete
|
||||
bounded AST also receives a `babel-ast-v1` structural fingerprint that ignores
|
||||
ordinary identifier names while retaining syntax, literals, operators, object
|
||||
|
||||
@@ -308,6 +308,23 @@ that reference for a summary, one section/module, or a stable page, or call
|
||||
bundle exports stream canonical JSON into an atomically published file. A broad
|
||||
follow-up or `get_evidence_bundle` can also exceed the response budget; exporting
|
||||
preserves the complete session without sending it through a single MCP frame.
|
||||
To discover existing records without their large payloads, call
|
||||
`get_evidence_bundle` with `detail: "summary"`. Its `result.records` contain
|
||||
Evidence IDs, artifact/provider identities, analysis-profile digests and native
|
||||
procedure metadata. Optional `filters` match exact `evidence_id`, `operation`,
|
||||
`target_sha256`, `analysis_profile_digest` or `procedure_address`. Use a discovered
|
||||
ID with `inspect_analysis_view`; neither discovery nor retained selection starts
|
||||
a provider. `retention: "complete-record"` describes stored Evidence, while
|
||||
`native_dossier.value_flow` independently reports truncation, unavailable or
|
||||
unknown metadata. Missing omission counts remain null. Summary output carries
|
||||
every matching record; actual transport admission still applies. Omitted
|
||||
`detail` preserves complete canonical bundle delivery; filters require summary.
|
||||
|
||||
For a portable exported bundle, the CLI shares this projection:
|
||||
`rea inspect-evidence-bundle '{"path":"/tmp/evidence.json","detail":"summary","filters":{"operation":"analyze_function"}}' --format json`.
|
||||
The CLI validates and materializes that local bundle under its existing memory
|
||||
budget; summary delivery does not make arbitrary large files cheap to import.
|
||||
|
||||
Cancelling an export stops further serialization and removes its staging file.
|
||||
The destination changes only when a complete export is atomically published.
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
+5
-1
@@ -21,7 +21,11 @@ Use [the rea-changelog-update skill](https://github.com/morluto/rea/blob/main/.a
|
||||
to turn the bot draft into grouped Highlights, Changes, and Fixes with inline
|
||||
`Thanks @...` credit. `npm run release:notes -- inventory` collects the full
|
||||
selected Git range and paginated GitHub PR/author/closing-issue metadata;
|
||||
`record` produces the complete contribution record, `check` validates credits
|
||||
`record --summary` produces a compact provenance link for public notes, while
|
||||
`record` produces the full record for review. Credit a contribution once even
|
||||
when it appears in highlights, details and migrations; omit maintainer
|
||||
`@morluto` self-thanks. Keep complete authorship metadata in the inventory
|
||||
instead of duplicating a credit ledger in the release body. `check` validates credits
|
||||
and optional PR-body equality, and `render` writes the verified release section.
|
||||
Run a subcommand without its required options to see its usage. GitHub reads
|
||||
require an authenticated `gh`; record/check/render work offline from the saved
|
||||
|
||||
+16
-9
@@ -450,15 +450,15 @@ installation (`application.java.min` through `application.java.max`). Current
|
||||
12.1 releases require JDK 21 or newer and set no maximum. The lanes below still
|
||||
prove behavior on the verified Ghidra 12.1.4 and JDK 21 build.
|
||||
|
||||
| Ghidra lane | Supported runner/target | Additional local tools |
|
||||
| ------------------------------------------ | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
|
||||
| `npm run verify:ghidra` | Linux x64/arm64 ELF or macOS x64/arm64 Mach-O | Host C compiler, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:seeds` | POSIX host; source-owned 32-bit x86 raw fixture; function order and BSS labels | Ghidra 12.1.4 and full JDK 21 |
|
||||
| `npm run verify:ghidra:swift` | macOS x64/arm64 Mach-O | Host Swift compiler, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:switch` | Linux x64 ELF; GCC/Clang optimized and stripped switch fixtures | GCC, Clang, GNU nm/objdump/strip, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:aarch64-jump-table` | Any supported Ghidra host; AArch64 ELF; byte/halfword tables; host ARM64 Mach-O | Clang with AArch64 target support, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:cross-format` | Any supported Ghidra host; also analyzes AArch64 ELF, x86-64 PE, and x86-64 Mach-O | `clang`, LLD, and `lld-link` in addition to host-lane prerequisites |
|
||||
| `npm run verify:ghidra:windows` | Controlled Windows x64 with native x86-64 PE; `-- --x86` selects native x86 PE | Ghidra 12.1.4, full JDK 21, and the matching native artifact |
|
||||
| Ghidra lane | Supported runner/target | Additional local tools |
|
||||
| ------------------------------------------ | ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
|
||||
| `npm run verify:ghidra` | Linux x64/arm64 ELF or macOS x64/arm64 Mach-O | Host C compiler, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:seeds` | POSIX host; source-owned 32-bit x86 raw fixture; function order and BSS labels | Ghidra 12.1.4 and full JDK 21 |
|
||||
| `npm run verify:ghidra:swift` | macOS x64/arm64 Mach-O | Host Swift compiler, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:switch` | Linux x64 ELF; GCC/Clang optimized and stripped switch fixtures | GCC, Clang, GNU nm/objdump/strip, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:aarch64-jump-table` | Any supported Ghidra host; AArch64 ELF; signed/unsigned byte/halfword tables; host ARM64 Mach-O | Clang with AArch64 target support, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:cross-format` | Any supported Ghidra host; also analyzes AArch64 ELF, x86-64 PE, and x86-64 Mach-O | `clang`, LLD, and `lld-link` in addition to host-lane prerequisites |
|
||||
| `npm run verify:ghidra:windows` | Controlled Windows x64 with native x86-64 PE; `-- --x86` selects native x86 PE | Ghidra 12.1.4, full JDK 21, and the matching native artifact |
|
||||
|
||||
The optional Swift lane compiles a source-owned fixture and checks modern and
|
||||
provider-demangled procedure names through real stdio MCP, with and without the
|
||||
@@ -672,6 +672,13 @@ boundaries and budgets.
|
||||
|
||||
## Optional NativeAOT Ghidra analysis
|
||||
|
||||
The source parser lane is independent of the optional annotation JAR. Its
|
||||
synthetic RTR 9.1 PE golden checks captured-byte digest admission, rehydration
|
||||
into an immutable parser overlay, MethodTable/slot relationships, and frozen
|
||||
literal extraction; provider tests exercise `inspect_native_load_image` and
|
||||
address-based `inspect_native_data_type` with the database result still
|
||||
unavailable. These tests do not establish Windows Ghidra host behavior.
|
||||
|
||||
This lane is separate from the default native lane. `build:fixtures:nativeaot`
|
||||
requires an existing .NET SDK 8.0.416, the platform NativeAOT compiler/linker and
|
||||
runtime pack 8.0.22. It builds benign sources into ignored `_reference/` and
|
||||
|
||||
@@ -21,9 +21,12 @@ additional permission flag or degraded mode is required.
|
||||
- the 25 read-only Ghidra inventory, memory, and function-analysis operations.
|
||||
|
||||
Loaded memory reads and file offsets preserve Ghidra's observed source mappings.
|
||||
PE load-image inspection returns measurements with an explicit unsupported
|
||||
attestation status; independent format-specific verification currently supports
|
||||
DOS MZ only.
|
||||
PE load-image inspection returns an explicit unsupported attestation status;
|
||||
independent load-image verification currently supports DOS MZ only. For x64 PE
|
||||
NativeAOT RTR 9.1, the read-only snapshot parser can report metadata through
|
||||
`inspect_native_load_image` and `inspect_native_data_type` without a JAR or
|
||||
database mutation. A real Windows-host NativeAOT Ghidra run is still required
|
||||
to establish the complete host workflow.
|
||||
|
||||
This boundary does not establish general Windows feature parity. Ghidra GUI
|
||||
state, annotations, other target architectures and formats, and Hopper-only
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
The MIT License (MIT)
|
||||
=====================
|
||||
|
||||
Copyright © 2023-2025 Washi
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the “Software”), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
|
||||
The read-only NativeAOT parser adapts format and hydration semantics from
|
||||
Washi1337/ghidra-nativeaot commit
|
||||
effeb734fc570c32650f88b159608979dc7b423e (MIT). MethodTable field layout was
|
||||
cross-checked against .NET runtime src/coreclr/nativeaot/Runtime/inc/MethodTable.h
|
||||
at dotnet/runtime commit a3b2d40328be0be3f8dd43f0e5cc925b8827b044 (MIT).
|
||||
@@ -1,5 +1,7 @@
|
||||
#include "authority.hpp"
|
||||
|
||||
#include <delayimp.h>
|
||||
|
||||
namespace rea {
|
||||
|
||||
static const napi_type_tag RESOURCE_TAG = {0x45fcbf0b49e84fe3ULL, 0x9e3e5074130de843ULL};
|
||||
@@ -121,3 +123,13 @@ static napi_value initialize(napi_env env, napi_value exports) {
|
||||
} // namespace rea
|
||||
|
||||
NAPI_MODULE(NODE_GYP_MODULE_NAME, rea::initialize)
|
||||
|
||||
// Node-API symbols are delay-imported from "node.exe". Resolve them against the
|
||||
// host executable so Node.js, Bun, and other Node-API hosts load the addon
|
||||
// regardless of their executable file name.
|
||||
static FARPROC WINAPI reaHostHook(unsigned event, PDelayLoadInfo info) {
|
||||
if (event != dliNotePreLoadLibrary || _stricmp(info->szDll, "node.exe") != 0) return nullptr;
|
||||
return reinterpret_cast<FARPROC>(GetModuleHandleW(nullptr));
|
||||
}
|
||||
|
||||
decltype(__pfnDliNotifyHook2) __pfnDliNotifyHook2 = reaHostHook;
|
||||
|
||||
@@ -219,9 +219,10 @@ std::unique_ptr<Runtime> createRuntime(const std::wstring& parentPath, const std
|
||||
require(BCryptGenRandom(nullptr, random.data(), random.size(), BCRYPT_USE_SYSTEM_PREFERRED_RNG) == 0,
|
||||
"Generate runtime identity failed", parentPath, ERROR_GEN_FAILURE);
|
||||
auto result = std::make_unique<Runtime>();
|
||||
result->path = parent->path + L"\\" + prefix + hex(random.data(), random.size());
|
||||
const auto name = prefix + hex(random.data(), random.size());
|
||||
result->path = parent->path + (parent->path.back() == L'\\' ? L"" : L"\\") + name;
|
||||
Security security;
|
||||
result->directory = createRelative(parent->get(), prefix + hex(random.data(), random.size()),
|
||||
result->directory = createRelative(parent->get(), name,
|
||||
DELETE | FILE_LIST_DIRECTORY | FILE_READ_ATTRIBUTES | READ_CONTROL,
|
||||
true, security, result->path);
|
||||
try {
|
||||
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "rea-agents",
|
||||
"version": "6.3.0",
|
||||
"version": "6.4.0",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "rea-agents",
|
||||
"version": "6.3.0",
|
||||
"version": "6.4.0",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@babel/parser": "8.0.4",
|
||||
|
||||
+3
-2
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "rea-agents",
|
||||
"mcpName": "io.github.morluto/rea",
|
||||
"version": "6.3.0",
|
||||
"version": "6.4.0",
|
||||
"description": "Reverse engineer anything from your terminal or agent with one CLI and MCP server.",
|
||||
"license": "MIT",
|
||||
"repository": {
|
||||
@@ -81,7 +81,8 @@
|
||||
"bridge/pwndbg/launch_impl.py",
|
||||
"bridge/pwndbg/core_context.py",
|
||||
"third_party/pwndbg/README.md",
|
||||
"third_party/pwndbg/LICENSE.md"
|
||||
"third_party/pwndbg/LICENSE.md",
|
||||
"licenses/nativeaot-parser-MIT.txt"
|
||||
],
|
||||
"scripts": {
|
||||
"release:notes": "node scripts/release-notes.mjs",
|
||||
|
||||
@@ -84,7 +84,8 @@ if (!(await exists(join(headers, "node_api.h")))) {
|
||||
}
|
||||
const artifact = "rea-windows-x64.node";
|
||||
// MSVC short-import libraries do not produce a correct Node import table with
|
||||
// GNU ld. Generate a GNU import library for the exact stable Node-API symbols.
|
||||
// GNU ld. Generate a GNU delay-import library for the exact stable Node-API
|
||||
// symbols; addon.cc binds it to the host executable, whatever its file name.
|
||||
const sources = ["addon.cc", "filesystem.cc", "process.cc"];
|
||||
const sourceText = (
|
||||
await Promise.all(
|
||||
@@ -106,7 +107,7 @@ await writeFile(
|
||||
definition,
|
||||
`LIBRARY node.exe\nEXPORTS\n${symbols.join("\n")}\n`,
|
||||
);
|
||||
run("x86_64-w64-mingw32-dlltool", ["-d", definition, "-l", library]);
|
||||
run("x86_64-w64-mingw32-dlltool", ["-d", definition, "-y", library]);
|
||||
run(compiler, [
|
||||
"-std=c++17",
|
||||
"-O2",
|
||||
|
||||
@@ -14,6 +14,24 @@ const flags = [
|
||||
"workflows",
|
||||
...Object.keys(SCOPES),
|
||||
];
|
||||
// Actionlint validates workflow syntax only. The semantic suite asserts these
|
||||
// workflows' release, publication and self-hosted runner safeguards.
|
||||
const GUARDED_WORKFLOWS = new Set([
|
||||
".github/workflows/release.yml",
|
||||
".github/workflows/real-ghidra-windows.yml",
|
||||
]);
|
||||
// Sources compiled by build-conformance-fixtures.mjs. The readiness journey
|
||||
// analyzes the built C fixture as well as its own applications.
|
||||
const FIXTURE_SOURCES = [
|
||||
"tests/conformance/c/**",
|
||||
"tests/conformance/napi/**",
|
||||
"tests/conformance/objc/**",
|
||||
"tests/conformance/swift/**",
|
||||
"tests/conformance/versions/**",
|
||||
];
|
||||
const READINESS_SOURCES = ["tests/conformance/readiness/**"];
|
||||
const matchesAny = (file, patterns) =>
|
||||
patterns.some((pattern) => matchesGlob(file, pattern));
|
||||
const selected = Object.fromEntries(flags.map((flag) => [flag, false]));
|
||||
const reasons = [];
|
||||
const full = (reason) => {
|
||||
@@ -122,6 +140,7 @@ if (
|
||||
file === ".github/actionlint.yaml"
|
||||
) {
|
||||
selected.workflows = true;
|
||||
if (GUARDED_WORKFLOWS.has(file)) selected.code = true;
|
||||
if (file !== ".github/workflows/ci.yml") continue;
|
||||
}
|
||||
if (owners.length === 1 && owners[0] === "website") continue;
|
||||
@@ -131,6 +150,11 @@ if (
|
||||
continue;
|
||||
}
|
||||
selected.code = true;
|
||||
if (matchesAny(file, FIXTURE_SOURCES)) {
|
||||
selected.fixtures = true;
|
||||
selected.readiness = true;
|
||||
}
|
||||
if (matchesAny(file, READINESS_SOURCES)) selected.readiness = true;
|
||||
if (file.startsWith("tests/")) continue;
|
||||
if (
|
||||
/^(src\/process\/|src\/filesystem\/|src\/contracts\/tool|src\/server\/createServer|src\/cli\.ts|scripts\/ci\/)/.test(
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
/* Keep the configured launcher identity alive while its resource-limited
|
||||
* Python child runs. Both remain in REA's owned process group. */
|
||||
#define _POSIX_C_SOURCE 200809L
|
||||
#include <errno.h>
|
||||
#include <signal.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <sys/resource.h>
|
||||
#include <sys/types.h>
|
||||
#include <sys/wait.h>
|
||||
#include <unistd.h>
|
||||
|
||||
int main(int argc, char **argv) {
|
||||
char *python = getenv("REA_VERIFY_LIMITED_PYTHON");
|
||||
char *limit = getenv("REA_VERIFY_LIMIT_OPTION");
|
||||
if (python == NULL || python[0] != '/' || limit == NULL) {
|
||||
fputs("limited Python fixture requires an absolute Python and limit\n", stderr);
|
||||
return 127;
|
||||
}
|
||||
struct rlimit core = {0, 0};
|
||||
if (setrlimit(RLIMIT_CORE, &core) != 0) {
|
||||
perror("setrlimit");
|
||||
return 127;
|
||||
}
|
||||
char **arguments = calloc((size_t)argc + 4, sizeof(char *));
|
||||
if (arguments == NULL) return 127;
|
||||
arguments[0] = "/usr/bin/prlimit";
|
||||
arguments[1] = limit;
|
||||
arguments[2] = "--";
|
||||
arguments[3] = python;
|
||||
for (int index = 1; index < argc; index++) arguments[index + 3] = argv[index];
|
||||
pid_t child = fork();
|
||||
if (child == 0) {
|
||||
execv(arguments[0], arguments);
|
||||
perror("execv");
|
||||
_exit(127);
|
||||
}
|
||||
free(arguments);
|
||||
if (child < 0) {
|
||||
perror("fork");
|
||||
return 127;
|
||||
}
|
||||
int status;
|
||||
while (waitpid(child, &status, 0) < 0) {
|
||||
if (errno == EINTR) continue;
|
||||
perror("waitpid");
|
||||
return 127;
|
||||
}
|
||||
if (WIFEXITED(status)) return WEXITSTATUS(status);
|
||||
if (WIFSIGNALED(status)) {
|
||||
int signum = WTERMSIG(status);
|
||||
signal(signum, SIG_DFL);
|
||||
sigset_t signals;
|
||||
sigemptyset(&signals);
|
||||
sigaddset(&signals, signum);
|
||||
sigprocmask(SIG_UNBLOCK, &signals, NULL);
|
||||
raise(signum);
|
||||
return 128 + signum;
|
||||
}
|
||||
return 127;
|
||||
}
|
||||
@@ -28,6 +28,11 @@ import {
|
||||
isolatedFixtureUrl,
|
||||
} from "./browser-runtime-world-e2e.mjs";
|
||||
|
||||
// Arming, progress delivery, isolated-world proof and the page action cross
|
||||
// separate transports. Leave time for their round trips on shared CI runners;
|
||||
// coverage can include a late action even after request collection has ended.
|
||||
const executionObservationMs = 2_000;
|
||||
|
||||
/** Exercise native browser runtime attribution through public CLI and stdio MCP, including arming. */
|
||||
export async function verifyBrowserRuntime(
|
||||
executable,
|
||||
@@ -210,7 +215,7 @@ async function executionMcp(
|
||||
const response = await client.callTool(
|
||||
{
|
||||
name: "observe_web_execution",
|
||||
arguments: { ...input, observation_ms: 300 },
|
||||
arguments: { ...input, observation_ms: executionObservationMs },
|
||||
},
|
||||
{
|
||||
timeout: 40_000,
|
||||
@@ -259,7 +264,7 @@ async function executionCli(entrypoint, input, env, action, isolatedContext) {
|
||||
input.cdp_endpoint,
|
||||
input.target_id,
|
||||
"--observation-ms",
|
||||
"300",
|
||||
String(executionObservationMs),
|
||||
"--json",
|
||||
],
|
||||
{ env, stdio: ["ignore", "pipe", "pipe"] },
|
||||
@@ -382,7 +387,14 @@ function assertExecution(evidence, site, requireBlock = true) {
|
||||
const request = result.requests.find((item) =>
|
||||
item.url.endsWith("/evidence?marker=chosen"),
|
||||
);
|
||||
assert.ok(request, "The selected request initiator must be retained");
|
||||
assert.ok(
|
||||
request,
|
||||
`The selected request initiator must be retained: ${JSON.stringify({
|
||||
window: result.window,
|
||||
requests: result.requests.map((item) => item.url),
|
||||
excluded_requests: result.excluded_requests,
|
||||
})}`,
|
||||
);
|
||||
assert.ok(
|
||||
request.callsites.some(
|
||||
(site) =>
|
||||
|
||||
@@ -99,6 +99,14 @@ const scenarioFor = (browser, origin) =>
|
||||
capture: { after_each_step: ["dom"], at_end: ["dom"] },
|
||||
});
|
||||
|
||||
const raceWithTimeout = (operation, timeoutMs, onTimeout = () => undefined) => {
|
||||
const controller = new AbortController();
|
||||
return Promise.race([
|
||||
operation,
|
||||
delay(timeoutMs, undefined, { signal: controller.signal }).then(onTimeout),
|
||||
]).finally(() => controller.abort());
|
||||
};
|
||||
|
||||
const storagePage = (kind, port) => {
|
||||
if (kind === "main")
|
||||
return `<!doctype html><html><body><pre id="main-report"></pre><pre id="frame-report"></pre><iframe src="http://localhost:${port}/storage-frame"></iframe><script>
|
||||
@@ -168,6 +176,9 @@ const startStorageSite = async () => {
|
||||
origin: `http://127.0.0.1:${port}`,
|
||||
close: async () => {
|
||||
server.close();
|
||||
// Chrome can leave speculative TCP connections without an HTTP request;
|
||||
// server.close() alone waits indefinitely for those owned sockets.
|
||||
server.closeAllConnections();
|
||||
await once(server, "close");
|
||||
},
|
||||
};
|
||||
@@ -303,11 +314,10 @@ const verifyConnectedStorage = async (endpoint, targetId, origin) => {
|
||||
await verifyCallerDebuggerPause(observer, origin, scenario);
|
||||
await verifyCancelledStorage(observer, endpoint, targetId, scenario);
|
||||
} finally {
|
||||
await Promise.race([
|
||||
restoreTarget(observer, target.url),
|
||||
delay(1_000),
|
||||
]).catch(() => undefined);
|
||||
await Promise.race([observer.close(), delay(1_000)]).catch(() => undefined);
|
||||
await raceWithTimeout(restoreTarget(observer, target.url), 1_000).catch(
|
||||
() => undefined,
|
||||
);
|
||||
await raceWithTimeout(observer.close(), 1_000).catch(() => undefined);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -500,14 +510,11 @@ const verifyCallerDebuggerPause = async (caller, origin, scenario) => {
|
||||
unsubscribe();
|
||||
});
|
||||
try {
|
||||
const callFrameId = await Promise.race([
|
||||
paused,
|
||||
delay(10_000).then(() => {
|
||||
throw new Error(
|
||||
`Caller debugger pause was not observed at ${debuggerUrl}; settled=${String(settled)} events=${JSON.stringify(protocolTrace)}`,
|
||||
);
|
||||
}),
|
||||
]);
|
||||
const callFrameId = await raceWithTimeout(paused, 10_000, () => {
|
||||
throw new Error(
|
||||
`Caller debugger pause was not observed at ${debuggerUrl}; settled=${String(settled)} events=${JSON.stringify(protocolTrace)}`,
|
||||
);
|
||||
});
|
||||
const pausedState = await caller.send("Debugger.evaluateOnCallFrame", {
|
||||
callFrameId,
|
||||
expression: "window.reaDebuggerAfter === true",
|
||||
@@ -536,10 +543,10 @@ const verifyCallerDebuggerPause = async (caller, origin, scenario) => {
|
||||
} catch (cause) {
|
||||
if (!pauseObserved) controller.abort(cause);
|
||||
else await caller.send("Debugger.resume").catch(() => undefined);
|
||||
const completion = await Promise.race([
|
||||
const completion = await raceWithTimeout(
|
||||
pending.catch((pendingCause) => ({ ok: false, error: pendingCause })),
|
||||
delay(5_000).then(() => undefined),
|
||||
]);
|
||||
5_000,
|
||||
);
|
||||
if (completion === undefined)
|
||||
throw new Error("Capture remained pending after cancellation", { cause });
|
||||
const completionError =
|
||||
@@ -579,7 +586,7 @@ const verifyCallerDebuggerPause = async (caller, origin, scenario) => {
|
||||
});
|
||||
} finally {
|
||||
unsubscribe();
|
||||
await Promise.race([caller.send("Debugger.disable"), delay(1_000)]).catch(
|
||||
await raceWithTimeout(caller.send("Debugger.disable"), 1_000).catch(
|
||||
() => undefined,
|
||||
);
|
||||
}
|
||||
|
||||
@@ -101,10 +101,13 @@ export async function readReleaseHistory(run, base, target) {
|
||||
// REA history maps Anthropic and Cursor's agent trailers to ordinary User actors.
|
||||
// Keep those source identities in the inventory while excluding automation credit.
|
||||
const AGENT_ACCOUNTS = new Set(["claude", "cursoragent"]);
|
||||
// REA's maintainer requested external-contributor thanks without self-credit.
|
||||
const MAINTAINER_ACCOUNTS = new Set(["morluto"]);
|
||||
const human = (value) =>
|
||||
value?.__typename === "User" &&
|
||||
!value.login.endsWith("[bot]") &&
|
||||
!AGENT_ACCOUNTS.has(value.login.toLowerCase())
|
||||
!AGENT_ACCOUNTS.has(value.login.toLowerCase()) &&
|
||||
!MAINTAINER_ACCOUNTS.has(value.login.toLowerCase())
|
||||
? value.login
|
||||
: undefined;
|
||||
const unique = (values) =>
|
||||
@@ -163,6 +166,12 @@ export function contributionRecord(raw) {
|
||||
return `${lines.join("\n")}\n`;
|
||||
}
|
||||
|
||||
/** Keep the full authorship snapshot available without duplicating public thanks. */
|
||||
export function contributionSummary(raw) {
|
||||
const inventory = releaseInventorySchema.parse(raw);
|
||||
return `Full contribution record: [${inventory.commits.length} commits and ${inventory.pullRequests.length} merged PR${inventory.pullRequests.length === 1 ? "" : "s"}](https://github.com/${inventory.repository}/blob/rea-agents-${inventory.version}/docs/releases/${inventory.version}.contributions.json).\n`;
|
||||
}
|
||||
|
||||
export function selectedReleaseNotes(text, version) {
|
||||
const headings = [...text.matchAll(/^## \[([^\]]+)\].*$/gmu)];
|
||||
if (headings[0]?.[1] !== version)
|
||||
@@ -236,7 +245,8 @@ function requiredCredits(inventory, bullet, references) {
|
||||
if (matches[0].parents.length > 1) continue;
|
||||
for (const identity of matches[0].authors) {
|
||||
const handle = human(identity.user);
|
||||
if (handle) required.push({ handle, context: `commit ${match[2]}` });
|
||||
if (handle)
|
||||
required.push({ handle, context: `commit ${matches[0].sha}` });
|
||||
}
|
||||
}
|
||||
return required;
|
||||
@@ -264,6 +274,8 @@ function verifyEditorialCredits(inventory, prose) {
|
||||
]).map((handle) => handle.toLowerCase()),
|
||||
);
|
||||
const errors = [];
|
||||
const requiredByContext = new Map();
|
||||
const thankedByContext = new Map();
|
||||
for (const bullet of proseBullets(prose)) {
|
||||
const references = bulletPrs(bullet, inventory.repository);
|
||||
const start = bullet.indexOf("Thanks ");
|
||||
@@ -282,13 +294,21 @@ function verifyEditorialCredits(inventory, prose) {
|
||||
bullet,
|
||||
references,
|
||||
)) {
|
||||
if (!thanked.has(handle.toLowerCase()))
|
||||
errors.push(`Missing Thanks @${handle} in bullet for ${context}`);
|
||||
const required = requiredByContext.get(context) ?? new Set();
|
||||
required.add(handle);
|
||||
requiredByContext.set(context, required);
|
||||
const credited = thankedByContext.get(context) ?? new Set();
|
||||
for (const login of thanked) credited.add(login);
|
||||
thankedByContext.set(context, credited);
|
||||
}
|
||||
for (const handle of thanked)
|
||||
if (!eligible.has(handle))
|
||||
errors.push(`Unverified human credit @${handle}`);
|
||||
}
|
||||
for (const [context, required] of requiredByContext)
|
||||
for (const handle of required)
|
||||
if (!thankedByContext.get(context)?.has(handle.toLowerCase()))
|
||||
errors.push(`Missing Thanks @${handle} for ${context}`);
|
||||
return errors;
|
||||
}
|
||||
|
||||
@@ -305,8 +325,15 @@ export function verifyReleaseNotes(raw, text, body) {
|
||||
"Release notes exceed GitHub's body budget; shorten prose without dropping credit",
|
||||
);
|
||||
const record = contributionRecord(inventory).trim();
|
||||
const start = notes.indexOf("### Complete contribution record");
|
||||
if (start < 0 || notes.slice(start).trim() !== record)
|
||||
const summary = contributionSummary(inventory).trim();
|
||||
const start = notes.endsWith(summary)
|
||||
? notes.length - summary.length
|
||||
: notes.indexOf("### Complete contribution record");
|
||||
if (
|
||||
start < 0 ||
|
||||
(notes.slice(start).trim() !== record &&
|
||||
notes.slice(start).trim() !== summary)
|
||||
)
|
||||
throw new Error(
|
||||
"Contribution record differs from inventory; regenerate it with release-notes record",
|
||||
);
|
||||
|
||||
@@ -6,6 +6,7 @@ import { collectGitHubCredits } from "./lib/release-notes-github.mjs";
|
||||
import { verifyNotesPublication } from "./lib/release-notes-publication.mjs";
|
||||
import {
|
||||
contributionRecord,
|
||||
contributionSummary,
|
||||
readReleaseHistory,
|
||||
releaseInventorySchema,
|
||||
selectedReleaseNotes,
|
||||
@@ -73,6 +74,7 @@ async function main() {
|
||||
"pr-body": { type: "string" },
|
||||
source: { type: "string" },
|
||||
output: { type: "string" },
|
||||
summary: { type: "boolean", default: false },
|
||||
},
|
||||
});
|
||||
const [command] = positionals;
|
||||
@@ -92,7 +94,7 @@ async function main() {
|
||||
!values.manifest
|
||||
)
|
||||
throw new Error(
|
||||
"Usage: release-notes inventory|record|check|render --manifest FILE [--repository OWNER/REPO --base TAG --target SHA --version VERSION] [--notes FILE --pr-body FILE --source SHA --output FILE]",
|
||||
"Usage: release-notes inventory|record|check|render --manifest FILE [--repository OWNER/REPO --base TAG --target SHA --version VERSION] [--notes FILE --pr-body FILE --source SHA --output FILE] [--summary for record]",
|
||||
);
|
||||
if (command === "inventory") {
|
||||
await createInventory(values);
|
||||
@@ -103,11 +105,14 @@ async function main() {
|
||||
);
|
||||
await verifyInventoryHistory(run, inventory, values.source);
|
||||
if (command === "record") {
|
||||
const record = values.summary
|
||||
? contributionSummary(inventory)
|
||||
: contributionRecord(inventory);
|
||||
if (values.output)
|
||||
await writeFile(values.output, contributionRecord(inventory), {
|
||||
await writeFile(values.output, record, {
|
||||
flag: "wx",
|
||||
});
|
||||
else process.stdout.write(contributionRecord(inventory));
|
||||
else process.stdout.write(record);
|
||||
return;
|
||||
}
|
||||
const notes = await readFile(values.notes, "utf8");
|
||||
|
||||
@@ -15,6 +15,10 @@ import { promisify } from "node:util";
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { PrivateRuntimeRoot } from "../dist/process/PrivateRuntimeRoot.js";
|
||||
import {
|
||||
OwnedCommandFailure,
|
||||
runOwnedCommand,
|
||||
} from "../dist/process/OwnedCommand.js";
|
||||
import { parseEvidence } from "../dist/domain/evidence.js";
|
||||
import { mcpTextValue } from "./lib/mcp-verifier-results.mjs";
|
||||
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
|
||||
@@ -56,6 +60,13 @@ const root = await PrivateRuntimeRoot.create({
|
||||
const environment = Object.fromEntries(
|
||||
Object.entries(process.env).filter(([, value]) => typeof value === "string"),
|
||||
);
|
||||
const limitedPython = join(root.path, "limited-python");
|
||||
const limitedEnvironment = (option) => ({
|
||||
...environment,
|
||||
REA_PWNTOOLS_PYTHON: limitedPython,
|
||||
REA_VERIFY_LIMITED_PYTHON: python,
|
||||
REA_VERIFY_LIMIT_OPTION: option,
|
||||
});
|
||||
const client = new Client({ name: "binary-layout-verifier", version: "1" });
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
@@ -73,6 +84,46 @@ let cases = 0;
|
||||
let bootstrapCases = 0;
|
||||
const failures = [];
|
||||
try {
|
||||
await execute(
|
||||
"gcc",
|
||||
[
|
||||
"-std=c11",
|
||||
"-Wall",
|
||||
"-Wextra",
|
||||
"-Werror",
|
||||
fileURLToPath(
|
||||
new URL("./fixtures/binary-layout-limited-python.c", import.meta.url),
|
||||
),
|
||||
"-o",
|
||||
limitedPython,
|
||||
],
|
||||
{ timeout: 30_000, maxBuffer: 1024 * 1024 },
|
||||
);
|
||||
// A live limited child must retain its configured launcher's identity so
|
||||
// timeout cleanup can still verify and terminate the entire owned group.
|
||||
await assert.rejects(
|
||||
runOwnedCommand(
|
||||
{
|
||||
command: limitedPython,
|
||||
arguments: [
|
||||
"-I",
|
||||
"-c",
|
||||
"import time; print('limited child ready', flush=True); time.sleep(60)",
|
||||
],
|
||||
cwd: root.path,
|
||||
runId: `${run.run_id}-limited-python-timeout`,
|
||||
hostEnvironment: limitedEnvironment("--as=2147483648"),
|
||||
},
|
||||
{ timeoutMs: 2000, diagnosticBytes: 1024 },
|
||||
),
|
||||
(error) => {
|
||||
assert.ok(error instanceof OwnedCommandFailure);
|
||||
assert.equal(error.reason, "timeout");
|
||||
assert.equal(error.cleanupFailure, null);
|
||||
assert.ok(error.snapshot.stdout.text.includes("limited child ready"));
|
||||
return true;
|
||||
},
|
||||
);
|
||||
const bootstrapRoot = join(root.path, "bootstrap-boundary");
|
||||
await mkdir(bootstrapRoot);
|
||||
const bootstrapPath = join(bootstrapRoot, "layout.py");
|
||||
@@ -796,19 +847,12 @@ try {
|
||||
assert.deepEqual(await readFile(extendedPath), extendedObject);
|
||||
cases++;
|
||||
}
|
||||
for (const [name, option, expected] of [
|
||||
["address-space", "--as=2147483648", '"address_space_bytes": 2147483648'],
|
||||
["cpu", "--cpu=20", '"cpu_seconds": 20'],
|
||||
["file-size", "--fsize=33554432", '"file_size_bytes": 33554432'],
|
||||
for (const [option, expected] of [
|
||||
["--as=2147483648", '"address_space_bytes": 2147483648'],
|
||||
["--cpu=20", '"cpu_seconds": 20'],
|
||||
["--fsize=33554432", '"file_size_bytes": 33554432'],
|
||||
]) {
|
||||
const wrapper = join(root.path, `python-limit-${name}`);
|
||||
const quotedPython = "'" + python.replaceAll("'", "'\"'\"'") + "'";
|
||||
await writeFile(
|
||||
wrapper,
|
||||
`#!/bin/sh\nexec /usr/bin/prlimit ${option} -- ${quotedPython} "$@"\n`,
|
||||
{ mode: 0o700 },
|
||||
);
|
||||
const limitedEnvironment = { ...environment, REA_PWNTOOLS_PYTHON: wrapper };
|
||||
const selectedEnvironment = limitedEnvironment(option);
|
||||
const limitedClient = new Client({
|
||||
name: "limited-layout-verifier",
|
||||
version: "1",
|
||||
@@ -816,7 +860,7 @@ try {
|
||||
const limitedTransport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [entrypoint, "mcp"],
|
||||
env: limitedEnvironment,
|
||||
env: selectedEnvironment,
|
||||
stderr: "pipe",
|
||||
});
|
||||
try {
|
||||
@@ -826,7 +870,7 @@ try {
|
||||
mode,
|
||||
join(root.path, "protected"),
|
||||
undefined,
|
||||
limitedEnvironment,
|
||||
selectedEnvironment,
|
||||
limitedClient,
|
||||
);
|
||||
assert.ok(value.limitations.some((item) => item.includes(expected)));
|
||||
@@ -862,17 +906,29 @@ try {
|
||||
sectionHeavy.writeUInt16LE(sectionCount, 60);
|
||||
const sectionHeavyPath = join(root.path, "section-heavy.o");
|
||||
await writeFile(sectionHeavyPath, sectionHeavy);
|
||||
const memoryWrapper = join(root.path, "python-memory-constraint");
|
||||
const quotedPython = "'" + python.replaceAll("'", "'\"'\"'") + "'";
|
||||
await writeFile(
|
||||
memoryWrapper,
|
||||
`#!/bin/sh\nexec /usr/bin/prlimit --as=100663296 -- ${quotedPython} "$@"\n`,
|
||||
{ mode: 0o700 },
|
||||
// pwntools materializes each PT_LOAD zero-filled tail during inspection.
|
||||
// One allocation larger than the whole address-space budget fails promptly;
|
||||
// a large section count instead races CPU work against the command deadline.
|
||||
const memoryHeavy = Buffer.from(await readFile(join(root.path, "protected")));
|
||||
const memoryPhOffset = Number(memoryHeavy.readBigUInt64LE(32));
|
||||
const memoryPhSize = memoryHeavy.readUInt16LE(54);
|
||||
const memoryPhCount = memoryHeavy.readUInt16LE(56);
|
||||
const lastLoadIndex = Array.from(
|
||||
{ length: memoryPhCount },
|
||||
(_, index) => index,
|
||||
).findLast(
|
||||
(index) =>
|
||||
memoryHeavy.readUInt32LE(memoryPhOffset + index * memoryPhSize) === 1,
|
||||
);
|
||||
const memoryEnvironment = {
|
||||
...environment,
|
||||
REA_PWNTOOLS_PYTHON: memoryWrapper,
|
||||
};
|
||||
assert.notEqual(lastLoadIndex, undefined);
|
||||
const memorySizeOffset = memoryPhOffset + lastLoadIndex * memoryPhSize + 40;
|
||||
memoryHeavy.writeBigUInt64LE(
|
||||
memoryHeavy.readBigUInt64LE(memorySizeOffset) + 128n * 1024n * 1024n,
|
||||
memorySizeOffset,
|
||||
);
|
||||
const memoryHeavyPath = join(root.path, "memory-zero-tail");
|
||||
await writeFile(memoryHeavyPath, memoryHeavy);
|
||||
const memoryEnvironment = limitedEnvironment("--as=100663296");
|
||||
const memoryClient = new Client({
|
||||
name: "memory-layout-verifier",
|
||||
version: "1",
|
||||
@@ -888,7 +944,7 @@ try {
|
||||
for (const mode of ["cli", "mcp"]) {
|
||||
const error = await inspect(
|
||||
mode,
|
||||
sectionHeavyPath,
|
||||
memoryHeavyPath,
|
||||
"resource_constraint",
|
||||
memoryEnvironment,
|
||||
memoryClient,
|
||||
@@ -903,7 +959,7 @@ try {
|
||||
assert.ok(error.remediation.action.includes("memory"));
|
||||
cases++;
|
||||
}
|
||||
assert.deepEqual(await readFile(sectionHeavyPath), sectionHeavy);
|
||||
assert.deepEqual(await readFile(memoryHeavyPath), memoryHeavy);
|
||||
} finally {
|
||||
try {
|
||||
await memoryClient.close();
|
||||
@@ -924,13 +980,7 @@ try {
|
||||
cpuHeavy.writeBigUInt64LE(BigInt(cpuSectionCount), sectionOffset + 32);
|
||||
const cpuHeavyPath = join(root.path, "cpu-section-heavy.o");
|
||||
await writeFile(cpuHeavyPath, cpuHeavy);
|
||||
const cpuWrapper = join(root.path, "python-cpu-constraint");
|
||||
await writeFile(
|
||||
cpuWrapper,
|
||||
`#!/bin/sh\nexec /usr/bin/prlimit --cpu=1: --core=0: -- ${quotedPython} "$@"\n`,
|
||||
{ mode: 0o700 },
|
||||
);
|
||||
const cpuEnvironment = { ...environment, REA_PWNTOOLS_PYTHON: cpuWrapper };
|
||||
const cpuEnvironment = limitedEnvironment("--cpu=1:");
|
||||
const cpuClient = new Client({ name: "cpu-layout-verifier", version: "1" });
|
||||
const cpuTransport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
@@ -963,13 +1013,7 @@ try {
|
||||
}
|
||||
}
|
||||
for (const limit of [1024, 1]) {
|
||||
const wrapper = join(root.path, `python-file-size-${limit}`);
|
||||
await writeFile(
|
||||
wrapper,
|
||||
`#!/bin/sh\nexec /usr/bin/prlimit --fsize=${limit}: --core=0: -- ${quotedPython} "$@"\n`,
|
||||
{ mode: 0o700 },
|
||||
);
|
||||
const fileEnvironment = { ...environment, REA_PWNTOOLS_PYTHON: wrapper };
|
||||
const fileEnvironment = limitedEnvironment(`--fsize=${limit}:`);
|
||||
const fileClient = new Client({
|
||||
name: "file-size-layout-verifier",
|
||||
version: "1",
|
||||
|
||||
+145
-97
@@ -38,104 +38,147 @@ let update;
|
||||
let repositorySkill;
|
||||
let report;
|
||||
|
||||
const runStage = async (name, action) => {
|
||||
const started = performance.now();
|
||||
process.stderr.write(`[verify:package] ${name}: start\n`);
|
||||
const result = await action();
|
||||
process.stderr.write(
|
||||
`[verify:package] ${name}: complete (${Math.round(performance.now() - started)} ms)\n`,
|
||||
);
|
||||
return result;
|
||||
};
|
||||
|
||||
try {
|
||||
repositorySkill = await verifyRepositorySkill({ root, workspace });
|
||||
({ tarball } = await verifyPackagePack({ root, workspace }));
|
||||
const environmentData = await verifyPackageEnvironment({
|
||||
root,
|
||||
workspace,
|
||||
evidenceRoot,
|
||||
referenceRoot,
|
||||
});
|
||||
const { cli, packageRunnerCli } = await verifyPackageInstall({
|
||||
tarball,
|
||||
prefix: environmentData.prefix,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
update = await verifyPackageUpdate({
|
||||
prefix: environmentData.prefix,
|
||||
packageRoot: join(
|
||||
environmentData.prefix,
|
||||
"lib",
|
||||
"node_modules",
|
||||
PRODUCT_IDENTITY.packageName,
|
||||
),
|
||||
tarball,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
mcpStartup = await measureMcpStartup({
|
||||
command: cli,
|
||||
args: ["mcp"],
|
||||
environment: {
|
||||
...environmentData.environment,
|
||||
},
|
||||
policy: MCP_STARTUP_POLICY,
|
||||
});
|
||||
mcpModuleLoading = await profileMcpModuleLoading({
|
||||
command: cli,
|
||||
args: ["mcp"],
|
||||
environment: {
|
||||
...environmentData.environment,
|
||||
},
|
||||
policy: MCP_STARTUP_POLICY,
|
||||
packageName: PRODUCT_IDENTITY.packageName,
|
||||
});
|
||||
const { supportedSetupHost, hopperSetupSupported } =
|
||||
await verifyPackageDiscovery({
|
||||
repositorySkill = await runStage("repository skill", () =>
|
||||
verifyRepositorySkill({ root, workspace }),
|
||||
);
|
||||
({ tarball } = await runStage("package archive", () =>
|
||||
verifyPackagePack({ root, workspace }),
|
||||
));
|
||||
const environmentData = await runStage("package environment", () =>
|
||||
verifyPackageEnvironment({
|
||||
root,
|
||||
workspace,
|
||||
evidenceRoot,
|
||||
referenceRoot,
|
||||
}),
|
||||
);
|
||||
const { cli, packageRunnerCli } = await runStage("package install", () =>
|
||||
verifyPackageInstall({
|
||||
tarball,
|
||||
prefix: environmentData.prefix,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
update = await runStage("package update", () =>
|
||||
verifyPackageUpdate({
|
||||
prefix: environmentData.prefix,
|
||||
packageRoot: join(
|
||||
environmentData.prefix,
|
||||
"lib",
|
||||
"node_modules",
|
||||
PRODUCT_IDENTITY.packageName,
|
||||
),
|
||||
tarball,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
mcpStartup = await runStage("MCP startup", () =>
|
||||
measureMcpStartup({
|
||||
command: cli,
|
||||
args: ["mcp"],
|
||||
environment: {
|
||||
...environmentData.environment,
|
||||
},
|
||||
policy: MCP_STARTUP_POLICY,
|
||||
}),
|
||||
);
|
||||
mcpModuleLoading = await runStage("MCP module loading", () =>
|
||||
profileMcpModuleLoading({
|
||||
command: cli,
|
||||
args: ["mcp"],
|
||||
environment: {
|
||||
...environmentData.environment,
|
||||
},
|
||||
policy: MCP_STARTUP_POLICY,
|
||||
packageName: PRODUCT_IDENTITY.packageName,
|
||||
}),
|
||||
);
|
||||
const { supportedSetupHost, hopperSetupSupported } = await runStage(
|
||||
"package discovery",
|
||||
() =>
|
||||
verifyPackageDiscovery({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
const { artifactArchive } = await runStage("artifact and Electron CLI", () =>
|
||||
verifyPackageArtifactAndElectron({
|
||||
cli,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
await runStage("managed analysis CLI", () =>
|
||||
verifyManaged({
|
||||
cli,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
await runStage("unknown provider CLI", () =>
|
||||
verifyUnknownProvider({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
const { artifactArchive } = await verifyPackageArtifactAndElectron({
|
||||
cli,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
await verifyManaged({
|
||||
cli,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
await verifyUnknownProvider({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
await verifyPackagePlatform({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
await verifyPackageCapabilitiesAndSearch({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
await verifyPackageEvidence({
|
||||
cli,
|
||||
evidenceRoot,
|
||||
referenceRoot,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
await verifyPackageSetup({
|
||||
cli,
|
||||
packageRunnerCli,
|
||||
environment: environmentData.environment,
|
||||
home: environmentData.home,
|
||||
npxLog: environmentData.npxLog,
|
||||
claudeConfig: environmentData.claudeConfig,
|
||||
codexConfig: environmentData.codexConfig,
|
||||
cursorConfig: environmentData.cursorConfig,
|
||||
codexTarget: environmentData.codexTarget,
|
||||
cursorTarget: environmentData.cursorTarget,
|
||||
supportedSetupHost,
|
||||
hopperSetupSupported,
|
||||
root,
|
||||
});
|
||||
await verifyPackageMcp({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
evidenceRoot,
|
||||
artifactArchive,
|
||||
});
|
||||
}),
|
||||
);
|
||||
await runStage("platform CLI", () =>
|
||||
verifyPackagePlatform({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
await runStage("capabilities and search CLI", () =>
|
||||
verifyPackageCapabilitiesAndSearch({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
await runStage("evidence CLI", () =>
|
||||
verifyPackageEvidence({
|
||||
cli,
|
||||
evidenceRoot,
|
||||
referenceRoot,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
await runStage("setup and uninstall CLI", () =>
|
||||
verifyPackageSetup({
|
||||
cli,
|
||||
packageRunnerCli,
|
||||
environment: environmentData.environment,
|
||||
home: environmentData.home,
|
||||
npxLog: environmentData.npxLog,
|
||||
claudeConfig: environmentData.claudeConfig,
|
||||
codexConfig: environmentData.codexConfig,
|
||||
cursorConfig: environmentData.cursorConfig,
|
||||
codexTarget: environmentData.codexTarget,
|
||||
cursorTarget: environmentData.cursorTarget,
|
||||
supportedSetupHost,
|
||||
hopperSetupSupported,
|
||||
root,
|
||||
}),
|
||||
);
|
||||
await runStage("MCP tools and evidence", () =>
|
||||
verifyPackageMcp({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
evidenceRoot,
|
||||
artifactArchive,
|
||||
}),
|
||||
);
|
||||
report = {
|
||||
cli: true,
|
||||
analysisCli: true,
|
||||
@@ -173,8 +216,13 @@ try {
|
||||
mcpModuleLoading,
|
||||
};
|
||||
} finally {
|
||||
await rm(workspace, { recursive: true, force: true });
|
||||
await runStage("workspace cleanup", () =>
|
||||
rm(workspace, { recursive: true, force: true }),
|
||||
);
|
||||
}
|
||||
const verifierLineage = await runStage("verifier lineage", () =>
|
||||
completeVerifierRun(verifierRun),
|
||||
);
|
||||
process.stdout.write(
|
||||
`${JSON.stringify({ verifier_run: await completeVerifierRun(verifierRun), ...report })}\n`,
|
||||
`${JSON.stringify({ verifier_run: verifierLineage, ...report })}\n`,
|
||||
);
|
||||
|
||||
@@ -277,11 +277,13 @@ try {
|
||||
throw new Error("Scenario attachment terminated its external browser");
|
||||
|
||||
const profilesBefore = await scenarioProfiles();
|
||||
process.stderr.write("Browser verifier: scenario environment\n");
|
||||
const scenarioEnvironment = await verifyScenarioEnvironment(
|
||||
endpoint,
|
||||
target,
|
||||
site.origin,
|
||||
);
|
||||
process.stderr.write("Browser verifier: scenario storage\n");
|
||||
const scenarioStorage = await verifyScenarioStorage({
|
||||
executable,
|
||||
endpoint,
|
||||
@@ -339,7 +341,9 @@ try {
|
||||
`Browser scenario included unselected structured value: ${unselectedValue}`,
|
||||
);
|
||||
|
||||
process.stderr.write("Browser verifier: large screenshot\n");
|
||||
const largeScreenshot = await verifyLargeScreenshotE2e(endpoint, site.origin);
|
||||
process.stderr.write("Browser verifier: popup coverage\n");
|
||||
const popupEvents = await verifyPopupEventCoverage(executable);
|
||||
const networkEvidence = await verifyBrowserNetworkEvidence(
|
||||
executable,
|
||||
|
||||
@@ -674,6 +674,113 @@ export async function verifyRelativeSwitch(client, procedures, entrySize) {
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Oracle for relative-switch-signed.S: signed halfword byte offsets from a base
|
||||
* inside the case bodies, a CBZ route for the zero entry, and a B.CS bound.
|
||||
*/
|
||||
const SIGNED_RELATIVE_SWITCH_CASES = [
|
||||
{ value: 0, immediate: 200 },
|
||||
{ value: 1, immediate: 201 },
|
||||
{ value: 2, immediate: 202 },
|
||||
{ value: 3, immediate: 250 },
|
||||
{ value: 4, immediate: 204 },
|
||||
];
|
||||
|
||||
export async function verifyRelativeSignedSwitch(client, selectorBias = 0) {
|
||||
const names = await inventoryCall(client, "list_names", {
|
||||
document: null,
|
||||
address: null,
|
||||
});
|
||||
const procedure = requireProcedure(names, "rea_relative_switch_signed");
|
||||
const dossier = await functionCall(client, "analyze_function", {
|
||||
procedure: procedure.address,
|
||||
});
|
||||
const table = dossier.native_api?.jump_tables.find((item) =>
|
||||
item.data_sources.some(
|
||||
(source) =>
|
||||
source.entry_size_bytes === 2 &&
|
||||
source.entry_count === SIGNED_RELATIVE_SWITCH_CASES.length,
|
||||
),
|
||||
);
|
||||
if (table === undefined)
|
||||
throw new Error(
|
||||
`Exact signed halfword relative table unavailable: ${JSON.stringify(dossier.native_api)}`,
|
||||
);
|
||||
const expectedValues = SIGNED_RELATIVE_SWITCH_CASES.map(
|
||||
({ value }) => value + selectorBias,
|
||||
);
|
||||
if (
|
||||
table.mappings.some(
|
||||
(mapping) =>
|
||||
mapping.case_value !== null &&
|
||||
!expectedValues.includes(mapping.case_value),
|
||||
)
|
||||
)
|
||||
throw new Error(
|
||||
`Signed relative switch emitted an unsupported case label: ${JSON.stringify(table)}`,
|
||||
);
|
||||
if (table.mappings.length !== SIGNED_RELATIVE_SWITCH_CASES.length)
|
||||
throw new Error(
|
||||
`Signed relative switch has unexpected mappings: ${JSON.stringify(table)}`,
|
||||
);
|
||||
if (selectorBias !== 0) {
|
||||
if (
|
||||
!table.limitations.some((limitation) =>
|
||||
limitation.includes("table indices are not proven"),
|
||||
) ||
|
||||
!table.mappings.some((mapping) => mapping.case_value === null) ||
|
||||
table.mappings.some((mapping) => mapping.case_value === selectorBias + 3)
|
||||
)
|
||||
throw new Error(
|
||||
`Normalized selector must retain unknown case evidence: ${JSON.stringify(table)}`,
|
||||
);
|
||||
}
|
||||
for (const { value: tableIndex, immediate } of SIGNED_RELATIVE_SWITCH_CASES) {
|
||||
const value = tableIndex + selectorBias;
|
||||
// Without a proven relationship between the machine index and the typed
|
||||
// switch selector, the CBZ-only case must stay unknown rather than being
|
||||
// assigned the normalized index or Ghidra's unreachable base destination.
|
||||
if (selectorBias !== 0 && tableIndex === 3) continue;
|
||||
const mapping = table.mappings.find((item) => item.case_value === value);
|
||||
if (
|
||||
mapping === undefined ||
|
||||
mapping.confidence !== "high" ||
|
||||
(selectorBias !== 0 &&
|
||||
!mapping.evidence.some(
|
||||
(evidence) => evidence.source === "ghidra-clang-case-token",
|
||||
))
|
||||
)
|
||||
throw new Error(
|
||||
`Signed relative switch case ${value} was not verified: ${JSON.stringify(table)}`,
|
||||
);
|
||||
const instruction = await functionCall(
|
||||
client,
|
||||
"inspect_native_instruction",
|
||||
{ address: mapping.target_address },
|
||||
);
|
||||
if (
|
||||
instruction.status !== "decoded" ||
|
||||
!instruction.operands.some((operand) =>
|
||||
operand.components.some(
|
||||
(component) =>
|
||||
component.kind === "immediate" &&
|
||||
component.value === `0x${immediate.toString(16)}`,
|
||||
),
|
||||
)
|
||||
)
|
||||
throw new Error(
|
||||
`Signed relative switch target does not implement source case ${value}: ${JSON.stringify(instruction)}`,
|
||||
);
|
||||
}
|
||||
return {
|
||||
entry_size: 2,
|
||||
signed: true,
|
||||
selector_bias: selectorBias,
|
||||
dispatch_address: table.dispatch_address,
|
||||
mappings: table.mappings.length,
|
||||
};
|
||||
}
|
||||
|
||||
export async function verifyNativeValueTrace(client, procedures, target) {
|
||||
const entry = requireProcedure(procedures, "rea_ghidra_inventory_entry");
|
||||
const names = await inventoryCall(client, "list_names", {
|
||||
|
||||
@@ -38,6 +38,7 @@ import {
|
||||
verifyJumpTableDenseSwitch,
|
||||
verifyNativeTypeLayout,
|
||||
verifyRelativeSwitch,
|
||||
verifyRelativeSignedSwitch,
|
||||
verifyNativeValueTrace,
|
||||
} from "./verify-real-ghidra-function.mjs";
|
||||
const exec = promisify(execFile);
|
||||
@@ -173,6 +174,36 @@ try {
|
||||
]);
|
||||
}
|
||||
}
|
||||
const signedRelativeSource = fileURLToPath(
|
||||
new URL(
|
||||
"../tests/conformance/ghidra/relative-switch-signed.S",
|
||||
import.meta.url,
|
||||
),
|
||||
);
|
||||
for (const selectorBias of [0, 2, 10]) {
|
||||
for (const format of [
|
||||
"elf",
|
||||
...(expectedNativeTarget.architecture === "arm64" ? ["mach-o"] : []),
|
||||
]) {
|
||||
const targetPath = join(
|
||||
fixtureRoot,
|
||||
`relative-signed-${selectorBias}-${format}.o`,
|
||||
);
|
||||
await exec(clang, [
|
||||
...(format === "elf" ? ["--target=aarch64-linux-gnu"] : []),
|
||||
`-DREA_SELECTOR_BIAS=${selectorBias}`,
|
||||
"-c",
|
||||
signedRelativeSource,
|
||||
"-o",
|
||||
targetPath,
|
||||
]);
|
||||
crossTargets.push([
|
||||
targetPath,
|
||||
`relative-signed-${selectorBias}`,
|
||||
{ format, architecture: "arm64" },
|
||||
]);
|
||||
}
|
||||
}
|
||||
|
||||
await exec(clang, [
|
||||
"--target=aarch64-linux-gnu",
|
||||
@@ -522,25 +553,30 @@ async function verifyTarget(targetPath, variant, expectedTarget = null) {
|
||||
client,
|
||||
})
|
||||
: null;
|
||||
const probes = variant.startsWith("relative-")
|
||||
? await verifyRelativeSwitch(
|
||||
const probes = variant.startsWith("relative-signed-")
|
||||
? await verifyRelativeSignedSwitch(
|
||||
client,
|
||||
procedures,
|
||||
Number(variant.slice(-1)),
|
||||
Number(variant.split("-").at(-1)),
|
||||
)
|
||||
: variant === "type-layout"
|
||||
? await verifyNativeTypeLayout(client, names)
|
||||
: variant === "jump-table"
|
||||
? await verifyJumpTableDenseSwitch(client, procedures)
|
||||
: variant === "custom" || variant === "aarch64-jump-table"
|
||||
? null
|
||||
: await verifyInventoryOperations({
|
||||
client,
|
||||
variant,
|
||||
procedures,
|
||||
names,
|
||||
strings,
|
||||
});
|
||||
: variant.startsWith("relative-")
|
||||
? await verifyRelativeSwitch(
|
||||
client,
|
||||
procedures,
|
||||
Number(variant.slice(-1)),
|
||||
)
|
||||
: variant === "type-layout"
|
||||
? await verifyNativeTypeLayout(client, names)
|
||||
: variant === "jump-table"
|
||||
? await verifyJumpTableDenseSwitch(client, procedures)
|
||||
: variant === "custom" || variant === "aarch64-jump-table"
|
||||
? null
|
||||
: await verifyInventoryOperations({
|
||||
client,
|
||||
variant,
|
||||
procedures,
|
||||
names,
|
||||
strings,
|
||||
});
|
||||
const nativeValues =
|
||||
variant === "debug"
|
||||
? await verifyNativeValueTrace(client, procedures, parsedTarget.value)
|
||||
|
||||
@@ -6,6 +6,7 @@ import { createHash } from "node:crypto";
|
||||
import { once } from "node:events";
|
||||
import {
|
||||
access,
|
||||
copyFile,
|
||||
mkdir,
|
||||
mkdtemp,
|
||||
readFile,
|
||||
@@ -16,7 +17,7 @@ import {
|
||||
open,
|
||||
} from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join, resolve } from "node:path";
|
||||
import { dirname, join, parse, resolve } from "node:path";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { verifyWindowsProtocolInput } from "./lib/windows-protocol-input.mjs";
|
||||
|
||||
@@ -32,6 +33,9 @@ const native = requireWindowsNativeAuthority();
|
||||
const { WindowsPrivateRuntime, windowsPrivateRuntime } = await import(
|
||||
pathToFileURL(join(packageRoot, "dist/windows/WindowsPrivateRuntime.js"))
|
||||
);
|
||||
const { ghidraSessionRoot } = await import(
|
||||
pathToFileURL(join(packageRoot, "dist/ghidra/GhidraSessionRoot.js"))
|
||||
);
|
||||
const workspace = await mkdtemp(join(tmpdir(), "rea-native-conformance-"));
|
||||
const environment = Object.entries(process.env)
|
||||
.filter(([, value]) => value !== undefined)
|
||||
@@ -77,6 +81,31 @@ const wait = async (child) => {
|
||||
};
|
||||
const report = { ok: false, artifact: "verified", controls: {}, processes: {} };
|
||||
try {
|
||||
// Exercise the loader in a process whose image is not named node.exe. The
|
||||
// ordinary Windows lane otherwise cannot detect eager Node-API imports.
|
||||
const renamedHost = join(workspace, "rea-renamed-node.exe");
|
||||
const renamedHostFixture = join(workspace, "renamed-host.mjs");
|
||||
const loaderUrl = pathToFileURL(
|
||||
join(packageRoot, "dist/windows/WindowsNativeLoader.js"),
|
||||
).href;
|
||||
await copyFile(process.execPath, renamedHost);
|
||||
await writeFile(
|
||||
renamedHostFixture,
|
||||
`import {basename} from 'node:path';
|
||||
import {requireWindowsNativeAuthority} from ${JSON.stringify(loaderUrl)};
|
||||
const {inspection}=requireWindowsNativeAuthority();
|
||||
process.stdout.write(JSON.stringify({executable:basename(process.execPath),abiVersion:inspection.abiVersion}));\n`,
|
||||
);
|
||||
const renamedHostResult = execFileSync(renamedHost, [renamedHostFixture], {
|
||||
encoding: "utf8",
|
||||
timeout: 20_000,
|
||||
windowsHide: true,
|
||||
});
|
||||
assert.deepEqual(JSON.parse(renamedHostResult), {
|
||||
executable: "rea-renamed-node.exe",
|
||||
abiVersion: 1,
|
||||
});
|
||||
report.controls.renamedHostNodeApi = true;
|
||||
const sourceDirectory = join(workspace, "source");
|
||||
await mkdir(sourceDirectory);
|
||||
const source = join(sourceDirectory, "target.bin");
|
||||
@@ -110,6 +139,47 @@ try {
|
||||
])
|
||||
assert.throws(() => native.call("open", [rejected]));
|
||||
report.controls.ordinaryDriveSeparatorsAndRequestedIdentity = true;
|
||||
const driveRoot = parse(workspace).root;
|
||||
const runtimeParent = ghidraSessionRoot({
|
||||
base: join(driveRoot, ".Tmp", "scratch"),
|
||||
});
|
||||
assert.equal(runtimeParent, driveRoot);
|
||||
const driveRootRuntime = native.call("runtime_create", [
|
||||
runtimeParent,
|
||||
"rea-drive-root-",
|
||||
]);
|
||||
try {
|
||||
assert.equal(driveRootRuntime.path.startsWith(`${driveRoot}\\`), false);
|
||||
assert.equal(dirname(driveRootRuntime.path), driveRoot);
|
||||
const snapshot = await native.call("runtime_snapshot", [
|
||||
driveRootRuntime.handle,
|
||||
source,
|
||||
"probe.bin",
|
||||
]);
|
||||
const sourceBytes = await readFile(source);
|
||||
assert.equal(
|
||||
snapshot.sha256,
|
||||
createHash("sha256").update(sourceBytes).digest("hex"),
|
||||
);
|
||||
assert.equal(snapshot.source.requestedPath, source);
|
||||
const readback = native.call("runtime_open", [
|
||||
driveRootRuntime.handle,
|
||||
"probe.bin",
|
||||
]);
|
||||
try {
|
||||
assert.equal(readback.size, sourceBytes.byteLength);
|
||||
assert.deepEqual(
|
||||
native.call("read", [readback.handle, 0, 65536]),
|
||||
sourceBytes.subarray(0, 65536),
|
||||
);
|
||||
} finally {
|
||||
native.call("close", [readback.handle]);
|
||||
}
|
||||
} finally {
|
||||
native.call("runtime_close", [driveRootRuntime.handle]);
|
||||
}
|
||||
assert.equal(await exists(driveRootRuntime.path), false);
|
||||
report.controls.driveRootRuntimeParent = true;
|
||||
const snapshotOwner = WindowsPrivateRuntime.create(
|
||||
workspace.replace(/\\/u, "/"),
|
||||
"single-flight-",
|
||||
|
||||
+2
-2
@@ -8,13 +8,13 @@
|
||||
"url": "https://github.com/morluto/rea",
|
||||
"source": "github"
|
||||
},
|
||||
"version": "6.3.0",
|
||||
"version": "6.4.0",
|
||||
"packages": [
|
||||
{
|
||||
"registryType": "npm",
|
||||
"registryBaseUrl": "https://registry.npmjs.org",
|
||||
"identifier": "rea-agents",
|
||||
"version": "6.3.0",
|
||||
"version": "6.4.0",
|
||||
"runtimeHint": "npx",
|
||||
"transport": {
|
||||
"type": "stdio"
|
||||
|
||||
@@ -1,17 +1,21 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { lstat, opendir, readFile, realpath, stat } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
import { parseBinary } from "plist";
|
||||
import { z } from "zod";
|
||||
|
||||
import { preflightPropertyListDecode } from "../artifacts/apple/InterfaceBuilderDecodeBudget.js";
|
||||
import { BinaryTargetError } from "../domain/configurationErrors.js";
|
||||
import { isPathWithinRoot } from "../domain/localPath.js";
|
||||
import { parseXmlPropertyList } from "../domain/propertyListKeys.js";
|
||||
import {
|
||||
omitPrototypeKeys,
|
||||
parseXmlPropertyList,
|
||||
} from "../domain/propertyListKeys.js";
|
||||
import { decodeXmlPlistText } from "../domain/propertyListXmlText.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
/** Decoded representation bound for one binary Info.plist. */
|
||||
const BINARY_PLIST_DECODE_BYTES = 16 * 1024 * 1024;
|
||||
|
||||
/** Where one app bundle layout keeps its Info.plist and program file. */
|
||||
interface BundleLayout {
|
||||
@@ -193,9 +197,8 @@ const resolveLayoutExecutable = async (
|
||||
try {
|
||||
name =
|
||||
plist.subarray(0, 6).toString("ascii") === "bplist"
|
||||
? await (fileSystem.decodeBinaryPlist ?? readBinaryPlistExecutable)(
|
||||
plistPath,
|
||||
)
|
||||
? await (fileSystem.decodeBinaryPlist?.(plistPath) ??
|
||||
parseBinaryPlistExecutable(plist))
|
||||
: parseXmlPlistExecutable(decodeXmlPlistText(plist));
|
||||
} catch (cause: unknown) {
|
||||
return err(
|
||||
@@ -248,6 +251,10 @@ const resolveLayoutExecutable = async (
|
||||
const parseXmlPlistExecutable = (plist: string): string => {
|
||||
// An unrelated `__proto__` entry must not make the bundle unreadable.
|
||||
const { value } = parseXmlPropertyList(plist);
|
||||
return executableName(value);
|
||||
};
|
||||
|
||||
const executableName = (value: unknown): string => {
|
||||
const executable = executableEntrySchema.safeParse(value);
|
||||
if (!executable.success) throw new Error("CFBundleExecutable is missing");
|
||||
return executable.data.CFBundleExecutable;
|
||||
@@ -255,19 +262,13 @@ const parseXmlPlistExecutable = (plist: string): string => {
|
||||
|
||||
const executableEntrySchema = z.looseObject({ CFBundleExecutable: z.string() });
|
||||
|
||||
const readBinaryPlistExecutable = async (plistPath: string): Promise<string> =>
|
||||
// `-n` strips only the newline plutil appends; it requires macOS 12+.
|
||||
(
|
||||
await execFileAsync("/usr/bin/plutil", [
|
||||
"-extract",
|
||||
"CFBundleExecutable",
|
||||
"raw",
|
||||
"-n",
|
||||
"-o",
|
||||
"-",
|
||||
plistPath,
|
||||
])
|
||||
).stdout;
|
||||
/** Decode a binary plist in-process, so no host plist tool is required. */
|
||||
const parseBinaryPlistExecutable = (plist: Buffer): string => {
|
||||
preflightPropertyListDecode(plist, BINARY_PLIST_DECODE_BYTES);
|
||||
// `plist` assigns a `__proto__` key as the prototype; copying drops it, as
|
||||
// the XML path does.
|
||||
return executableName(omitPrototypeKeys(parseBinary(plist)).value);
|
||||
};
|
||||
|
||||
const isSafeExecutableName = (name: string): boolean =>
|
||||
name.length > 0 &&
|
||||
|
||||
@@ -138,7 +138,7 @@ export const runManagedProviderExecution = async (
|
||||
signal?: AbortSignal,
|
||||
): Promise<Result<AnalysisExecution, AnalysisError>> =>
|
||||
withProcessCancellation(signal, async (operationSignal) => {
|
||||
const session = dependencies.createManagedBinarySession();
|
||||
const session = await dependencies.createManagedBinarySession();
|
||||
return withSessionCleanup(
|
||||
session,
|
||||
async () => {
|
||||
@@ -194,7 +194,7 @@ const runAnalysis = async (
|
||||
const { logger, signal, snapshotPath } = options;
|
||||
const config = dependencies.readConfiguration();
|
||||
if (!config.ok) return cliError(config.error);
|
||||
const session = dependencies.createBinarySession(config.value, logger);
|
||||
const session = await dependencies.createBinarySession(config.value, logger);
|
||||
return withSessionCleanup(
|
||||
session,
|
||||
async () => {
|
||||
|
||||
@@ -10,6 +10,8 @@ export interface DirectAnalysisDependencies {
|
||||
readonly createBinarySession: (
|
||||
config: AppConfig,
|
||||
logger: Logger,
|
||||
) => BinarySession;
|
||||
readonly createManagedBinarySession: () => BinarySession;
|
||||
) => BinarySession | Promise<BinarySession>;
|
||||
readonly createManagedBinarySession: () =>
|
||||
| BinarySession
|
||||
| Promise<BinarySession>;
|
||||
}
|
||||
|
||||
@@ -15,7 +15,7 @@ export const runSessionStatus = async (
|
||||
): Promise<JsonValue> => {
|
||||
const config = dependencies.readConfiguration();
|
||||
if (!config.ok) return { error: projectAnalysisError(config.error) };
|
||||
const session = dependencies.createBinarySession(config.value, logger);
|
||||
const session = await dependencies.createBinarySession(config.value, logger);
|
||||
try {
|
||||
return {
|
||||
...jsonObjectSchema.parse(session.status()),
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
import type { EvidenceBundle } from "../../domain/evidenceBundle.js";
|
||||
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
|
||||
import { err, ok, type Result } from "../../domain/result.js";
|
||||
import {
|
||||
summarizeEvidenceBundle,
|
||||
type EvidenceBundleSummary,
|
||||
type EvidenceRecordFilters,
|
||||
} from "../../domain/evidenceRecordSummary.js";
|
||||
|
||||
/** Select complete bundle delivery or discovery metadata without analysis payloads. */
|
||||
export const inspectEvidenceBundle = (
|
||||
bundle: EvidenceBundle,
|
||||
options: {
|
||||
readonly detail: "complete" | "summary";
|
||||
readonly filters?: EvidenceRecordFilters | undefined;
|
||||
},
|
||||
): Result<EvidenceBundle | EvidenceBundleSummary, AnalysisInputError> => {
|
||||
if (options.detail === "complete" && options.filters !== undefined)
|
||||
return err(
|
||||
new AnalysisInputError("get_evidence_bundle", undefined, [
|
||||
{
|
||||
path: ["filters"],
|
||||
reason: "invalid_value",
|
||||
message:
|
||||
"Record filters require detail: summary; complete returns the canonical bundle.",
|
||||
expected: "detail: summary",
|
||||
},
|
||||
]),
|
||||
);
|
||||
return ok(
|
||||
options.detail === "summary"
|
||||
? summarizeEvidenceBundle(bundle, options.filters)
|
||||
: bundle,
|
||||
);
|
||||
};
|
||||
@@ -26,6 +26,7 @@ import {
|
||||
staticInferenceEvidence,
|
||||
} from "./JavaScriptArtifactGraphEvidence.js";
|
||||
import { resolveArtifactPathByContext } from "./JavaScriptArtifactPathResolution.js";
|
||||
import { compareUnicodeCodePoints } from "../../domain/unicodeCodePointOrder.js";
|
||||
|
||||
interface PackageRoleInput {
|
||||
readonly packageNode: ApplicationNode;
|
||||
@@ -281,6 +282,7 @@ export const addJavaScriptPackageNodes = (
|
||||
context: JavaScriptArtifactGraphContext,
|
||||
): ApplicationNode[] => {
|
||||
const roots: ApplicationNode[] = [];
|
||||
const rootPath = applicationManifestPath(context);
|
||||
for (const packageValue of context.analysis.packages) {
|
||||
const file = context.filesByPath.get(packageValue.path);
|
||||
if (file === undefined) continue;
|
||||
@@ -309,7 +311,7 @@ export const addJavaScriptPackageNodes = (
|
||||
},
|
||||
],
|
||||
});
|
||||
if (roots.length === 0) {
|
||||
if (file.path === rootPath) {
|
||||
roots.push(node);
|
||||
context.accumulator.addEdge({
|
||||
source_node_id: node.node_id,
|
||||
@@ -330,6 +332,7 @@ export const addJavaScriptPackageNodes = (
|
||||
file,
|
||||
operation: "inventory-package",
|
||||
});
|
||||
if (dependencyManifest(file.path)) continue;
|
||||
addPackageRole(context, {
|
||||
packageNode: node,
|
||||
packageFile: file,
|
||||
@@ -346,6 +349,24 @@ export const addJavaScriptPackageNodes = (
|
||||
return roots;
|
||||
};
|
||||
|
||||
/** Dependency-only artifacts retain their inventory root. */
|
||||
const applicationManifestPath = (
|
||||
context: JavaScriptArtifactGraphContext,
|
||||
): string | undefined =>
|
||||
context.analysis.packages
|
||||
.map(({ path }) => path)
|
||||
.filter(
|
||||
(path) => context.filesByPath.has(path) && !dependencyManifest(path),
|
||||
)
|
||||
.sort(
|
||||
(left, right) =>
|
||||
left.split("/").length - right.split("/").length ||
|
||||
compareUnicodeCodePoints(left, right),
|
||||
)[0];
|
||||
|
||||
const dependencyManifest = (path: string): boolean =>
|
||||
path.split("/").slice(0, -1).includes("node_modules");
|
||||
|
||||
const createFileTarget = (
|
||||
context: JavaScriptArtifactGraphContext,
|
||||
file: JavaScriptArtifactFile,
|
||||
|
||||
@@ -43,6 +43,7 @@ import {
|
||||
type WebScreenshotDiff,
|
||||
} from "../domain/webScreenshot.js";
|
||||
import { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import { AnalysisInputError } from "../domain/analysisErrorCore.js";
|
||||
import { BrowserObservationError } from "../domain/browserObservationError.js";
|
||||
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
|
||||
import { type BrowserObservationOperation } from "../domain/browserObservationErrors.js";
|
||||
@@ -314,6 +315,12 @@ export class CdpBrowserProvider implements BrowserObservationPort {
|
||||
try {
|
||||
return ok(webScreenshotDiffSchema.parse(comparePngScreenshots(input)));
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof TypeError)
|
||||
return err(
|
||||
new AnalysisInputError("compare_web_screenshots", { cause }, [
|
||||
{ path: [], reason: "invalid_format", message: cause.message },
|
||||
]),
|
||||
);
|
||||
return err(providerError(cause, "compare_web_screenshots"));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -606,3 +606,402 @@ describe("SVG xlink:href context", () => {
|
||||
expect(dangling.urls).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
const SVG_NAMESPACE = "http://www.w3.org/2000/svg";
|
||||
const XLINK_NAMESPACE = "http://www.w3.org/1999/xlink";
|
||||
const XHTML_NAMESPACE = "http://www.w3.org/1999/xhtml";
|
||||
|
||||
const xhtmlRoot = (
|
||||
attributes: readonly (readonly [string, string])[] = [
|
||||
["xmlns", XHTML_NAMESPACE],
|
||||
["xmlns:svg", SVG_NAMESPACE],
|
||||
["xmlns:xlink", XLINK_NAMESPACE],
|
||||
],
|
||||
): SvgSnapshotNode => ({
|
||||
name: "html",
|
||||
parent: 0,
|
||||
attributes,
|
||||
});
|
||||
|
||||
const approvedHref = (
|
||||
nodeIndex: number,
|
||||
url: string,
|
||||
): {
|
||||
readonly node_index: number;
|
||||
readonly attribute: "href";
|
||||
readonly url: string;
|
||||
readonly destination_scope: "approved";
|
||||
} => ({
|
||||
node_index: nodeIndex,
|
||||
attribute: "href",
|
||||
url,
|
||||
destination_scope: "approved",
|
||||
});
|
||||
|
||||
describe("XML SVG roots and namespace scope restoration", () => {
|
||||
it.each(["svg", "document"])(
|
||||
"resolves namespaces under an unprefixed XML %s root",
|
||||
(name) => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
{
|
||||
name,
|
||||
parent: 0,
|
||||
attributes: [
|
||||
["xmlns", SVG_NAMESPACE],
|
||||
["xmlns:p", XLINK_NAMESPACE],
|
||||
],
|
||||
},
|
||||
{ name: "a", parent: 1, attributes: [["p:href", "/linked"]] },
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([approvedHref(2, `${origin}/linked`)]);
|
||||
},
|
||||
);
|
||||
|
||||
it.each([2, 3, 99])(
|
||||
"rejects self-parent, cyclic and dangling XML ancestry (%s)",
|
||||
(parent) => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot(),
|
||||
{
|
||||
name: "svg:a",
|
||||
parent,
|
||||
attributes: [
|
||||
["xmlns:svg", SVG_NAMESPACE],
|
||||
["xmlns:p", XLINK_NAMESPACE],
|
||||
["p:href", "/invalid"],
|
||||
],
|
||||
},
|
||||
{ name: "svg:g", parent: 2 },
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([]);
|
||||
},
|
||||
);
|
||||
|
||||
it("restores namespace declarations and undeclarations before visiting siblings", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot(),
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [
|
||||
["xmlns:xlink", ""],
|
||||
["xlink:href", "/undeclared"],
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [["xlink:href", "/inherited"]],
|
||||
},
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [
|
||||
["xmlns:svg", "other"],
|
||||
["xlink:href", "/rebound"],
|
||||
],
|
||||
},
|
||||
{ name: "svg:a", parent: 1, attributes: [["xlink:href", "/restored"]] },
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([
|
||||
approvedHref(3, `${origin}/inherited`),
|
||||
approvedHref(5, `${origin}/restored`),
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("XHTML and XML SVG namespace prefixes", () => {
|
||||
it("reports prefixed svg:a, svg:image, and an svg:a with no svg parent", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot(),
|
||||
{ name: "svg:svg", parent: 1 },
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 2,
|
||||
attributes: [
|
||||
["xlink:href", "/xl-pa1"],
|
||||
["xlink:href", "/later"],
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "svg:image",
|
||||
parent: 2,
|
||||
attributes: [["xlink:href", "guide"]],
|
||||
},
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [["xlink:href", "/xl-orphan"]],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(result.nodes[3]?.node_name).toBe("svg:a");
|
||||
expect(result.nodes[3]?.attribute_names).toEqual(["xlink:href"]);
|
||||
expect(result.urls).toEqual([
|
||||
approvedHref(3, `${origin}/xl-pa1`),
|
||||
approvedHref(4, `${origin}/screens/assets/guide`),
|
||||
approvedHref(5, `${origin}/xl-orphan`),
|
||||
]);
|
||||
});
|
||||
|
||||
it("reports svg:a under svg:foreignObject and skips XHTML anchors there", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot([
|
||||
["xmlns", XHTML_NAMESPACE],
|
||||
["xmlns:svg", SVG_NAMESPACE],
|
||||
["xmlns:xlink", XLINK_NAMESPACE],
|
||||
["xmlns:h", XHTML_NAMESPACE],
|
||||
]),
|
||||
{ name: "svg:svg", parent: 1 },
|
||||
{ name: "svg:foreignObject", parent: 2 },
|
||||
{
|
||||
name: "a",
|
||||
parent: 3,
|
||||
attributes: [["xlink:href", "/xl-html-fo"]],
|
||||
},
|
||||
{
|
||||
name: "h:a",
|
||||
parent: 3,
|
||||
attributes: [["xlink:href", "/xl-h"]],
|
||||
},
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 3,
|
||||
attributes: [["xlink:href", "/xl-fo-svg"]],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(urlFor(result, 4)).toBeUndefined();
|
||||
expect(urlFor(result, 5)).toBeUndefined();
|
||||
expect(result.urls).toEqual([approvedHref(6, `${origin}/xl-fo-svg`)]);
|
||||
});
|
||||
|
||||
it("reads XLink by namespace, including a prefix other than xlink", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot([["xmlns", XHTML_NAMESPACE]]),
|
||||
{
|
||||
name: "svg",
|
||||
parent: 1,
|
||||
attributes: [
|
||||
["xmlns", SVG_NAMESPACE],
|
||||
["xmlns:xlink", "http://example.com/not-xlink"],
|
||||
["xmlns:x", XLINK_NAMESPACE],
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "a",
|
||||
parent: 2,
|
||||
attributes: [
|
||||
["xlink:href", "/not-xlink"],
|
||||
["x:href", "/xl-custom"],
|
||||
["x:href", "/later"],
|
||||
],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(result.nodes[3]?.attribute_names).toEqual(["xlink:href", "x:href"]);
|
||||
expect(result.urls).toEqual([approvedHref(3, `${origin}/xl-custom`)]);
|
||||
});
|
||||
|
||||
it("keeps plain href ahead of a namespaced XLink href, including an empty href", () => {
|
||||
const baseUrl = `${origin}/screens/assets/`;
|
||||
const anchor = (
|
||||
parent: number,
|
||||
attributes: readonly (readonly [string, string])[],
|
||||
): SvgSnapshotNode => ({
|
||||
name: "svg:a",
|
||||
parent,
|
||||
attributes,
|
||||
});
|
||||
const result = captureSvg(
|
||||
svgDocument(
|
||||
[
|
||||
xhtmlRoot(),
|
||||
{ name: "svg:svg", parent: 1 },
|
||||
anchor(2, [
|
||||
["xlink:href", "/xlink-loses"],
|
||||
["href", "/href-wins"],
|
||||
]),
|
||||
anchor(2, [
|
||||
["href", "/href-wins"],
|
||||
["xlink:href", "/xlink-loses"],
|
||||
]),
|
||||
anchor(2, [
|
||||
["href", ""],
|
||||
["xlink:href", "/xlink-should-lose"],
|
||||
]),
|
||||
],
|
||||
baseUrl,
|
||||
),
|
||||
);
|
||||
expect(result.urls).toEqual([
|
||||
approvedHref(3, `${origin}/href-wins`),
|
||||
approvedHref(4, `${origin}/href-wins`),
|
||||
approvedHref(5, baseUrl),
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("XHTML SVG exclusions, HTML documents, and prefix scope", () => {
|
||||
it("classifies a prefixed SVG xlink:href outside the allowed origins as outside_policy", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot(),
|
||||
{ name: "svg:svg", parent: 1 },
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 2,
|
||||
attributes: [["xlink:href", "https://cdn.example.test/out"]],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([
|
||||
{
|
||||
node_index: 3,
|
||||
attribute: "href",
|
||||
url: null,
|
||||
destination_scope: "outside_policy",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("does not treat bogus:svg in another namespace as SVG", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot([
|
||||
["xmlns", XHTML_NAMESPACE],
|
||||
["xmlns:svg", SVG_NAMESPACE],
|
||||
["xmlns:xlink", XLINK_NAMESPACE],
|
||||
["xmlns:bogus", "http://example.com/not-svg"],
|
||||
]),
|
||||
{
|
||||
name: "bogus:svg",
|
||||
parent: 1,
|
||||
attributes: [["xlink:href", "/xl-bogus"]],
|
||||
},
|
||||
{
|
||||
name: "bogus:a",
|
||||
parent: 2,
|
||||
attributes: [["xlink:href", "/xl-bogus-a"]],
|
||||
},
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [["xlink:href", "/xl-kept"]],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(result.nodes[2]?.node_name).toBe("bogus:svg");
|
||||
expect(result.urls).toEqual([approvedHref(4, `${origin}/xl-kept`)]);
|
||||
});
|
||||
|
||||
it("leaves HTML documents on literal svg and xlink:href names", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
{ name: "HTML", parent: 0 },
|
||||
{
|
||||
name: "svg",
|
||||
parent: 1,
|
||||
attributes: [
|
||||
["xmlns", SVG_NAMESPACE],
|
||||
["xmlns:x", XLINK_NAMESPACE],
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "a",
|
||||
parent: 2,
|
||||
attributes: [["xlink:href", "/xl-literal"]],
|
||||
},
|
||||
{
|
||||
name: "a",
|
||||
parent: 2,
|
||||
attributes: [["x:href", "/xl-custom-html"]],
|
||||
},
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [
|
||||
["xmlns:svg", SVG_NAMESPACE],
|
||||
["xmlns:xlink", XLINK_NAMESPACE],
|
||||
["xlink:href", "/xl-prefixed-html"],
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "A",
|
||||
parent: 1,
|
||||
attributes: [["xlink:href", "/xl-html-anchor"]],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([approvedHref(3, `${origin}/xl-literal`)]);
|
||||
});
|
||||
|
||||
it("applies an element's own xmlns declarations to its name and href", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
{ name: "html", parent: 0, attributes: [["xmlns", XHTML_NAMESPACE]] },
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [
|
||||
["xmlns:svg", SVG_NAMESPACE],
|
||||
["xmlns:xlink", XLINK_NAMESPACE],
|
||||
["xlink:href", "/xl-self"],
|
||||
],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([approvedHref(2, `${origin}/xl-self`)]);
|
||||
});
|
||||
|
||||
it("resolves a prefix declared on a later ancestor", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 2,
|
||||
attributes: [["xlink:href", "/forward"]],
|
||||
},
|
||||
{ name: "svg:svg", parent: 3 },
|
||||
xhtmlRoot(),
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([approvedHref(1, `${origin}/forward`)]);
|
||||
});
|
||||
|
||||
it("resolves a deep chain with distinct namespace declarations without quadratic expansion", () => {
|
||||
const depth = 12_000;
|
||||
const elements: SvgSnapshotNode[] = [
|
||||
xhtmlRoot(),
|
||||
{ name: "svg:svg", parent: 1 },
|
||||
];
|
||||
let parent = 2;
|
||||
for (let index = 0; index < depth; index += 1) {
|
||||
elements.push({
|
||||
name: "svg:g",
|
||||
parent,
|
||||
attributes: [[`xmlns:p${index}`, `urn:prefix:${index}`]],
|
||||
});
|
||||
parent = elements.length;
|
||||
}
|
||||
elements.push({
|
||||
name: "svg:a",
|
||||
parent,
|
||||
attributes: [["xlink:href", "/deep"]],
|
||||
});
|
||||
const snapshot = svgDocument(elements);
|
||||
const result = captureSvg(snapshot);
|
||||
expect(result.urls).toEqual([
|
||||
approvedHref(elements.length, `${origin}/deep`),
|
||||
]);
|
||||
}, 30_000);
|
||||
});
|
||||
|
||||
@@ -11,6 +11,7 @@ import {
|
||||
} from "./CdpCaptureValues.js";
|
||||
import { exclusionReasonForUrl } from "./CdpCaptureEventHelpers.js";
|
||||
import type { CdpCaptureCompleteness } from "./CdpCaptureCompleteness.js";
|
||||
import { createDocumentSvgAncestry, domSvgLink } from "./CdpCaptureDomSvg.js";
|
||||
|
||||
export type CapturedResource = Omit<
|
||||
WebPageInspection["resources"][number],
|
||||
@@ -171,24 +172,20 @@ export const captureDom = (
|
||||
? documentNodes.attributes
|
||||
: [];
|
||||
const baseIndex = nodes.length;
|
||||
const svgContexts = new Map<number, boolean>();
|
||||
const ancestry = createDocumentSvgAncestry({
|
||||
nodeTypes,
|
||||
nodeNames,
|
||||
parents,
|
||||
strings,
|
||||
attributes,
|
||||
});
|
||||
total += nodeTypes.length;
|
||||
for (let index = 0; index < nodeTypes.length; index += 1) {
|
||||
const attributeIndexes = numberArray(attributes[index]);
|
||||
const parent = Math.trunc(parents[index] ?? -1);
|
||||
const nodeIndex = nodes.length;
|
||||
const nodeName = indexedString(strings, nodeNames[index]);
|
||||
const rawAttributeNames = attributeIndexes
|
||||
.filter((_value, attributeIndex) => attributeIndex % 2 === 0)
|
||||
.map((value) => indexedString(strings, value));
|
||||
const seenAttributeNames = new Set<string>();
|
||||
const attributeNames: string[] = [];
|
||||
for (const name of rawAttributeNames) {
|
||||
if (!seenAttributeNames.has(name)) {
|
||||
seenAttributeNames.add(name);
|
||||
attributeNames.push(name);
|
||||
}
|
||||
}
|
||||
const attributeNames = uniqueAttributeNames(attributeIndexes, strings);
|
||||
nodes.push({
|
||||
index: nodeIndex,
|
||||
parent_index: parent < 0 ? -1 : baseIndex + parent,
|
||||
@@ -206,13 +203,7 @@ export const captureDom = (
|
||||
nodeIndex,
|
||||
nodeName,
|
||||
allowedOrigins,
|
||||
svgElement: elementInSvgContext(
|
||||
index,
|
||||
nodeNames,
|
||||
parents,
|
||||
strings,
|
||||
svgContexts,
|
||||
),
|
||||
...domSvgLink(ancestry, index),
|
||||
});
|
||||
for (const url of metadata.urls) {
|
||||
urls.push(url);
|
||||
@@ -367,6 +358,21 @@ const indexedString = (strings: readonly string[], index: unknown): string => {
|
||||
return integer === undefined ? "" : (strings[Math.trunc(integer)] ?? "");
|
||||
};
|
||||
|
||||
const uniqueAttributeNames = (
|
||||
attributeIndexes: readonly number[],
|
||||
strings: readonly string[],
|
||||
): string[] => {
|
||||
const names: string[] = [];
|
||||
const seen = new Set<string>();
|
||||
for (let index = 0; index < attributeIndexes.length; index += 2) {
|
||||
const name = indexedString(strings, attributeIndexes[index]);
|
||||
if (seen.has(name)) continue;
|
||||
seen.add(name);
|
||||
names.push(name);
|
||||
}
|
||||
return names;
|
||||
};
|
||||
|
||||
const numberArray = (value: unknown): readonly number[] =>
|
||||
Array.isArray(value)
|
||||
? value.flatMap((item) => {
|
||||
@@ -384,41 +390,22 @@ interface DomMetadataOptions {
|
||||
readonly nodeName: string;
|
||||
readonly allowedOrigins: ReadonlySet<string>;
|
||||
readonly svgElement: boolean;
|
||||
readonly xmlDocument: boolean;
|
||||
readonly xlinkHref: string | undefined;
|
||||
}
|
||||
|
||||
// DOMSnapshot carries no namespace URI. Chrome's HTML parser records SVG
|
||||
// local names (`svg`, `foreignObject`). Descendants of foreignObject are HTML
|
||||
// until a nested `svg`.
|
||||
const elementInSvgContext = (
|
||||
index: number,
|
||||
nodeNames: readonly number[],
|
||||
parents: readonly number[],
|
||||
strings: readonly string[],
|
||||
contexts: Map<number, boolean>,
|
||||
): boolean => {
|
||||
let current = index;
|
||||
const seen = new Set<number>();
|
||||
let inSvg = false;
|
||||
while (current >= 0 && current < nodeNames.length && !seen.has(current)) {
|
||||
const cached = contexts.get(current);
|
||||
if (cached !== undefined) {
|
||||
inSvg = cached;
|
||||
break;
|
||||
}
|
||||
seen.add(current);
|
||||
if (indexedString(strings, nodeNames[current]) === "svg") {
|
||||
inSvg = true;
|
||||
break;
|
||||
}
|
||||
const parent = Math.trunc(parents[current] ?? -1);
|
||||
if (parent < 0 || parent === current) break;
|
||||
if (indexedString(strings, nodeNames[parent]) === "foreignObject") break;
|
||||
current = parent;
|
||||
}
|
||||
// Cache the entire examined chain, including forward references and cycles,
|
||||
// so shared ancestors are examined once per document.
|
||||
for (const node of seen) contexts.set(node, inSvg);
|
||||
return inSvg;
|
||||
// Chrome reads an SVG XLink href only when href itself is absent, and an
|
||||
// empty href still wins. HTML keeps the literal attribute name `xlink:href`.
|
||||
// XML uses any prefix bound to the XLink namespace. Both are stored as href.
|
||||
const hrefAttributeValue = (
|
||||
pairs: ReadonlyMap<string, string>,
|
||||
attribute: string,
|
||||
options: DomMetadataOptions,
|
||||
): string | undefined => {
|
||||
const declared = pairs.get(attribute);
|
||||
if (declared !== undefined) return declared;
|
||||
if (attribute !== "href" || !options.svgElement) return undefined;
|
||||
return options.xmlDocument ? options.xlinkHref : pairs.get("xlink:href");
|
||||
};
|
||||
|
||||
const domMetadata = (
|
||||
@@ -435,7 +422,6 @@ const domMetadata = (
|
||||
nodeIndex,
|
||||
nodeName,
|
||||
allowedOrigins,
|
||||
svgElement,
|
||||
} = options;
|
||||
const pairs = new Map<string, string>();
|
||||
for (let index = 0; index + 1 < attributes.length; index += 2) {
|
||||
@@ -447,13 +433,7 @@ const domMetadata = (
|
||||
}
|
||||
const urls: WebPageInspection["metadata"]["dom_urls"] = [];
|
||||
for (const attribute of domUrlAttributes) {
|
||||
// Chrome reads SVG xlink:href only when href itself is absent, and an
|
||||
// empty href still wins. Report the fallback as href.
|
||||
const value =
|
||||
pairs.get(attribute) ??
|
||||
(attribute === "href" && svgElement
|
||||
? pairs.get("xlink:href")
|
||||
: undefined);
|
||||
const value = hrefAttributeValue(pairs, attribute, options);
|
||||
if (value === undefined) continue;
|
||||
const tagName = nodeName.toLowerCase();
|
||||
// Chrome strips HTML whitespace for FORM.action, while submit controls
|
||||
|
||||
@@ -0,0 +1,275 @@
|
||||
import { numberValue } from "./CdpCaptureValues.js";
|
||||
|
||||
const SVG_NAMESPACE = "http://www.w3.org/2000/svg";
|
||||
const XLINK_NAMESPACE = "http://www.w3.org/1999/xlink";
|
||||
|
||||
interface DomSnapshotNodes {
|
||||
readonly nodeTypes: readonly number[];
|
||||
readonly nodeNames: readonly number[];
|
||||
readonly parents: readonly number[];
|
||||
readonly strings: readonly string[];
|
||||
readonly attributes: readonly unknown[];
|
||||
}
|
||||
|
||||
interface XmlSvgLink {
|
||||
readonly svgElement: boolean;
|
||||
readonly xlinkHref: string | undefined;
|
||||
}
|
||||
|
||||
interface DocumentSvgAncestry {
|
||||
readonly xml: boolean;
|
||||
readonly nodes: DomSnapshotNodes;
|
||||
readonly svgContexts: Map<number, boolean>;
|
||||
readonly xmlLinks: Map<number, XmlSvgLink>;
|
||||
}
|
||||
|
||||
interface DomSvgLink {
|
||||
readonly svgElement: boolean;
|
||||
readonly xmlDocument: boolean;
|
||||
readonly xlinkHref: string | undefined;
|
||||
}
|
||||
|
||||
const UNRESOLVED: XmlSvgLink = {
|
||||
svgElement: false,
|
||||
xlinkHref: undefined,
|
||||
};
|
||||
|
||||
const stringAt = (strings: readonly string[], index: unknown): string => {
|
||||
const integer = numberValue(index);
|
||||
return integer === undefined ? "" : (strings[Math.trunc(integer)] ?? "");
|
||||
};
|
||||
|
||||
const numberArray = (value: unknown): readonly number[] =>
|
||||
Array.isArray(value)
|
||||
? value.flatMap((item) => {
|
||||
const number = numberValue(item);
|
||||
return number === undefined ? [] : [number];
|
||||
})
|
||||
: [];
|
||||
|
||||
// DOMSnapshot has no namespace URI. HTML documents use uppercase `HTML` and
|
||||
// the parser's local names. XML documents preserve prefixed names and xmlns
|
||||
// declarations, including those on standalone SVG roots.
|
||||
const xmlDocument = (nodes: DomSnapshotNodes): boolean => {
|
||||
const documentIndex = nodes.nodeTypes.findIndex(
|
||||
(nodeType) => Math.trunc(nodeType) === 9,
|
||||
);
|
||||
if (documentIndex < 0) return false;
|
||||
for (let index = 0; index < nodes.nodeTypes.length; index += 1) {
|
||||
if (Math.trunc(nodes.nodeTypes[index] ?? 0) !== 1) continue;
|
||||
if (Math.trunc(nodes.parents[index] ?? -1) !== documentIndex) continue;
|
||||
const name = stringAt(nodes.strings, nodes.nodeNames[index]);
|
||||
if (name === "HTML") return false;
|
||||
if (name === "html" || name.includes(":")) return true;
|
||||
// Explicit namespaces also identify standalone SVG and other XML roots.
|
||||
// Namespace-free synthetic snapshots retain the legacy HTML behavior.
|
||||
const attributes = numberArray(nodes.attributes[index]);
|
||||
return attributes.some(
|
||||
(value, offset) =>
|
||||
offset % 2 === 0 &&
|
||||
xmlnsPrefix(stringAt(nodes.strings, value)) !== undefined,
|
||||
);
|
||||
}
|
||||
return false;
|
||||
};
|
||||
|
||||
export const createDocumentSvgAncestry = (
|
||||
nodes: DomSnapshotNodes,
|
||||
): DocumentSvgAncestry => {
|
||||
const xml = xmlDocument(nodes);
|
||||
return {
|
||||
xml,
|
||||
nodes,
|
||||
svgContexts: new Map(),
|
||||
xmlLinks: xml ? resolveXmlElements(nodes) : new Map(),
|
||||
};
|
||||
};
|
||||
|
||||
export const domSvgLink = (
|
||||
ancestry: DocumentSvgAncestry,
|
||||
index: number,
|
||||
): DomSvgLink => {
|
||||
if (!ancestry.xml) {
|
||||
return {
|
||||
svgElement: elementInSvgContext(
|
||||
index,
|
||||
ancestry.nodes,
|
||||
ancestry.svgContexts,
|
||||
),
|
||||
xmlDocument: false,
|
||||
xlinkHref: undefined,
|
||||
};
|
||||
}
|
||||
const resolved = ancestry.xmlLinks.get(index) ?? UNRESOLVED;
|
||||
return {
|
||||
svgElement: resolved.svgElement,
|
||||
xmlDocument: true,
|
||||
xlinkHref: resolved.xlinkHref,
|
||||
};
|
||||
};
|
||||
|
||||
// Chrome's HTML parser records SVG local names (`svg`, `foreignObject`).
|
||||
// Descendants of foreignObject are HTML until a nested `svg`.
|
||||
const elementInSvgContext = (
|
||||
index: number,
|
||||
nodes: DomSnapshotNodes,
|
||||
contexts: Map<number, boolean>,
|
||||
): boolean => {
|
||||
let current = index;
|
||||
const seen = new Set<number>();
|
||||
let inSvg = false;
|
||||
while (
|
||||
current >= 0 &&
|
||||
current < nodes.nodeNames.length &&
|
||||
!seen.has(current)
|
||||
) {
|
||||
const cached = contexts.get(current);
|
||||
if (cached !== undefined) {
|
||||
inSvg = cached;
|
||||
break;
|
||||
}
|
||||
seen.add(current);
|
||||
if (stringAt(nodes.strings, nodes.nodeNames[current]) === "svg") {
|
||||
inSvg = true;
|
||||
break;
|
||||
}
|
||||
const parent = Math.trunc(nodes.parents[current] ?? -1);
|
||||
if (parent < 0 || parent === current) break;
|
||||
if (stringAt(nodes.strings, nodes.nodeNames[parent]) === "foreignObject")
|
||||
break;
|
||||
current = parent;
|
||||
}
|
||||
// Cache the entire examined chain, including forward references and cycles,
|
||||
// so shared ancestors are examined once per document.
|
||||
for (const node of seen) contexts.set(node, inSvg);
|
||||
return inSvg;
|
||||
};
|
||||
|
||||
interface NamespaceRestore {
|
||||
readonly prefix: string;
|
||||
readonly previous: string | undefined;
|
||||
}
|
||||
|
||||
type XmlWalkStep =
|
||||
| { readonly node: number }
|
||||
| { readonly restore: readonly NamespaceRestore[] };
|
||||
|
||||
// Walk parent-before-child with one mutable namespace environment. Restore
|
||||
// declarations on exit so siblings retain their own scope. Only declarations
|
||||
// on the active path are retained, rather than cumulative maps at every node.
|
||||
const resolveXmlElements = (
|
||||
nodes: DomSnapshotNodes,
|
||||
): Map<number, XmlSvgLink> => {
|
||||
const children = new Map<number, number[]>();
|
||||
const pending: XmlWalkStep[] = [];
|
||||
for (let node = 0; node < nodes.nodeNames.length; node += 1) {
|
||||
const parent = Math.trunc(nodes.parents[node] ?? -1);
|
||||
if (parent < 0) pending.push({ node });
|
||||
else if (parent < nodes.nodeNames.length && parent !== node) {
|
||||
const siblings = children.get(parent);
|
||||
if (siblings === undefined) children.set(parent, [node]);
|
||||
else siblings.push(node);
|
||||
}
|
||||
}
|
||||
const bindings = new Map<string, string>();
|
||||
const resolved = new Map<number, XmlSvgLink>();
|
||||
while (pending.length > 0) {
|
||||
const step = pending.pop();
|
||||
if (step === undefined) break;
|
||||
if ("restore" in step) {
|
||||
for (const { prefix, previous } of step.restore) {
|
||||
if (previous === undefined) bindings.delete(prefix);
|
||||
else bindings.set(prefix, previous);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
const { node } = step;
|
||||
const attributes = numberArray(nodes.attributes[node]);
|
||||
const restore = applyDeclarations(bindings, attributes, nodes.strings);
|
||||
const svgElement =
|
||||
Math.trunc(nodes.nodeTypes[node] ?? 0) === 1 &&
|
||||
elementNamespace(
|
||||
stringAt(nodes.strings, nodes.nodeNames[node]),
|
||||
bindings,
|
||||
) === SVG_NAMESPACE;
|
||||
resolved.set(node, {
|
||||
svgElement,
|
||||
xlinkHref: svgElement
|
||||
? xlinkHrefValue(attributes, nodes.strings, bindings)
|
||||
: undefined,
|
||||
});
|
||||
pending.push({ restore });
|
||||
for (const child of children.get(node) ?? []) pending.push({ node: child });
|
||||
}
|
||||
// Cycles (including self-parents) and dangling chains cannot reach a root,
|
||||
// so remain unresolved instead of deriving URLs from an invalid ancestry.
|
||||
return resolved;
|
||||
};
|
||||
|
||||
const applyDeclarations = (
|
||||
bindings: Map<string, string>,
|
||||
attributes: readonly number[],
|
||||
strings: readonly string[],
|
||||
): readonly NamespaceRestore[] => {
|
||||
const restore: NamespaceRestore[] = [];
|
||||
const seen = new Set<string>();
|
||||
for (let index = 0; index + 1 < attributes.length; index += 2) {
|
||||
const prefix = xmlnsPrefix(stringAt(strings, attributes[index]));
|
||||
if (prefix === undefined || seen.has(prefix)) continue;
|
||||
seen.add(prefix);
|
||||
restore.push({ prefix, previous: bindings.get(prefix) });
|
||||
const value = stringAt(strings, attributes[index + 1]);
|
||||
if (value.length === 0) bindings.delete(prefix);
|
||||
else bindings.set(prefix, value);
|
||||
}
|
||||
return restore;
|
||||
};
|
||||
|
||||
const xmlnsPrefix = (name: string): string | undefined => {
|
||||
if (name === "xmlns") return "";
|
||||
if (!name.startsWith("xmlns:")) return undefined;
|
||||
const prefix = name.slice("xmlns:".length);
|
||||
if (prefix.length === 0 || prefix === "xmlns" || prefix.includes(":"))
|
||||
return undefined;
|
||||
return prefix;
|
||||
};
|
||||
|
||||
const elementNamespace = (
|
||||
nodeName: string,
|
||||
bindings: ReadonlyMap<string, string>,
|
||||
): string => {
|
||||
const colon = nodeName.indexOf(":");
|
||||
if (colon < 0) return bindings.get("") ?? "";
|
||||
const prefix = nodeName.slice(0, colon);
|
||||
const local = nodeName.slice(colon + 1);
|
||||
if (
|
||||
prefix.length === 0 ||
|
||||
local.length === 0 ||
|
||||
local.includes(":") ||
|
||||
prefix === "xmlns"
|
||||
)
|
||||
return "";
|
||||
return bindings.get(prefix) ?? "";
|
||||
};
|
||||
|
||||
// First prefixed href bound to the XLink namespace. Duplicate names keep the
|
||||
// earlier value. Plain href is applied by the caller and wins even when empty.
|
||||
const xlinkHrefValue = (
|
||||
attributes: readonly number[],
|
||||
strings: readonly string[],
|
||||
bindings: ReadonlyMap<string, string>,
|
||||
): string | undefined => {
|
||||
const seen = new Set<string>();
|
||||
for (let index = 0; index + 1 < attributes.length; index += 2) {
|
||||
const name = stringAt(strings, attributes[index]);
|
||||
if (seen.has(name)) continue;
|
||||
seen.add(name);
|
||||
const colon = name.indexOf(":");
|
||||
if (colon <= 0) continue;
|
||||
const prefix = name.slice(0, colon);
|
||||
const local = name.slice(colon + 1);
|
||||
if (local !== "href" || bindings.get(prefix) !== XLINK_NAMESPACE) continue;
|
||||
return stringAt(strings, attributes[index + 1]);
|
||||
}
|
||||
return undefined;
|
||||
};
|
||||
@@ -3,6 +3,10 @@ import { crc32, deflateSync } from "node:zlib";
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { comparePngScreenshots } from "./PngVisualDiff.js";
|
||||
import {
|
||||
AnalysisResourceConstraintError,
|
||||
AnalysisUnsupportedTargetError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import {
|
||||
compareWebScreenshotsInputSchema,
|
||||
createWebScreenshotArtifact,
|
||||
@@ -117,7 +121,7 @@ describe("PNG visual diff", () => {
|
||||
});
|
||||
|
||||
it("rejects malformed PNG dimensions after validating image data", () => {
|
||||
const image = artifact(32_000_001, 1, [0, 0, 0, 255]);
|
||||
const image = artifact(2, 1, [0, 0, 0, 255]);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
@@ -149,6 +153,256 @@ describe("PNG visual diff", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("PNG comparison memory budget", () => {
|
||||
it("classifies legal huge dimensions as a memory constraint before inflate", () => {
|
||||
const image = createWebScreenshotArtifact(headerOnlyPng(0x7fffffff, 1));
|
||||
const input = compareWebScreenshotsInputSchema.parse({
|
||||
before: image,
|
||||
after: image,
|
||||
});
|
||||
|
||||
try {
|
||||
comparePngScreenshots(input);
|
||||
throw new Error("expected PNG working-memory rejection");
|
||||
} catch (cause: unknown) {
|
||||
expect(cause).toBeInstanceOf(AnalysisResourceConstraintError);
|
||||
expect(cause).toMatchObject({
|
||||
resource: "memory",
|
||||
reportedLimits: {
|
||||
maximum_working_memory_bytes: 256 * 1024 * 1024,
|
||||
estimated_working_memory_bytes: "34359738499",
|
||||
before_dimensions: { width: 0x7fffffff, height: 1 },
|
||||
after_dimensions: { width: 0x7fffffff, height: 1 },
|
||||
},
|
||||
});
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("PNG chunk integrity", () => {
|
||||
it.each(["IHDR", "IDAT", "IEND"])(
|
||||
"rejects a corrupt %s chunk checksum",
|
||||
(type) => {
|
||||
const bytes = png(1, 1, [0, 0, 0, 255]);
|
||||
let offset = 8;
|
||||
while (offset < bytes.length) {
|
||||
const length = bytes.readUInt32BE(offset);
|
||||
if (bytes.subarray(offset + 4, offset + 8).toString("ascii") === type) {
|
||||
bytes[offset + 8 + length] = (bytes[offset + 8 + length] ?? 0) ^ 1;
|
||||
break;
|
||||
}
|
||||
offset += length + 12;
|
||||
}
|
||||
const image = createWebScreenshotArtifact(bytes);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: image,
|
||||
after: image,
|
||||
}),
|
||||
),
|
||||
).toThrow("Invalid PNG chunk checksum");
|
||||
},
|
||||
);
|
||||
|
||||
it("reports a CRC-valid but corrupt compressed stream as invalid PNG data", () => {
|
||||
const header = Buffer.from(png(1, 1, [0, 0, 0, 255]).subarray(16, 29));
|
||||
const image = createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]),
|
||||
chunk("IHDR", header),
|
||||
chunk("IDAT", Buffer.from("bad")),
|
||||
chunk("IEND", Buffer.alloc(0)),
|
||||
]),
|
||||
);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({ before: image, after: image }),
|
||||
),
|
||||
).toThrow("Invalid PNG image data");
|
||||
});
|
||||
|
||||
it("rejects high-bit bytes in a CRC-valid PNG chunk name", () => {
|
||||
const bytes = png(1, 1, [0, 0, 0, 255]);
|
||||
const headerChunk = Buffer.from(bytes.subarray(8, 33));
|
||||
headerChunk[4] = 0xc9;
|
||||
headerChunk.writeUInt32BE(crc32(headerChunk.subarray(4, 21)), 21);
|
||||
const malformed = createWebScreenshotArtifact(
|
||||
Buffer.concat([bytes.subarray(0, 8), headerChunk, bytes.subarray(33)]),
|
||||
);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: malformed,
|
||||
after: malformed,
|
||||
}),
|
||||
),
|
||||
).toThrow("Invalid PNG chunk type");
|
||||
});
|
||||
|
||||
it("rejects duplicate image headers and unknown critical chunks while preserving ancillary chunks", () => {
|
||||
const bytes = png(1, 1, [0, 0, 0, 255]);
|
||||
const duplicate = Buffer.concat([
|
||||
bytes.subarray(0, 33),
|
||||
bytes.subarray(8, 33),
|
||||
bytes.subarray(33),
|
||||
]);
|
||||
const unknown = Buffer.concat([
|
||||
bytes.subarray(0, 33),
|
||||
chunk("ABCD", Buffer.alloc(0)),
|
||||
bytes.subarray(33),
|
||||
]);
|
||||
for (const malformed of [duplicate, unknown]) {
|
||||
const image = createWebScreenshotArtifact(malformed);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: image,
|
||||
after: image,
|
||||
}),
|
||||
),
|
||||
).toThrow(/Invalid PNG header order|Unsupported critical PNG chunk/u);
|
||||
}
|
||||
const ancillary = createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
bytes.subarray(0, 33),
|
||||
chunk("tEXt", Buffer.from("Comment\0original capture")),
|
||||
bytes.subarray(33),
|
||||
]),
|
||||
);
|
||||
expect(
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: ancillary,
|
||||
after: createWebScreenshotArtifact(bytes),
|
||||
}),
|
||||
).status,
|
||||
).toBe("identical");
|
||||
});
|
||||
|
||||
it("rejects bytes after IEND and reports valid unsupported PNG encodings separately", () => {
|
||||
const bytes = png(1, 1, [0, 0, 0, 255]);
|
||||
const trailing = createWebScreenshotArtifact(
|
||||
Buffer.concat([bytes, Buffer.from([0])]),
|
||||
);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: trailing,
|
||||
after: trailing,
|
||||
}),
|
||||
),
|
||||
).toThrow("Invalid PNG end chunk");
|
||||
|
||||
const grayscaleHeader = Buffer.from(bytes.subarray(16, 29));
|
||||
grayscaleHeader[9] = 0;
|
||||
const grayscale = createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]),
|
||||
chunk("IHDR", grayscaleHeader),
|
||||
chunk("IDAT", deflateSync(Buffer.from([0, 30]))),
|
||||
chunk("IEND", Buffer.alloc(0)),
|
||||
]),
|
||||
);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: grayscale,
|
||||
after: grayscale,
|
||||
}),
|
||||
),
|
||||
).toThrow(AnalysisUnsupportedTargetError);
|
||||
});
|
||||
});
|
||||
|
||||
describe("PNG chunk ordering", () => {
|
||||
it("rejects malformed palettes, interrupted data and misplaced transparency while retaining valid chunks", () => {
|
||||
const header = Buffer.from(png(1, 1, [10, 20, 30, 255]).subarray(16, 29));
|
||||
header[9] = 2;
|
||||
const ihdr = chunk("IHDR", header);
|
||||
const compressed = deflateSync(Buffer.from([0, 10, 20, 30]));
|
||||
const idat = chunk("IDAT", compressed);
|
||||
const iend = chunk("IEND", Buffer.alloc(0));
|
||||
const palette = chunk("PLTE", Buffer.from([10, 20, 30]));
|
||||
const transparency = chunk("tRNS", Buffer.from([0, 10, 0, 20, 0, 30]));
|
||||
const ancillary = chunk("tEXt", Buffer.from("Comment\0retained"));
|
||||
const image = (...chunks: readonly Buffer[]) =>
|
||||
createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]),
|
||||
...chunks,
|
||||
]),
|
||||
);
|
||||
const original = image(ihdr, idat, iend);
|
||||
const cases = [
|
||||
[ihdr, chunk("PLTE", Buffer.alloc(1)), idat, iend],
|
||||
[ihdr, palette, palette, idat, iend],
|
||||
[ihdr, idat, palette, iend],
|
||||
[ihdr, transparency, palette, idat, iend],
|
||||
[ihdr, transparency, transparency, idat, iend],
|
||||
[ihdr, idat, transparency, iend],
|
||||
[
|
||||
ihdr,
|
||||
chunk("IDAT", compressed.subarray(0, 6)),
|
||||
ancillary,
|
||||
chunk("IDAT", compressed.subarray(6)),
|
||||
iend,
|
||||
],
|
||||
[ihdr, idat, chunk("IEND", Buffer.from([1]))],
|
||||
];
|
||||
for (const chunks of cases) {
|
||||
const malformed = image(...chunks);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: malformed,
|
||||
after: original,
|
||||
}),
|
||||
),
|
||||
).toThrow(
|
||||
/Invalid PNG palette|Invalid PNG transparency order|Nonconsecutive PNG image data|Invalid PNG end chunk/u,
|
||||
);
|
||||
}
|
||||
const indexedHeader = Buffer.from(header);
|
||||
indexedHeader[8] = 1;
|
||||
indexedHeader[9] = 3;
|
||||
const emptyTransparency = image(
|
||||
chunk("IHDR", indexedHeader),
|
||||
chunk("PLTE", Buffer.from([10, 20, 30])),
|
||||
chunk("tRNS", Buffer.alloc(0)),
|
||||
chunk("IDAT", deflateSync(Buffer.from([0, 0]))),
|
||||
iend,
|
||||
);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: emptyTransparency,
|
||||
after: original,
|
||||
}),
|
||||
),
|
||||
).toThrow("Unsupported PNG transparency");
|
||||
for (const valid of [
|
||||
image(ihdr, palette, idat, iend),
|
||||
image(ihdr, ancillary, idat, iend),
|
||||
image(
|
||||
ihdr,
|
||||
chunk("IDAT", compressed.subarray(0, 6)),
|
||||
chunk("IDAT", compressed.subarray(6)),
|
||||
iend,
|
||||
),
|
||||
]) {
|
||||
expect(
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: valid,
|
||||
after: original,
|
||||
}),
|
||||
).status,
|
||||
).toBe("identical");
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
const artifact = (width: number, height: number, pixels: readonly number[]) =>
|
||||
createWebScreenshotArtifact(png(width, height, pixels));
|
||||
|
||||
@@ -173,6 +427,19 @@ const png = (
|
||||
]);
|
||||
};
|
||||
|
||||
const headerOnlyPng = (width: number, height: number): Buffer => {
|
||||
const header = Buffer.alloc(13);
|
||||
header.writeUInt32BE(width, 0);
|
||||
header.writeUInt32BE(height, 4);
|
||||
header.set([8, 6, 0, 0, 0], 8);
|
||||
return Buffer.concat([
|
||||
Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]),
|
||||
chunk("IHDR", header),
|
||||
chunk("IDAT", deflateSync(Buffer.from([0]))),
|
||||
chunk("IEND", Buffer.alloc(0)),
|
||||
]);
|
||||
};
|
||||
|
||||
const chunk = (type: string, data: Buffer): Buffer => {
|
||||
const result = Buffer.alloc(12 + data.byteLength);
|
||||
result.writeUInt32BE(data.byteLength, 0);
|
||||
|
||||
+267
-31
@@ -1,10 +1,14 @@
|
||||
import { inflateSync } from "node:zlib";
|
||||
import { constants as zlibConstants, crc32, inflateSync } from "node:zlib";
|
||||
|
||||
import type {
|
||||
CompareWebScreenshotsInput,
|
||||
WebScreenshotArtifact,
|
||||
WebScreenshotDiff,
|
||||
} from "../domain/webScreenshot.js";
|
||||
import {
|
||||
AnalysisResourceConstraintError,
|
||||
AnalysisUnsupportedTargetError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import { decodeCanonicalBase64 } from "../domain/webScreenshot.js";
|
||||
|
||||
interface DecodedPng {
|
||||
@@ -13,12 +17,32 @@ interface DecodedPng {
|
||||
readonly rgba: Buffer;
|
||||
}
|
||||
|
||||
interface ParsedPng {
|
||||
readonly bytes: Buffer;
|
||||
readonly width: number;
|
||||
readonly height: number;
|
||||
readonly channels: number;
|
||||
readonly compressed: readonly Buffer[];
|
||||
readonly transparentColor: readonly number[] | undefined;
|
||||
readonly expectedRawBytes: bigint;
|
||||
readonly unfilteredBytes: bigint;
|
||||
readonly rgbaBytes: bigint;
|
||||
}
|
||||
|
||||
/** Conservative allowance for the simultaneously live PNG comparison buffers. */
|
||||
const PNG_COMPARISON_WORKING_MEMORY_BYTES = 256 * 1024 * 1024;
|
||||
const PNG_INFLATE_CHUNK_BYTES = zlibConstants.Z_DEFAULT_CHUNK;
|
||||
|
||||
/** Compare two validated screenshots using deterministic RGBA metrics. */
|
||||
export const comparePngScreenshots = (
|
||||
input: CompareWebScreenshotsInput,
|
||||
): WebScreenshotDiff => {
|
||||
const before = decodePng(input.before);
|
||||
const after = decodePng(input.after);
|
||||
admitEncodedInputPair(input.before, input.after);
|
||||
const beforePlan = parsePng(input.before, "before");
|
||||
const afterPlan = parsePng(input.after, "after");
|
||||
admitPngPair(beforePlan, afterPlan);
|
||||
const before = decodePng(beforePlan);
|
||||
const after = decodePng(afterPlan);
|
||||
if (before.width !== after.width || before.height !== after.height)
|
||||
return {
|
||||
status: "dimension_mismatch",
|
||||
@@ -73,7 +97,10 @@ export const comparePngScreenshots = (
|
||||
};
|
||||
};
|
||||
|
||||
const decodePng = (artifact: WebScreenshotArtifact): DecodedPng => {
|
||||
const parsePng = (
|
||||
artifact: WebScreenshotArtifact,
|
||||
field: "before" | "after",
|
||||
): ParsedPng => {
|
||||
const bytes = decodeCanonicalBase64(artifact.data_base64);
|
||||
if (bytes === undefined || !bytes.subarray(0, 8).equals(PNG_SIGNATURE))
|
||||
throw new TypeError("Invalid PNG signature");
|
||||
@@ -81,68 +108,277 @@ const decodePng = (artifact: WebScreenshotArtifact): DecodedPng => {
|
||||
let header: ReturnType<typeof parseHeader> | undefined;
|
||||
const compressed: Buffer[] = [];
|
||||
let sawEnd = false;
|
||||
let sawPalette = false;
|
||||
let imageDataEnded = false;
|
||||
let paletteEntries = 0;
|
||||
let transparentColor: readonly number[] | undefined;
|
||||
let sawTransparency = false;
|
||||
let unsupportedReason: string | undefined;
|
||||
while (offset < bytes.length) {
|
||||
if (offset + 12 > bytes.length) throw new TypeError("Truncated PNG chunk");
|
||||
const length = bytes.readUInt32BE(offset);
|
||||
const type = bytes.subarray(offset + 4, offset + 8).toString("ascii");
|
||||
if (length > 0x7fffffff) throw new TypeError("Invalid PNG chunk length");
|
||||
const type = bytes.subarray(offset + 4, offset + 8).toString("latin1");
|
||||
const dataStart = offset + 8;
|
||||
const dataEnd = dataStart + length;
|
||||
if (dataEnd + 4 > bytes.length) throw new TypeError("Truncated PNG data");
|
||||
const data = bytes.subarray(dataStart, dataEnd);
|
||||
if (type === "IHDR") header = parseHeader(data);
|
||||
if (
|
||||
crc32(bytes.subarray(offset + 4, dataEnd)) !== bytes.readUInt32BE(dataEnd)
|
||||
)
|
||||
throw new TypeError("Invalid PNG chunk checksum");
|
||||
if (!/^[A-Za-z]{4}$/u.test(type) || type[2] !== type[2]?.toUpperCase())
|
||||
throw new TypeError("Invalid PNG chunk type");
|
||||
if (type !== "IDAT" && compressed.length > 0) imageDataEnded = true;
|
||||
if (type === "IHDR") {
|
||||
if (offset !== 8 || header !== undefined)
|
||||
throw new TypeError("Invalid PNG header order");
|
||||
header = parseHeader(data);
|
||||
} else if (header === undefined)
|
||||
throw new TypeError("Invalid PNG header order");
|
||||
else if (type === "tRNS") {
|
||||
if (header?.channels !== 3 || data.length !== 6)
|
||||
if (sawTransparency || compressed.length > 0)
|
||||
throw new TypeError("Invalid PNG transparency order");
|
||||
sawTransparency = true;
|
||||
if (header.colorType === 2 && data.length === 6) {
|
||||
transparentColor = [
|
||||
data.readUInt16BE(0),
|
||||
data.readUInt16BE(2),
|
||||
data.readUInt16BE(4),
|
||||
];
|
||||
} else if (
|
||||
(header.colorType === 0 && data.length === 2) ||
|
||||
(header.colorType === 3 &&
|
||||
sawPalette &&
|
||||
data.length > 0 &&
|
||||
data.length <= paletteEntries)
|
||||
) {
|
||||
unsupportedReason ??=
|
||||
"PNG transparency for this color type is not supported.";
|
||||
} else {
|
||||
throw new TypeError("Unsupported PNG transparency");
|
||||
transparentColor = [
|
||||
data.readUInt16BE(0),
|
||||
data.readUInt16BE(2),
|
||||
data.readUInt16BE(4),
|
||||
];
|
||||
} else if (type === "IDAT") compressed.push(data);
|
||||
else if (type === "IEND") {
|
||||
}
|
||||
} else if (type === "PLTE") {
|
||||
if (
|
||||
sawPalette ||
|
||||
sawTransparency ||
|
||||
compressed.length > 0 ||
|
||||
length === 0 ||
|
||||
length > 768 ||
|
||||
length % 3 !== 0 ||
|
||||
(header.colorType === 3 && length / 3 > 2 ** header.bitDepth) ||
|
||||
(header.colorType !== 2 &&
|
||||
header.colorType !== 3 &&
|
||||
header.colorType !== 6)
|
||||
)
|
||||
throw new TypeError("Invalid PNG palette");
|
||||
sawPalette = true;
|
||||
paletteEntries = length / 3;
|
||||
} else if (type === "IDAT") {
|
||||
if (imageDataEnded) throw new TypeError("Nonconsecutive PNG image data");
|
||||
compressed.push(data);
|
||||
} else if (type === "IEND") {
|
||||
if (length !== 0 || dataEnd + 4 !== bytes.length)
|
||||
throw new TypeError("Invalid PNG end chunk");
|
||||
sawEnd = true;
|
||||
break;
|
||||
} else if (/^[A-Z]/u.test(type)) {
|
||||
unsupportedReason ??= `Unsupported critical PNG chunk ${type}.`;
|
||||
}
|
||||
offset = dataEnd + 4;
|
||||
}
|
||||
if (header === undefined || !sawEnd || compressed.length === 0)
|
||||
throw new TypeError("Incomplete PNG");
|
||||
const rowBytes = header.width * header.channels;
|
||||
const expected = (rowBytes + 1) * header.height;
|
||||
const raw = inflateSync(Buffer.concat(compressed), {
|
||||
maxOutputLength: expected + 1,
|
||||
});
|
||||
if (raw.byteLength !== expected) throw new TypeError("Unexpected PNG size");
|
||||
if (header.colorType === 3 && !sawPalette)
|
||||
throw new TypeError("Missing PNG palette");
|
||||
if (unsupportedReason !== undefined)
|
||||
throw new AnalysisUnsupportedTargetError(
|
||||
"compare_web_screenshots",
|
||||
field,
|
||||
unsupportedReason,
|
||||
);
|
||||
if (!header.supported)
|
||||
throw new AnalysisUnsupportedTargetError(
|
||||
"compare_web_screenshots",
|
||||
field,
|
||||
"PNG encoding is not supported for pixel comparison.",
|
||||
);
|
||||
const pixels = BigInt(header.width) * BigInt(header.height);
|
||||
const expectedRawBytes =
|
||||
(BigInt(header.width) * BigInt(header.channels) + 1n) *
|
||||
BigInt(header.height);
|
||||
return {
|
||||
bytes,
|
||||
width: header.width,
|
||||
height: header.height,
|
||||
channels: header.channels,
|
||||
compressed,
|
||||
transparentColor,
|
||||
expectedRawBytes,
|
||||
unfilteredBytes: pixels * BigInt(header.channels),
|
||||
rgbaBytes: pixels * 4n,
|
||||
};
|
||||
};
|
||||
|
||||
const decodePng = (plan: ParsedPng): DecodedPng => {
|
||||
const expected = Number(plan.expectedRawBytes);
|
||||
const raw = inflatePng(Buffer.concat(plan.compressed), expected + 1);
|
||||
if (raw.byteLength !== expected) throw new TypeError("Unexpected PNG size");
|
||||
return {
|
||||
width: plan.width,
|
||||
height: plan.height,
|
||||
rgba: expandRgba(
|
||||
unfilter(raw, header.width, header.height, header.channels),
|
||||
header.channels,
|
||||
transparentColor,
|
||||
unfilter(raw, plan.width, plan.height, plan.channels),
|
||||
plan.channels,
|
||||
plan.transparentColor,
|
||||
),
|
||||
};
|
||||
};
|
||||
|
||||
const admitEncodedInputPair = (
|
||||
before: WebScreenshotArtifact,
|
||||
after: WebScreenshotArtifact,
|
||||
): void => {
|
||||
const sourceBytes = BigInt(before.bytes) + BigInt(after.bytes);
|
||||
if (sourceBytes > BigInt(PNG_COMPARISON_WORKING_MEMORY_BYTES))
|
||||
throw pngWorkingMemoryError(sourceBytes, null, null);
|
||||
};
|
||||
|
||||
const admitPngPair = (before: ParsedPng, after: ParsedPng): void => {
|
||||
// Input buffers remain live for both plans. For each decode, count the
|
||||
// concatenated IDAT, rounded zlib output chunks, any concatenated output and
|
||||
// unfiltered rows, plus RGB-to-RGBA output. The first completed RGBA image
|
||||
// remains live while the second is decoded.
|
||||
const inputBytes = BigInt(before.bytes.byteLength + after.bytes.byteLength);
|
||||
const estimate = (current: ParsedPng, retained: ParsedPng | undefined) => {
|
||||
const compressedBytes = current.compressed.reduce(
|
||||
(total, chunk) => total + BigInt(chunk.byteLength),
|
||||
0n,
|
||||
);
|
||||
const rgbaExpansion = current.channels === 3 ? current.rgbaBytes : 0n;
|
||||
return (
|
||||
inputBytes +
|
||||
(retained?.rgbaBytes ?? 0n) +
|
||||
compressedBytes +
|
||||
inflateBufferAllowance(current.expectedRawBytes) +
|
||||
current.unfilteredBytes +
|
||||
rgbaExpansion
|
||||
);
|
||||
};
|
||||
const estimateBytes = [
|
||||
estimate(before, undefined),
|
||||
estimate(after, before),
|
||||
].reduce((maximum, value) => (value > maximum ? value : maximum), 0n);
|
||||
if (estimateBytes > BigInt(PNG_COMPARISON_WORKING_MEMORY_BYTES))
|
||||
throw pngWorkingMemoryError(estimateBytes, before, after);
|
||||
};
|
||||
|
||||
const inflateBufferAllowance = (expectedRawBytes: bigint): bigint => {
|
||||
const chunkBytes = BigInt(PNG_INFLATE_CHUNK_BYTES);
|
||||
const maximumOutputBytes = expectedRawBytes + 1n;
|
||||
const roundUpToChunk = (bytes: bigint): bigint =>
|
||||
((bytes + chunkBytes - 1n) / chunkBytes) * chunkBytes;
|
||||
|
||||
// Node checks maxOutputLength after writing a chunk, so a rejected stream can
|
||||
// retain one chunk's worth of output past the limit before it throws.
|
||||
const rejectedOutputChunks = roundUpToChunk(
|
||||
maximumOutputBytes + chunkBytes - 1n,
|
||||
);
|
||||
|
||||
// Sync zlib retains each output chunk while Buffer.concat creates the final
|
||||
// result. A full final chunk also allocates the next scratch chunk first.
|
||||
const returnedOutputChunks = roundUpToChunk(maximumOutputBytes);
|
||||
const scratchChunk =
|
||||
expectedRawBytes % chunkBytes === 0n ||
|
||||
maximumOutputBytes % chunkBytes === 0n
|
||||
? chunkBytes
|
||||
: 0n;
|
||||
const concatenatedOutput =
|
||||
maximumOutputBytes > chunkBytes ? maximumOutputBytes : 0n;
|
||||
const returnedOutput =
|
||||
returnedOutputChunks + scratchChunk + concatenatedOutput;
|
||||
|
||||
return rejectedOutputChunks > returnedOutput
|
||||
? rejectedOutputChunks
|
||||
: returnedOutput;
|
||||
};
|
||||
|
||||
const pngWorkingMemoryError = (
|
||||
estimateBytes: bigint,
|
||||
before: Pick<ParsedPng, "width" | "height"> | null,
|
||||
after: Pick<ParsedPng, "width" | "height"> | null,
|
||||
) =>
|
||||
new AnalysisResourceConstraintError(
|
||||
"compare_web_screenshots",
|
||||
"memory",
|
||||
"Screenshot comparison exceeds REA's 256 MiB estimated working-memory budget.",
|
||||
{
|
||||
boundary: "png-screenshot-comparison",
|
||||
estimated_working_memory_bytes: estimateBytes.toString(),
|
||||
maximum_working_memory_bytes: PNG_COMPARISON_WORKING_MEMORY_BYTES,
|
||||
before_dimensions:
|
||||
before === null ? null : { width: before.width, height: before.height },
|
||||
after_dimensions:
|
||||
after === null ? null : { width: after.width, height: after.height },
|
||||
},
|
||||
{
|
||||
remediationAction:
|
||||
"Compare smaller screenshots or reduce their viewport dimensions before capture.",
|
||||
},
|
||||
);
|
||||
|
||||
const inflatePng = (compressed: Buffer, maxOutputLength: number): Buffer => {
|
||||
try {
|
||||
return inflateSync(compressed, {
|
||||
maxOutputLength,
|
||||
chunkSize: PNG_INFLATE_CHUNK_BYTES,
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
if (
|
||||
cause instanceof Error &&
|
||||
"code" in cause &&
|
||||
(cause.code === "Z_DATA_ERROR" ||
|
||||
cause.code === "Z_BUF_ERROR" ||
|
||||
cause.code === "ERR_BUFFER_TOO_LARGE")
|
||||
)
|
||||
throw new TypeError("Invalid PNG image data", { cause });
|
||||
throw cause;
|
||||
}
|
||||
};
|
||||
|
||||
const parseHeader = (data: Buffer) => {
|
||||
if (data.byteLength !== 13) throw new TypeError("Invalid PNG header");
|
||||
const width = data.readUInt32BE(0);
|
||||
const height = data.readUInt32BE(4);
|
||||
const bitDepth = data[8];
|
||||
const colorType = data[9];
|
||||
const bitDepth = data[8] ?? 0;
|
||||
const colorType = data[9] ?? -1;
|
||||
const validDepth =
|
||||
(colorType === 0 && [1, 2, 4, 8, 16].includes(bitDepth)) ||
|
||||
(colorType === 2 && [8, 16].includes(bitDepth)) ||
|
||||
(colorType === 3 && [1, 2, 4, 8].includes(bitDepth)) ||
|
||||
(colorType === 4 && [8, 16].includes(bitDepth)) ||
|
||||
(colorType === 6 && [8, 16].includes(bitDepth));
|
||||
if (
|
||||
width === 0 ||
|
||||
height === 0 ||
|
||||
!Number.isSafeInteger(width * height) ||
|
||||
bitDepth !== 8 ||
|
||||
(colorType !== 2 && colorType !== 6) ||
|
||||
width > 0x7fffffff ||
|
||||
height > 0x7fffffff ||
|
||||
!validDepth ||
|
||||
data[10] !== 0 ||
|
||||
data[11] !== 0 ||
|
||||
data[12] !== 0
|
||||
(data[12] !== 0 && data[12] !== 1)
|
||||
)
|
||||
throw new TypeError("Unsupported PNG");
|
||||
return { width, height, channels: colorType === 6 ? 4 : 3 };
|
||||
throw new TypeError("Invalid PNG header");
|
||||
return {
|
||||
width,
|
||||
height,
|
||||
colorType,
|
||||
bitDepth,
|
||||
interlace: data[12],
|
||||
channels: colorType === 6 ? 4 : 3,
|
||||
supported:
|
||||
bitDepth === 8 && (colorType === 2 || colorType === 6) && data[12] === 0,
|
||||
};
|
||||
};
|
||||
|
||||
const unfilter = (
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { access } from "node:fs/promises";
|
||||
import { access, stat } from "node:fs/promises";
|
||||
import { constants } from "node:fs";
|
||||
import { isAbsolute } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
@@ -32,6 +32,8 @@ export class MitmproxyCaptureAdapter implements HistoricalCaptureFormatAdapter {
|
||||
);
|
||||
try {
|
||||
await access(command, constants.R_OK | constants.X_OK);
|
||||
if (!(await stat(command)).isFile())
|
||||
throw new TypeError("Configured path is not a regular file");
|
||||
} catch (cause: unknown) {
|
||||
const reason = `Configured mitmdump executable is unavailable: ${command}.`;
|
||||
throw new AnalysisCapabilityUnavailableError(
|
||||
|
||||
@@ -1,6 +1,12 @@
|
||||
import { z } from "incur";
|
||||
|
||||
import { inspectAnalysisViewValidated } from "../application/analysisView/AnalysisViewService.js";
|
||||
import { readEvidenceBundle } from "../application/EvidenceBundleFiles.js";
|
||||
import { inspectEvidenceBundle } from "../application/investigation/InspectEvidenceBundle.js";
|
||||
import {
|
||||
getEvidenceBundleInputSchema,
|
||||
importEvidenceBundleInputSchema,
|
||||
} from "../contracts/sessionToolSchemas.js";
|
||||
import { CLI_COMMANDS } from "../cliCommandNames.js";
|
||||
import { parseCliJsonInput } from "../cliJsonInput.js";
|
||||
import { logCliCommand } from "../cliLogging.js";
|
||||
@@ -16,6 +22,53 @@ export const registerAnalysisViewCommands = (
|
||||
cli: CliInstance,
|
||||
logger: Logger,
|
||||
): void => {
|
||||
cli.command(CLI_COMMANDS.inspectEvidenceBundle, {
|
||||
description:
|
||||
"Inspect complete or selected discovery metadata from a local Evidence bundle",
|
||||
args: z.object({
|
||||
inputJson: z
|
||||
.string()
|
||||
.describe(
|
||||
"JSON or JSON file with absolute path, detail and optional record filters",
|
||||
),
|
||||
}),
|
||||
run: ({ args }) =>
|
||||
logCliCommand(logger, CLI_COMMANDS.inspectEvidenceBundle, async () => {
|
||||
const input = await parseCliJsonInput(
|
||||
args.inputJson,
|
||||
CLI_COMMANDS.inspectEvidenceBundle,
|
||||
);
|
||||
if (!input.ok) return input.error;
|
||||
const schema = getEvidenceBundleInputSchema.safeExtend({
|
||||
path: importEvidenceBundleInputSchema.shape.path,
|
||||
});
|
||||
const parsed = schema.safeParse(input.value);
|
||||
if (!parsed.success)
|
||||
return {
|
||||
error: "Evidence bundle inspection failed",
|
||||
...projectAnalysisError(
|
||||
analysisInputErrorFromIssues(
|
||||
CLI_COMMANDS.inspectEvidenceBundle,
|
||||
parsed.error.issues,
|
||||
input.value,
|
||||
),
|
||||
),
|
||||
};
|
||||
const loaded = await readEvidenceBundle(parsed.data.path);
|
||||
if (!loaded.ok)
|
||||
return {
|
||||
error: "Evidence bundle inspection failed",
|
||||
...projectAnalysisError(loaded.error),
|
||||
};
|
||||
const result = inspectEvidenceBundle(loaded.value, parsed.data);
|
||||
return result.ok
|
||||
? result.value
|
||||
: {
|
||||
error: "Evidence bundle inspection failed",
|
||||
...projectAnalysisError(result.error),
|
||||
};
|
||||
}),
|
||||
});
|
||||
cli.command(CLI_COMMANDS.inspectAnalysisView, {
|
||||
description:
|
||||
"Project a selected view of completed layout, JavaScript application or native function Evidence JSON",
|
||||
|
||||
@@ -43,6 +43,7 @@ export const CLI_COMMANDS = Object.freeze({
|
||||
inspectBinaryLayout: "inspect-binary-layout",
|
||||
inspectPeResources: "inspect-pe-resources",
|
||||
inspectAnalysisView: "inspect-analysis-view",
|
||||
inspectEvidenceBundle: "inspect-evidence-bundle",
|
||||
inspectFirmwareRegions: "inspect-firmware-regions",
|
||||
extractFirmware: "extract-firmware",
|
||||
inspectAndroidPackage: "inspect-android-package",
|
||||
|
||||
@@ -7,7 +7,6 @@ import {
|
||||
} from "../application/DirectAnalysis.js";
|
||||
import { runSessionStatus as executeSessionStatus } from "../application/DirectAnalysisStatus.js";
|
||||
import type { DirectAnalysisDependencies } from "../application/DirectAnalysisDependencies.js";
|
||||
import { createBinarySession, createManagedBinarySession } from "./binary.js";
|
||||
|
||||
/** Bind one-shot analysis and status commands to one caller-selected environment. */
|
||||
export const createDirectAnalysis = (
|
||||
@@ -16,9 +15,14 @@ export const createDirectAnalysis = (
|
||||
const environment = snapshotEnvironment(selectedEnvironment);
|
||||
const dependencies: DirectAnalysisDependencies = {
|
||||
readConfiguration: () => parseConfig(environment),
|
||||
createBinarySession: (config, logger) =>
|
||||
createBinarySession(config, logger, environment),
|
||||
createManagedBinarySession,
|
||||
createBinarySession: async (config, logger) => {
|
||||
const { createBinarySession } = await import("./binary.js");
|
||||
return createBinarySession(config, logger, environment);
|
||||
},
|
||||
createManagedBinarySession: async () => {
|
||||
const { createManagedBinarySession } = await import("./binary.js");
|
||||
return createManagedBinarySession();
|
||||
},
|
||||
};
|
||||
return {
|
||||
runDirectAnalysis: executeDirectAnalysis.bind(undefined, dependencies),
|
||||
@@ -34,6 +38,9 @@ export type DirectAnalysis = ReturnType<typeof createDirectAnalysis>;
|
||||
export const runManagedProviderExecution = executeManagedProvider.bind(
|
||||
undefined,
|
||||
{
|
||||
createManagedBinarySession,
|
||||
createManagedBinarySession: async () => {
|
||||
const { createManagedBinarySession } = await import("./binary.js");
|
||||
return createManagedBinarySession();
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
@@ -159,7 +159,7 @@ export const SESSION_TOOL_CONTRACTS = [
|
||||
),
|
||||
session(
|
||||
"get_evidence_bundle",
|
||||
"Return every Evidence record and residual unknown currently retained by this session as one inline bundle for direct inspection or follow-up workflows.",
|
||||
"Return every retained Evidence record and residual unknown as a complete canonical bundle by default. detail=summary discovers retained records without copying or transferring their payloads; optionally filter by exact Evidence ID, operation, target SHA-256, analysis-profile digest or canonical native procedure address. Metadata distinguishes complete record retention from native value-flow truncation or unavailable/unknown coverage. Use a discovered evidence_id with inspect_analysis_view without restarting a provider. Filters apply only to summary delivery.",
|
||||
getEvidenceBundleInputSchema,
|
||||
),
|
||||
session(
|
||||
|
||||
@@ -6,9 +6,13 @@ import { localPathStringSchema } from "../domain/localPath.js";
|
||||
import { processTraceSpecificationSchema } from "../domain/process/processTraceSpecification.js";
|
||||
import { evidenceSchema } from "../domain/evidence.js";
|
||||
import { prefixedDigestSchema } from "./../domain/digests.js";
|
||||
import { evidenceRecordFiltersSchema } from "../domain/evidenceRecordSummary.js";
|
||||
|
||||
/** Return the current canonical Evidence bundle inline. */
|
||||
export const getEvidenceBundleInputSchema = z.strictObject({});
|
||||
/** Complete retained Evidence, or selected discovery metadata without payloads. */
|
||||
export const getEvidenceBundleInputSchema = z.strictObject({
|
||||
detail: z.enum(["complete", "summary"]).default("complete"),
|
||||
filters: evidenceRecordFiltersSchema.optional(),
|
||||
});
|
||||
|
||||
/** Optional document selection for volatile navigation context. */
|
||||
export const navigationContextInputSchema = z.strictObject({
|
||||
|
||||
@@ -53,6 +53,10 @@ export const TOOL_EXAMPLE_OVERRIDES: Readonly<
|
||||
trace_native_ui_action: { action: "buildTapped:" },
|
||||
open_binary: { path: "/tmp/fixture" },
|
||||
export_evidence_bundle: { path: "/tmp/evidence.json" },
|
||||
get_evidence_bundle: {
|
||||
detail: "summary",
|
||||
filters: { operation: "analyze_function", procedure_address: "0x1000" },
|
||||
},
|
||||
inspect_address_context: { address: "0x1000" },
|
||||
import_evidence_bundle: { path: "/tmp/evidence.json" },
|
||||
capture_process_scenario: {
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { z } from "zod";
|
||||
import { evidenceBundleSummarySchema } from "../domain/evidenceRecordSummary.js";
|
||||
import { BINARY_ARCHITECTURES } from "../domain/binaryTargetTypes.js";
|
||||
import { nativeFunctionAnnotationsSchema } from "../domain/native/nativeFunctionAnnotations.js";
|
||||
import { nativeLoadImageSchema } from "../domain/native/nativeLoadImage.js";
|
||||
@@ -470,7 +471,9 @@ export const sessionOutputSchemas = {
|
||||
unknowns: z.number().int().min(0),
|
||||
}),
|
||||
),
|
||||
get_evidence_bundle: lifecycleResultOf(evidenceBundleSchema),
|
||||
get_evidence_bundle: lifecycleResultOf(
|
||||
z.union([evidenceBundleSchema, evidenceBundleSummarySchema]),
|
||||
),
|
||||
get_navigation_context: lifecycleResultOf(
|
||||
z.object({
|
||||
document: z.string(),
|
||||
|
||||
@@ -47,7 +47,7 @@ export const analysisErrorRemediationAction = (
|
||||
return (
|
||||
error.remediationAction ??
|
||||
(error.resource === "transport"
|
||||
? "Export retained session evidence through export_evidence_bundle to a caller-selected path, or use complete CLI JSON output. Tools that accept retained-evidence references can inspect compatible records. The connection remains usable."
|
||||
? "Discover retained records with get_evidence_bundle and detail: summary, then inspect compatible records through inspect_analysis_view using an exact Evidence reference. Export complete retained session evidence through export_evidence_bundle to a caller-selected path, or use complete CLI JSON output. The connection remains usable."
|
||||
: error.resource === "cpu"
|
||||
? "Review the reported worker CPU limits and observed signal. Retry with sufficient CPU time or a smaller artifact; REA retains tighter inherited limits."
|
||||
: error.resource === "file-size"
|
||||
|
||||
@@ -21,6 +21,12 @@ describe("cooperative canonical JSON digest", () => {
|
||||
),
|
||||
{ payload: `${"x".repeat(8191)}𐀀${"\n\ud800".repeat(20_000)}` },
|
||||
{ [`${"x".repeat(8191)}𐀀${"\n".repeat(20_000)}`]: "value" },
|
||||
Object.fromEntries(
|
||||
Array.from({ length: 512 }, (_, index) => [
|
||||
`escaped\n\"\\𐀀${"x".repeat(32)}${index}`,
|
||||
{ repeated: index },
|
||||
]),
|
||||
),
|
||||
];
|
||||
it.each(values.map((value) => ({ value })))(
|
||||
"preserves complete canonical meaning (%#)",
|
||||
@@ -55,6 +61,18 @@ describe("cooperative canonical JSON digest", () => {
|
||||
expect(steps.next().done).toBe(true);
|
||||
});
|
||||
|
||||
it("continues object hashing when a pending property is omitted", () => {
|
||||
const value: { [key: string]: JsonValue } = {
|
||||
a: "x".repeat(32_768),
|
||||
b: 1,
|
||||
c: { retained: true },
|
||||
};
|
||||
const steps = canonicalJsonDigestSteps(value);
|
||||
expect(steps.next().done).toBe(false);
|
||||
delete value.b;
|
||||
expect(complete(steps)).toBe(digestCanonicalValue(value));
|
||||
});
|
||||
|
||||
it("rejects sparse arrays, nonfinite numbers and cycles", () => {
|
||||
const sparse: JsonValue[] = [];
|
||||
sparse.length = 2;
|
||||
|
||||
@@ -26,6 +26,9 @@ export function* canonicalJsonDigestSteps(
|
||||
const hash = createHash("sha256");
|
||||
const ancestors = new Set<object>();
|
||||
const stack: DigestFrame[] = [{ kind: "value", value }];
|
||||
const encodedKeys = new Map<string, string>();
|
||||
// Keep reusable key encodings within one existing digest-buffer quantum.
|
||||
let remainingKeyCodeUnits = 8192;
|
||||
let buffered = "";
|
||||
let flushed = false;
|
||||
const emit = (part: string): void => {
|
||||
@@ -69,22 +72,22 @@ export function* canonicalJsonDigestSteps(
|
||||
stack.push(frame);
|
||||
continue;
|
||||
}
|
||||
let item: JsonValue;
|
||||
if (frame.kind === "array") {
|
||||
if (frame.index === frame.length) {
|
||||
emit("]");
|
||||
ancestors.delete(frame.value);
|
||||
continue;
|
||||
}
|
||||
const item = frame.value[frame.index];
|
||||
if (item === undefined)
|
||||
const element = frame.value[frame.index];
|
||||
if (element === undefined)
|
||||
throw new TypeError(
|
||||
"Canonical JSON digest requires dense JSON arrays",
|
||||
);
|
||||
if (frame.index++ > 0) emit(",");
|
||||
stack.push(frame, { kind: "value", value: item });
|
||||
continue;
|
||||
}
|
||||
if (frame.kind === "object") {
|
||||
stack.push(frame);
|
||||
item = element;
|
||||
} else if (frame.kind === "object") {
|
||||
if (frame.index === frame.keys.length) {
|
||||
emit("}");
|
||||
ancestors.delete(frame.value);
|
||||
@@ -93,25 +96,42 @@ export function* canonicalJsonDigestSteps(
|
||||
const key = frame.keys[frame.index++];
|
||||
if (key === undefined)
|
||||
throw new TypeError("Canonical JSON object changed during hashing");
|
||||
const item = frame.value[key];
|
||||
if (item === undefined) continue;
|
||||
const property = frame.value[key];
|
||||
if (property === undefined) {
|
||||
stack.push(frame);
|
||||
continue;
|
||||
}
|
||||
if (frame.written++ > 0) emit(",");
|
||||
if (key.length > 8192) {
|
||||
emit('"');
|
||||
stack.push(
|
||||
frame,
|
||||
{ kind: "value", value: item },
|
||||
{ kind: "value", value: property },
|
||||
{ kind: "separator" },
|
||||
{ kind: "string", value: key, index: 0 },
|
||||
);
|
||||
continue;
|
||||
}
|
||||
emit(JSON.stringify(key));
|
||||
let encodedKey = encodedKeys.get(key);
|
||||
if (encodedKey === undefined) {
|
||||
encodedKey = JSON.stringify(key);
|
||||
if (encodedKey.length <= remainingKeyCodeUnits) {
|
||||
encodedKeys.set(key, encodedKey);
|
||||
remainingKeyCodeUnits -= encodedKey.length;
|
||||
}
|
||||
}
|
||||
emit(encodedKey);
|
||||
emit(":");
|
||||
stack.push(frame, { kind: "value", value: item });
|
||||
continue;
|
||||
stack.push(frame);
|
||||
item = property;
|
||||
} else {
|
||||
item = frame.value;
|
||||
}
|
||||
// Resume containers directly; scalar members need no temporary value frame.
|
||||
if (flushed) {
|
||||
flushed = false;
|
||||
yield;
|
||||
}
|
||||
const item = frame.value;
|
||||
if (typeof item === "string" && item.length > 8192) {
|
||||
emit('"');
|
||||
stack.push({ kind: "string", value: item, index: 0 });
|
||||
@@ -151,6 +171,7 @@ export function* canonicalJsonDigestSteps(
|
||||
} finally {
|
||||
stack.length = 0;
|
||||
ancestors.clear();
|
||||
encodedKeys.clear();
|
||||
buffered = "";
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import { createEvidence } from "./evidence.js";
|
||||
import { createEvidenceBundle } from "./evidenceBundle.js";
|
||||
import { summarizeEvidenceBundle } from "./evidenceRecordSummary.js";
|
||||
|
||||
it("preserves native identity independently of missing or incompatible value-flow metadata", () => {
|
||||
const records = [
|
||||
undefined,
|
||||
{ available: true, truncated: true },
|
||||
{ unexpected: "legacy" },
|
||||
{ available: false, reason: "Decompiler unavailable" },
|
||||
].map((flow) =>
|
||||
createEvidence(
|
||||
{ path: "/fixtures/native.exe", format: "pe", sha256: "a".repeat(64) },
|
||||
{ id: "provider", name: "Provider", version: "1" },
|
||||
{
|
||||
operation: "analyze_function",
|
||||
parameters: {},
|
||||
result: {
|
||||
procedure: { address: "space:0x10", name: "entry" },
|
||||
...(flow === undefined ? {} : { native_value_flow: flow }),
|
||||
},
|
||||
},
|
||||
),
|
||||
);
|
||||
const summary = summarizeEvidenceBundle(createEvidenceBundle(records), {
|
||||
procedure_address: "space:0x10",
|
||||
});
|
||||
expect(summary.matching_records).toBe(4);
|
||||
const native = summary.records.map((r) => r.native_dossier);
|
||||
expect(
|
||||
native.every((r) => r.available && r.procedure_address === "space:0x10"),
|
||||
).toBe(true);
|
||||
expect(native).toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.objectContaining({ value_flow: { status: "not-recorded" } }),
|
||||
expect.objectContaining({
|
||||
value_flow: {
|
||||
status: "observed",
|
||||
truncated: true,
|
||||
omitted_operations_lower_bound: null,
|
||||
known_omitted_inputs: null,
|
||||
known_omitted_edges: null,
|
||||
},
|
||||
}),
|
||||
expect.objectContaining({
|
||||
value_flow: { status: "unknown", reason: expect.any(String) },
|
||||
}),
|
||||
expect.objectContaining({
|
||||
value_flow: { status: "unavailable", reason: "Decompiler unavailable" },
|
||||
}),
|
||||
]),
|
||||
);
|
||||
});
|
||||
@@ -0,0 +1,166 @@
|
||||
import { z } from "zod";
|
||||
|
||||
import { digestSchema, prefixedDigestSchema } from "./digests.js";
|
||||
import { evidenceSchema, type Evidence } from "./evidence.js";
|
||||
import type { EvidenceBundle } from "./evidenceBundle.js";
|
||||
|
||||
/** Exact retained-record selectors; selection never resolves a provider address. */
|
||||
export const evidenceRecordFiltersSchema = z.strictObject({
|
||||
evidence_id: prefixedDigestSchema("ev").optional(),
|
||||
operation: z.string().min(1).optional(),
|
||||
target_sha256: digestSchema.optional(),
|
||||
analysis_profile_digest: digestSchema.optional(),
|
||||
procedure_address: z.string().min(1).optional(),
|
||||
});
|
||||
export type EvidenceRecordFilters = z.infer<typeof evidenceRecordFiltersSchema>;
|
||||
|
||||
const countSchema = z.number().int().nonnegative();
|
||||
const nativeMetadataSchema = z.object({
|
||||
procedure: z.object({ address: z.string(), name: z.string().nullish() }),
|
||||
native_value_flow: z.unknown().optional(),
|
||||
});
|
||||
const valueFlowMetadataSchema = z.union([
|
||||
z.object({
|
||||
available: z.literal(true),
|
||||
truncated: z.boolean(),
|
||||
omitted_operations_lower_bound: countSchema.optional(),
|
||||
known_omitted_inputs: countSchema.optional(),
|
||||
known_omitted_edges: countSchema.optional(),
|
||||
}),
|
||||
z.object({ available: z.literal(false), reason: z.string() }),
|
||||
]);
|
||||
|
||||
const nativeSummarySchema = z.discriminatedUnion("available", [
|
||||
z.strictObject({
|
||||
available: z.literal(true),
|
||||
procedure_address: z.string(),
|
||||
procedure_name: z.string().nullable(),
|
||||
value_flow: z.discriminatedUnion("status", [
|
||||
z.strictObject({
|
||||
status: z.literal("observed"),
|
||||
truncated: z.boolean(),
|
||||
omitted_operations_lower_bound: countSchema.nullable(),
|
||||
known_omitted_inputs: countSchema.nullable(),
|
||||
known_omitted_edges: countSchema.nullable(),
|
||||
}),
|
||||
z.strictObject({ status: z.literal("unavailable"), reason: z.string() }),
|
||||
z.strictObject({ status: z.literal("not-recorded") }),
|
||||
z.strictObject({ status: z.literal("unknown"), reason: z.string() }),
|
||||
]),
|
||||
}),
|
||||
z.strictObject({
|
||||
available: z.literal(false),
|
||||
reason: z.enum(["different-operation", "not-recorded-or-incompatible"]),
|
||||
}),
|
||||
]);
|
||||
|
||||
/** Metadata for complete retained records, independent of analysis completeness. */
|
||||
export const evidenceBundleSummarySchema = z.strictObject({
|
||||
kind: z.literal("evidence-bundle-summary"),
|
||||
filters: z.record(z.string(), z.string()),
|
||||
total_retained_records: countSchema,
|
||||
matching_records: countSchema,
|
||||
unknown_revisions: countSchema,
|
||||
records: z.array(
|
||||
z.strictObject({
|
||||
evidence_id: evidenceSchema.shape.evidence_id,
|
||||
subject: evidenceSchema.shape.subject,
|
||||
provider: evidenceSchema.shape.provider,
|
||||
operation: evidenceSchema.shape.operation,
|
||||
confidence: evidenceSchema.shape.confidence,
|
||||
authority: evidenceSchema.shape.authority,
|
||||
analysis_profile_digest: digestSchema.nullable(),
|
||||
retention: z.literal("complete-record"),
|
||||
limitation_count: countSchema,
|
||||
native_dossier: nativeSummarySchema,
|
||||
}),
|
||||
),
|
||||
});
|
||||
export type EvidenceBundleSummary = z.infer<typeof evidenceBundleSummarySchema>;
|
||||
|
||||
const nativeSummary = (
|
||||
evidence: Evidence,
|
||||
): z.infer<typeof nativeSummarySchema> => {
|
||||
if (evidence.operation !== "analyze_function")
|
||||
return { available: false, reason: "different-operation" };
|
||||
// Read only producer metadata. Parsing the full dossier would allocate every
|
||||
// large collection merely to discover its already authenticated parent ID.
|
||||
const parsed = nativeMetadataSchema.safeParse(evidence.normalized_result);
|
||||
if (!parsed.success)
|
||||
return { available: false, reason: "not-recorded-or-incompatible" };
|
||||
const flow = valueFlowMetadataSchema.safeParse(parsed.data.native_value_flow);
|
||||
return {
|
||||
available: true,
|
||||
procedure_address: parsed.data.procedure.address,
|
||||
procedure_name: parsed.data.procedure.name ?? null,
|
||||
value_flow:
|
||||
parsed.data.native_value_flow == null
|
||||
? { status: "not-recorded" }
|
||||
: !flow.success
|
||||
? {
|
||||
status: "unknown",
|
||||
reason: "Value-flow metadata has an incompatible representation.",
|
||||
}
|
||||
: flow.data.available
|
||||
? {
|
||||
status: "observed",
|
||||
truncated: flow.data.truncated,
|
||||
omitted_operations_lower_bound:
|
||||
flow.data.omitted_operations_lower_bound ?? null,
|
||||
known_omitted_inputs: flow.data.known_omitted_inputs ?? null,
|
||||
known_omitted_edges: flow.data.known_omitted_edges ?? null,
|
||||
}
|
||||
: { status: "unavailable", reason: flow.data.reason },
|
||||
};
|
||||
};
|
||||
|
||||
/** Discover retained Evidence without reading or copying its complete payloads. */
|
||||
export const summarizeEvidenceBundle = (
|
||||
bundle: EvidenceBundle,
|
||||
filters: EvidenceRecordFilters = {},
|
||||
): EvidenceBundleSummary => {
|
||||
const selectedFilters: Record<string, string> = {};
|
||||
for (const [field, value] of Object.entries(filters))
|
||||
if (value !== undefined) selectedFilters[field] = value;
|
||||
const records: EvidenceBundleSummary["records"] = [];
|
||||
for (const evidence of bundle.records) {
|
||||
if (
|
||||
(filters.evidence_id !== undefined &&
|
||||
filters.evidence_id !== evidence.evidence_id) ||
|
||||
(filters.operation !== undefined &&
|
||||
filters.operation !== evidence.operation) ||
|
||||
(filters.target_sha256 !== undefined &&
|
||||
filters.target_sha256 !== evidence.subject?.digest.sha256) ||
|
||||
(filters.analysis_profile_digest !== undefined &&
|
||||
filters.analysis_profile_digest !== evidence.analysis_profile?.digest)
|
||||
)
|
||||
continue;
|
||||
const native = nativeSummary(evidence);
|
||||
if (
|
||||
filters.procedure_address !== undefined &&
|
||||
(!native.available ||
|
||||
filters.procedure_address !== native.procedure_address)
|
||||
)
|
||||
continue;
|
||||
records.push({
|
||||
evidence_id: evidence.evidence_id,
|
||||
subject: evidence.subject,
|
||||
provider: evidence.provider,
|
||||
operation: evidence.operation,
|
||||
confidence: evidence.confidence,
|
||||
authority: evidence.authority,
|
||||
analysis_profile_digest: evidence.analysis_profile?.digest ?? null,
|
||||
retention: "complete-record",
|
||||
limitation_count: evidence.limitations.length,
|
||||
native_dossier: native,
|
||||
});
|
||||
}
|
||||
return {
|
||||
kind: "evidence-bundle-summary",
|
||||
filters: selectedFilters,
|
||||
total_retained_records: bundle.records.length,
|
||||
matching_records: records.length,
|
||||
unknown_revisions: bundle.unknowns.length,
|
||||
records,
|
||||
};
|
||||
};
|
||||
@@ -33,7 +33,7 @@ export const inspectElectronBrowserNode = (
|
||||
context: JavaScriptFindingContext,
|
||||
): void => {
|
||||
if (t.isNewExpression(node)) inspectBrowserWindow(node, context);
|
||||
if (!t.isCallExpression(node)) return;
|
||||
if (!t.isCallExpression(node) && !t.isOptionalCallExpression(node)) return;
|
||||
inspectContextBridge(node, context);
|
||||
inspectUtilityProcess(node, context);
|
||||
};
|
||||
@@ -162,7 +162,7 @@ const collectWebPreferences = (
|
||||
};
|
||||
|
||||
const inspectContextBridge = (
|
||||
node: t.CallExpression,
|
||||
node: t.CallExpression | t.OptionalCallExpression,
|
||||
context: JavaScriptFindingContext,
|
||||
): void => {
|
||||
const path = electronCalleePath(node.callee, context);
|
||||
@@ -208,7 +208,7 @@ const inspectContextBridge = (
|
||||
};
|
||||
|
||||
const inspectUtilityProcess = (
|
||||
node: t.CallExpression,
|
||||
node: t.CallExpression | t.OptionalCallExpression,
|
||||
context: JavaScriptFindingContext,
|
||||
): void => {
|
||||
const path = electronCalleePath(node.callee, context);
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { analyzeJavaScriptStaticSource } from "./javascriptStaticAnalysis.js";
|
||||
|
||||
describe("Electron API identity", () => {
|
||||
it("recovers APIs through an ES module default import of Electron", () => {
|
||||
const analysis = analyzeJavaScriptStaticSource(`
|
||||
import electron from "electron";
|
||||
const { ipcMain } = electron;
|
||||
new electron.BrowserWindow({ webPreferences: { preload: "./preload.js" } });
|
||||
electron.contextBridge.exposeInMainWorld("api", { ping: true });
|
||||
electron.ipcRenderer.invoke("default:ping");
|
||||
ipcMain.handle("destructured:ping", () => "pong");
|
||||
electron.default.ipcMain.handle("nested-default", () => "local");
|
||||
`);
|
||||
|
||||
expect(analysis.electron.browser_windows).toMatchObject([
|
||||
{ preload_path: "./preload.js" },
|
||||
]);
|
||||
expect(analysis.electron.context_bridge_apis).toMatchObject([
|
||||
{ api_key: "api", members: ["ping"] },
|
||||
]);
|
||||
expect(
|
||||
analysis.electron.ipc.map(({ side, operation, channel }) => ({
|
||||
side,
|
||||
operation,
|
||||
channel,
|
||||
})),
|
||||
).toEqual([
|
||||
{ side: "renderer", operation: "invoke", channel: "default:ping" },
|
||||
{ side: "main", operation: "handle", channel: "destructured:ping" },
|
||||
]);
|
||||
});
|
||||
|
||||
it("recovers APIs and sender checks written with optional chaining", () => {
|
||||
const analysis = analyzeJavaScriptStaticSource(`
|
||||
const { contextBridge, ipcMain, utilityProcess } = require("electron");
|
||||
contextBridge?.exposeInMainWorld("api", {});
|
||||
utilityProcess?.fork("./worker.js");
|
||||
ipcMain?.handle("optional:ping", (event) => {
|
||||
if (!event.senderFrame?.url.startsWith("file://")) return null;
|
||||
return event.sender?.getURL() === "file:///index.html";
|
||||
});
|
||||
`);
|
||||
|
||||
expect(analysis.electron.context_bridge_apis).toMatchObject([
|
||||
{ api_key: "api" },
|
||||
]);
|
||||
expect(analysis.electron.utility_processes).toMatchObject([
|
||||
{ module_path: "./worker.js" },
|
||||
]);
|
||||
expect(analysis.electron.ipc).toMatchObject([
|
||||
{ side: "main", operation: "handle", channel: "optional:ping" },
|
||||
]);
|
||||
expect(
|
||||
analysis.electron.sender_validations.map(
|
||||
({ subject, mechanism, expected }) => ({
|
||||
subject,
|
||||
mechanism,
|
||||
expected: expected.value,
|
||||
}),
|
||||
),
|
||||
).toEqual([
|
||||
{
|
||||
subject: "sender-url",
|
||||
mechanism: "call:startsWith",
|
||||
expected: "file://",
|
||||
},
|
||||
{
|
||||
subject: "sender-url",
|
||||
mechanism: "comparison:===",
|
||||
expected: "file:///index.html",
|
||||
},
|
||||
]);
|
||||
});
|
||||
});
|
||||
@@ -26,7 +26,7 @@ export const inspectElectronIpcNode = (
|
||||
node: t.Node,
|
||||
context: JavaScriptFindingContext,
|
||||
): void => {
|
||||
if (t.isCallExpression(node)) {
|
||||
if (t.isCallExpression(node) || t.isOptionalCallExpression(node)) {
|
||||
inspectIpcCall(node, context);
|
||||
inspectValidationCall(node, context);
|
||||
}
|
||||
@@ -34,7 +34,7 @@ export const inspectElectronIpcNode = (
|
||||
};
|
||||
|
||||
const inspectIpcCall = (
|
||||
node: t.CallExpression,
|
||||
node: t.CallExpression | t.OptionalCallExpression,
|
||||
context: JavaScriptFindingContext,
|
||||
): void => {
|
||||
const descriptor = ipcDescriptor(electronCalleePath(node.callee, context));
|
||||
@@ -117,7 +117,7 @@ const inspectValidationComparison = (
|
||||
};
|
||||
|
||||
const inspectValidationCall = (
|
||||
node: t.CallExpression,
|
||||
node: t.CallExpression | t.OptionalCallExpression,
|
||||
context: JavaScriptFindingContext,
|
||||
): void => {
|
||||
if (
|
||||
@@ -225,9 +225,10 @@ const ipcDescriptor = (
|
||||
const senderSubject = (
|
||||
node: t.Node,
|
||||
): ElectronSenderValidationFinding["subject"] | undefined => {
|
||||
const name = t.isCallExpression(node)
|
||||
? calleeName(node.callee)
|
||||
: calleeName(node);
|
||||
const name =
|
||||
t.isCallExpression(node) || t.isOptionalCallExpression(node)
|
||||
? calleeName(node.callee)
|
||||
: calleeName(node);
|
||||
if (/(?:^|\.)sender\.getURL$/u.test(name)) return "sender-url";
|
||||
if (/(?:^|\.)senderFrame\.origin$/u.test(name)) return "sender-origin";
|
||||
if (/(?:^|\.)senderFrame\.url$/u.test(name)) return "sender-url";
|
||||
|
||||
@@ -12,6 +12,16 @@ import { traverseJavaScriptAstSteps } from "./javascriptSemanticTraversal.js";
|
||||
export const ELECTRON_MODULE =
|
||||
/^electron(?:\/(?:common|main|renderer|utility))?$/u;
|
||||
|
||||
/**
|
||||
* Electron's CommonJS exports define no `default` member, so a leading
|
||||
* `default` segment can only denote the module object itself, as an ES module
|
||||
* default import (`import electron from "electron"`) binds it.
|
||||
*/
|
||||
export const electronExportPath = (
|
||||
importedPath: readonly string[],
|
||||
): readonly string[] =>
|
||||
importedPath[0] === "default" ? importedPath.slice(1) : importedPath;
|
||||
|
||||
/** Static identity recovery cannot establish the absence of Electron usage. */
|
||||
export const ELECTRON_IDENTITY_LIMITATION =
|
||||
"Electron API identity follows proven lexical module aliases. Reassigned bindings, dynamic selections, and visibly overwritten namespace members are unresolved; missing findings do not establish absence. Arbitrary call side effects and runtime registration are not evaluated.";
|
||||
|
||||
@@ -0,0 +1,329 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { analyzeJavaScriptStaticSource } from "./javascriptStaticAnalysis.js";
|
||||
|
||||
const endpoints = (source: string) =>
|
||||
analyzeJavaScriptStaticSource(source).endpoints.map(
|
||||
({ kind, value, mechanism }) => ({ kind, value, mechanism }),
|
||||
);
|
||||
|
||||
const rpcEndpoints = (source: string) =>
|
||||
endpoints(source).filter(
|
||||
({ mechanism }) =>
|
||||
mechanism.startsWith("trpc:") || mechanism.startsWith("graphql:"),
|
||||
);
|
||||
|
||||
describe("tRPC procedure recognition", () => {
|
||||
it("recognizes React hook procedures on a renamed proxy root", () => {
|
||||
expect(
|
||||
rpcEndpoints(`
|
||||
const a = n.post.byId.useQuery({ id: 1 });
|
||||
const b = n.post.add.useMutation();
|
||||
n.feed.useInfiniteQuery({ limit: 10 });
|
||||
n.onEvent.useSubscription(undefined, { onData() {} });
|
||||
n.invoice.list.useSuspenseQuery();
|
||||
`),
|
||||
).toEqual([
|
||||
{
|
||||
kind: "network",
|
||||
value: "post.byId",
|
||||
mechanism: "trpc:procedure:useQuery",
|
||||
},
|
||||
{
|
||||
kind: "network",
|
||||
value: "post.add",
|
||||
mechanism: "trpc:procedure:useMutation",
|
||||
},
|
||||
{
|
||||
kind: "network",
|
||||
value: "feed",
|
||||
mechanism: "trpc:procedure:useInfiniteQuery",
|
||||
},
|
||||
{
|
||||
kind: "network",
|
||||
value: "onEvent",
|
||||
mechanism: "trpc:procedure:useSubscription",
|
||||
},
|
||||
{
|
||||
kind: "network",
|
||||
value: "invoice.list",
|
||||
mechanism: "trpc:procedure:useSuspenseQuery",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("recognizes TanStack React Query integration option factories", () => {
|
||||
expect(
|
||||
rpcEndpoints(`
|
||||
useQuery(e.invoice.list.queryOptions({ page: 1 }));
|
||||
useMutation(e.invoice.create.mutationOptions());
|
||||
client.invalidateQueries({ queryKey: e.invoice.list.queryKey() });
|
||||
`),
|
||||
).toEqual([
|
||||
{
|
||||
kind: "network",
|
||||
value: "invoice.list",
|
||||
mechanism: "trpc:procedure:queryOptions",
|
||||
},
|
||||
{
|
||||
kind: "network",
|
||||
value: "invoice.create",
|
||||
mechanism: "trpc:procedure:mutationOptions",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("recognizes vanilla client calls and link URLs only in sources carrying a tRPC marker", () => {
|
||||
expect(
|
||||
rpcEndpoints(`
|
||||
const client = createTRPCClient({
|
||||
links: [httpBatchLink({ url: "/api/trpc" })],
|
||||
});
|
||||
const bilbo = await client.getUser.query("id_bilbo");
|
||||
await client.post.add.mutate({ title: "x" });
|
||||
client.onPost.subscribe(undefined, { onData() {} });
|
||||
`),
|
||||
).toEqual([
|
||||
{
|
||||
kind: "network",
|
||||
value: "/api/trpc",
|
||||
mechanism: "trpc:link:httpBatchLink",
|
||||
},
|
||||
{ kind: "network", value: "getUser", mechanism: "trpc:procedure:query" },
|
||||
{
|
||||
kind: "network",
|
||||
value: "post.add",
|
||||
mechanism: "trpc:procedure:mutate",
|
||||
},
|
||||
{
|
||||
kind: "network",
|
||||
value: "onPost",
|
||||
mechanism: "trpc:procedure:subscribe",
|
||||
},
|
||||
]);
|
||||
expect(
|
||||
rpcEndpoints(`
|
||||
const rows = await pool.users.query("select 1");
|
||||
emitter.changes.subscribe(listener);
|
||||
store.cart.mutate(update);
|
||||
`),
|
||||
).toEqual([]);
|
||||
});
|
||||
|
||||
it("requires corroboration for request methods shared with Apollo, RxJS and stores", () => {
|
||||
// The source carries a tRPC marker, so only the call shape can decide.
|
||||
expect(
|
||||
rpcEndpoints(`
|
||||
class TRPCClientError extends Error {}
|
||||
this.client.query({ query: q });
|
||||
e.queryManager.query(options);
|
||||
this.untypedClient.query("path", input);
|
||||
this.route.params.subscribe((params) => params);
|
||||
t.current.subscribe(listener);
|
||||
r.store.subscribe(() => {});
|
||||
n.cache.mutate(update);
|
||||
o.k.endpoints.getPosts.useQuery();
|
||||
e.api.endpoints.list.useQuery();
|
||||
this.trpc.invoice.list.useQuery();
|
||||
`),
|
||||
).toEqual([]);
|
||||
expect(
|
||||
rpcEndpoints(`
|
||||
class TRPCClientError extends Error {}
|
||||
e.invoice.list.query({ page: 1 });
|
||||
e.invoice.onChange.subscribe(undefined, { onData() {} });
|
||||
`),
|
||||
).toEqual([
|
||||
{
|
||||
kind: "network",
|
||||
value: "invoice.list",
|
||||
mechanism: "trpc:procedure:query",
|
||||
},
|
||||
{
|
||||
kind: "network",
|
||||
value: "invoice.onChange",
|
||||
mechanism: "trpc:procedure:subscribe",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("never reports browser globals or RTK Query endpoints as procedures", () => {
|
||||
expect(
|
||||
rpcEndpoints(`
|
||||
class TRPCClientError extends Error {}
|
||||
navigator.permissions.query({ name: "geolocation" });
|
||||
window.navigator.permissions.query({ name: "camera" });
|
||||
api.endpoints.getPosts.useQuery();
|
||||
api.useGetPostsQuery();
|
||||
useQuery({ queryKey: ["x"] });
|
||||
`),
|
||||
).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("tRPC binding identity and bundled syntax", () => {
|
||||
it("keeps client bindings local to their lexical declaration", () => {
|
||||
expect(
|
||||
rpcEndpoints(`
|
||||
function first() { const c = createTRPCClient({}); c.users.query(); }
|
||||
function unrelated(c) { c.users.query("select 1"); }
|
||||
const c = createTRPCClient({});
|
||||
function closure() { c.posts.query(); }
|
||||
function shadowed() { c.users.query(); var c = pool; }
|
||||
{ const c = pool; c.users.query(); }
|
||||
try {} catch (c) { c.users.query(); }
|
||||
`),
|
||||
).toEqual([
|
||||
{ kind: "network", value: "users", mechanism: "trpc:procedure:query" },
|
||||
{ kind: "network", value: "posts", mechanism: "trpc:procedure:query" },
|
||||
]);
|
||||
});
|
||||
|
||||
it("does not corroborate overwritten or destructured client bindings", () => {
|
||||
expect(
|
||||
rpcEndpoints(`
|
||||
let c = createTRPCClient({}); c = pool; c.users.query();
|
||||
const { client } = createTRPCClient({}); client.users.query();
|
||||
`),
|
||||
).toEqual([]);
|
||||
expect(
|
||||
rpcEndpoints(`
|
||||
var c = createTRPCClient({}); c.users.query();
|
||||
`),
|
||||
).toEqual([
|
||||
{ kind: "network", value: "users", mechanism: "trpc:procedure:query" },
|
||||
]);
|
||||
});
|
||||
|
||||
it("recognizes surviving factory names inside bundled call wrappers", () => {
|
||||
expect(
|
||||
rpcEndpoints(`
|
||||
const c = (0, t.createTRPCClient)({
|
||||
links: [(0, t.httpBatchLink)({ url: "/api/rpc" })],
|
||||
});
|
||||
c.users.query();
|
||||
const p = (t.createTRPCProxyClient as any)({});
|
||||
p.posts.query();
|
||||
(0, t.httpLink)({ url: "/api/other" });
|
||||
`),
|
||||
).toEqual([
|
||||
{
|
||||
kind: "network",
|
||||
value: "/api/rpc",
|
||||
mechanism: "trpc:link:httpBatchLink",
|
||||
},
|
||||
{ kind: "network", value: "users", mechanism: "trpc:procedure:query" },
|
||||
{ kind: "network", value: "posts", mechanism: "trpc:procedure:query" },
|
||||
{ kind: "network", value: "/api/other", mechanism: "trpc:link:httpLink" },
|
||||
]);
|
||||
});
|
||||
|
||||
it("preserves procedure paths beyond the former nesting cutoff", () => {
|
||||
const path = Array.from({ length: 64 }, (_, i) => `router${i}`).join(".");
|
||||
expect(rpcEndpoints(`root.${path}.useQuery();`)).toEqual([
|
||||
{ kind: "network", value: path, mechanism: "trpc:procedure:useQuery" },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("GraphQL operation recognition", () => {
|
||||
it("recognizes named operations in tagged templates and string literals", () => {
|
||||
expect(
|
||||
rpcEndpoints(`
|
||||
const GET_USER = gql\`
|
||||
# comment
|
||||
query GetUser($id: ID!) { user(id: $id) { name } }
|
||||
\`;
|
||||
const UPDATE = "mutation UpdateInvoice($input: InvoiceInput!) { updateInvoice(input: $input) { id } }";
|
||||
const ON = \`subscription OnInvoice { invoiceChanged { id } } \${fragment}\`;
|
||||
`),
|
||||
).toEqual([
|
||||
{ kind: "network", value: "GetUser", mechanism: "graphql:query" },
|
||||
{
|
||||
kind: "network",
|
||||
value: "UpdateInvoice",
|
||||
mechanism: "graphql:mutation",
|
||||
},
|
||||
{
|
||||
kind: "network",
|
||||
value: "OnInvoice",
|
||||
mechanism: "graphql:subscription",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("retains operations after long ignored prefixes and long headers", () => {
|
||||
const name = "Operation" + "x".repeat(512);
|
||||
const text =
|
||||
"#" +
|
||||
"comment".repeat(700) +
|
||||
"\nquery " +
|
||||
name +
|
||||
" ".repeat(600) +
|
||||
"{ users { id } }";
|
||||
expect(rpcEndpoints(`const doc = ${JSON.stringify(text)};`)).toEqual([
|
||||
{ kind: "network", value: name, mechanism: "graphql:query" },
|
||||
]);
|
||||
});
|
||||
|
||||
it("recognizes precompiled GraphQL document objects", () => {
|
||||
expect(
|
||||
rpcEndpoints(`
|
||||
const doc = { kind: "Document", definitions: [{
|
||||
kind: "OperationDefinition",
|
||||
operation: "query",
|
||||
name: { kind: "Name", value: "ListInvoices" },
|
||||
selectionSet: { kind: "SelectionSet", selections: [] },
|
||||
}] };
|
||||
`),
|
||||
).toEqual([
|
||||
{ kind: "network", value: "ListInvoices", mechanism: "graphql:query" },
|
||||
]);
|
||||
});
|
||||
|
||||
it("ignores anonymous operations and ordinary prose", () => {
|
||||
expect(
|
||||
rpcEndpoints(`
|
||||
const a = "query { viewer { id } }";
|
||||
const b = "query string parameters";
|
||||
const c = "mutation observer started";
|
||||
const d = { kind: "OperationDefinition", operation: "query" };
|
||||
const e = "mutation Observer {";
|
||||
const f = "query Search(";
|
||||
const g = "subscription Plan (monthly)";
|
||||
`),
|
||||
).toEqual([]);
|
||||
});
|
||||
|
||||
it("reports a precompiled graphql-tag document once, not again from loc.source.body", () => {
|
||||
expect(
|
||||
rpcEndpoints(`
|
||||
const doc = { kind: "Document", definitions: [{
|
||||
kind: "OperationDefinition",
|
||||
operation: "query",
|
||||
name: { kind: "Name", value: "GetUser" },
|
||||
selectionSet: { kind: "SelectionSet", selections: [] },
|
||||
}], loc: { start: 0, end: 40, source: {
|
||||
body: "query GetUser($id: ID!) { user(id: $id) { name } }",
|
||||
name: "GraphQL request",
|
||||
locationOffset: { line: 1, column: 1 },
|
||||
} } };
|
||||
`),
|
||||
).toEqual([
|
||||
{ kind: "network", value: "GetUser", mechanism: "graphql:query" },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps existing network endpoint recognition unchanged", () => {
|
||||
expect(
|
||||
endpoints(`fetch("/api/invoices"); n.invoice.list.useQuery();`),
|
||||
).toEqual([
|
||||
{ kind: "network", value: "/api/invoices", mechanism: "call:fetch" },
|
||||
{
|
||||
kind: "network",
|
||||
value: "invoice.list",
|
||||
mechanism: "trpc:procedure:useQuery",
|
||||
},
|
||||
]);
|
||||
});
|
||||
@@ -0,0 +1,322 @@
|
||||
import * as t from "@babel/types";
|
||||
|
||||
import {
|
||||
unwrapJavaScriptExpression,
|
||||
semanticStaticPropertyKey,
|
||||
} from "./javascriptAstValues.js";
|
||||
import { calleeName } from "./javascriptStaticAnalysisHelpers.js";
|
||||
|
||||
/** One RPC operation or transport URL recovered from client-side syntax. */
|
||||
export interface JavaScriptRpcEndpoint {
|
||||
readonly value: string;
|
||||
readonly mechanism: string;
|
||||
}
|
||||
|
||||
/** Per-source RPC recognition state; feed it every AST node in pre-order. */
|
||||
export interface JavaScriptRpcScan {
|
||||
inspect(node: t.Node): JavaScriptRpcEndpoint | undefined;
|
||||
}
|
||||
|
||||
// tRPC proxies preserve router and procedure names as property names, so they
|
||||
// survive identifier minification. Method names follow @trpc/react-query and
|
||||
// @trpc/tanstack-react-query 11.19 documentation; cache-key helpers such as
|
||||
// queryKey are excluded because they never issue a request.
|
||||
const TRPC_HOOK_METHODS: ReadonlySet<string> = new Set([
|
||||
"useQuery",
|
||||
"useSuspenseQuery",
|
||||
"useInfiniteQuery",
|
||||
"useSuspenseInfiniteQuery",
|
||||
"usePrefetchQuery",
|
||||
"usePrefetchInfiniteQuery",
|
||||
"useMutation",
|
||||
"useSubscription",
|
||||
"queryOptions",
|
||||
"infiniteQueryOptions",
|
||||
"mutationOptions",
|
||||
"subscriptionOptions",
|
||||
]);
|
||||
|
||||
// @trpc/client 11.19 clientCallTypeMap methods; common outside tRPC (Apollo,
|
||||
// database pools, stores), so they need corroborating evidence.
|
||||
const TRPC_REQUEST_METHODS: ReadonlySet<string> = new Set(["query", "mutate"]);
|
||||
// @trpc/client 11.19 TRPCSubscriptionObserver callbacks; RxJS observers use
|
||||
// next/error/complete instead.
|
||||
const TRPC_SUBSCRIPTION_CALLBACKS: ReadonlySet<string> = new Set([
|
||||
"onStarted",
|
||||
"onData",
|
||||
"onError",
|
||||
"onStopped",
|
||||
"onComplete",
|
||||
"onConnectionStateChange",
|
||||
]);
|
||||
// @trpc/client 11.19 client factories (createTRPCProxyClient is the
|
||||
// deprecated alias of createTRPCClient).
|
||||
const TRPC_CLIENT_FACTORIES: ReadonlySet<string> = new Set([
|
||||
"createTRPCClient",
|
||||
"createTRPCProxyClient",
|
||||
]);
|
||||
// Built-in HTTP links documented by tRPC 11.19 that carry a `url` option.
|
||||
const TRPC_URL_LINKS: ReadonlySet<string> = new Set([
|
||||
"httpLink",
|
||||
"httpBatchLink",
|
||||
"httpBatchStreamLink",
|
||||
"httpSubscriptionLink",
|
||||
]);
|
||||
// Literals retained by @trpc/client 11.19 bundles (`this.name`, the default
|
||||
// accept header key) or by the conventional transport path.
|
||||
const TRPC_SOURCE_MARKERS = ["TRPCClientError", "trpc-accept", "/trpc"];
|
||||
|
||||
// Browser-owned roots whose methods share tRPC names, such as
|
||||
// navigator.permissions.query.
|
||||
const BROWSER_GLOBAL_NAMES: ReadonlySet<string> = new Set([
|
||||
"window",
|
||||
"self",
|
||||
"globalThis",
|
||||
"document",
|
||||
"navigator",
|
||||
"location",
|
||||
"history",
|
||||
"console",
|
||||
"performance",
|
||||
"screen",
|
||||
"top",
|
||||
"parent",
|
||||
"frames",
|
||||
]);
|
||||
|
||||
const PROCEDURE_SEGMENT = /^[A-Za-z_$][\w$]*$/u;
|
||||
|
||||
interface MemberPath {
|
||||
readonly root: t.Node;
|
||||
/** Non-computed property names from the root outward. */
|
||||
readonly properties: readonly string[];
|
||||
}
|
||||
|
||||
/** Read a static member path and the expression it starts from. */
|
||||
const memberPath = (callee: t.Node): MemberPath | undefined => {
|
||||
const reversed: string[] = [];
|
||||
let current: t.Node = callee;
|
||||
while (
|
||||
t.isMemberExpression(current) ||
|
||||
t.isOptionalMemberExpression(current)
|
||||
) {
|
||||
const property = semanticStaticPropertyKey(
|
||||
current.property,
|
||||
current.computed,
|
||||
);
|
||||
if (property === null || !PROCEDURE_SEGMENT.test(property))
|
||||
return undefined;
|
||||
reversed.push(property);
|
||||
current = current.object;
|
||||
}
|
||||
return { root: current, properties: reversed.reverse() };
|
||||
};
|
||||
|
||||
const objectKeys = (node: t.Node | undefined): readonly string[] =>
|
||||
t.isObjectExpression(node)
|
||||
? node.properties.flatMap((property) => {
|
||||
if (!t.isObjectProperty(property) && !t.isObjectMethod(property))
|
||||
return [];
|
||||
const key = semanticStaticPropertyKey(property.key, property.computed);
|
||||
return key === null ? [] : [key];
|
||||
})
|
||||
: [];
|
||||
|
||||
const objectStringProperty = (
|
||||
node: t.ObjectExpression,
|
||||
key: string,
|
||||
): string | undefined => {
|
||||
for (const property of node.properties)
|
||||
if (
|
||||
t.isObjectProperty(property) &&
|
||||
semanticStaticPropertyKey(property.key, property.computed) === key &&
|
||||
t.isStringLiteral(property.value)
|
||||
)
|
||||
return property.value.value;
|
||||
return undefined;
|
||||
};
|
||||
|
||||
const objectProperty = (
|
||||
node: t.ObjectExpression,
|
||||
key: string,
|
||||
): t.Node | undefined => {
|
||||
for (const property of node.properties)
|
||||
if (
|
||||
t.isObjectProperty(property) &&
|
||||
semanticStaticPropertyKey(property.key, property.computed) === key
|
||||
)
|
||||
return property.value;
|
||||
return undefined;
|
||||
};
|
||||
|
||||
const GRAPHQL_OPERATION_TYPES: ReadonlySet<string> = new Set([
|
||||
"query",
|
||||
"mutation",
|
||||
"subscription",
|
||||
]);
|
||||
// A named GraphQL operation header (GraphQL October 2021, OperationDefinition):
|
||||
// type, name, then variable definitions starting with `$`, a directive, or a
|
||||
// selection set starting with a field name.
|
||||
const GRAPHQL_OPERATION_HEADER =
|
||||
/^(query|mutation|subscription)[\s,]+([_A-Za-z][_0-9A-Za-z]*)[\s,]*(?:\([\s,]*\$|@[_A-Za-z]|\{[\s,]*[_A-Za-z])/u;
|
||||
|
||||
const isGraphqlIgnoredCharacter = (character: string): boolean =>
|
||||
character === "," || character === "" || /\s/u.test(character);
|
||||
|
||||
/** Recover a named operation from GraphQL document text. */
|
||||
export const graphqlOperationFromText = (
|
||||
text: string,
|
||||
): JavaScriptRpcEndpoint | undefined => {
|
||||
// Skip GraphQL ignored tokens: whitespace, commas, BOM and # comments.
|
||||
const limit = text.length;
|
||||
let index = 0;
|
||||
while (index < limit) {
|
||||
const character = text.charAt(index);
|
||||
if (isGraphqlIgnoredCharacter(character)) index += 1;
|
||||
else if (character === "#") {
|
||||
while (index < limit && !/[\n\r]/u.test(text.charAt(index))) index += 1;
|
||||
} else break;
|
||||
}
|
||||
if (index >= limit) return undefined;
|
||||
const header = GRAPHQL_OPERATION_HEADER.exec(text.slice(index));
|
||||
const type = header?.[1];
|
||||
const name = header?.[2];
|
||||
return type === undefined || name === undefined
|
||||
? undefined
|
||||
: { value: name, mechanism: `graphql:${type}` };
|
||||
};
|
||||
|
||||
/** Recover a named operation from a precompiled GraphQL document object. */
|
||||
export const graphqlOperationFromDocumentNode = (
|
||||
node: t.ObjectExpression,
|
||||
): JavaScriptRpcEndpoint | undefined => {
|
||||
if (objectStringProperty(node, "kind") !== "OperationDefinition")
|
||||
return undefined;
|
||||
const type = objectStringProperty(node, "operation");
|
||||
if (type === undefined || !GRAPHQL_OPERATION_TYPES.has(type))
|
||||
return undefined;
|
||||
const name = objectProperty(node, "name");
|
||||
if (
|
||||
!t.isObjectExpression(name) ||
|
||||
objectStringProperty(name, "kind") !== "Name"
|
||||
)
|
||||
return undefined;
|
||||
const value = objectStringProperty(name, "value");
|
||||
return value === undefined
|
||||
? undefined
|
||||
: { value, mechanism: `graphql:${type}` };
|
||||
};
|
||||
|
||||
/** Bundlers use `(0, imported.factory)` to call an import without a receiver. */
|
||||
const rpcFactoryName = (callee: t.Node): string => {
|
||||
let current = unwrapJavaScriptExpression(callee).node;
|
||||
while (t.isSequenceExpression(current)) {
|
||||
const last = current.expressions.at(-1);
|
||||
if (last === undefined) return "";
|
||||
current = unwrapJavaScriptExpression(last).node;
|
||||
}
|
||||
return calleeName(current);
|
||||
};
|
||||
|
||||
/** Whether an initializer has the syntax of a documented tRPC client factory. */
|
||||
export const isJavaScriptRpcClientFactory = (node: t.Node): boolean => {
|
||||
const call = unwrapJavaScriptExpression(node).node;
|
||||
return (
|
||||
(t.isCallExpression(call) || t.isOptionalCallExpression(call)) &&
|
||||
TRPC_CLIENT_FACTORIES.has(
|
||||
rpcFactoryName(call.callee).split(".").pop() ?? "",
|
||||
)
|
||||
);
|
||||
};
|
||||
|
||||
/** Recover the literal `url` option passed to a documented tRPC HTTP link. */
|
||||
const trpcLinkUrl = (
|
||||
node: t.CallExpression | t.OptionalCallExpression,
|
||||
): JavaScriptRpcEndpoint | undefined => {
|
||||
const name = rpcFactoryName(node.callee);
|
||||
const factory = name.slice(name.lastIndexOf(".") + 1);
|
||||
if (!TRPC_URL_LINKS.has(factory)) return undefined;
|
||||
const options = node.arguments[0];
|
||||
if (!t.isObjectExpression(options)) return undefined;
|
||||
const url = objectStringProperty(options, "url");
|
||||
return url === undefined
|
||||
? undefined
|
||||
: { value: url, mechanism: `trpc:link:${factory}` };
|
||||
};
|
||||
|
||||
/**
|
||||
* Create the per-source scan. Hook and option-factory methods are distinctive.
|
||||
* `query`/`mutate` need a root bound to a tRPC client factory, or a tRPC source
|
||||
* marker with a router path of at least two segments; `subscribe` needs the
|
||||
* marker and a tRPC observer argument. `this` roots, browser globals and RTK
|
||||
* Query `endpoints` chains are never procedures.
|
||||
*/
|
||||
export const createJavaScriptRpcScan = (
|
||||
source: string,
|
||||
clientRoots: ReadonlySet<t.Node>,
|
||||
): JavaScriptRpcScan => {
|
||||
let marker: boolean | undefined;
|
||||
const hasMarker = (): boolean =>
|
||||
(marker ??= TRPC_SOURCE_MARKERS.some((value) => source.includes(value)));
|
||||
// graphql-tag documents retain their source text in loc.source.body; that
|
||||
// text repeats the OperationDefinition already recovered from the object.
|
||||
const documentSourceBodies = new WeakSet<t.Node>();
|
||||
|
||||
const procedure = (
|
||||
node: t.CallExpression | t.OptionalCallExpression,
|
||||
): JavaScriptRpcEndpoint | undefined => {
|
||||
const path = memberPath(node.callee);
|
||||
if (path === undefined || t.isThisExpression(path.root)) return undefined;
|
||||
if (t.isIdentifier(path.root) && BROWSER_GLOBAL_NAMES.has(path.root.name))
|
||||
return undefined;
|
||||
const method = path.properties[path.properties.length - 1];
|
||||
const segments = path.properties.slice(0, -1);
|
||||
const first = segments[0];
|
||||
if (method === undefined || first === undefined) return undefined;
|
||||
if (BROWSER_GLOBAL_NAMES.has(first) || segments.includes("endpoints"))
|
||||
return undefined;
|
||||
const bound = clientRoots.has(path.root);
|
||||
const recognized = TRPC_HOOK_METHODS.has(method)
|
||||
? true
|
||||
: TRPC_REQUEST_METHODS.has(method)
|
||||
? bound || (segments.length >= 2 && hasMarker())
|
||||
: method === "subscribe" &&
|
||||
(bound || hasMarker()) &&
|
||||
objectKeys(node.arguments[1]).some((key) =>
|
||||
TRPC_SUBSCRIPTION_CALLBACKS.has(key),
|
||||
);
|
||||
return recognized
|
||||
? { value: segments.join("."), mechanism: `trpc:procedure:${method}` }
|
||||
: undefined;
|
||||
};
|
||||
|
||||
return {
|
||||
inspect: (node) => {
|
||||
if (t.isCallExpression(node) || t.isOptionalCallExpression(node))
|
||||
return trpcLinkUrl(node) ?? procedure(node);
|
||||
if (t.isObjectExpression(node)) {
|
||||
const body = objectProperty(node, "body");
|
||||
if (
|
||||
t.isStringLiteral(body) &&
|
||||
objectProperty(node, "locationOffset") !== undefined
|
||||
) {
|
||||
documentSourceBodies.add(body);
|
||||
return undefined;
|
||||
}
|
||||
return graphqlOperationFromDocumentNode(node);
|
||||
}
|
||||
if (t.isStringLiteral(node))
|
||||
return documentSourceBodies.has(node)
|
||||
? undefined
|
||||
: graphqlOperationFromText(node.value);
|
||||
if (t.isTemplateLiteral(node)) {
|
||||
// A template recovered without a cooked value stays dynamic.
|
||||
const cooked = node.quasis[0]?.value.cooked;
|
||||
return cooked === undefined || cooked === null
|
||||
? undefined
|
||||
: graphqlOperationFromText(cooked);
|
||||
}
|
||||
return undefined;
|
||||
},
|
||||
};
|
||||
};
|
||||
@@ -47,6 +47,7 @@ import { collectJavaScriptDerivedSemanticsSteps } from "./javascriptSemanticDeri
|
||||
import { evaluateSemanticProvenance } from "./javascriptSemanticValues.js";
|
||||
import {
|
||||
collectElectronMemberWritesSteps,
|
||||
electronExportPath,
|
||||
electronMemberPath,
|
||||
ELECTRON_MODULE,
|
||||
} from "./javascriptElectronMemberWrites.js";
|
||||
@@ -59,6 +60,8 @@ import {
|
||||
type ParsedJavaScriptSource,
|
||||
} from "./javascriptSourceParser.js";
|
||||
|
||||
import { isJavaScriptRpcClientFactory } from "./javascriptRpcEndpoints.js";
|
||||
|
||||
interface BindPatternInput {
|
||||
readonly pattern: t.Node;
|
||||
readonly initializer: t.Node | null;
|
||||
@@ -181,6 +184,40 @@ export function* analyzeParsedJavaScriptSemanticsSteps(
|
||||
};
|
||||
}
|
||||
|
||||
/** Resolve tRPC client roots by lexical identity, excluding overwritten bindings. */
|
||||
export function* classifyParsedJavaScriptRpcClientRootsSteps(
|
||||
file: ParsedJavaScriptSource,
|
||||
): Generator<void, ReadonlySet<t.Node>> {
|
||||
const state = createState(file.program);
|
||||
yield* collectDefinitionsSteps(file.program, state);
|
||||
const roots = new Set<t.Node>();
|
||||
const clients = new Map<string, boolean>();
|
||||
yield* traverseJavaScriptAstSteps(file.program, {
|
||||
enter: (node) => {
|
||||
if (!t.isMemberExpression(node) && !t.isOptionalMemberExpression(node))
|
||||
return;
|
||||
const root = node.object;
|
||||
if (!t.isIdentifier(root)) return;
|
||||
const binding = resolveSemanticBindingState(state, root, root.name);
|
||||
if (binding === undefined) return;
|
||||
let client = clients.get(binding.bindingId);
|
||||
if (client === undefined) {
|
||||
const [initializer] = binding.initializers;
|
||||
client =
|
||||
initializer !== undefined &&
|
||||
initializer.projection.length === 0 &&
|
||||
binding.initializers.length === 1 &&
|
||||
!binding.definitions.some(({ kind }) => kind === "assignment") &&
|
||||
!state.conditionalInitializers.has(initializer.node) &&
|
||||
isJavaScriptRpcClientFactory(initializer.node);
|
||||
clients.set(binding.bindingId, client);
|
||||
}
|
||||
if (client) roots.add(root);
|
||||
},
|
||||
});
|
||||
return roots;
|
||||
}
|
||||
|
||||
/** Receiver identity facts for the overloaded `.open` syntax only. */
|
||||
export type JavaScriptOpenReceiverFact =
|
||||
| "window"
|
||||
@@ -349,7 +386,7 @@ export function* classifyParsedJavaScriptElectronBindingsSteps(
|
||||
}
|
||||
const origin = semanticRequireOrigin(root, state);
|
||||
return origin !== undefined && ELECTRON_MODULE.test(origin.specifier)
|
||||
? origin.importedPath
|
||||
? electronExportPath(origin.importedPath)
|
||||
: undefined;
|
||||
};
|
||||
const writes = yield* collectElectronMemberWritesSteps(
|
||||
@@ -400,7 +437,7 @@ const electronExport = (
|
||||
? provenance.origins[0]
|
||||
: undefined);
|
||||
return origin !== undefined && ELECTRON_MODULE.test(origin.specifier)
|
||||
? origin.importedPath
|
||||
? electronExportPath(origin.importedPath)
|
||||
: null;
|
||||
};
|
||||
|
||||
|
||||
@@ -23,6 +23,7 @@ import {
|
||||
inspectCall,
|
||||
inspectRouteProperty,
|
||||
inspectRoleProperty,
|
||||
inspectRpcNode,
|
||||
} from "./javascriptStaticAnalysisCalls.js";
|
||||
import {
|
||||
inspectBundlerRegistration,
|
||||
@@ -37,6 +38,7 @@ import {
|
||||
import type { JavaScriptStaticAnalysis } from "./javascriptStaticAnalysisTypes.js";
|
||||
import {
|
||||
classifyParsedJavaScriptElectronBindingsSteps,
|
||||
classifyParsedJavaScriptRpcClientRootsSteps,
|
||||
classifyParsedJavaScriptOpenReceivers,
|
||||
type JavaScriptOpenReceiverFact,
|
||||
} from "./javascriptSemanticAnalysis.js";
|
||||
@@ -45,6 +47,8 @@ import {
|
||||
type ParsedJavaScriptSource,
|
||||
} from "./javascriptSourceParser.js";
|
||||
|
||||
import { createJavaScriptRpcScan } from "./javascriptRpcEndpoints.js";
|
||||
|
||||
/** Parse one JavaScript artifact and recover static structure only. */
|
||||
export const analyzeJavaScriptStaticSource = (
|
||||
source: string,
|
||||
@@ -78,11 +82,13 @@ export function* analyzeParsedJavaScriptStaticSourceSteps(
|
||||
const accumulator = createJavaScriptAnalysisAccumulator(source.length);
|
||||
const electronBindings =
|
||||
yield* classifyParsedJavaScriptElectronBindingsSteps(file);
|
||||
const clientRoots = yield* classifyParsedJavaScriptRpcClientRootsSteps(file);
|
||||
yield* traverseStaticSourceSteps(file, {
|
||||
source,
|
||||
accumulator,
|
||||
...(openReceiverFacts === undefined ? {} : { openReceiverFacts }),
|
||||
electronBindings,
|
||||
rpcScan: createJavaScriptRpcScan(source, clientRoots),
|
||||
});
|
||||
addSourceMapDirectives(source, file.comments ?? [], accumulator);
|
||||
return finalizeStaticAnalysis(source, file, accumulator);
|
||||
@@ -223,4 +229,5 @@ const inspectNode = (
|
||||
inspectRouteProperty(node, findings);
|
||||
inspectRoleProperty(node, findings);
|
||||
}
|
||||
inspectRpcNode(node, findings);
|
||||
};
|
||||
|
||||
@@ -201,6 +201,13 @@ const inspectEndpointCall = (
|
||||
});
|
||||
};
|
||||
|
||||
/** Record tRPC procedures, tRPC link URLs and named GraphQL operations. */
|
||||
export const inspectRpcNode = (node: t.Node, context: FindingContext): void => {
|
||||
const rpc = context.rpcScan.inspect(node);
|
||||
if (rpc !== undefined)
|
||||
addEndpoint(context, { node, kind: "network", ...rpc });
|
||||
};
|
||||
|
||||
export const inspectRouteProperty = (
|
||||
node: t.ObjectProperty,
|
||||
context: FindingContext,
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import type * as t from "@babel/types";
|
||||
import type { JavaScriptRpcScan } from "./javascriptRpcEndpoints.js";
|
||||
|
||||
import type {
|
||||
JavaScriptBundlerRegistration,
|
||||
@@ -57,6 +58,7 @@ export interface JavaScriptAnalysisAccumulator {
|
||||
|
||||
/** Shared source and accumulator for helper inspections. */
|
||||
export interface JavaScriptFindingContext {
|
||||
readonly rpcScan: JavaScriptRpcScan;
|
||||
readonly source: string;
|
||||
readonly accumulator: JavaScriptAnalysisAccumulator;
|
||||
readonly openReceiverFacts?: ReadonlyMap<
|
||||
|
||||
@@ -0,0 +1,176 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type { NativeAotParserLimits } from "./nativeAotPe.js";
|
||||
import {
|
||||
NATIVE_AOT_MAX_REPORT_BYTES,
|
||||
parseNativeAotPe,
|
||||
} from "./nativeAotPe.js";
|
||||
import {
|
||||
nativeAotPeDigest,
|
||||
nativeAotPeFixture,
|
||||
} from "../../../tests/fixtures/nativeaotPe.js";
|
||||
|
||||
const REPORT_BUDGET = NATIVE_AOT_MAX_REPORT_BYTES;
|
||||
|
||||
describe("NativeAOT parser resource budgets", () => {
|
||||
it("reports source-cap rejection without hashing or inventing omitted-byte coverage", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const sha = nativeAotPeDigest(bytes);
|
||||
const limits: NativeAotParserLimits = {
|
||||
sourceBytes: bytes.length - 1,
|
||||
workUnits: 1024,
|
||||
workingMemoryBytes: 1024,
|
||||
};
|
||||
const report = parseNativeAotPe(
|
||||
bytes,
|
||||
sha,
|
||||
REPORT_BUDGET,
|
||||
undefined,
|
||||
limits,
|
||||
).summary;
|
||||
expect(report).toMatchObject({
|
||||
status: "partial",
|
||||
analysis_artifact_sha256: sha,
|
||||
header_address: null,
|
||||
coverage: {
|
||||
source_bytes_examined: 0,
|
||||
source_byte_budget_bytes: bytes.length - 1,
|
||||
truncation_reason: "source-byte-budget",
|
||||
},
|
||||
});
|
||||
expect(report.coverage.source_bytes_omitted).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe("NativeAOT parser report admission", () => {
|
||||
it("never returns an over-budget report from early exits", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const sha = nativeAotPeDigest(bytes);
|
||||
expect(() =>
|
||||
parseNativeAotPe(bytes, sha, 0, undefined, {
|
||||
sourceBytes: bytes.length - 1,
|
||||
workUnits: 1024,
|
||||
workingMemoryBytes: 1024,
|
||||
}),
|
||||
).toThrow(/minimum truthful report/iu);
|
||||
const nonPe = Buffer.from("not a PE image");
|
||||
expect(() => parseNativeAotPe(nonPe, nativeAotPeDigest(nonPe), 0)).toThrow(
|
||||
/minimum truthful report/iu,
|
||||
);
|
||||
});
|
||||
|
||||
it("does not claim a missing RTR header when signature coverage hits the work limit", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const limits: NativeAotParserLimits = {
|
||||
sourceBytes: bytes.length,
|
||||
workUnits: 8,
|
||||
workingMemoryBytes: 1024,
|
||||
};
|
||||
const report = parseNativeAotPe(
|
||||
bytes,
|
||||
nativeAotPeDigest(bytes),
|
||||
REPORT_BUDGET,
|
||||
undefined,
|
||||
limits,
|
||||
).summary;
|
||||
expect(report).toMatchObject({
|
||||
status: "partial",
|
||||
header_address: null,
|
||||
truncated: true,
|
||||
coverage: {
|
||||
rtr_signature_scan_bytes_examined: 8,
|
||||
rtr_signature_scan_bytes_omitted: expect.any(Number),
|
||||
truncation_reason: "work-unit-budget",
|
||||
},
|
||||
});
|
||||
expect(report.reason).not.toMatch(/No supported ReadyToRun directory/iu);
|
||||
});
|
||||
|
||||
it("does not claim a missing root when table discovery is incomplete", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const report = parseNativeAotPe(
|
||||
bytes,
|
||||
nativeAotPeDigest(bytes),
|
||||
REPORT_BUDGET,
|
||||
undefined,
|
||||
{
|
||||
sourceBytes: bytes.length,
|
||||
workUnits: 10_000,
|
||||
workingMemoryBytes: 1024,
|
||||
},
|
||||
).summary;
|
||||
expect(report.truncated).toBe(true);
|
||||
expect(report.reason).toMatch(/before determining whether.*root exists/iu);
|
||||
expect(report.reason).not.toMatch(
|
||||
/no unique System\.Object.*fully examined/iu,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("NativeAOT detail and summary byte accounting", () => {
|
||||
it("admits serialized MethodTable memory descriptors before emitting them", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const sha = nativeAotPeDigest(bytes);
|
||||
const detail = (budget: number) =>
|
||||
parseNativeAotPe(bytes, sha, REPORT_BUDGET).readTypeDetail(
|
||||
"0x140002280",
|
||||
budget,
|
||||
);
|
||||
const complete = detail(REPORT_BUDGET);
|
||||
expect(complete?.derived_memory).not.toBeNull();
|
||||
let low = 0;
|
||||
let high = Buffer.byteLength(JSON.stringify(complete));
|
||||
while (low < high) {
|
||||
const middle = Math.floor((low + high) / 2);
|
||||
if (detail(middle)?.derived_memory != null) high = middle;
|
||||
else low = middle + 1;
|
||||
}
|
||||
const atBoundary = detail(low);
|
||||
const belowBoundary = detail(low - 1);
|
||||
expect(atBoundary?.derived_memory).not.toBeNull();
|
||||
expect(Buffer.byteLength(JSON.stringify(atBoundary))).toBeLessThanOrEqual(
|
||||
low,
|
||||
);
|
||||
expect(belowBoundary?.derived_memory ?? null).toBeNull();
|
||||
expect(belowBoundary?.diagnostics.join(" ")).toContain(
|
||||
"omitted because the final serialized report exceeded its byte budget",
|
||||
);
|
||||
expect(
|
||||
belowBoundary === undefined
|
||||
? 0
|
||||
: Buffer.byteLength(JSON.stringify(belowBoundary)),
|
||||
).toBeLessThanOrEqual(low - 1);
|
||||
});
|
||||
|
||||
it("truncates report rows before exceeding its JSON byte budget", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const sha = nativeAotPeDigest(bytes);
|
||||
const complete = parseNativeAotPe(bytes, sha, REPORT_BUDGET);
|
||||
const fullBytes = Buffer.byteLength(JSON.stringify(complete.summary));
|
||||
let budget = fullBytes - 1;
|
||||
let summary: ReturnType<typeof parseNativeAotPe>["summary"] | undefined;
|
||||
while (budget > 0) {
|
||||
try {
|
||||
const candidate = parseNativeAotPe(bytes, sha, budget);
|
||||
if (candidate.summary.truncated) {
|
||||
summary = candidate.summary;
|
||||
break;
|
||||
}
|
||||
} catch {
|
||||
// Budgets below the fixed report header cannot carry an honest summary.
|
||||
}
|
||||
budget--;
|
||||
}
|
||||
if (summary === undefined)
|
||||
throw new Error("No partial summary budget found");
|
||||
expect(summary.truncated).toBe(true);
|
||||
expect(
|
||||
(summary.types_omitted ?? 0) +
|
||||
(summary.frozen_strings_omitted ?? 0) +
|
||||
(summary.coverage.scan_bytes_omitted ?? 0) +
|
||||
(summary.coverage.frozen_scan_bytes_omitted ?? 0),
|
||||
).toBeGreaterThan(0);
|
||||
expect(Buffer.byteLength(JSON.stringify(summary))).toBeLessThanOrEqual(
|
||||
budget,
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,161 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { nativeMetadataRecoverySummarySchema } from "./nativeMetadataRecovery.js";
|
||||
import {
|
||||
NATIVE_AOT_MAX_REPORT_BYTES,
|
||||
parseNativeAotPe,
|
||||
} from "./nativeAotPe.js";
|
||||
import {
|
||||
nativeAotPeDigest,
|
||||
nativeAotPeFixture,
|
||||
} from "../../../tests/fixtures/nativeaotPe.js";
|
||||
const SUMMARY_BUDGET = NATIVE_AOT_MAX_REPORT_BYTES;
|
||||
|
||||
describe("read-only NativeAOT PE parser", () => {
|
||||
it("recovers a bounded RTR9.1 type graph and frozen literal from captured bytes", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const before = Buffer.from(bytes);
|
||||
const parsed = parseNativeAotPe(
|
||||
bytes,
|
||||
nativeAotPeDigest(bytes),
|
||||
SUMMARY_BUDGET,
|
||||
"0x140000000",
|
||||
);
|
||||
expect(bytes).toEqual(before);
|
||||
expect(parsed.summary).toMatchObject({
|
||||
format: "dotnet-nativeaot",
|
||||
status: "complete",
|
||||
analysis_mode: "read-only-derived-overlay",
|
||||
format_major: 9,
|
||||
format_minor: 1,
|
||||
method_tables: 3,
|
||||
frozen_strings: [
|
||||
{ address: "0x140002300", value: "REA_NATIVEAOT_FROZEN" },
|
||||
],
|
||||
derived_overlay: { address: "0x140002380", size_bytes: 1 },
|
||||
coverage: {
|
||||
basis: "raw-image-and-rehydrated-pointer-scan",
|
||||
method_tables_recovered: 3,
|
||||
frozen_string_candidates: 1,
|
||||
},
|
||||
});
|
||||
expect(parsed.readTypeDetail("0x140002280", SUMMARY_BUDGET)).toMatchObject({
|
||||
source: "read-only-derived-overlay",
|
||||
method_table_address: "0x140002280",
|
||||
related_type: { address: "0x140002200", type: null },
|
||||
virtual_slots: [{ slot: 0, target_address: "0x140001040" }],
|
||||
});
|
||||
expect(nativeMetadataRecoverySummarySchema.parse(parsed.summary)).toEqual(
|
||||
parsed.summary,
|
||||
);
|
||||
});
|
||||
|
||||
it("fails closed on snapshot identity mismatch", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
expect(() =>
|
||||
parseNativeAotPe(bytes, "0".repeat(64), SUMMARY_BUDGET),
|
||||
).toThrow(/digest/iu);
|
||||
});
|
||||
|
||||
it("reports a detected but unsupported RTR version as partial without guessed types", () => {
|
||||
const bytes = nativeAotPeFixture(10, 0);
|
||||
const parsed = parseNativeAotPe(
|
||||
bytes,
|
||||
nativeAotPeDigest(bytes),
|
||||
SUMMARY_BUDGET,
|
||||
);
|
||||
expect(parsed.summary).toMatchObject({
|
||||
status: "partial",
|
||||
format_major: 10,
|
||||
format_minor: 0,
|
||||
method_tables: 0,
|
||||
types: [],
|
||||
});
|
||||
});
|
||||
|
||||
it("does not discover an RTR directory in raw padding beyond VirtualSize", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const sectionTable = 0x98 + 0xf0;
|
||||
const dataSection = sectionTable + 40;
|
||||
bytes.writeUInt32LE(0x20, dataSection + 8);
|
||||
const parsed = parseNativeAotPe(
|
||||
bytes,
|
||||
nativeAotPeDigest(bytes),
|
||||
SUMMARY_BUDGET,
|
||||
);
|
||||
expect(parsed.summary).toMatchObject({
|
||||
status: "not_applicable",
|
||||
reason:
|
||||
"No supported ReadyToRun directory signature was found in initialized non-executable PE sections.",
|
||||
method_tables: 0,
|
||||
});
|
||||
});
|
||||
|
||||
it("uses raw size as the virtual extent when VirtualSize is zero", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const sectionTable = 0x98 + 0xf0;
|
||||
bytes.writeUInt32LE(0, sectionTable + 40 + 8);
|
||||
const parsed = parseNativeAotPe(
|
||||
bytes,
|
||||
nativeAotPeDigest(bytes),
|
||||
SUMMARY_BUDGET,
|
||||
);
|
||||
expect(parsed.summary).toMatchObject({
|
||||
status: "complete",
|
||||
method_tables: 3,
|
||||
derived_overlay: { address: "0x140002380", size_bytes: 1 },
|
||||
});
|
||||
});
|
||||
|
||||
it("accepts executable-section targets in the zero-filled virtual tail", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const sectionTable = 0x98 + 0xf0;
|
||||
bytes.writeUInt32LE(0x300, sectionTable + 8);
|
||||
const slotOffset = 0x600 + (0x2280 - 0x2000) + 24;
|
||||
// RVA 0x1200 is executable-section zero fill: virtual size 0x300, raw size 0x200.
|
||||
bytes.writeBigUInt64LE(0x140001200n, slotOffset);
|
||||
const parsed = parseNativeAotPe(
|
||||
bytes,
|
||||
nativeAotPeDigest(bytes),
|
||||
SUMMARY_BUDGET,
|
||||
"0x140000000",
|
||||
);
|
||||
expect(parsed.summary).toMatchObject({
|
||||
status: "complete",
|
||||
method_tables: 3,
|
||||
});
|
||||
expect(parsed.readTypeDetail("0x140002280", SUMMARY_BUDGET)).toMatchObject({
|
||||
virtual_slots: [{ slot: 0, target_address: "0x140001200" }],
|
||||
});
|
||||
});
|
||||
|
||||
it("does not claim an image mapping when Ghidra rebased the image", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
expect(
|
||||
parseNativeAotPe(
|
||||
bytes,
|
||||
nativeAotPeDigest(bytes),
|
||||
SUMMARY_BUDGET,
|
||||
"0x180000000",
|
||||
).summary.reason,
|
||||
).toMatch(/preferred image base/iu);
|
||||
});
|
||||
|
||||
it("reports an unsupported hydration opcode as partial without mutating the snapshot", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
bytes.writeUInt8(6, 0x704);
|
||||
const before = Buffer.from(bytes);
|
||||
const parsed = parseNativeAotPe(
|
||||
bytes,
|
||||
nativeAotPeDigest(bytes),
|
||||
SUMMARY_BUDGET,
|
||||
);
|
||||
expect(bytes).toEqual(before);
|
||||
expect(parsed.summary).toMatchObject({
|
||||
status: "partial",
|
||||
derived_overlay: null,
|
||||
diagnostics: [
|
||||
expect.stringContaining("Unsupported DEHYDRATED_DATA command"),
|
||||
],
|
||||
});
|
||||
});
|
||||
});
|
||||
File diff suppressed because it is too large
Load Diff
@@ -6,6 +6,8 @@ const relatedType = z.strictObject({ address, type: z.string().nullable() });
|
||||
/** Runtime-format metadata recovered in an analysis image, with explicit inference boundaries. */
|
||||
export const nativeMetadataRecoverySchema = z.strictObject({
|
||||
format: z.literal("dotnet-nativeaot"),
|
||||
/** Identifies parser-only facts without claiming Ghidra defined a DataType. */
|
||||
source: z.enum(["analysis-database", "read-only-derived-overlay"]).optional(),
|
||||
status: z.enum(["complete", "partial"]),
|
||||
header_address: address,
|
||||
format_major: z.number().int().nonnegative(),
|
||||
@@ -25,19 +27,29 @@ export const nativeMetadataRecoverySchema = z.strictObject({
|
||||
basis: z.literal("method-table-pointer"),
|
||||
}),
|
||||
),
|
||||
derived_memory: z.strictObject({
|
||||
address,
|
||||
size_bytes: z.number().int().nonnegative(),
|
||||
sha256: z.string().regex(/^[a-f0-9]{64}$/u),
|
||||
file_offset: z.null(),
|
||||
}),
|
||||
derived_memory: z
|
||||
.strictObject({
|
||||
address,
|
||||
size_bytes: z.number().int().nonnegative(),
|
||||
sha256: z.string().regex(/^[a-f0-9]{64}$/u),
|
||||
file_offset: z.null(),
|
||||
})
|
||||
.nullable(),
|
||||
truncated: z.boolean().optional(),
|
||||
virtual_slots_omitted: z.number().int().nonnegative().optional(),
|
||||
interfaces_omitted: z.number().int().nonnegative().optional(),
|
||||
output_budget_bytes: z.number().int().nonnegative().optional(),
|
||||
diagnostics: z.array(z.string()),
|
||||
limitations: z.array(z.string()),
|
||||
});
|
||||
export type NativeMetadataRecovery = z.infer<
|
||||
typeof nativeMetadataRecoverySchema
|
||||
>;
|
||||
|
||||
/** Discover a recovered runtime format and reusable type identities from the loaded image. */
|
||||
export const nativeMetadataRecoverySummarySchema = z.strictObject({
|
||||
format: z.literal("dotnet-nativeaot"),
|
||||
analysis_mode: z.literal("read-only-derived-overlay").optional(),
|
||||
status: z.enum(["complete", "partial", "not_applicable"]),
|
||||
reason: z.string().nullable(),
|
||||
analysis_artifact_sha256: z.string().regex(/^[a-f0-9]{64}$/u),
|
||||
@@ -47,14 +59,76 @@ export const nativeMetadataRecoverySummarySchema = z.strictObject({
|
||||
format_minor: z.number().int().nonnegative().nullable(),
|
||||
method_tables: z.number().int().nonnegative(),
|
||||
types: z.array(relatedType),
|
||||
derived_memory: nativeMetadataRecoverySchema.shape.derived_memory.nullable(),
|
||||
frozen_strings: z
|
||||
.array(z.strictObject({ address, value: z.string() }))
|
||||
.optional(),
|
||||
truncated: z.boolean().optional(),
|
||||
types_omitted: z.number().int().nonnegative().optional(),
|
||||
frozen_strings_omitted: z.number().int().nonnegative().optional(),
|
||||
output_budget_bytes: z.number().int().nonnegative().optional(),
|
||||
derived_overlay: z
|
||||
.strictObject({
|
||||
address,
|
||||
size_bytes: z.number().int().nonnegative(),
|
||||
sha256: z.string().regex(/^[a-f0-9]{64}$/u),
|
||||
})
|
||||
.nullable()
|
||||
.optional(),
|
||||
derived_memory: nativeMetadataRecoverySchema.shape.derived_memory,
|
||||
coverage: z
|
||||
.strictObject({
|
||||
frozen_object_candidates: z.number().int().nonnegative(),
|
||||
frozen_objects_annotated: z.number().int().nonnegative(),
|
||||
basis: z.literal(
|
||||
frozen_object_candidates: z.number().int().nonnegative().optional(),
|
||||
frozen_objects_annotated: z.number().int().nonnegative().optional(),
|
||||
pointer_candidates_examined: z.number().int().nonnegative().optional(),
|
||||
pointer_slots_examined: z.number().int().nonnegative().optional(),
|
||||
scan_bytes_examined: z.number().int().nonnegative().optional(),
|
||||
scan_bytes_omitted: z.number().int().nonnegative().optional(),
|
||||
rtr_signature_scan_bytes_examined: z
|
||||
.number()
|
||||
.int()
|
||||
.nonnegative()
|
||||
.optional(),
|
||||
rtr_signature_scan_bytes_omitted: z
|
||||
.number()
|
||||
.int()
|
||||
.nonnegative()
|
||||
.optional(),
|
||||
method_table_validation_items_examined: z
|
||||
.number()
|
||||
.int()
|
||||
.nonnegative()
|
||||
.optional(),
|
||||
work_units_examined: z.number().int().nonnegative().optional(),
|
||||
method_tables_recovered: z.number().int().nonnegative().optional(),
|
||||
frozen_string_candidates: z.number().int().nonnegative().optional(),
|
||||
frozen_scan_bytes_examined: z.number().int().nonnegative().optional(),
|
||||
frozen_scan_bytes_omitted: z.number().int().nonnegative().optional(),
|
||||
frozen_string_data_bytes_examined: z
|
||||
.number()
|
||||
.int()
|
||||
.nonnegative()
|
||||
.optional(),
|
||||
source_bytes_examined: z.number().int().nonnegative().optional(),
|
||||
source_bytes_omitted: z.number().int().nonnegative().optional(),
|
||||
source_byte_budget_bytes: z.number().int().nonnegative().optional(),
|
||||
work_budget_units: z.number().int().nonnegative().optional(),
|
||||
working_memory_budget_bytes: z.number().int().nonnegative().optional(),
|
||||
working_memory_bytes_peak: z.number().int().nonnegative().optional(),
|
||||
truncation_reason: z
|
||||
.enum([
|
||||
"source-byte-budget",
|
||||
"work-unit-budget",
|
||||
"working-memory-budget",
|
||||
"response-budget",
|
||||
])
|
||||
.optional(),
|
||||
basis: z.enum([
|
||||
"rehydrated-pointer-candidates-and-committed-instance-types",
|
||||
),
|
||||
"raw-image-pointer-scan",
|
||||
"raw-image-and-rehydrated-pointer-scan",
|
||||
"unsupported-layout",
|
||||
"no-rtr-header",
|
||||
]),
|
||||
})
|
||||
.nullable(),
|
||||
diagnostics: z.array(z.string()),
|
||||
|
||||
@@ -603,3 +603,56 @@ it("retains direct and literal computed call names", () => {
|
||||
expect.objectContaining({ name: "literal-tool" }),
|
||||
]);
|
||||
});
|
||||
|
||||
it("recognizes tRPC procedures and named GraphQL operations in captured scripts", () => {
|
||||
const result = analyzeCapturedWebBundle(
|
||||
inspection(`
|
||||
const n = createTRPCClient({ links: [httpBatchLink({ url: "/api/trpc" })] });
|
||||
e.invoice.list.useQuery({ competencia: "2026-09" });
|
||||
n.empresa.byCnpj.query("00000000000191");
|
||||
const q = "query ListarNotas($competencia: String!) { notas { id } }";
|
||||
this.client.query({ query: q });
|
||||
`),
|
||||
);
|
||||
expect(
|
||||
result.observations.endpoints.map(({ value, mechanism }) => ({
|
||||
value,
|
||||
mechanism,
|
||||
})),
|
||||
).toEqual([
|
||||
{ value: "/api/trpc", mechanism: "trpc:link:httpBatchLink" },
|
||||
{ value: "invoice.list", mechanism: "trpc:procedure:useQuery" },
|
||||
{ value: "empresa.byCnpj", mechanism: "trpc:procedure:query" },
|
||||
{ value: "ListarNotas", mechanism: "graphql:query" },
|
||||
]);
|
||||
});
|
||||
|
||||
it("keeps RPC lexical identity and bundled factory recognition in captured scripts", () => {
|
||||
const longOperation = "Operation" + "x".repeat(512);
|
||||
const document =
|
||||
"#" +
|
||||
"comment".repeat(700) +
|
||||
"\nquery " +
|
||||
longOperation +
|
||||
" { users { id } }";
|
||||
const result = analyzeCapturedWebBundle(
|
||||
inspection(`
|
||||
function first() { const c = createTRPCClient({}); }
|
||||
function unrelated(c) { c.users.query("select 1"); }
|
||||
const c = (0, t.createTRPCClient)({ links: [(0, t.httpBatchLink)({ url: "/api/rpc" })] });
|
||||
c.posts.query();
|
||||
const doc = ${JSON.stringify(document)};
|
||||
`),
|
||||
);
|
||||
expect(
|
||||
result.observations.endpoints.map(({ value, mechanism }) => ({
|
||||
value,
|
||||
mechanism,
|
||||
})),
|
||||
).toEqual([
|
||||
{ value: "/api/rpc", mechanism: "trpc:link:httpBatchLink" },
|
||||
{ value: "posts", mechanism: "trpc:procedure:query" },
|
||||
{ value: longOperation, mechanism: "graphql:query" },
|
||||
]);
|
||||
expect(result.completeness.status).toBe("complete");
|
||||
});
|
||||
|
||||
@@ -5,8 +5,14 @@ import { sanitizeEndpointCandidate } from "./browserObservation.js";
|
||||
import { compositeKey } from "./unicodeCodePointOrder.js";
|
||||
import type { WebPageInspection } from "./browserObservationSchemas.js";
|
||||
import type { WebBundleAnalysis } from "./webBundleAnalysis.js";
|
||||
import { traverseJavaScriptAst } from "./javascript/javascriptSemanticTraversal.js";
|
||||
import { classifyParsedJavaScriptOpenReceivers } from "./javascript/javascriptSemanticAnalysis.js";
|
||||
import {
|
||||
completeSemanticSteps,
|
||||
traverseJavaScriptAst,
|
||||
} from "./javascript/javascriptSemanticTraversal.js";
|
||||
import {
|
||||
classifyParsedJavaScriptOpenReceivers,
|
||||
classifyParsedJavaScriptRpcClientRootsSteps,
|
||||
} from "./javascript/javascriptSemanticAnalysis.js";
|
||||
import type { JavaScriptOpenReceiverFact } from "./javascript/javascriptSemanticAnalysis.js";
|
||||
import { semanticStaticPropertyName } from "./javascript/javascriptAstValues.js";
|
||||
import {
|
||||
@@ -22,6 +28,7 @@ import {
|
||||
resolveSpecifier,
|
||||
isUrlLikeModuleSpecifier,
|
||||
} from "./webBundleAnalyzerAst.js";
|
||||
import { createJavaScriptRpcScan } from "./javascript/javascriptRpcEndpoints.js";
|
||||
|
||||
type BundleObservations = WebBundleAnalysis["observations"];
|
||||
type ChunkEdge = BundleObservations["chunks"]["edges"][number];
|
||||
@@ -66,10 +73,24 @@ export const analyzeScript = (
|
||||
accumulator.parsedScripts += 1;
|
||||
detectVendorFingerprints(script, accumulator);
|
||||
const openReceiverFacts = classifyParsedJavaScriptOpenReceivers(file);
|
||||
const rpc = createJavaScriptRpcScan(
|
||||
script.source.artifact.text,
|
||||
completeSemanticSteps(classifyParsedJavaScriptRpcClientRootsSteps(file)),
|
||||
);
|
||||
traverseJavaScriptAst(file, {
|
||||
enter: (node) => {
|
||||
accumulator.visitedNodes += 1;
|
||||
inspectNode(script, node, accumulator, openReceiverFacts);
|
||||
const endpoint = rpc.inspect(node);
|
||||
if (endpoint !== undefined)
|
||||
addFinding({
|
||||
collection: accumulator.endpoints,
|
||||
script,
|
||||
rawValue: endpoint.value,
|
||||
mechanism: endpoint.mechanism,
|
||||
node,
|
||||
accumulator,
|
||||
});
|
||||
},
|
||||
});
|
||||
};
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
/** Generated from package.json and package-lock.json; do not edit. */
|
||||
export const PACKAGE_METADATA = {
|
||||
name: "rea-agents",
|
||||
version: "6.3.0", // x-release-please-version
|
||||
version: "6.4.0", // x-release-please-version
|
||||
serverSdkVersion: "2.3.1",
|
||||
clientSdkVersion: "2.3.1",
|
||||
coreSdkVersion: "2.3.1",
|
||||
|
||||
@@ -17,6 +17,13 @@ import {
|
||||
ghidraMipsProfileParameters,
|
||||
ghidraMipsUnsupportedReason,
|
||||
} from "./GhidraMipsProfile.js";
|
||||
import {
|
||||
NATIVE_AOT_MAX_REPORT_BYTES,
|
||||
NATIVE_AOT_MAX_SOURCE_BYTES,
|
||||
NATIVE_AOT_MAX_WORKING_MEMORY_BYTES,
|
||||
NATIVE_AOT_MAX_WORK_UNITS,
|
||||
NATIVE_AOT_PROFILE_CONTRACT_REVISION,
|
||||
} from "../domain/native/nativeAotPe.js";
|
||||
|
||||
/** Resolve version-bound, deterministic semantics before Ghidra imports a target. */
|
||||
export const resolveGhidraAnalysisProfile = (
|
||||
@@ -69,6 +76,13 @@ export const resolveGhidraAnalysisProfile = (
|
||||
? {
|
||||
executable_role: target.executableRole ?? null,
|
||||
managed: target.managed ?? null,
|
||||
native_aot_metadata: {
|
||||
contract_revision: NATIVE_AOT_PROFILE_CONTRACT_REVISION,
|
||||
source_bytes: NATIVE_AOT_MAX_SOURCE_BYTES,
|
||||
work_units: NATIVE_AOT_MAX_WORK_UNITS,
|
||||
working_memory_bytes: NATIVE_AOT_MAX_WORKING_MEMORY_BYTES,
|
||||
report_bytes: NATIVE_AOT_MAX_REPORT_BYTES,
|
||||
},
|
||||
}
|
||||
: {}),
|
||||
architecture: target.architecture ?? null,
|
||||
|
||||
@@ -43,6 +43,21 @@ import {
|
||||
} from "./GhidraSessionValues.js";
|
||||
import { createGhidraTargetSnapshot } from "./GhidraTargetSnapshot.js";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { readBoundedFileBytes } from "../process/BoundedFileBytes.js";
|
||||
import {
|
||||
openRegularFile,
|
||||
RegularFileChangedError,
|
||||
sameRegularFileState,
|
||||
} from "../filesystem/RegularFile.js";
|
||||
import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
|
||||
|
||||
export type GhidraTargetSnapshot =
|
||||
| { readonly kind: "captured"; readonly bytes: Buffer }
|
||||
| {
|
||||
readonly kind: "over-capacity";
|
||||
readonly sourceBytesAtLeast: number;
|
||||
readonly maximumBytes: number;
|
||||
};
|
||||
import { connectGhidraSocket } from "./GhidraSocketConnection.js";
|
||||
import {
|
||||
createGhidraEndpoint,
|
||||
@@ -79,6 +94,7 @@ export class GhidraClient {
|
||||
#socketRoot: PrivateRuntimeRoot | undefined;
|
||||
#endpointPath: string | undefined;
|
||||
#snapshotPath: string | undefined;
|
||||
readonly #snapshotOwners = new Set<OwnedFileHandle>();
|
||||
#targetAdmission: JsonValue | undefined;
|
||||
#token: string | undefined;
|
||||
// Retain authentication identities for diagnostics from late request settlement.
|
||||
@@ -227,7 +243,10 @@ export class GhidraClient {
|
||||
}
|
||||
|
||||
/** Read the private immutable source for independent import verification. */
|
||||
async readTargetSnapshot(): Promise<Result<Buffer, GhidraSessionError>> {
|
||||
async readTargetSnapshot(
|
||||
maximumBytes?: number,
|
||||
signal?: AbortSignal,
|
||||
): Promise<Result<GhidraTargetSnapshot, GhidraSessionError>> {
|
||||
if (this.#snapshotPath === undefined)
|
||||
return err(
|
||||
this.#failure(
|
||||
@@ -236,11 +255,80 @@ export class GhidraClient {
|
||||
),
|
||||
);
|
||||
try {
|
||||
return ok(await readFile(this.#snapshotPath));
|
||||
if (maximumBytes !== undefined) {
|
||||
signal?.throwIfAborted();
|
||||
if (!Number.isSafeInteger(maximumBytes) || maximumBytes < 0)
|
||||
throw new RangeError("Snapshot byte limit must be a safe integer");
|
||||
const handle = await openRegularFile(this.#snapshotPath, {
|
||||
symlinks: "reject",
|
||||
});
|
||||
const owner = new OwnedFileHandle(handle);
|
||||
this.#snapshotOwners.add(owner);
|
||||
let result: GhidraTargetSnapshot;
|
||||
let readFailure: unknown;
|
||||
try {
|
||||
const metadata = await handle.stat();
|
||||
signal?.throwIfAborted();
|
||||
if (metadata.size > maximumBytes)
|
||||
result = {
|
||||
kind: "over-capacity",
|
||||
sourceBytesAtLeast: metadata.size,
|
||||
maximumBytes,
|
||||
};
|
||||
else {
|
||||
const bytes = await readBoundedFileBytes(
|
||||
handle,
|
||||
maximumBytes,
|
||||
signal,
|
||||
metadata.size,
|
||||
);
|
||||
signal?.throwIfAborted();
|
||||
const after = await handle.stat();
|
||||
if (
|
||||
bytes !== undefined &&
|
||||
bytes.length === metadata.size &&
|
||||
sameRegularFileState(metadata, after)
|
||||
) {
|
||||
result = { kind: "captured", bytes };
|
||||
} else if (
|
||||
bytes === undefined &&
|
||||
after.isFile() &&
|
||||
after.size > maximumBytes
|
||||
) {
|
||||
result = {
|
||||
kind: "over-capacity",
|
||||
sourceBytesAtLeast: after.size,
|
||||
maximumBytes,
|
||||
};
|
||||
} else {
|
||||
throw new RegularFileChangedError(this.#snapshotPath);
|
||||
}
|
||||
}
|
||||
} catch (cause: unknown) {
|
||||
readFailure = cause;
|
||||
}
|
||||
try {
|
||||
await owner.close();
|
||||
this.#snapshotOwners.delete(owner);
|
||||
} catch (closeFailure: unknown) {
|
||||
throw new AggregateError(
|
||||
readFailure === undefined
|
||||
? [closeFailure]
|
||||
: [readFailure, closeFailure],
|
||||
"Ghidra target snapshot read or descriptor cleanup failed",
|
||||
);
|
||||
}
|
||||
if (readFailure !== undefined) throw readFailure;
|
||||
return ok(result!);
|
||||
}
|
||||
return ok({
|
||||
kind: "captured",
|
||||
bytes: await readFile(this.#snapshotPath),
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
return err(
|
||||
this.#failure(
|
||||
"protocol",
|
||||
signal?.aborted === true ? "cancelled" : "protocol",
|
||||
"Ghidra target snapshot could not be read for load-image verification",
|
||||
cause,
|
||||
),
|
||||
@@ -602,6 +690,10 @@ export class GhidraClient {
|
||||
this.#lastDiagnostics = this.#diagnostics();
|
||||
this.#process = undefined;
|
||||
this.#launch = undefined;
|
||||
for (const owner of this.#snapshotOwners) {
|
||||
await owner.close();
|
||||
this.#snapshotOwners.delete(owner);
|
||||
}
|
||||
try {
|
||||
await this.#removeRuntimeRoots();
|
||||
} catch (cause: unknown) {
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import type { AnalysisOperation } from "../application/AnalysisProvider.js";
|
||||
import { AnalysisCancelledError } from "../domain/analysisErrorCore.js";
|
||||
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
|
||||
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js";
|
||||
@@ -12,23 +13,91 @@ import {
|
||||
attestGhidraDosComLoadImage,
|
||||
attestGhidraDosLoadImage,
|
||||
} from "./GhidraLoadImageValues.js";
|
||||
import { type NativeAotPeResult } from "../domain/native/nativeAotPe.js";
|
||||
import type { GhidraTargetSnapshot } from "./GhidraClient.js";
|
||||
import type { GhidraSessionError } from "./GhidraSessionError.js";
|
||||
|
||||
type LoadImageSnapshotClient = {
|
||||
readonly readTargetSnapshot?: () => Promise<
|
||||
Result<Buffer, GhidraSessionError>
|
||||
>;
|
||||
readonly readTargetSnapshot?: (
|
||||
maximumBytes?: number,
|
||||
signal?: AbortSignal,
|
||||
) => Promise<Result<GhidraTargetSnapshot, GhidraSessionError>>;
|
||||
};
|
||||
|
||||
type NativeLoadImageAttestationInput = {
|
||||
readonly target: BinaryTarget;
|
||||
readonly operation: AnalysisOperation;
|
||||
readonly measured: JsonValue;
|
||||
readonly client: LoadImageSnapshotClient;
|
||||
readonly mapSessionError: (failure: GhidraSessionError) => AnalysisError;
|
||||
readonly nativeAotResult?: NativeAotPeResult | undefined;
|
||||
readonly signal?: AbortSignal | undefined;
|
||||
};
|
||||
|
||||
const isSignalAborted = (signal: AbortSignal | undefined): boolean =>
|
||||
signal?.aborted === true;
|
||||
|
||||
/** Independently verify measured load-image observations for admitted DOS targets. */
|
||||
export const attestGhidraNativeLoadImage = async (
|
||||
target: BinaryTarget,
|
||||
operation: AnalysisOperation,
|
||||
measured: JsonValue,
|
||||
client: LoadImageSnapshotClient,
|
||||
mapSessionError: (failure: GhidraSessionError) => AnalysisError,
|
||||
input: NativeLoadImageAttestationInput,
|
||||
): Promise<Result<JsonValue, AnalysisError>> => {
|
||||
const {
|
||||
target,
|
||||
operation,
|
||||
measured,
|
||||
client,
|
||||
mapSessionError,
|
||||
nativeAotResult,
|
||||
signal,
|
||||
} = input;
|
||||
if (isSignalAborted(signal))
|
||||
return err(new AnalysisCancelledError(operation));
|
||||
const observations = nativeLoadImageObservationSchema.parse(measured);
|
||||
if (target.format === "pe") {
|
||||
if (nativeAotResult === undefined)
|
||||
return err(
|
||||
new ProviderAdapterError("ghidra", operation, {
|
||||
diagnostics: {
|
||||
reason:
|
||||
"The Ghidra provider did not produce read-only NativeAOT metadata for this admitted PE snapshot.",
|
||||
},
|
||||
}),
|
||||
);
|
||||
if (isSignalAborted(signal))
|
||||
return err(new AnalysisCancelledError(operation));
|
||||
try {
|
||||
return ok(
|
||||
jsonValueSchema.parse(
|
||||
nativeLoadImageSchema.parse({
|
||||
status: "unsupported",
|
||||
reason:
|
||||
"Independent load-image verification currently supports DOS MZ and explicit DOS COM targets only.",
|
||||
observations: {
|
||||
...observations,
|
||||
metadata_recovery: [
|
||||
...(observations.metadata_recovery ?? []),
|
||||
nativeAotResult.summary,
|
||||
],
|
||||
},
|
||||
limitations: [
|
||||
"Load-image verification is not performed for PE targets; NativeAOT metadata observations are derived separately from the exact admitted snapshot.",
|
||||
],
|
||||
}),
|
||||
),
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
return err(
|
||||
new ProviderAdapterError("ghidra", operation, {
|
||||
diagnostics: {
|
||||
reason:
|
||||
cause instanceof Error
|
||||
? cause.message
|
||||
: "NativeAOT metadata could not be represented in the load-image result",
|
||||
},
|
||||
}),
|
||||
);
|
||||
}
|
||||
}
|
||||
if (target.format !== "dos-mz" && target.format !== "dos-com") {
|
||||
return ok(
|
||||
jsonValueSchema.parse(
|
||||
@@ -55,11 +124,20 @@ export const attestGhidraNativeLoadImage = async (
|
||||
);
|
||||
const snapshot = await client.readTargetSnapshot();
|
||||
if (!snapshot.ok) return err(mapSessionError(snapshot.error));
|
||||
if (snapshot.value.kind !== "captured")
|
||||
return err(
|
||||
new ProviderAdapterError("ghidra", operation, {
|
||||
diagnostics: {
|
||||
reason:
|
||||
"The uncapped DOS load-image snapshot was unexpectedly not captured.",
|
||||
},
|
||||
}),
|
||||
);
|
||||
const attested = (
|
||||
target.format === "dos-com"
|
||||
? attestGhidraDosComLoadImage
|
||||
: attestGhidraDosLoadImage
|
||||
)(snapshot.value, target.sha256 ?? "", observations);
|
||||
)(snapshot.value.bytes, target.sha256 ?? "", observations);
|
||||
if (!attested.ok)
|
||||
return err(
|
||||
new ProviderAdapterError("ghidra", operation, {
|
||||
|
||||
@@ -10,16 +10,22 @@ import { fileURLToPath } from "node:url";
|
||||
import { promisify } from "node:util";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { fixtureDosLoadImage } from "./GhidraLoadImage.fixture.js";
|
||||
import {
|
||||
nativeAotPeDigest,
|
||||
nativeAotPeFixture,
|
||||
} from "../../tests/fixtures/nativeaotPe.js";
|
||||
import { jsonValueSchema } from "../domain/jsonValue.js";
|
||||
|
||||
import { parseConfig } from "../config/parseConfig.js";
|
||||
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
|
||||
import { parseExecutableHeader } from "../domain/binaryTarget.js";
|
||||
import { GhidraProvider } from "./GhidraProvider.js";
|
||||
import type { GhidraTargetSnapshot } from "./GhidraClient.js";
|
||||
import type { GhidraProviderClientFactory } from "./GhidraProviderClient.js";
|
||||
import type { GhidraInstallationHost } from "./GhidraInstallation.js";
|
||||
import { GHIDRA_SESSION_CAPABILITIES } from "./GhidraSessionValues.js";
|
||||
import { err, ok } from "../domain/result.js";
|
||||
import type { Result } from "../domain/result.js";
|
||||
import { GhidraSessionError } from "./GhidraSessionError.js";
|
||||
import { silentLogger } from "../logger.js";
|
||||
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
|
||||
@@ -219,10 +225,25 @@ describe("Ghidra platform support", () => {
|
||||
expect(profile.parameters).toMatchObject({
|
||||
executable_role: target.executableRole,
|
||||
managed: target.managed,
|
||||
native_aot_metadata: {
|
||||
contract_revision: "rtr-9.1-x64-pe-read-only-v3",
|
||||
source_bytes: 128 * 1024 * 1024,
|
||||
work_units: 64 * 1024 * 1024,
|
||||
working_memory_bytes: 64 * 1024 * 1024,
|
||||
report_bytes: 4 * 1024 * 1024,
|
||||
},
|
||||
});
|
||||
digests.add(profile.digest);
|
||||
}
|
||||
expect(digests.size).toBe(3);
|
||||
const nonPe = await ghidra.resolveAnalysisProfile(
|
||||
executableTarget("elf", "x86_64"),
|
||||
);
|
||||
if (!nonPe.ok || nonPe.value.profile === null)
|
||||
throw new Error("Expected a non-PE analysis profile");
|
||||
expect(nonPe.value.profile.parameters).not.toHaveProperty(
|
||||
"native_aot_metadata",
|
||||
);
|
||||
});
|
||||
|
||||
it("keeps Windows annotation mutation unavailable independently of native controls", () => {
|
||||
@@ -717,7 +738,8 @@ describe("Ghidra measured load-image projection", () => {
|
||||
...(state === "missing-snapshot"
|
||||
? {}
|
||||
: {
|
||||
readTargetSnapshot: () => Promise.resolve(ok(fixture.bytes)),
|
||||
readTargetSnapshot: () =>
|
||||
Promise.resolve(ok({ kind: "captured", bytes: fixture.bytes })),
|
||||
}),
|
||||
});
|
||||
const ghidra = provider(installationHost(), factory);
|
||||
@@ -757,6 +779,505 @@ describe("Ghidra measured load-image projection", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("returns read-only NativeAOT metadata inline and drills down by derived MethodTable address", async () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const observations = fixtureDosLoadImage().observation;
|
||||
observations.image_base = "0x140000000";
|
||||
const factory: GhidraProviderClientFactory = () => ({
|
||||
start: () =>
|
||||
Promise.resolve(
|
||||
ok({
|
||||
...sessionInfo(),
|
||||
target: { ...sessionInfo().target, image_base: "0x140000000" },
|
||||
}),
|
||||
),
|
||||
callTool: (operation) =>
|
||||
Promise.resolve(
|
||||
ok(
|
||||
jsonValueSchema.parse(
|
||||
operation === "inspect_native_load_image"
|
||||
? observations
|
||||
: {
|
||||
status: "unavailable",
|
||||
reason: "No Ghidra DataType is defined at this address.",
|
||||
id: null,
|
||||
name: null,
|
||||
kind: "unavailable",
|
||||
source: "analysis-database",
|
||||
source_archive: null,
|
||||
address: null,
|
||||
size_bytes: null,
|
||||
alignment_bytes: null,
|
||||
packing_enabled: null,
|
||||
referenced_type: null,
|
||||
array_count: null,
|
||||
array_stride_bytes: null,
|
||||
fields: [],
|
||||
members: [],
|
||||
total_fields: 0,
|
||||
truncated: false,
|
||||
limitations: [],
|
||||
},
|
||||
),
|
||||
),
|
||||
),
|
||||
readTargetSnapshot: () => Promise.resolve(ok({ kind: "captured", bytes })),
|
||||
close: () => Promise.resolve(ok(null)),
|
||||
});
|
||||
const ghidra = provider(installationHost(), factory);
|
||||
const target = { ...peTarget("x86_64"), sha256: nativeAotPeDigest(bytes) };
|
||||
const profile = await ghidra.resolveAnalysisProfile(target);
|
||||
if (!profile.ok || profile.value.profile === null)
|
||||
throw new Error("Expected admitted PE profile");
|
||||
const client = ghidra.createClient(target, profile.value.profile);
|
||||
const image = await client.execute("inspect_native_load_image", {});
|
||||
expect(image).toMatchObject({
|
||||
ok: true,
|
||||
value: {
|
||||
result: {
|
||||
status: "unsupported",
|
||||
observations: {
|
||||
metadata_recovery: [
|
||||
{
|
||||
status: "complete",
|
||||
analysis_mode: "read-only-derived-overlay",
|
||||
method_tables: 3,
|
||||
frozen_strings: [{ value: "REA_NATIVEAOT_FROZEN" }],
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
rawResult: observations,
|
||||
},
|
||||
});
|
||||
const detail = await client.execute("inspect_native_data_type", {
|
||||
address: "0x140002280",
|
||||
});
|
||||
expect(detail).toMatchObject({
|
||||
ok: true,
|
||||
value: {
|
||||
result: {
|
||||
status: "unavailable",
|
||||
source: "analysis-database",
|
||||
metadata_recovery: {
|
||||
source: "read-only-derived-overlay",
|
||||
method_table_address: "0x140002280",
|
||||
related_type: { address: "0x140002200", type: null },
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
await client.close();
|
||||
});
|
||||
|
||||
it("derives PE metadata for a direct type-address query without a load-image call", async () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const operations: string[] = [];
|
||||
const factory: GhidraProviderClientFactory = () => ({
|
||||
start: () =>
|
||||
Promise.resolve(
|
||||
ok({
|
||||
...sessionInfo(),
|
||||
target: { ...sessionInfo().target, image_base: "0x140000000" },
|
||||
}),
|
||||
),
|
||||
callTool: (operation) => {
|
||||
operations.push(operation);
|
||||
return Promise.resolve(
|
||||
ok(
|
||||
jsonValueSchema.parse({
|
||||
status: "unavailable",
|
||||
reason: "No Ghidra DataType is defined at this address.",
|
||||
id: null,
|
||||
name: null,
|
||||
kind: "unavailable",
|
||||
source: "analysis-database",
|
||||
source_archive: null,
|
||||
address: null,
|
||||
size_bytes: null,
|
||||
alignment_bytes: null,
|
||||
packing_enabled: null,
|
||||
referenced_type: null,
|
||||
array_count: null,
|
||||
array_stride_bytes: null,
|
||||
fields: [],
|
||||
members: [],
|
||||
total_fields: 0,
|
||||
truncated: false,
|
||||
limitations: [],
|
||||
}),
|
||||
),
|
||||
);
|
||||
},
|
||||
readTargetSnapshot: () => Promise.resolve(ok({ kind: "captured", bytes })),
|
||||
close: () => Promise.resolve(ok(null)),
|
||||
});
|
||||
const ghidra = provider(installationHost(), factory);
|
||||
const target = { ...peTarget("x86_64"), sha256: nativeAotPeDigest(bytes) };
|
||||
const profile = await ghidra.resolveAnalysisProfile(target);
|
||||
if (!profile.ok || profile.value.profile === null)
|
||||
throw new Error("Expected admitted PE profile");
|
||||
const client = ghidra.createClient(target, profile.value.profile);
|
||||
const detail = await client.execute("inspect_native_data_type", {
|
||||
address: "0x140002280",
|
||||
});
|
||||
expect(detail).toMatchObject({
|
||||
ok: true,
|
||||
value: {
|
||||
result: {
|
||||
metadata_recovery: {
|
||||
source: "read-only-derived-overlay",
|
||||
method_table_address: "0x140002280",
|
||||
related_type: { address: "0x140002200", type: null },
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(operations).toEqual(["inspect_native_data_type"]);
|
||||
await client.close();
|
||||
});
|
||||
|
||||
it("shares one session snapshot across callers while isolating cancellation and rejecting a changed base", async () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const baseObservation = fixtureDosLoadImage().observation;
|
||||
baseObservation.image_base = "0x140000000";
|
||||
let calls = 0;
|
||||
let snapshotReads = 0;
|
||||
let snapshotSignal: AbortSignal | undefined;
|
||||
let releaseSnapshot:
|
||||
| ((value: Result<GhidraTargetSnapshot, GhidraSessionError>) => void)
|
||||
| undefined;
|
||||
let markSnapshotStarted: (() => void) | undefined;
|
||||
const snapshotStarted = new Promise<void>((resolve) => {
|
||||
markSnapshotStarted = resolve;
|
||||
});
|
||||
const snapshotResult = new Promise<
|
||||
Result<GhidraTargetSnapshot, GhidraSessionError>
|
||||
>((resolve) => {
|
||||
releaseSnapshot = resolve;
|
||||
});
|
||||
const factory: GhidraProviderClientFactory = () => ({
|
||||
start: () => Promise.resolve(ok(sessionInfo())),
|
||||
callTool: (operation) => {
|
||||
calls += 1;
|
||||
const observation = {
|
||||
...baseObservation,
|
||||
source_files: [
|
||||
{ ...baseObservation.source_files[0]!, name: `caller-${calls}` },
|
||||
],
|
||||
};
|
||||
return Promise.resolve(
|
||||
ok(
|
||||
jsonValueSchema.parse(
|
||||
operation === "inspect_native_load_image"
|
||||
? observation
|
||||
: {
|
||||
status: "unavailable",
|
||||
reason: "No Ghidra DataType is defined at this address.",
|
||||
id: null,
|
||||
name: null,
|
||||
kind: "unavailable",
|
||||
source: "analysis-database",
|
||||
source_archive: null,
|
||||
address: null,
|
||||
size_bytes: null,
|
||||
alignment_bytes: null,
|
||||
packing_enabled: null,
|
||||
referenced_type: null,
|
||||
array_count: null,
|
||||
array_stride_bytes: null,
|
||||
fields: [],
|
||||
members: [],
|
||||
total_fields: 0,
|
||||
truncated: false,
|
||||
limitations: [],
|
||||
},
|
||||
),
|
||||
),
|
||||
);
|
||||
},
|
||||
readTargetSnapshot: (_maximumBytes, signal) => {
|
||||
snapshotReads += 1;
|
||||
snapshotSignal = signal;
|
||||
markSnapshotStarted?.();
|
||||
return snapshotResult;
|
||||
},
|
||||
close: () => Promise.resolve(ok(null)),
|
||||
});
|
||||
const ghidra = provider(installationHost(), factory);
|
||||
const target = { ...peTarget("x86_64"), sha256: nativeAotPeDigest(bytes) };
|
||||
const profile = await ghidra.resolveAnalysisProfile(target);
|
||||
if (!profile.ok || profile.value.profile === null)
|
||||
throw new Error("Expected admitted PE profile");
|
||||
const client = ghidra.createClient(target, profile.value.profile);
|
||||
const firstCaller = new AbortController();
|
||||
const secondCaller = new AbortController();
|
||||
const first = client.execute(
|
||||
"inspect_native_load_image",
|
||||
{},
|
||||
{
|
||||
signal: firstCaller.signal,
|
||||
},
|
||||
);
|
||||
const second = client.execute(
|
||||
"inspect_native_load_image",
|
||||
{},
|
||||
{
|
||||
signal: secondCaller.signal,
|
||||
},
|
||||
);
|
||||
await snapshotStarted;
|
||||
expect(snapshotReads).toBe(1);
|
||||
expect(snapshotSignal).toBeDefined();
|
||||
expect(snapshotSignal).not.toBe(firstCaller.signal);
|
||||
expect(snapshotSignal).not.toBe(secondCaller.signal);
|
||||
firstCaller.abort();
|
||||
await expect(first).resolves.toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisCancelledError" },
|
||||
});
|
||||
releaseSnapshot?.(ok({ kind: "captured", bytes }));
|
||||
const secondResult = await second;
|
||||
expect(secondResult).toMatchObject({
|
||||
ok: true,
|
||||
value: {
|
||||
rawResult: {
|
||||
source_files: [{ name: "caller-2" }],
|
||||
},
|
||||
result: {
|
||||
observations: {
|
||||
source_files: [{ name: "caller-2" }],
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(snapshotReads).toBe(1);
|
||||
expect(calls).toBe(2);
|
||||
|
||||
baseObservation.image_base = "0x150000000";
|
||||
const conflictingBase = await client.execute("inspect_native_load_image", {});
|
||||
expect(conflictingBase).toMatchObject({
|
||||
ok: false,
|
||||
error: {
|
||||
_tag: "ProviderAdapterError",
|
||||
diagnostics: {
|
||||
reason: expect.stringContaining("different loaded image base"),
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(snapshotReads).toBe(1);
|
||||
await client.close();
|
||||
expect(snapshotSignal?.aborted).toBe(true);
|
||||
});
|
||||
|
||||
it("retries a failed NativeAOT snapshot read without retaining a failed derivation", async () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const observations = fixtureDosLoadImage().observation;
|
||||
observations.image_base = "0x140000000";
|
||||
let snapshotReads = 0;
|
||||
const factory: GhidraProviderClientFactory = () => ({
|
||||
start: () => Promise.resolve(ok(sessionInfo())),
|
||||
callTool: () => Promise.resolve(ok(jsonValueSchema.parse(observations))),
|
||||
readTargetSnapshot: () => {
|
||||
snapshotReads += 1;
|
||||
return Promise.resolve(
|
||||
snapshotReads === 1
|
||||
? err(
|
||||
new GhidraSessionError(
|
||||
"process",
|
||||
"temporary snapshot read failure",
|
||||
),
|
||||
)
|
||||
: ok({ kind: "captured", bytes }),
|
||||
);
|
||||
},
|
||||
close: () => Promise.resolve(ok(null)),
|
||||
});
|
||||
const ghidra = provider(installationHost(), factory);
|
||||
const target = { ...peTarget("x86_64"), sha256: nativeAotPeDigest(bytes) };
|
||||
const profile = await ghidra.resolveAnalysisProfile(target);
|
||||
if (!profile.ok || profile.value.profile === null)
|
||||
throw new Error("Expected admitted PE profile");
|
||||
const client = ghidra.createClient(target, profile.value.profile);
|
||||
await expect(
|
||||
client.execute("inspect_native_load_image", {}),
|
||||
).resolves.toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "ProviderAdapterError" },
|
||||
});
|
||||
await expect(
|
||||
client.execute("inspect_native_load_image", {}),
|
||||
).resolves.toMatchObject({
|
||||
ok: true,
|
||||
value: {
|
||||
result: {
|
||||
observations: {
|
||||
metadata_recovery: [{ status: "complete" }],
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(snapshotReads).toBe(2);
|
||||
await client.close();
|
||||
});
|
||||
|
||||
it("rejects all waiters when the loaded image base changes during derivation", async () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const observation = fixtureDosLoadImage().observation;
|
||||
let calls = 0;
|
||||
let resolveSnapshot:
|
||||
| ((value: Result<GhidraTargetSnapshot, GhidraSessionError>) => void)
|
||||
| undefined;
|
||||
let markSnapshotStarted: (() => void) | undefined;
|
||||
const snapshotStarted = new Promise<void>((resolve) => {
|
||||
markSnapshotStarted = resolve;
|
||||
});
|
||||
const snapshot = new Promise<
|
||||
Result<GhidraTargetSnapshot, GhidraSessionError>
|
||||
>((resolve) => {
|
||||
resolveSnapshot = resolve;
|
||||
});
|
||||
const factory: GhidraProviderClientFactory = () => ({
|
||||
start: () => Promise.resolve(ok(sessionInfo())),
|
||||
callTool: () => {
|
||||
calls += 1;
|
||||
return Promise.resolve(
|
||||
ok(
|
||||
jsonValueSchema.parse({
|
||||
...observation,
|
||||
image_base: calls === 1 ? "0x140000000" : "0x150000000",
|
||||
}),
|
||||
),
|
||||
);
|
||||
},
|
||||
readTargetSnapshot: () => {
|
||||
markSnapshotStarted?.();
|
||||
return snapshot;
|
||||
},
|
||||
close: () => Promise.resolve(ok(null)),
|
||||
});
|
||||
const ghidra = provider(installationHost(), factory);
|
||||
const target = { ...peTarget("x86_64"), sha256: nativeAotPeDigest(bytes) };
|
||||
const profile = await ghidra.resolveAnalysisProfile(target);
|
||||
if (!profile.ok || profile.value.profile === null)
|
||||
throw new Error("Expected admitted PE profile");
|
||||
const client = ghidra.createClient(target, profile.value.profile);
|
||||
const originalBase = client.execute("inspect_native_load_image", {});
|
||||
await snapshotStarted;
|
||||
const changedBase = await client.execute("inspect_native_load_image", {});
|
||||
expect(changedBase).toMatchObject({
|
||||
ok: false,
|
||||
error: {
|
||||
_tag: "ProviderAdapterError",
|
||||
diagnostics: {
|
||||
observed_image_base: "0x150000000",
|
||||
measured_observation: { image_base: "0x150000000" },
|
||||
},
|
||||
},
|
||||
});
|
||||
resolveSnapshot?.(ok({ kind: "captured", bytes }));
|
||||
await expect(originalBase).resolves.toMatchObject({
|
||||
ok: false,
|
||||
error: {
|
||||
_tag: "ProviderAdapterError",
|
||||
diagnostics: {
|
||||
reason: expect.stringContaining("conflicting loaded image bases"),
|
||||
measured_observation: { image_base: "0x140000000" },
|
||||
conflicting_observation: { image_base: "0x150000000" },
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(calls).toBe(2);
|
||||
await client.close();
|
||||
});
|
||||
|
||||
it("cancels an in-flight NativeAOT waiter when its owning session closes", async () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const observations = fixtureDosLoadImage().observation;
|
||||
observations.image_base = "0x140000000";
|
||||
let resolveSnapshot:
|
||||
| ((value: Result<GhidraTargetSnapshot, GhidraSessionError>) => void)
|
||||
| undefined;
|
||||
let snapshotSignal: AbortSignal | undefined;
|
||||
let markSnapshotStarted: (() => void) | undefined;
|
||||
const snapshotStarted = new Promise<void>((resolve) => {
|
||||
markSnapshotStarted = resolve;
|
||||
});
|
||||
const snapshot = new Promise<
|
||||
Result<GhidraTargetSnapshot, GhidraSessionError>
|
||||
>((resolve) => {
|
||||
resolveSnapshot = resolve;
|
||||
});
|
||||
const factory: GhidraProviderClientFactory = () => ({
|
||||
start: () => Promise.resolve(ok(sessionInfo())),
|
||||
callTool: () => Promise.resolve(ok(jsonValueSchema.parse(observations))),
|
||||
readTargetSnapshot: (_maximumBytes, signal) => {
|
||||
snapshotSignal = signal;
|
||||
markSnapshotStarted?.();
|
||||
return snapshot;
|
||||
},
|
||||
close: () => Promise.resolve(ok(null)),
|
||||
});
|
||||
const ghidra = provider(installationHost(), factory);
|
||||
const target = { ...peTarget("x86_64"), sha256: nativeAotPeDigest(bytes) };
|
||||
const profile = await ghidra.resolveAnalysisProfile(target);
|
||||
if (!profile.ok || profile.value.profile === null)
|
||||
throw new Error("Expected admitted PE profile");
|
||||
const client = ghidra.createClient(target, profile.value.profile);
|
||||
const pending = client.execute("inspect_native_load_image", {});
|
||||
await snapshotStarted;
|
||||
await client.close();
|
||||
expect(snapshotSignal?.aborted).toBe(true);
|
||||
resolveSnapshot?.(ok({ kind: "captured", bytes }));
|
||||
await expect(pending).resolves.toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisCancelledError" },
|
||||
});
|
||||
});
|
||||
|
||||
describe("Ghidra NativeAOT snapshot budget", () => {
|
||||
it("reports snapshot-capacity omissions inline through the load-image result", async () => {
|
||||
const observations = fixtureDosLoadImage().observation;
|
||||
const target = peTarget("x86_64");
|
||||
const factory: GhidraProviderClientFactory = () => ({
|
||||
start: () => Promise.resolve(ok(sessionInfo())),
|
||||
callTool: () => Promise.resolve(ok(jsonValueSchema.parse(observations))),
|
||||
readTargetSnapshot: () =>
|
||||
Promise.resolve(
|
||||
ok({
|
||||
kind: "over-capacity",
|
||||
sourceBytesAtLeast: 128 * 1024 * 1024 + 1,
|
||||
maximumBytes: 128 * 1024 * 1024,
|
||||
}),
|
||||
),
|
||||
close: () => Promise.resolve(ok(null)),
|
||||
});
|
||||
const ghidra = provider(installationHost(), factory);
|
||||
const profile = await ghidra.resolveAnalysisProfile(target);
|
||||
if (!profile.ok || profile.value.profile === null)
|
||||
throw new Error("Expected admitted PE profile");
|
||||
const client = ghidra.createClient(target, profile.value.profile);
|
||||
const image = await client.execute("inspect_native_load_image", {});
|
||||
expect(image).toMatchObject({
|
||||
ok: true,
|
||||
value: {
|
||||
result: {
|
||||
observations: {
|
||||
metadata_recovery: [
|
||||
{
|
||||
status: "partial",
|
||||
truncated: true,
|
||||
coverage: { truncation_reason: "source-byte-budget" },
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
await client.close();
|
||||
});
|
||||
});
|
||||
|
||||
const sessionInfo = () => ({
|
||||
name: "REA Ghidra bridge" as const,
|
||||
run_id: "11111111-1111-4111-8111-111111111111",
|
||||
|
||||
@@ -8,7 +8,6 @@ import {
|
||||
type AnalysisOperation,
|
||||
} from "../application/AnalysisProvider.js";
|
||||
import type { AppConfig } from "../config/types.js";
|
||||
import type { JsonValue } from "../domain/jsonValue.js";
|
||||
import {
|
||||
ghidraSeedCommitmentSchema,
|
||||
ghidraSeedFailure,
|
||||
@@ -30,6 +29,7 @@ import {
|
||||
AnalysisTimeoutError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import { type AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js";
|
||||
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import type { Logger } from "pino";
|
||||
@@ -52,6 +52,18 @@ import {
|
||||
import { unverifiedGhidraBuildLimitation } from "./GhidraInstallationPolicy.js";
|
||||
import { GhidraHeadlessLauncher } from "./GhidraLauncher.js";
|
||||
import { attestGhidraNativeLoadImage } from "./GhidraLoadImageAttest.js";
|
||||
import {
|
||||
NATIVE_AOT_MAX_REPORT_BYTES,
|
||||
NATIVE_AOT_MAX_SOURCE_BYTES,
|
||||
nativeAotSnapshotOverCapacity,
|
||||
parseNativeAotPe,
|
||||
type NativeAotPeResult,
|
||||
} from "../domain/native/nativeAotPe.js";
|
||||
import {
|
||||
ABORTED,
|
||||
waitForAbortable,
|
||||
} from "../application/binary/AbortablePromise.js";
|
||||
import { nativeLoadImageObservationSchema } from "../domain/native/nativeLoadImage.js";
|
||||
import {
|
||||
GHIDRA_PROVIDER_IDENTITY,
|
||||
healthLimitations,
|
||||
@@ -77,6 +89,250 @@ export type GhidraProviderClientFactory = (
|
||||
) => Pick<GhidraClient, "start" | "callTool" | "close"> &
|
||||
Partial<Pick<GhidraClient, "runtimeLineage" | "readTargetSnapshot">>;
|
||||
|
||||
type GhidraProviderClientInstance = ReturnType<GhidraProviderClientFactory>;
|
||||
|
||||
type NativeAotSnapshotOwner = {
|
||||
readonly derive: (
|
||||
imageBase: string,
|
||||
operation: AnalysisOperation,
|
||||
callerSignal: AbortSignal | undefined,
|
||||
observations: JsonValue,
|
||||
) => Promise<Result<NativeAotPeResult, AnalysisError>>;
|
||||
readonly readTypeDetail: (
|
||||
address: string,
|
||||
) => ReturnType<NativeAotPeResult["readTypeDetail"]>;
|
||||
readonly close: () => void;
|
||||
};
|
||||
|
||||
/** Own one session's captured PE snapshot and immutable NativeAOT derivation. */
|
||||
const createNativeAotSnapshotOwner = (input: {
|
||||
readonly client: GhidraProviderClientInstance;
|
||||
readonly target: BinaryTarget;
|
||||
readonly startupTimeoutMs: number;
|
||||
}): NativeAotSnapshotOwner => {
|
||||
const { client, target, startupTimeoutMs } = input;
|
||||
const controller = new AbortController();
|
||||
let identity: string | undefined;
|
||||
let result: NativeAotPeResult | undefined;
|
||||
let pending:
|
||||
| {
|
||||
readonly identity: string;
|
||||
readonly promise: Promise<Result<NativeAotPeResult, AnalysisError>>;
|
||||
}
|
||||
| undefined;
|
||||
let closed = false;
|
||||
let identityConflict = false;
|
||||
let identityConflictObservation:
|
||||
| { readonly imageBase: string; readonly observations: JsonValue }
|
||||
| undefined;
|
||||
const identityError = (
|
||||
operation: AnalysisOperation,
|
||||
reason: string,
|
||||
observedBase: string,
|
||||
observations: JsonValue,
|
||||
): AnalysisError =>
|
||||
new ProviderAdapterError("ghidra", operation, {
|
||||
diagnostics: {
|
||||
reason,
|
||||
artifact_sha256: target.sha256,
|
||||
bound_image_base:
|
||||
identity === undefined
|
||||
? null
|
||||
: `0x${identity.slice(target.sha256.length + 1)}`,
|
||||
observed_image_base: observedBase,
|
||||
measured_observation: observations,
|
||||
conflicting_image_base: identityConflictObservation?.imageBase ?? null,
|
||||
conflicting_observation:
|
||||
identityConflictObservation?.observations ?? null,
|
||||
},
|
||||
});
|
||||
const identityConflictFailure = (
|
||||
operation: AnalysisOperation,
|
||||
observedBase: string,
|
||||
observations: JsonValue,
|
||||
): Result<NativeAotPeResult, AnalysisError> => {
|
||||
return err(
|
||||
identityError(
|
||||
operation,
|
||||
"The Ghidra session reported conflicting loaded image bases; reopen the analysis session to establish a new identity.",
|
||||
observedBase,
|
||||
observations,
|
||||
),
|
||||
);
|
||||
};
|
||||
const derive = async (
|
||||
imageBase: string,
|
||||
operation: AnalysisOperation,
|
||||
callerSignal: AbortSignal | undefined,
|
||||
observations: JsonValue,
|
||||
): Promise<Result<NativeAotPeResult, AnalysisError>> => {
|
||||
if (callerSignal?.aborted || closed)
|
||||
return err(new AnalysisCancelledError(operation));
|
||||
if (identityConflict)
|
||||
return identityConflictFailure(operation, imageBase, observations);
|
||||
if (controller.signal.aborted)
|
||||
return err(new AnalysisCancelledError(operation));
|
||||
let normalizedBase: string;
|
||||
try {
|
||||
normalizedBase = BigInt(imageBase).toString(16);
|
||||
} catch (cause: unknown) {
|
||||
return err(
|
||||
new ProviderAdapterError("ghidra", operation, {
|
||||
diagnostics: {
|
||||
reason:
|
||||
"Ghidra returned an invalid loaded image base for NativeAOT identity binding.",
|
||||
artifact_sha256: target.sha256,
|
||||
observed_image_base: imageBase,
|
||||
measured_observation: observations,
|
||||
},
|
||||
cause,
|
||||
}),
|
||||
);
|
||||
}
|
||||
const key = `${target.sha256}:${normalizedBase}`;
|
||||
if (identity !== undefined && identity !== key) {
|
||||
identityConflict = true;
|
||||
identityConflictObservation = { imageBase, observations };
|
||||
result = undefined;
|
||||
controller.abort();
|
||||
return err(
|
||||
identityError(
|
||||
operation,
|
||||
"The Ghidra session reported a different loaded image base after NativeAOT metadata was bound to its captured snapshot; reopen the analysis session to establish a new identity.",
|
||||
imageBase,
|
||||
observations,
|
||||
),
|
||||
);
|
||||
}
|
||||
identity ??= key;
|
||||
if (result !== undefined) {
|
||||
const settled = await waitForAbortable(
|
||||
Promise.resolve(ok(result)),
|
||||
callerSignal,
|
||||
);
|
||||
if (closed) return err(new AnalysisCancelledError(operation));
|
||||
if (identityConflict)
|
||||
return identityConflictFailure(operation, imageBase, observations);
|
||||
if (controller.signal.aborted)
|
||||
return err(new AnalysisCancelledError(operation));
|
||||
return settled === ABORTED
|
||||
? err(new AnalysisCancelledError(operation))
|
||||
: settled;
|
||||
}
|
||||
if (pending !== undefined && pending.identity !== key)
|
||||
return err(
|
||||
identityError(
|
||||
operation,
|
||||
"A NativeAOT snapshot derivation is already bound to a different loaded image base.",
|
||||
imageBase,
|
||||
observations,
|
||||
),
|
||||
);
|
||||
if (pending === undefined) {
|
||||
const promise: Promise<Result<NativeAotPeResult, AnalysisError>> =
|
||||
(async () => {
|
||||
try {
|
||||
if (client.readTargetSnapshot === undefined)
|
||||
return err(
|
||||
new ProviderAdapterError("ghidra", operation, {
|
||||
diagnostics: {
|
||||
reason:
|
||||
"The Ghidra client does not expose its immutable target snapshot for NativeAOT metadata inspection.",
|
||||
},
|
||||
}),
|
||||
);
|
||||
const snapshot = await client.readTargetSnapshot(
|
||||
NATIVE_AOT_MAX_SOURCE_BYTES,
|
||||
controller.signal,
|
||||
);
|
||||
if (!snapshot.ok)
|
||||
return err(
|
||||
projectSessionError(
|
||||
operation,
|
||||
snapshot.error,
|
||||
startupTimeoutMs,
|
||||
),
|
||||
);
|
||||
if (closed) return err(new AnalysisCancelledError(operation));
|
||||
if (identityConflict)
|
||||
return identityConflictFailure(
|
||||
operation,
|
||||
imageBase,
|
||||
observations,
|
||||
);
|
||||
if (controller.signal.aborted)
|
||||
return err(new AnalysisCancelledError(operation));
|
||||
const parsed =
|
||||
snapshot.value.kind === "over-capacity"
|
||||
? {
|
||||
summary: nativeAotSnapshotOverCapacity(
|
||||
target.sha256,
|
||||
snapshot.value.sourceBytesAtLeast,
|
||||
NATIVE_AOT_MAX_REPORT_BYTES,
|
||||
),
|
||||
readTypeDetail: () => undefined,
|
||||
}
|
||||
: parseNativeAotPe(
|
||||
snapshot.value.bytes,
|
||||
target.sha256,
|
||||
NATIVE_AOT_MAX_REPORT_BYTES,
|
||||
imageBase,
|
||||
);
|
||||
if (closed) return err(new AnalysisCancelledError(operation));
|
||||
if (identityConflict)
|
||||
return identityConflictFailure(
|
||||
operation,
|
||||
imageBase,
|
||||
observations,
|
||||
);
|
||||
if (controller.signal.aborted)
|
||||
return err(new AnalysisCancelledError(operation));
|
||||
return ok(parsed);
|
||||
} catch (cause: unknown) {
|
||||
return err(
|
||||
new ProviderAdapterError("ghidra", operation, {
|
||||
diagnostics: {
|
||||
reason:
|
||||
cause instanceof Error
|
||||
? cause.message
|
||||
: "NativeAOT metadata could not be derived from the captured target snapshot.",
|
||||
},
|
||||
cause,
|
||||
}),
|
||||
);
|
||||
}
|
||||
})();
|
||||
const current = { identity: key, promise };
|
||||
pending = current;
|
||||
void promise.then((settled) => {
|
||||
if (pending !== current) return;
|
||||
pending = undefined;
|
||||
if (settled.ok && !closed && !identityConflict) result = settled.value;
|
||||
});
|
||||
}
|
||||
const settled = await waitForAbortable(pending.promise, callerSignal);
|
||||
if (closed) return err(new AnalysisCancelledError(operation));
|
||||
if (identityConflict)
|
||||
return identityConflictFailure(operation, imageBase, observations);
|
||||
if (controller.signal.aborted)
|
||||
return err(new AnalysisCancelledError(operation));
|
||||
return settled === ABORTED
|
||||
? err(new AnalysisCancelledError(operation))
|
||||
: settled;
|
||||
};
|
||||
return {
|
||||
derive,
|
||||
readTypeDetail: (address) =>
|
||||
result?.readTypeDetail(address, NATIVE_AOT_MAX_REPORT_BYTES),
|
||||
close: () => {
|
||||
closed = true;
|
||||
controller.abort();
|
||||
result = undefined;
|
||||
pending = undefined;
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
/** Build one AnalysisClient for an admitted Ghidra target and profile. */
|
||||
export const createGhidraProviderClient = (input: {
|
||||
readonly config: AppConfig;
|
||||
@@ -258,6 +514,14 @@ export const createGhidraProviderClient = (input: {
|
||||
...seedLimitations,
|
||||
...(releaseLimitation === undefined ? [] : [releaseLimitation]),
|
||||
];
|
||||
const nativeAotOwner =
|
||||
target.format === "pe"
|
||||
? createNativeAotSnapshotOwner({
|
||||
client,
|
||||
target,
|
||||
startupTimeoutMs: config.ghidraStartupTimeoutMs,
|
||||
})
|
||||
: undefined;
|
||||
return {
|
||||
execute: async (operation, parameters, options) => {
|
||||
if (extensionFailure !== undefined) return err(extensionFailure);
|
||||
@@ -334,19 +598,83 @@ export const createGhidraProviderClient = (input: {
|
||||
if (!result.ok) return result;
|
||||
normalized = result.value;
|
||||
}
|
||||
if (operation === "inspect_native_data_type") {
|
||||
const address = input.value.address;
|
||||
if (typeof address === "string") {
|
||||
if (target.format === "pe" && nativeAotOwner !== undefined) {
|
||||
const started = await client.start(options?.signal);
|
||||
if (!started.ok)
|
||||
return err(
|
||||
projectSessionError(
|
||||
operation,
|
||||
started.error,
|
||||
config.ghidraStartupTimeoutMs,
|
||||
),
|
||||
);
|
||||
const derived = await nativeAotOwner.derive(
|
||||
started.value.target.image_base,
|
||||
operation,
|
||||
options?.signal,
|
||||
normalized,
|
||||
);
|
||||
if (!derived.ok) return derived;
|
||||
}
|
||||
let metadata:
|
||||
| ReturnType<NativeAotPeResult["readTypeDetail"]>
|
||||
| undefined;
|
||||
metadata = nativeAotOwner?.readTypeDetail(address);
|
||||
if (
|
||||
metadata !== undefined &&
|
||||
normalized !== null &&
|
||||
typeof normalized === "object" &&
|
||||
!Array.isArray(normalized)
|
||||
) {
|
||||
const nativeResult = normalized as Record<string, JsonValue>;
|
||||
if (nativeResult.metadata_recovery === undefined)
|
||||
normalized = jsonValueSchema.parse({
|
||||
...nativeResult,
|
||||
metadata_recovery: metadata,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
if (operation === "inspect_native_load_image") {
|
||||
const attested = await attestGhidraNativeLoadImage(
|
||||
let derived: NativeAotPeResult | undefined;
|
||||
if (target.format === "pe") {
|
||||
const observations =
|
||||
nativeLoadImageObservationSchema.parse(normalized);
|
||||
if (nativeAotOwner === undefined)
|
||||
return err(
|
||||
new ProviderAdapterError("ghidra", operation, {
|
||||
diagnostics: {
|
||||
reason:
|
||||
"The admitted PE session has no read-only NativeAOT snapshot owner.",
|
||||
},
|
||||
}),
|
||||
);
|
||||
const result = await nativeAotOwner.derive(
|
||||
observations.image_base,
|
||||
operation,
|
||||
options?.signal,
|
||||
normalized,
|
||||
);
|
||||
if (!result.ok) return result;
|
||||
derived = result.value;
|
||||
}
|
||||
const attested = await attestGhidraNativeLoadImage({
|
||||
target,
|
||||
operation,
|
||||
normalized,
|
||||
measured: normalized,
|
||||
client,
|
||||
(failure) =>
|
||||
mapSessionError: (failure) =>
|
||||
projectSessionError(
|
||||
operation,
|
||||
failure,
|
||||
config.ghidraStartupTimeoutMs,
|
||||
),
|
||||
);
|
||||
nativeAotResult: derived,
|
||||
signal: options?.signal,
|
||||
});
|
||||
if (!attested.ok) return attested;
|
||||
normalized = attested.value;
|
||||
}
|
||||
@@ -368,7 +696,10 @@ export const createGhidraProviderClient = (input: {
|
||||
? []
|
||||
: [{ provider: committedProfile.provider, observation }];
|
||||
},
|
||||
close: () => client.close(),
|
||||
close: async () => {
|
||||
nativeAotOwner?.close();
|
||||
return client.close();
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
|
||||
@@ -44,6 +44,15 @@ describe("ghidraSessionRoot", () => {
|
||||
).toBe(posixBase("Users", "operator"));
|
||||
});
|
||||
|
||||
it("can select a filesystem root when the inherited temp starts with a dot-prefixed element", () => {
|
||||
expect(
|
||||
ghidraSessionRoot({
|
||||
base: posixBase(".Tmp", "scratch"),
|
||||
platform: "win32",
|
||||
}),
|
||||
).toBe(resolve(sep));
|
||||
});
|
||||
|
||||
it.skipIf(!POSIX)("prefers the platform temp directory by default", () => {
|
||||
expect(
|
||||
ghidraSessionRoot({ base: posixBase("home", "operator", ".cache") }),
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { constants } from "node:fs";
|
||||
import { access, realpath } from "node:fs/promises";
|
||||
import { access, realpath, stat } from "node:fs/promises";
|
||||
import { isAbsolute } from "node:path";
|
||||
import {
|
||||
AnalysisCancelledError,
|
||||
@@ -87,6 +87,8 @@ export const resolveWakaruCommand = async (
|
||||
if (!isAbsolute(limiter))
|
||||
throw new TypeError("REA_JAVASCRIPT_PRLIMIT_COMMAND must be absolute");
|
||||
await access(limiter, constants.X_OK);
|
||||
if (!(await stat(limiter)).isFile())
|
||||
throw new TypeError(`Resource limiter is not a regular file: ${limiter}`);
|
||||
return { command, limiter, ...fingerprint };
|
||||
} catch (cause: unknown) {
|
||||
const reason = `Configured Wakaru or util-linux prlimit is unavailable: ${recoveryFailureMessage(cause)}`;
|
||||
|
||||
@@ -28,6 +28,12 @@ interface Section {
|
||||
readonly size: number;
|
||||
}
|
||||
|
||||
// RVA ownership follows the declared virtual extent. Raw size describes
|
||||
// initialized file bytes and can include file-alignment padding; loaders may
|
||||
// map page-rounded tails, but those bytes are outside the declared section.
|
||||
const virtualExtent = (section: Section): number =>
|
||||
section.virtualSize === 0 ? section.size : section.virtualSize;
|
||||
|
||||
/** Header admission and unambiguous file-backed RVA mapping, independent of section names. */
|
||||
export const readPeResourceLayout = (bytes: Buffer) => {
|
||||
requirePeRange(bytes, 0, 64);
|
||||
@@ -64,7 +70,7 @@ export const readPeResourceLayout = (bytes: Buffer) => {
|
||||
offset: bytes.readUInt32LE(at + 20),
|
||||
size: bytes.readUInt32LE(at + 16),
|
||||
};
|
||||
if (section.rva + Math.max(section.virtualSize, section.size) > 0x100000000)
|
||||
if (section.rva + virtualExtent(section) > 0x100000000)
|
||||
peFailure("PE section RVA range overflows.");
|
||||
if (section.size > 0) {
|
||||
requirePeRange(bytes, section.offset, section.size);
|
||||
@@ -90,15 +96,14 @@ export const readPeResourceLayout = (bytes: Buffer) => {
|
||||
if (rva + size > 0x100000000) peFailure("Resource RVA range overflows.");
|
||||
const candidates = sections.filter(
|
||||
(section) =>
|
||||
rva >= section.rva &&
|
||||
rva - section.rva < Math.max(section.size, section.virtualSize),
|
||||
rva >= section.rva && rva - section.rva < virtualExtent(section),
|
||||
);
|
||||
if (rva < headerSize) {
|
||||
if (
|
||||
sections.some(
|
||||
(section) =>
|
||||
section.rva < rva + size &&
|
||||
section.rva + Math.max(section.size, section.virtualSize) > rva,
|
||||
section.rva + virtualExtent(section) > rva,
|
||||
) ||
|
||||
size > headerSize - rva
|
||||
)
|
||||
@@ -111,6 +116,8 @@ export const readPeResourceLayout = (bytes: Buffer) => {
|
||||
const section = candidates[0];
|
||||
if (section === undefined) return peFailure("Missing resource section.");
|
||||
const within = rva - section.rva;
|
||||
if (size > virtualExtent(section) - within)
|
||||
peFailure("Resource range leaves its section's virtual size.");
|
||||
if (within > section.size || size > section.size - within)
|
||||
peFailure("Resource range is not entirely file-backed in one section.");
|
||||
// A second section starting inside the requested interval is ambiguous too.
|
||||
@@ -119,7 +126,7 @@ export const readPeResourceLayout = (bytes: Buffer) => {
|
||||
(other) =>
|
||||
other !== section &&
|
||||
other.rva < rva + size &&
|
||||
other.rva + Math.max(other.size, other.virtualSize) > rva,
|
||||
other.rva + virtualExtent(other) > rva,
|
||||
)
|
||||
)
|
||||
peFailure("Resource interval intersects overlapping virtual sections.");
|
||||
|
||||
@@ -259,3 +259,62 @@ describe("PE header admission and bounded traversal", () => {
|
||||
await expect(parsing).rejects.toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe("PE resource section virtual extent", () => {
|
||||
const sectionSizes = (fixture: ReturnType<typeof peResourceFixture>) => {
|
||||
const virtualSize = fixture.bytes.readUInt32LE(fixture.sectionAt + 8);
|
||||
const rawSize = fixture.bytes.readUInt32LE(fixture.sectionAt + 16);
|
||||
expect(rawSize - virtualSize).toBeGreaterThanOrEqual(8);
|
||||
return { virtualSize, rawSize };
|
||||
};
|
||||
|
||||
it.each([
|
||||
[0, "Resource RVA has missing or overlapping section mappings."],
|
||||
[-2, "Resource range leaves its section's virtual size."],
|
||||
])(
|
||||
"rejects a payload reaching raw alignment padding past VirtualSize (start %i)",
|
||||
async (start, message) => {
|
||||
const fixture = peResourceFixture();
|
||||
const { virtualSize } = sectionSizes(fixture);
|
||||
const rva = 0x2000 + virtualSize + start;
|
||||
fixture.bytes.writeUInt32LE(rva, fixture.dataOffsets[0]);
|
||||
fixture.bytes.writeUInt32LE(4, (fixture.dataOffsets[0] ?? 0) + 4);
|
||||
await expect(parse(fixture.bytes)).rejects.toMatchObject({
|
||||
reason: "format",
|
||||
message,
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("maps raw bytes when VirtualSize is zero, as the image loader does", async () => {
|
||||
const fixture = peResourceFixture();
|
||||
fixture.bytes.writeUInt32LE(0, fixture.sectionAt + 8);
|
||||
expect(await parse(fixture.bytes)).toMatchObject({
|
||||
coverage: { status: "complete", resources: 5 },
|
||||
});
|
||||
});
|
||||
|
||||
it("selects a later section that starts inside earlier raw padding", async () => {
|
||||
const fixture = peResourceFixture();
|
||||
const { virtualSize } = sectionSizes(fixture);
|
||||
const laterRva = 0x2000 + Math.ceil(virtualSize / 4) * 4;
|
||||
const rawOffset = fixture.bytes.length;
|
||||
const payload = Buffer.from("LATE");
|
||||
const bytes = Buffer.concat([fixture.bytes, payload]);
|
||||
bytes.writeUInt16LE(2, 0x86);
|
||||
const later = fixture.sectionAt + 40;
|
||||
bytes.writeUInt32LE(payload.length, later + 8);
|
||||
bytes.writeUInt32LE(laterRva, later + 12);
|
||||
bytes.writeUInt32LE(payload.length, later + 16);
|
||||
bytes.writeUInt32LE(rawOffset, later + 20);
|
||||
bytes.writeUInt32LE(laterRva, fixture.dataOffsets[0]);
|
||||
bytes.writeUInt32LE(payload.length, (fixture.dataOffsets[0] ?? 0) + 4);
|
||||
|
||||
const result = peResourcesSchema.parse(await parse(bytes));
|
||||
expect(result.resources[0]?.payload).toMatchObject({
|
||||
rva: laterRva,
|
||||
location: { offset: rawOffset, bytes: payload.length },
|
||||
sha256: createHash("sha256").update(payload).digest("hex"),
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -5,7 +5,9 @@ import type { ExecutionOptions } from "../../application/AnalysisProvider.js";
|
||||
import { ArtifactOperationError } from "../../domain/artifactOperationError.js";
|
||||
import {
|
||||
AnalysisAccessDeniedError,
|
||||
AnalysisArtifactChangedError,
|
||||
AnalysisCancelledError,
|
||||
AnalysisInputError,
|
||||
AnalysisResourceConstraintError,
|
||||
} from "../../domain/analysisErrorCore.js";
|
||||
import type { InspectPeResourcesInput } from "../../domain/native/peResources.js";
|
||||
@@ -37,47 +39,70 @@ export class PeResourcesProvider implements PeResourcesPort {
|
||||
),
|
||||
);
|
||||
} catch (cause) {
|
||||
if (options?.signal?.aborted)
|
||||
return err(
|
||||
new AnalysisCancelledError("inspect_pe_resources", { cause }),
|
||||
);
|
||||
if (cause instanceof ArtifactReaderFailure && cause.reason === "limit")
|
||||
return err(
|
||||
new AnalysisResourceConstraintError(
|
||||
"inspect_pe_resources",
|
||||
"memory",
|
||||
cause.message,
|
||||
{
|
||||
max_file_bytes: input.max_file_bytes,
|
||||
max_entries: input.max_entries,
|
||||
},
|
||||
{ cause },
|
||||
),
|
||||
);
|
||||
if (
|
||||
cause instanceof Error &&
|
||||
"code" in cause &&
|
||||
(cause.code === "EACCES" || cause.code === "EPERM")
|
||||
)
|
||||
return err(
|
||||
new AnalysisAccessDeniedError(
|
||||
"inspect_pe_resources",
|
||||
input.path,
|
||||
cause.code,
|
||||
{ cause },
|
||||
),
|
||||
);
|
||||
return err(
|
||||
new ArtifactOperationError(
|
||||
"inspect_pe_resources",
|
||||
cause instanceof ArtifactReaderFailure ? cause.reason : "io",
|
||||
undefined,
|
||||
cause instanceof Error
|
||||
? cause.message
|
||||
: "PE artifact inspection failed.",
|
||||
{ cause },
|
||||
),
|
||||
);
|
||||
return err(peResourcesFailure(cause, input, options?.signal));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const peResourcesFailure = (
|
||||
cause: unknown,
|
||||
input: InspectPeResourcesInput,
|
||||
signal: AbortSignal | undefined,
|
||||
) => {
|
||||
const operation = "inspect_pe_resources";
|
||||
if (signal?.aborted) return new AnalysisCancelledError(operation, { cause });
|
||||
|
||||
if (cause instanceof ArtifactReaderFailure) {
|
||||
if (cause.reason === "limit")
|
||||
return new AnalysisResourceConstraintError(
|
||||
operation,
|
||||
"memory",
|
||||
cause.message,
|
||||
{
|
||||
max_file_bytes: input.max_file_bytes,
|
||||
max_entries: input.max_entries,
|
||||
},
|
||||
{ cause },
|
||||
);
|
||||
if (cause.reason === "integrity")
|
||||
return new AnalysisArtifactChangedError(
|
||||
operation,
|
||||
input.path,
|
||||
cause.message,
|
||||
{ cause },
|
||||
);
|
||||
if (cause.reason === "path" || cause.reason === "format")
|
||||
return new AnalysisInputError(operation, { cause }, [
|
||||
{
|
||||
path: ["path"],
|
||||
reason: "invalid_format",
|
||||
message: cause.message,
|
||||
},
|
||||
]);
|
||||
if (cause.reason === "cancelled")
|
||||
return new AnalysisCancelledError(operation, { cause });
|
||||
}
|
||||
|
||||
if (cause instanceof Error && "code" in cause) {
|
||||
if (cause.code === "EACCES" || cause.code === "EPERM")
|
||||
return new AnalysisAccessDeniedError(operation, input.path, cause.code, {
|
||||
cause,
|
||||
});
|
||||
if (cause.code === "ENOENT" || cause.code === "ENOTDIR")
|
||||
return new AnalysisInputError(operation, { cause }, [
|
||||
{
|
||||
path: ["path"],
|
||||
reason: "invalid_value",
|
||||
message: `Selected PE image could not be read (${cause.code}): ${input.path}.`,
|
||||
},
|
||||
]);
|
||||
}
|
||||
|
||||
return new ArtifactOperationError(
|
||||
operation,
|
||||
cause instanceof ArtifactReaderFailure ? cause.reason : "io",
|
||||
undefined,
|
||||
cause instanceof Error ? cause.message : "PE artifact inspection failed.",
|
||||
{ cause },
|
||||
);
|
||||
};
|
||||
|
||||
@@ -1,27 +1,53 @@
|
||||
import type { FileHandle } from "node:fs/promises";
|
||||
|
||||
/** Read through EOF, returning undefined on overflow; consume at most maxBytes + 1 bytes without closing the handle. */
|
||||
/** Read through EOF, returning undefined on overflow; leave the handle open. */
|
||||
export const readBoundedFileBytes = async (
|
||||
handle: FileHandle,
|
||||
maxBytes: number,
|
||||
signal?: AbortSignal,
|
||||
expectedSize?: number,
|
||||
): Promise<Buffer | undefined> => {
|
||||
if (!Number.isSafeInteger(maxBytes) || maxBytes < 0)
|
||||
throw new RangeError("File byte limit must be a nonnegative safe integer");
|
||||
const chunks: Buffer[] = [];
|
||||
const buffer = Buffer.allocUnsafe(Math.min(64 * 1_024, maxBytes + 1));
|
||||
if (
|
||||
expectedSize !== undefined &&
|
||||
(!Number.isSafeInteger(expectedSize) ||
|
||||
expectedSize < 0 ||
|
||||
expectedSize > maxBytes)
|
||||
)
|
||||
throw new RangeError("Expected file size must fit the byte limit");
|
||||
if (expectedSize === undefined) {
|
||||
const chunks: Buffer[] = [];
|
||||
const buffer = Buffer.allocUnsafe(Math.min(64 * 1_024, maxBytes + 1));
|
||||
let total = 0;
|
||||
for (;;) {
|
||||
signal?.throwIfAborted();
|
||||
const { bytesRead } = await handle.read(
|
||||
buffer,
|
||||
0,
|
||||
Math.min(buffer.length, maxBytes - total + 1),
|
||||
null,
|
||||
);
|
||||
if (bytesRead === 0) return Buffer.concat(chunks, total);
|
||||
total += bytesRead;
|
||||
if (total > maxBytes) return undefined;
|
||||
chunks.push(Buffer.from(buffer.subarray(0, bytesRead)));
|
||||
}
|
||||
}
|
||||
const buffer = Buffer.allocUnsafe(expectedSize);
|
||||
if (expectedSize === 0) return buffer;
|
||||
let total = 0;
|
||||
for (;;) {
|
||||
signal?.throwIfAborted();
|
||||
const { bytesRead } = await handle.read(
|
||||
buffer,
|
||||
0,
|
||||
Math.min(buffer.length, maxBytes - total + 1),
|
||||
total,
|
||||
expectedSize - total,
|
||||
null,
|
||||
);
|
||||
if (bytesRead === 0) return Buffer.concat(chunks, total);
|
||||
if (bytesRead === 0) return buffer.subarray(0, total);
|
||||
total += bytesRead;
|
||||
if (total > maxBytes) return undefined;
|
||||
chunks.push(Buffer.from(buffer.subarray(0, bytesRead)));
|
||||
if (total === expectedSize) return buffer;
|
||||
}
|
||||
};
|
||||
|
||||
@@ -339,11 +339,61 @@ interface OwnedCleanupPlan {
|
||||
readonly unverified: readonly OwnershipSweepUnverifiedProcess[];
|
||||
}
|
||||
|
||||
const createOwnedCleanupPlan = async (
|
||||
// A process-table read can race exit and retain the old state with an altered
|
||||
// command. Only a fresh table proving that PID is no longer live can settle
|
||||
// that mismatch; surviving group members still require normal token checks.
|
||||
const settleLauncherExit = async (
|
||||
ownership: OwnedProcessGroup,
|
||||
processes: readonly ProcessTableEntry[],
|
||||
host: ProcessOwnershipHost,
|
||||
): Promise<
|
||||
| readonly ProcessTableEntry[]
|
||||
| Extract<ProcessCleanupResult, { readonly cleaned: false }>
|
||||
> => {
|
||||
const launcher = processes.find(({ pid }) => pid === ownership.leaderPid);
|
||||
if (launcher === undefined) return processes;
|
||||
const reason = launcherIdentityFailure(launcher, ownership);
|
||||
if (reason === null) return processes;
|
||||
const refreshed = await readLiveProcessTable(host);
|
||||
if (
|
||||
refreshed.available &&
|
||||
!refreshed.processes.some(({ pid }) => pid === launcher.pid)
|
||||
) {
|
||||
// Refreshing must not discard a surviving detached descendant's lineage,
|
||||
// including a child that changed groups while the launcher exited.
|
||||
const descendants = descendantsOf([launcher.pid], processes);
|
||||
const descendantPids = new Set(descendants.map(({ pid }) => pid));
|
||||
const descendantGroupIds = new Set(
|
||||
descendants.map(({ processGroupId }) => processGroupId),
|
||||
);
|
||||
const refreshedDescendantPids = new Set(
|
||||
descendantsOf(
|
||||
descendants.map(({ pid }) => pid),
|
||||
refreshed.processes,
|
||||
).map(({ pid }) => pid),
|
||||
);
|
||||
if (
|
||||
refreshed.processes.some(
|
||||
(process) =>
|
||||
process.processGroupId !== ownership.processGroupId &&
|
||||
(refreshedDescendantPids.has(process.pid) ||
|
||||
descendantPids.has(process.pid) ||
|
||||
descendantGroupIds.has(process.processGroupId)),
|
||||
)
|
||||
)
|
||||
return { cleaned: false, reason };
|
||||
return refreshed.processes;
|
||||
}
|
||||
return { cleaned: false, reason };
|
||||
};
|
||||
|
||||
const createOwnedCleanupPlan = async (
|
||||
ownership: OwnedProcessGroup,
|
||||
initialProcesses: readonly ProcessTableEntry[],
|
||||
host: ProcessOwnershipHost,
|
||||
): Promise<OwnedCleanupPlan | ProcessCleanupResult> => {
|
||||
const processes = await settleLauncherExit(ownership, initialProcesses, host);
|
||||
if ("cleaned" in processes) return processes;
|
||||
const sampledProcessGroupIds = new Set(
|
||||
(ownership.sampledProcessGroupIds ?? []).filter(
|
||||
(processGroupId) =>
|
||||
@@ -397,10 +447,7 @@ const createOwnedCleanupPlan = async (
|
||||
};
|
||||
let descendants: readonly ProcessTableEntry[] = [];
|
||||
if (launcher !== undefined) {
|
||||
const identityFailure = launcherIdentityFailure(launcher, ownership);
|
||||
if (identityFailure !== null)
|
||||
return { cleaned: false, reason: identityFailure };
|
||||
descendants = descendantsOf(launcher.pid, processes);
|
||||
descendants = descendantsOf([launcher.pid], processes);
|
||||
}
|
||||
const descendantPids = new Set(descendants.map(({ pid }) => pid));
|
||||
const processGroupIds = new Set<number>([ownership.processGroupId]);
|
||||
@@ -496,7 +543,13 @@ const revalidateOwnedProcessGroup = async (
|
||||
): Promise<{ readonly empty: boolean } | ProcessCleanupResult> => {
|
||||
let members: readonly ProcessTableEntry[];
|
||||
try {
|
||||
members = liveProcesses(await host.listProcesses()).filter(
|
||||
const processes = await settleLauncherExit(
|
||||
ownership,
|
||||
liveProcesses(await host.listProcesses()),
|
||||
host,
|
||||
);
|
||||
if ("cleaned" in processes) return processes;
|
||||
members = processes.filter(
|
||||
({ processGroupId: observedGroupId }) =>
|
||||
observedGroupId === processGroupId,
|
||||
);
|
||||
@@ -513,14 +566,6 @@ const revalidateOwnedProcessGroup = async (
|
||||
expectedIdentities,
|
||||
);
|
||||
if (failures.length > 0) return cleanupValidationFailure(failures);
|
||||
if (processGroupId === ownership.processGroupId) {
|
||||
const launcher = members.find(({ pid }) => pid === ownership.leaderPid);
|
||||
if (launcher !== undefined) {
|
||||
const identityFailure = launcherIdentityFailure(launcher, ownership);
|
||||
if (identityFailure !== null)
|
||||
return { cleaned: false, reason: identityFailure };
|
||||
}
|
||||
}
|
||||
if (
|
||||
processGroupId !== ownership.processGroupId &&
|
||||
members.length > 0 &&
|
||||
@@ -850,7 +895,7 @@ const scanTokenOwnedProcesses = async (
|
||||
const relatedGroupIds = new Set<number>();
|
||||
if (relation !== undefined) {
|
||||
relatedPids.add(relation.leaderPid);
|
||||
for (const descendant of descendantsOf(relation.leaderPid, processes))
|
||||
for (const descendant of descendantsOf([relation.leaderPid], processes))
|
||||
relatedPids.add(descendant.pid);
|
||||
relatedGroupIds.add(relation.processGroupId);
|
||||
for (const processGroupId of relation.sampledProcessGroupIds ?? [])
|
||||
|
||||
@@ -3,6 +3,7 @@ import {
|
||||
cleanupOwnedProcessGroup,
|
||||
cleanupWindowsProcessTree,
|
||||
type ProcessOwnershipHost,
|
||||
type ProcessTableEntry,
|
||||
type WindowsProcessTreeHost,
|
||||
} from "./ProcessOwnership.js";
|
||||
import {
|
||||
@@ -11,18 +12,20 @@ import {
|
||||
} from "./ProcessOwnershipObservation.js";
|
||||
import { host, ownership } from "./ProcessOwnership.fixture.js";
|
||||
|
||||
const launcher: ProcessTableEntry = {
|
||||
pid: 100,
|
||||
parentPid: 1,
|
||||
processGroupId: 100,
|
||||
state: "S",
|
||||
command: "fixture",
|
||||
};
|
||||
|
||||
describe("owned process-group cleanup validation: ownership and lineage", () => {
|
||||
it("fails closed when a descendant in another process group lacks the token", async () => {
|
||||
const adapter: ProcessOwnershipHost = {
|
||||
listProcesses: () =>
|
||||
Promise.resolve([
|
||||
{
|
||||
pid: 100,
|
||||
parentPid: 1,
|
||||
processGroupId: 100,
|
||||
state: "S",
|
||||
command: "fixture",
|
||||
},
|
||||
launcher,
|
||||
{
|
||||
pid: 101,
|
||||
parentPid: 100,
|
||||
@@ -211,16 +214,9 @@ describe("owned process-group cleanup liveness rechecks", () => {
|
||||
);
|
||||
|
||||
it("accepts a member that exits during ownership revalidation", async () => {
|
||||
const liveLauncher = {
|
||||
pid: 100,
|
||||
parentPid: 1,
|
||||
processGroupId: 100,
|
||||
state: "S",
|
||||
command: "fixture",
|
||||
};
|
||||
const listProcesses = vi
|
||||
.fn<ProcessOwnershipHost["listProcesses"]>()
|
||||
.mockResolvedValueOnce([liveLauncher])
|
||||
.mockResolvedValueOnce([launcher])
|
||||
.mockResolvedValue([]);
|
||||
const signalGroup = vi.fn();
|
||||
const adapter: ProcessOwnershipHost = {
|
||||
@@ -239,6 +235,181 @@ describe("owned process-group cleanup liveness rechecks", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("owned process-group cleanup validation: refreshed descendant lineage", () => {
|
||||
it.each(["planning", "signaling"])(
|
||||
"retains cleanup uncertainty for newly detached descendants during %s",
|
||||
async (phase) => {
|
||||
const child = { ...launcher, pid: 101, parentPid: 100 };
|
||||
const stale = { ...launcher, command: "[MainThread]" };
|
||||
const signalGroup = vi.fn();
|
||||
let reads = 0;
|
||||
const adapter: ProcessOwnershipHost = {
|
||||
listProcesses: () => {
|
||||
reads += 1;
|
||||
if (phase === "signaling" && reads === 1)
|
||||
return Promise.resolve([launcher, child]);
|
||||
if (reads === (phase === "planning" ? 1 : 2))
|
||||
return Promise.resolve([stale, child]);
|
||||
return Promise.resolve([
|
||||
{ ...child, parentPid: 1 },
|
||||
{ ...child, pid: 102, parentPid: 101 },
|
||||
{ ...child, pid: 103, parentPid: 102, processGroupId: 103 },
|
||||
]);
|
||||
},
|
||||
environment: () =>
|
||||
Promise.resolve({ REA_PROCESS_RUN_ID: ownership.runId }),
|
||||
signalGroup,
|
||||
};
|
||||
await expect(
|
||||
cleanupOwnedProcessGroup(
|
||||
{ ...ownership, expectedCommand: "fixture" },
|
||||
adapter,
|
||||
),
|
||||
).resolves.toMatchObject({ cleaned: false });
|
||||
expect(signalGroup).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("owned process-group cleanup validation: launcher exit races", () => {
|
||||
it.each([101, 999])(
|
||||
"retains cleanup uncertainty for a detached descendant in group %s after launcher exit",
|
||||
async (processGroupId) => {
|
||||
const child = {
|
||||
...launcher,
|
||||
pid: 101,
|
||||
parentPid: 100,
|
||||
processGroupId: 101,
|
||||
};
|
||||
const signalGroup = vi.fn();
|
||||
const listProcesses = vi
|
||||
.fn<ProcessOwnershipHost["listProcesses"]>()
|
||||
.mockResolvedValueOnce([
|
||||
{ ...launcher, command: "[MainThread]" },
|
||||
child,
|
||||
])
|
||||
.mockResolvedValue([{ ...child, parentPid: 1, processGroupId }]);
|
||||
await expect(
|
||||
cleanupOwnedProcessGroup(
|
||||
{
|
||||
...ownership,
|
||||
expectedCommand: "fixture",
|
||||
},
|
||||
{
|
||||
listProcesses,
|
||||
environment: () =>
|
||||
Promise.resolve({ REA_PROCESS_RUN_ID: "run-token" }),
|
||||
signalGroup,
|
||||
},
|
||||
),
|
||||
).resolves.toMatchObject({ cleaned: false });
|
||||
expect(signalGroup).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
it.each(["planning", "signaling"])(
|
||||
"settles a stale launcher command during %s without skipping surviving group ownership",
|
||||
async (phase) => {
|
||||
for (const remaining of ["absent", "zombie", "owned", "foreign"]) {
|
||||
const stale = { ...launcher, command: "[MainThread]" };
|
||||
const child = { ...launcher, pid: 101, command: "child" };
|
||||
let reads = 0;
|
||||
const signalGroup = vi.fn();
|
||||
const adapter: ProcessOwnershipHost = {
|
||||
listProcesses: () => {
|
||||
reads += 1;
|
||||
if (phase === "signaling" && reads === 1)
|
||||
return Promise.resolve([launcher]);
|
||||
if (reads === (phase === "planning" ? 1 : 2))
|
||||
return Promise.resolve([stale]);
|
||||
return Promise.resolve(
|
||||
remaining === "absent"
|
||||
? []
|
||||
: remaining === "zombie"
|
||||
? [{ ...stale, state: "Z" }]
|
||||
: [child],
|
||||
);
|
||||
},
|
||||
environment: () =>
|
||||
Promise.resolve({
|
||||
REA_PROCESS_RUN_ID:
|
||||
remaining === "foreign" && reads > 1
|
||||
? "other-run"
|
||||
: "run-token",
|
||||
}),
|
||||
signalGroup,
|
||||
};
|
||||
const result = await cleanupOwnedProcessGroup(
|
||||
{ ...ownership, expectedCommand: "fixture" },
|
||||
adapter,
|
||||
);
|
||||
if (remaining === "foreign")
|
||||
expect(result).toMatchObject({
|
||||
cleaned: false,
|
||||
reason: "process tree contains an unowned or PID-reused process",
|
||||
});
|
||||
else
|
||||
expect(result).toEqual({
|
||||
cleaned: true,
|
||||
signaled: remaining === "owned",
|
||||
});
|
||||
if (remaining === "owned")
|
||||
expect(signalGroup).toHaveBeenCalledWith(100, "SIGKILL");
|
||||
else expect(signalGroup).not.toHaveBeenCalled();
|
||||
}
|
||||
},
|
||||
);
|
||||
it("fails closed when a mismatched launcher's exit cannot be inspected", async () => {
|
||||
const { adapter, signalGroup } = host({
|
||||
100: { REA_PROCESS_RUN_ID: "run-token" },
|
||||
});
|
||||
const listProcesses = vi
|
||||
.fn<ProcessOwnershipHost["listProcesses"]>()
|
||||
.mockResolvedValueOnce(await adapter.listProcesses())
|
||||
.mockRejectedValue(new Error("process table unavailable"));
|
||||
await expect(
|
||||
cleanupOwnedProcessGroup(
|
||||
{ ...ownership, expectedCommand: "/owned/hopper" },
|
||||
{ ...adapter, listProcesses },
|
||||
),
|
||||
).resolves.toMatchObject({
|
||||
cleaned: false,
|
||||
reason: expect.stringContaining("command identity did not match"),
|
||||
});
|
||||
expect(signalGroup).not.toHaveBeenCalled();
|
||||
});
|
||||
it.each(["planning", "signaling"])(
|
||||
"rejects a surviving launcher PID in another group during %s",
|
||||
async (phase) => {
|
||||
let reads = 0;
|
||||
const signalGroup = vi.fn();
|
||||
const adapter: ProcessOwnershipHost = {
|
||||
listProcesses: () => {
|
||||
reads += 1;
|
||||
return Promise.resolve([
|
||||
phase === "signaling" && reads === 1
|
||||
? launcher
|
||||
: reads === (phase === "planning" ? 1 : 2)
|
||||
? { ...launcher, command: "[MainThread]" }
|
||||
: { ...launcher, processGroupId: 999 },
|
||||
]);
|
||||
},
|
||||
environment: () => Promise.resolve({ REA_PROCESS_RUN_ID: "run-token" }),
|
||||
signalGroup,
|
||||
};
|
||||
await expect(
|
||||
cleanupOwnedProcessGroup(
|
||||
{ ...ownership, expectedCommand: "fixture" },
|
||||
adapter,
|
||||
),
|
||||
).resolves.toMatchObject({
|
||||
cleaned: false,
|
||||
reason: expect.stringContaining("command identity did not match"),
|
||||
});
|
||||
expect(signalGroup).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("owned process-group cleanup validation: exited members", () => {
|
||||
it("preserves cancellation during process-table observation instead of reporting unverifiable ownership", async () => {
|
||||
const controller = new AbortController();
|
||||
@@ -293,13 +464,7 @@ describe("owned process-group cleanup validation: exited members", () => {
|
||||
const adapter: ProcessOwnershipHost = {
|
||||
listProcesses: () =>
|
||||
Promise.resolve([
|
||||
{
|
||||
pid: 100,
|
||||
parentPid: 1,
|
||||
processGroupId: 100,
|
||||
state: "S",
|
||||
command: "fixture",
|
||||
},
|
||||
launcher,
|
||||
{
|
||||
pid: 101,
|
||||
parentPid: 100,
|
||||
|
||||
@@ -116,7 +116,7 @@ export const observeOwnedProcessLineage = async (
|
||||
const identityFailure = launcherIdentityFailure(launcher, ownership);
|
||||
if (identityFailure !== null)
|
||||
return unavailableLineage(ownership, identityFailure);
|
||||
const descendants = descendantsOf(launcher.pid, processes);
|
||||
const descendants = descendantsOf([launcher.pid], processes);
|
||||
const verifiedDescendants: ProcessTableEntry[] = [];
|
||||
for await (const { process: member, observation } of readProcessRunTokens(
|
||||
host,
|
||||
|
||||
@@ -6,9 +6,9 @@ export const liveProcesses = (
|
||||
): readonly ProcessTableEntry[] =>
|
||||
members.filter(({ state }) => !state.startsWith("Z"));
|
||||
|
||||
/** Return every live-table descendant of a launcher in breadth-first order. */
|
||||
/** Return every live-table descendant of the selected roots in breadth-first order. */
|
||||
export const descendantsOf = (
|
||||
launcherPid: number,
|
||||
rootPids: readonly number[],
|
||||
processes: readonly ProcessTableEntry[],
|
||||
): ProcessTableEntry[] => {
|
||||
const childrenByParent = new Map<number, ProcessTableEntry[]>();
|
||||
@@ -18,8 +18,8 @@ export const descendantsOf = (
|
||||
childrenByParent.set(process.parentPid, siblings);
|
||||
}
|
||||
const descendants: ProcessTableEntry[] = [];
|
||||
const pending = [...(childrenByParent.get(launcherPid) ?? [])];
|
||||
const visited = new Set<number>([launcherPid]);
|
||||
const pending = rootPids.flatMap((pid) => childrenByParent.get(pid) ?? []);
|
||||
const visited = new Set<number>(rootPids);
|
||||
for (const process of pending) {
|
||||
if (visited.has(process.pid)) continue;
|
||||
visited.add(process.pid);
|
||||
|
||||
@@ -8,6 +8,7 @@ import {
|
||||
import { toolContract } from "../contracts/toolContracts.js";
|
||||
import type { EvidenceBundle } from "../domain/evidenceBundle.js";
|
||||
import { ok } from "../domain/result.js";
|
||||
import { inspectEvidenceBundle } from "../application/investigation/InspectEvidenceBundle.js";
|
||||
import { toolRegistrationOptions } from "./toolRegistrationOptions.js";
|
||||
|
||||
interface EvidenceToolRegistration {
|
||||
@@ -72,9 +73,9 @@ const registerSnapshotEvidenceTool = ({
|
||||
server.registerTool(
|
||||
snapshotContract.name,
|
||||
toolRegistrationOptions(snapshotContract),
|
||||
() =>
|
||||
(input) =>
|
||||
server.delivery.toCallToolResult(
|
||||
ok(bundleForSerialization(session)),
|
||||
inspectEvidenceBundle(bundleForSerialization(session), input),
|
||||
snapshotContract,
|
||||
),
|
||||
);
|
||||
|
||||
@@ -0,0 +1,120 @@
|
||||
import { mkdir, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { expect } from "vitest";
|
||||
import { z } from "zod";
|
||||
|
||||
import { javascriptApplicationAnalysisResultSchema } from "../../../src/domain/javascript/javascriptApplicationAnalysis.js";
|
||||
import { connectLocalToolsMcp } from "../../fixtures/localToolsMcp.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
import { cliTest } from "../../support/cli/cliFixture.js";
|
||||
|
||||
cliTest.for(["CLI", "stdio MCP"] as const)(
|
||||
"keeps dependency manifests out of application roots and Electron entries through %s",
|
||||
async (surface, { cli }) => {
|
||||
const root = await createTestTempDirectory("rea-application-manifest-");
|
||||
const dependency = join(root, "node_modules", "dep");
|
||||
const nested = join(root, "a");
|
||||
await Promise.all([mkdir(dependency, { recursive: true }), mkdir(nested)]);
|
||||
await Promise.all([
|
||||
writeFile(
|
||||
join(nested, "package.json"),
|
||||
JSON.stringify({ name: "nested" }),
|
||||
),
|
||||
writeFile(
|
||||
join(root, "package.json"),
|
||||
JSON.stringify({ name: "app", main: "main.js" }),
|
||||
),
|
||||
writeFile(
|
||||
join(root, "main.js"),
|
||||
"throw new Error('source must remain inert');",
|
||||
),
|
||||
writeFile(
|
||||
join(dependency, "package.json"),
|
||||
JSON.stringify({
|
||||
name: "dep",
|
||||
main: "index.js",
|
||||
browser: "browser.js",
|
||||
}),
|
||||
),
|
||||
writeFile(join(dependency, "index.js"), "module.exports = {};"),
|
||||
writeFile(join(dependency, "browser.js"), "export const browser = true;"),
|
||||
]);
|
||||
|
||||
let document: unknown;
|
||||
if (surface === "CLI") {
|
||||
const response = await cli.run({
|
||||
arguments: ["analyze-javascript-application", root, "--json"],
|
||||
});
|
||||
expect(response.exitCode).toBe(0);
|
||||
document = response.json;
|
||||
} else {
|
||||
const { call } = await connectLocalToolsMcp();
|
||||
const response = await call("analyze_javascript_application", {
|
||||
input_path: root,
|
||||
});
|
||||
expect(response.isError, JSON.stringify(response)).not.toBe(true);
|
||||
document = response.structuredContent;
|
||||
}
|
||||
const { graph } = z
|
||||
.object({ normalized_result: javascriptApplicationAnalysisResultSchema })
|
||||
.parse(document).normalized_result;
|
||||
expect(
|
||||
graph.nodes
|
||||
.filter(({ node_id }) => graph.root_node_ids.includes(node_id))
|
||||
.flatMap(({ observations }) => observations.map(({ label }) => label)),
|
||||
).toEqual(["app"]);
|
||||
expect(
|
||||
graph.nodes
|
||||
.filter(({ kind }) => kind === "electron-main")
|
||||
.flatMap(({ observations }) =>
|
||||
observations.map(({ properties }) => properties.declared_path),
|
||||
),
|
||||
).toEqual(["main.js"]);
|
||||
expect(
|
||||
graph.nodes.filter(({ kind }) => kind === "electron-renderer"),
|
||||
).toEqual([]);
|
||||
expect(
|
||||
graph.nodes
|
||||
.filter(({ kind }) => kind === "package")
|
||||
.flatMap(({ observations }) => observations.map(({ label }) => label))
|
||||
.sort(),
|
||||
).toEqual(["app", "dep", "nested"]);
|
||||
},
|
||||
);
|
||||
|
||||
cliTest(
|
||||
"keeps the artifact root when only dependency manifests are present",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-dependency-only-");
|
||||
const dependency = join(root, "node_modules", "dep");
|
||||
await mkdir(dependency, { recursive: true });
|
||||
await writeFile(
|
||||
join(dependency, "package.json"),
|
||||
JSON.stringify({ name: "dep", main: "index.js" }),
|
||||
);
|
||||
await writeFile(join(dependency, "index.js"), "module.exports = {};");
|
||||
const response = await cli.run({
|
||||
arguments: ["analyze-javascript-application", root, "--json"],
|
||||
});
|
||||
expect(response.exitCode, response.stderr).toBe(0);
|
||||
const { graph } = z
|
||||
.object({ normalized_result: javascriptApplicationAnalysisResultSchema })
|
||||
.parse(response.json).normalized_result;
|
||||
expect(
|
||||
graph.nodes
|
||||
.filter(({ node_id }) => graph.root_node_ids.includes(node_id))
|
||||
.map(({ kind }) => kind),
|
||||
).toEqual(["artifact"]);
|
||||
expect(
|
||||
graph.nodes
|
||||
.filter(({ kind }) => kind === "package")
|
||||
.flatMap(({ observations }) => observations.map(({ label }) => label)),
|
||||
).toEqual(["dep"]);
|
||||
expect(
|
||||
graph.nodes.some(
|
||||
({ kind }) => kind === "electron-main" || kind === "electron-renderer",
|
||||
),
|
||||
).toBe(false);
|
||||
},
|
||||
);
|
||||
@@ -6,7 +6,7 @@ import { promisify } from "node:util";
|
||||
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { afterAll, describe, expect, it } from "vitest";
|
||||
import { z } from "zod";
|
||||
|
||||
import { javascriptApplicationAnalysisResultSchema } from "../../../src/domain/javascript/javascriptApplicationAnalysis.js";
|
||||
@@ -18,6 +18,27 @@ const requireFixture = createRequire(import.meta.url);
|
||||
const comparisonEnvelope = z.object({
|
||||
normalized_result: javaScriptExportShapeComparisonResultSchema,
|
||||
});
|
||||
let mcpComparisonConnection:
|
||||
| {
|
||||
readonly client: Client;
|
||||
readonly transport: StdioClientTransport;
|
||||
readonly ready: Promise<void>;
|
||||
}
|
||||
| undefined;
|
||||
|
||||
afterAll(async () => {
|
||||
if (mcpComparisonConnection === undefined) return;
|
||||
try {
|
||||
await mcpComparisonConnection.ready;
|
||||
} catch {
|
||||
// Keep the failed connection owned until both resources are closed below.
|
||||
}
|
||||
try {
|
||||
await mcpComparisonConnection.client.close();
|
||||
} finally {
|
||||
await mcpComparisonConnection.transport.close();
|
||||
}
|
||||
});
|
||||
|
||||
describe("exported function binding resolution through the CLI", () => {
|
||||
it.each([
|
||||
@@ -289,6 +310,113 @@ describe("CommonJS assignment-chain receiver identity through the CLI", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("CommonJS export identity and scope regressions", () => {
|
||||
it("does not export a named write through the original exports alias after replacement", async () => {
|
||||
const fixture = await analyzeVersions(
|
||||
(count) =>
|
||||
`const VERSION = ${String(count)};\nexports.parse = (module.exports = () => ({ count: VERSION }));`,
|
||||
"parse",
|
||||
"app.cjs",
|
||||
);
|
||||
for (const path of [fixture.leftPath, fixture.rightPath]) {
|
||||
const value = requireFixture(path);
|
||||
expect(typeof value).toBe("function");
|
||||
expect(Reflect.ownKeys(value)).toEqual(["length", "name"]);
|
||||
}
|
||||
const cli = await compareThroughCli(fixture);
|
||||
expect(cli).toMatchObject({
|
||||
left: { status: "missing" },
|
||||
right: { status: "missing" },
|
||||
summary: { added: 0, removed: 0, changed: 0 },
|
||||
});
|
||||
expect(await compareThroughMcp(fixture.input)).toEqual(cli);
|
||||
});
|
||||
|
||||
it("keeps a dead var shadow of the alias-chain RHS unavailable", async () => {
|
||||
const fixture = await analyzeVersions(
|
||||
(count) =>
|
||||
`function parse() { return { count: ${String(count)} }; }\nfunction configure() { if (false) { var parse = () => ({ count: 7 }); } module.exports = exports = parse; }\nconfigure();`,
|
||||
"default",
|
||||
"app.cjs",
|
||||
);
|
||||
expect(requireFixture(fixture.leftPath)).toBeUndefined();
|
||||
expect(requireFixture(fixture.rightPath)).toBeUndefined();
|
||||
const cli = await compareThroughCli(fixture);
|
||||
expectUncertainComparison(cli);
|
||||
expect(await compareThroughMcp(fixture.input)).toEqual(cli);
|
||||
});
|
||||
|
||||
it.each(["+=", "||=", "??="] as const)(
|
||||
"does not treat module.exports %s as a definite function export",
|
||||
async (operator) => {
|
||||
const fixture = await analyzeVersions(
|
||||
(count) =>
|
||||
`module.exports ${operator} (() => ({ kind: "result", count: ${String(count)} }));`,
|
||||
"default",
|
||||
"app.cjs",
|
||||
);
|
||||
const left = requireFixture(fixture.leftPath);
|
||||
const right = requireFixture(fixture.rightPath);
|
||||
if (operator === "+=") {
|
||||
expect(typeof left).toBe("string");
|
||||
expect(typeof right).toBe("string");
|
||||
} else {
|
||||
expect(left).toEqual({});
|
||||
expect(right).toEqual({});
|
||||
}
|
||||
const cli = await compareThroughCli(fixture);
|
||||
expectUncertainComparison(cli);
|
||||
expect(await compareThroughMcp(fixture.input)).toEqual(cli);
|
||||
},
|
||||
);
|
||||
|
||||
it("keeps a parameter-default mutation unknown in a CommonJS return shape", async () => {
|
||||
const fixture = await analyzeVersions(
|
||||
(count) =>
|
||||
`const source = { value: 0 }; module.exports = exports = function current(arg = null, set = (arg = source)) { var arg; arg.value = ${String(count)}; return { kind: "result", count: source.value }; };`,
|
||||
"default",
|
||||
"app.cjs",
|
||||
);
|
||||
expect(requireFixture(fixture.leftPath)()).toEqual({
|
||||
kind: "result",
|
||||
count: 1,
|
||||
});
|
||||
expect(requireFixture(fixture.rightPath)()).toEqual({
|
||||
kind: "result",
|
||||
count: 2,
|
||||
});
|
||||
for (const evidence of [fixture.input.left, fixture.input.right]) {
|
||||
const analysis = z
|
||||
.object({
|
||||
normalized_result: javascriptApplicationAnalysisResultSchema,
|
||||
})
|
||||
.parse(evidence).normalized_result;
|
||||
const shapes = analysis.graph.nodes
|
||||
.flatMap(({ observations }) => observations)
|
||||
.find(
|
||||
({ properties }) =>
|
||||
properties.semantic_role === "export-return-shapes" &&
|
||||
properties.exported_name === "default",
|
||||
)?.properties.static_return_shapes;
|
||||
expect(shapes).toEqual([
|
||||
expect.objectContaining({
|
||||
fields: expect.arrayContaining([
|
||||
expect.objectContaining({ path: "/count", state: "unknown" }),
|
||||
]),
|
||||
}),
|
||||
]);
|
||||
}
|
||||
const cli = await compareThroughCli(fixture);
|
||||
expect(cli).toMatchObject({
|
||||
left: { status: "selected" },
|
||||
right: { status: "selected" },
|
||||
summary: { added: 0, removed: 0, changed: 0 },
|
||||
changes: [],
|
||||
});
|
||||
expect(await compareThroughMcp(fixture.input)).toEqual(cli);
|
||||
});
|
||||
});
|
||||
|
||||
describe("stable exported function bindings through the CLI", () => {
|
||||
it.each([
|
||||
{
|
||||
@@ -798,34 +926,37 @@ const compareThroughCli = async (
|
||||
};
|
||||
|
||||
const compareThroughMcp = async (input: Record<string, unknown>) => {
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [resolve("scripts/rea.mjs"), "mcp"],
|
||||
cwd: process.cwd(),
|
||||
env: { PATH: process.env.PATH ?? "", REA_LOG_LEVEL: "silent" },
|
||||
stderr: "pipe",
|
||||
const client = await getMcpComparisonClient();
|
||||
const response = await client.callTool({
|
||||
name: "compare_javascript_export_shapes",
|
||||
arguments: input,
|
||||
});
|
||||
const client = new Client({
|
||||
name: "exported-function-bindings",
|
||||
version: "1",
|
||||
});
|
||||
try {
|
||||
await client.connect(transport);
|
||||
const response = await client.callTool({
|
||||
name: "compare_javascript_export_shapes",
|
||||
arguments: input,
|
||||
expect(response.isError).not.toBe(true);
|
||||
await client.ping();
|
||||
return comparisonEnvelope.parse(response.structuredContent).normalized_result;
|
||||
};
|
||||
|
||||
const getMcpComparisonClient = async (): Promise<Client> => {
|
||||
if (mcpComparisonConnection === undefined) {
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [resolve("scripts/rea.mjs"), "mcp"],
|
||||
cwd: process.cwd(),
|
||||
env: { PATH: process.env.PATH ?? "", REA_LOG_LEVEL: "silent" },
|
||||
stderr: "pipe",
|
||||
});
|
||||
expect(response.isError).not.toBe(true);
|
||||
await client.ping();
|
||||
return comparisonEnvelope.parse(response.structuredContent)
|
||||
.normalized_result;
|
||||
} finally {
|
||||
try {
|
||||
await client.close();
|
||||
} finally {
|
||||
await transport.close();
|
||||
}
|
||||
const client = new Client({
|
||||
name: "exported-function-bindings",
|
||||
version: "1",
|
||||
});
|
||||
mcpComparisonConnection = {
|
||||
client,
|
||||
transport,
|
||||
ready: client.connect(transport),
|
||||
};
|
||||
}
|
||||
await mcpComparisonConnection.ready;
|
||||
return mcpComparisonConnection.client;
|
||||
};
|
||||
|
||||
const runtimeExport = async (
|
||||
|
||||
@@ -8,7 +8,87 @@ import { toolContract } from "../../../src/contracts/toolContracts.js";
|
||||
import { peResourcesSchema } from "../../../src/domain/native/peResources.js";
|
||||
import { peResourceFixture } from "../../../src/native/pe/PeResources.fixture.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
import { cliTest } from "../../support/cli/cliFixture.js";
|
||||
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
|
||||
import { cliTest, type TestCli } from "../../support/cli/cliFixture.js";
|
||||
|
||||
const expectPeInputFailureAcrossAdapters = async (input: {
|
||||
readonly cli: TestCli;
|
||||
readonly client: Client;
|
||||
readonly toolName: string;
|
||||
readonly path: string;
|
||||
readonly reason: "invalid_format" | "invalid_value";
|
||||
}) => {
|
||||
const { cli, client, toolName, path, reason } = input;
|
||||
const result = await cli.run({
|
||||
arguments: ["inspect-pe-resources", path, "--json"],
|
||||
});
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.json).toMatchObject({
|
||||
code: "invalid_request",
|
||||
category: "invalid_input",
|
||||
details: {
|
||||
operation: "inspect_pe_resources",
|
||||
issues: [{ path: ["path"], reason }],
|
||||
},
|
||||
});
|
||||
const response = await client.callTool({
|
||||
name: toolName,
|
||||
arguments: { path },
|
||||
});
|
||||
expect(response.isError).toBe(true);
|
||||
expect(parseMcpToolError(response)).toEqual({ error: result.json });
|
||||
};
|
||||
|
||||
const expectPePaddingFailureAcrossAdapters = async (input: {
|
||||
readonly cli: TestCli;
|
||||
readonly client: Client;
|
||||
readonly toolName: string;
|
||||
readonly path: string;
|
||||
}) => {
|
||||
const { cli, client, toolName, path } = input;
|
||||
const fixture = peResourceFixture();
|
||||
const virtualSize = fixture.bytes.readUInt32LE(fixture.sectionAt + 8);
|
||||
fixture.bytes.writeUInt32LE(
|
||||
0x2000 + virtualSize,
|
||||
fixture.dataOffsets[0] ?? 0,
|
||||
);
|
||||
fixture.bytes.writeUInt32LE(4, (fixture.dataOffsets[0] ?? 0) + 4);
|
||||
await writeFile(path, fixture.bytes);
|
||||
const cliResult = await cli.run({
|
||||
arguments: ["inspect-pe-resources", path, "--json"],
|
||||
});
|
||||
expect(cliResult.exitCode).not.toBe(0);
|
||||
const mcpResult = await client.callTool({
|
||||
name: toolName,
|
||||
arguments: { path },
|
||||
});
|
||||
expect(mcpResult.isError).toBe(true);
|
||||
expect(peResourcesSchema.safeParse(mcpResult.structuredContent).success).toBe(
|
||||
false,
|
||||
);
|
||||
};
|
||||
|
||||
const expectPeInputFailuresAcrossAdapters = async (input: {
|
||||
readonly cli: TestCli;
|
||||
readonly client: Client;
|
||||
readonly toolName: string;
|
||||
readonly root: string;
|
||||
}) => {
|
||||
const { cli, client, toolName, root } = input;
|
||||
const malformedPath = join(root, "malformed.exe");
|
||||
await writeFile(malformedPath, Buffer.from("not a PE image"));
|
||||
for (const [path, reason] of [
|
||||
[join(root, "missing.exe"), "invalid_value"],
|
||||
[malformedPath, "invalid_format"],
|
||||
] as const)
|
||||
await expectPeInputFailureAcrossAdapters({
|
||||
cli,
|
||||
client,
|
||||
toolName,
|
||||
path,
|
||||
reason,
|
||||
});
|
||||
};
|
||||
|
||||
cliTest(
|
||||
"inspects PE32/PE32+ resources through compiled CLI and real stdio MCP without an active target",
|
||||
@@ -92,6 +172,18 @@ cliTest(
|
||||
).toBe(evidence.evidence_id);
|
||||
expect(await readFile(path)).toEqual(fixture.bytes);
|
||||
}
|
||||
await expectPePaddingFailureAcrossAdapters({
|
||||
cli,
|
||||
client,
|
||||
toolName: contract.name,
|
||||
path: join(root, "raw-padding.exe"),
|
||||
});
|
||||
await expectPeInputFailuresAcrossAdapters({
|
||||
cli,
|
||||
client,
|
||||
toolName: contract.name,
|
||||
root,
|
||||
});
|
||||
const absent = peResourceFixture();
|
||||
absent.bytes.fill(0, absent.directoryAt, absent.directoryAt + 8);
|
||||
const path = join(root, "absent.exe");
|
||||
|
||||
@@ -30,8 +30,9 @@ const chromeExecutable = [
|
||||
);
|
||||
|
||||
// Issue 1586's page, plus the HTML, image, use, and foreign-origin cases the
|
||||
// real snapshot must distinguish. The test is skipped when no Chrome-family
|
||||
// executable is installed.
|
||||
// real snapshot must distinguish. The XHTML page is the prefixed-svg gap
|
||||
// reported on PR 1602. The tests are skipped when no Chrome-family executable
|
||||
// is installed.
|
||||
const pageHtml = `<!doctype html><title>x</title>
|
||||
<svg width="10" height="10">
|
||||
<a id="s2" xlink:href="/xl-s2h"><text>s2</text></a>
|
||||
@@ -159,6 +160,161 @@ it.skipIf(chromeExecutable === undefined)(
|
||||
},
|
||||
);
|
||||
|
||||
const xhtmlPage = `<?xml version="1.0" encoding="UTF-8"?>
|
||||
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:svg="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
|
||||
<head><title>x</title></head>
|
||||
<body>
|
||||
<svg:svg width="10" height="10">
|
||||
<svg:a id="pa1" xlink:href="/xl-pa1"><svg:text>p</svg:text></svg:a>
|
||||
<svg:image id="img" xlink:href="/xl-image"/>
|
||||
<svg:foreignObject width="10" height="10">
|
||||
<a id="fo-a" xlink:href="/xl-html-fo">no</a>
|
||||
<h:a xmlns:h="http://www.w3.org/1999/xhtml" id="fo-h" xlink:href="/xl-h">no</h:a>
|
||||
<svg:a id="fo-svg" xlink:href="/xl-fo-svg"><svg:text>y</svg:text></svg:a>
|
||||
</svg:foreignObject>
|
||||
</svg:svg>
|
||||
<svg:a id="orphan" xlink:href="/xl-orphan"><svg:text>o</svg:text></svg:a>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" xmlns:x="http://www.w3.org/1999/xlink" width="10" height="10">
|
||||
<a id="custom" x:href="/xl-custom"><text>c</text></a>
|
||||
<a id="both" href="/href-wins" x:href="/xlink-loses"><text>b</text></a>
|
||||
<a id="empty" href="" x:href="/xlink-empty"><text>e</text></a>
|
||||
</svg>
|
||||
<svg:a id="out" xlink:href="https://cdn.example.test/out"><svg:text>out</svg:text></svg:a>
|
||||
<bogus:svg xmlns:bogus="http://example.com/not-svg" id="bogus" xlink:href="/xl-bogus"/>
|
||||
<a id="xhtml-a" xlink:href="/xl-xhtml-a">no</a>
|
||||
</body>
|
||||
</html>
|
||||
`;
|
||||
|
||||
for (const { format, page, contentType, nodeNames, link, destinations } of [
|
||||
{
|
||||
format: "XHTML",
|
||||
page: xhtmlPage,
|
||||
contentType: "application/xhtml+xml",
|
||||
nodeNames: ["html", "svg:a", "bogus:svg"],
|
||||
link: { name: "svg:a", attribute: "xlink:href" },
|
||||
destinations: [
|
||||
"/xl-pa1",
|
||||
"/xl-image",
|
||||
"/xl-fo-svg",
|
||||
"/xl-orphan",
|
||||
"/xl-custom",
|
||||
"/href-wins",
|
||||
"",
|
||||
null,
|
||||
],
|
||||
},
|
||||
{
|
||||
format: "SVG",
|
||||
page: `<svg xmlns="http://www.w3.org/2000/svg" xmlns:p="http://www.w3.org/1999/xlink"><a p:href="/linked"><text>link</text></a></svg>`,
|
||||
contentType: "image/svg+xml",
|
||||
nodeNames: ["svg", "a"],
|
||||
link: { name: "a", attribute: "p:href" },
|
||||
destinations: ["/linked"],
|
||||
},
|
||||
]) {
|
||||
it.skipIf(chromeExecutable === undefined)(
|
||||
`reports namespaced SVG links from a standalone ${format} document through headless Chrome`,
|
||||
async ({ processes }) => {
|
||||
const executable = chromeExecutable;
|
||||
if (executable === undefined) return;
|
||||
const profile = await createTestTempDirectory("rea-svg-xhtml-");
|
||||
const server = createServer((_request, response) => {
|
||||
response.writeHead(200, {
|
||||
"content-type": `${contentType}; charset=utf-8`,
|
||||
});
|
||||
response.end(page);
|
||||
});
|
||||
await new Promise<void>((resolve) => {
|
||||
server.listen(0, "127.0.0.1", resolve);
|
||||
});
|
||||
const address = server.address();
|
||||
if (address === null || typeof address === "string")
|
||||
throw new Error("XHTML fixture server did not bind a TCP port");
|
||||
const origin = `http://127.0.0.1:${String(address.port)}`;
|
||||
const pageUrl = `${origin}/page.xhtml`;
|
||||
let stderr = "";
|
||||
const child = processes.spawn(executable, [
|
||||
"--headless=new",
|
||||
...(process.env.REA_BROWSER_NO_SANDBOX === "true"
|
||||
? ["--no-sandbox"]
|
||||
: []),
|
||||
"--remote-debugging-address=127.0.0.1",
|
||||
"--remote-debugging-port=0",
|
||||
`--user-data-dir=${profile}`,
|
||||
"--no-first-run",
|
||||
"--no-default-browser-check",
|
||||
"--disable-background-networking",
|
||||
"--disable-component-update",
|
||||
"--disable-dev-shm-usage",
|
||||
"--disable-sync",
|
||||
"--metrics-recording-only",
|
||||
pageUrl,
|
||||
]);
|
||||
child.stdout.on("data", () => undefined);
|
||||
child.stderr.on("data", (chunk: Buffer) => {
|
||||
if (stderr.length < 8_192) stderr += chunk.toString("utf8");
|
||||
});
|
||||
try {
|
||||
const port = await waitForBrowserDevtoolsPort({
|
||||
child,
|
||||
executable,
|
||||
activePortPath: join(profile, "DevToolsActivePort"),
|
||||
stderr: () => stderr,
|
||||
timeoutMs: 20_000,
|
||||
});
|
||||
const endpoint = `http://127.0.0.1:${String(port)}`;
|
||||
const provider = new CdpBrowserProvider();
|
||||
const targetId = await pageTarget(provider, endpoint, origin, pageUrl);
|
||||
const result = await provider.inspectPage(
|
||||
inspectWebPageInputSchema.parse({
|
||||
cdp_endpoint: endpoint,
|
||||
allowed_origins: [origin],
|
||||
target_id: targetId,
|
||||
observation_ms: 0,
|
||||
}),
|
||||
);
|
||||
if (!result.ok) throw result.error;
|
||||
const urls = result.value.metadata.dom_urls;
|
||||
for (const name of nodeNames) {
|
||||
expect(
|
||||
result.value.dom.nodes.some((node) => node.node_name === name),
|
||||
).toBe(true);
|
||||
}
|
||||
expect(
|
||||
result.value.dom.nodes.some(
|
||||
(node) =>
|
||||
node.node_name === link.name &&
|
||||
node.attribute_names.includes(link.attribute),
|
||||
),
|
||||
).toBe(true);
|
||||
expect(urls).toEqual(
|
||||
destinations.map((destination) =>
|
||||
expect.objectContaining({
|
||||
attribute: "href",
|
||||
url:
|
||||
destination === null
|
||||
? null
|
||||
: destination === ""
|
||||
? pageUrl
|
||||
: `${origin}${destination}`,
|
||||
destination_scope:
|
||||
destination === null ? "outside_policy" : "approved",
|
||||
}),
|
||||
),
|
||||
);
|
||||
} finally {
|
||||
child.kill("SIGTERM");
|
||||
if (!(await waitForExit(child, 2_000))) child.kill("SIGKILL");
|
||||
await waitForExit(child, 2_000);
|
||||
await new Promise<void>((resolve) => {
|
||||
server.close(() => resolve());
|
||||
});
|
||||
}
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
const pageTarget = async (
|
||||
provider: CdpBrowserProvider,
|
||||
endpoint: string,
|
||||
|
||||
@@ -2,8 +2,10 @@ import { PrivateRuntimeRoot } from "../../../src/process/PrivateRuntimeRoot.js";
|
||||
import {
|
||||
access,
|
||||
chmod,
|
||||
mkdir,
|
||||
mkdtemp,
|
||||
rm,
|
||||
symlink,
|
||||
truncate,
|
||||
writeFile,
|
||||
} from "node:fs/promises";
|
||||
@@ -11,6 +13,7 @@ import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { expect, it, onTestFinished } from "vitest";
|
||||
import { HistoricalCaptureDecoder } from "../../../src/browser/history/HistoricalCaptureDecoder.js";
|
||||
import { MitmproxyCaptureAdapter } from "../../../src/browser/history/MitmproxyCaptureAdapter.js";
|
||||
import { HAR_CAPTURE_PROVIDER_IDENTITY } from "../../../src/browser/history/CaptureRelease.js";
|
||||
import {
|
||||
inspectWebNetworkCaptureInputSchema,
|
||||
@@ -20,6 +23,41 @@ import { historicalHar } from "../../fixtures/historicalHar.js";
|
||||
import { decodeHarCapture } from "../../../src/browser/history/HarCapture.js";
|
||||
import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js";
|
||||
|
||||
const linux = it.skipIf(process.platform !== "linux");
|
||||
|
||||
linux("refuses a directory configured as the mitmdump executable", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "rea-mitmdump-command-"));
|
||||
onTestFinished(() => rm(root, { recursive: true, force: true }));
|
||||
const command = join(root, "mitmdump");
|
||||
await mkdir(command);
|
||||
|
||||
await expect(
|
||||
new MitmproxyCaptureAdapter({ REA_MITMDUMP_COMMAND: command }).command(
|
||||
join(root, "request.json"),
|
||||
join(root, "runtime"),
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
_tag: "AnalysisCapabilityUnavailableError",
|
||||
reason: expect.stringContaining(command),
|
||||
});
|
||||
});
|
||||
|
||||
linux("allows a symlink to a regular mitmdump executable", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "rea-mitmdump-command-symlink-"));
|
||||
onTestFinished(() => rm(root, { recursive: true, force: true }));
|
||||
const command = join(root, "mitmdump");
|
||||
const target = join(root, "mitmdump-bin");
|
||||
await writeFile(target, "#!/bin/sh\n", { mode: 0o700 });
|
||||
await symlink(target, command);
|
||||
|
||||
await expect(
|
||||
new MitmproxyCaptureAdapter({ REA_MITMDUMP_COMMAND: command }).command(
|
||||
join(root, "request.json"),
|
||||
join(root, "runtime"),
|
||||
),
|
||||
).resolves.toMatchObject({ command });
|
||||
});
|
||||
|
||||
it.each([true, false])(
|
||||
"binds decoder replies to the complete selected provider identity (matching name: %s)",
|
||||
async (matching) => {
|
||||
|
||||
@@ -9,6 +9,53 @@ import { cliTest } from "../../support/cli/cliFixture.js";
|
||||
import { analysisViewLayoutEvidence } from "../../fixtures/analysisView.js";
|
||||
import { createEvidence } from "../../../src/domain/evidence.js";
|
||||
import { ghidraFunctionDossier } from "../../../src/domain/ghidraValues.fixture.js";
|
||||
import { createEvidenceBundle } from "../../../src/domain/evidenceBundle.js";
|
||||
import { inspectEvidenceBundle } from "../../../src/application/investigation/InspectEvidenceBundle.js";
|
||||
|
||||
cliTest(
|
||||
"discovers native records from a complete portable bundle using the shared workflow",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-evidence-discovery-cli-");
|
||||
const parent = createEvidence(
|
||||
{ path: "/fixtures/native.exe", format: "pe", sha256: "a".repeat(64) },
|
||||
{ id: "ghidra", name: "Ghidra", version: "12.1.4" },
|
||||
{
|
||||
operation: "analyze_function",
|
||||
parameters: {},
|
||||
result: ghidraFunctionDossier(),
|
||||
},
|
||||
);
|
||||
const bundle = createEvidenceBundle([parent]);
|
||||
const path = join(root, "evidence.json");
|
||||
await writeFile(path, JSON.stringify(bundle));
|
||||
const options = {
|
||||
detail: "summary" as const,
|
||||
filters: { evidence_id: parent.evidence_id },
|
||||
};
|
||||
const result = await cli.run({
|
||||
arguments: [
|
||||
"inspect-evidence-bundle",
|
||||
JSON.stringify({ path, ...options }),
|
||||
"--json",
|
||||
],
|
||||
environment: { HOME: root, XDG_CONFIG_HOME: root, XDG_CACHE_HOME: root },
|
||||
});
|
||||
expect(result.exitCode).toBe(0);
|
||||
const expected = inspectEvidenceBundle(bundle, options);
|
||||
if (!expected.ok) throw expected.error;
|
||||
expect(result.json).toEqual(expected.value);
|
||||
const complete = await cli.run({
|
||||
arguments: [
|
||||
"inspect-evidence-bundle",
|
||||
JSON.stringify({ path }),
|
||||
"--json",
|
||||
],
|
||||
environment: { HOME: root, XDG_CONFIG_HOME: root, XDG_CACHE_HOME: root },
|
||||
});
|
||||
expect(complete.exitCode).toBe(0);
|
||||
expect(complete.json).toEqual(bundle);
|
||||
},
|
||||
);
|
||||
cliTest(
|
||||
"projects one section from a JSON file of inline layout Evidence",
|
||||
async ({ cli }) => {
|
||||
|
||||
@@ -97,6 +97,14 @@ describe("browser comparison JSON file inputs", () => {
|
||||
INTEGRATION_TEST_TIMEOUT_MS,
|
||||
);
|
||||
|
||||
it(
|
||||
"reports malformed PNG chunks as invalid input and valid unsupported PNGs as unsupported targets",
|
||||
async () => {
|
||||
await assertPngInputErrorProjections();
|
||||
},
|
||||
INTEGRATION_TEST_TIMEOUT_MS,
|
||||
);
|
||||
|
||||
it(
|
||||
"identifies a missing capture file instead of reporting malformed JSON",
|
||||
async () => {
|
||||
@@ -123,3 +131,105 @@ describe("browser comparison JSON file inputs", () => {
|
||||
INTEGRATION_TEST_TIMEOUT_MS,
|
||||
);
|
||||
});
|
||||
|
||||
const assertPngInputErrorProjections = async (): Promise<void> => {
|
||||
const damagedBytes = noisePng(1, 1);
|
||||
damagedBytes[29] = (damagedBytes[29] ?? 0) ^ 1;
|
||||
const damaged = JSON.stringify(createWebScreenshotArtifact(damagedBytes));
|
||||
expect(
|
||||
await runCli(["compare-web-screenshots", damaged, damaged, "--json"]),
|
||||
).toMatchObject({
|
||||
code: "invalid_request",
|
||||
details: {
|
||||
operation: "compare_web_screenshots",
|
||||
issues: [
|
||||
{ path: [], reason: "invalid_format", message: expect.any(String) },
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const header = Buffer.alloc(13);
|
||||
header.writeUInt32BE(1, 0);
|
||||
header.writeUInt32BE(1, 4);
|
||||
header.set([8, 2, 0, 0, 0], 8);
|
||||
const malformedDeflate = JSON.stringify(
|
||||
createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
|
||||
pngChunk("IHDR", header),
|
||||
pngChunk("IDAT", Buffer.from("bad")),
|
||||
pngChunk("IEND", Buffer.alloc(0)),
|
||||
]),
|
||||
),
|
||||
);
|
||||
expect(
|
||||
await runCli([
|
||||
"compare-web-screenshots",
|
||||
malformedDeflate,
|
||||
malformedDeflate,
|
||||
"--json",
|
||||
]),
|
||||
).toMatchObject({
|
||||
code: "invalid_request",
|
||||
details: {
|
||||
operation: "compare_web_screenshots",
|
||||
issues: [{ path: [], reason: "invalid_format" }],
|
||||
},
|
||||
});
|
||||
|
||||
const grayscaleHeader = Buffer.alloc(13);
|
||||
grayscaleHeader.writeUInt32BE(1, 0);
|
||||
grayscaleHeader.writeUInt32BE(1, 4);
|
||||
grayscaleHeader.set([8, 0, 0, 0, 0], 8);
|
||||
const grayscale = JSON.stringify(
|
||||
createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
|
||||
pngChunk("IHDR", grayscaleHeader),
|
||||
pngChunk("IDAT", deflateSync(Buffer.from([0, 30]))),
|
||||
pngChunk("IEND", Buffer.alloc(0)),
|
||||
]),
|
||||
),
|
||||
);
|
||||
expect(
|
||||
await runCli(["compare-web-screenshots", grayscale, grayscale, "--json"]),
|
||||
).toMatchObject({
|
||||
code: "unsupported_target",
|
||||
details: { operation: "compare_web_screenshots" },
|
||||
});
|
||||
|
||||
const oversizedHeader = Buffer.alloc(13);
|
||||
oversizedHeader.writeUInt32BE(0x7fffffff, 0);
|
||||
oversizedHeader.writeUInt32BE(1, 4);
|
||||
oversizedHeader.set([8, 6, 0, 0, 0], 8);
|
||||
const hugeDimensions = JSON.stringify(
|
||||
createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
|
||||
pngChunk("IHDR", oversizedHeader),
|
||||
pngChunk("IDAT", deflateSync(Buffer.from([0]))),
|
||||
pngChunk("IEND", Buffer.alloc(0)),
|
||||
]),
|
||||
),
|
||||
);
|
||||
expect(
|
||||
await runCli([
|
||||
"compare-web-screenshots",
|
||||
hugeDimensions,
|
||||
hugeDimensions,
|
||||
"--json",
|
||||
]),
|
||||
).toMatchObject({
|
||||
code: "resource_constraint",
|
||||
details: {
|
||||
operation: "compare_web_screenshots",
|
||||
resource: "memory",
|
||||
reported_limits: {
|
||||
maximum_working_memory_bytes: 256 * 1024 * 1024,
|
||||
estimated_working_memory_bytes: "34359738499",
|
||||
before_dimensions: { width: 0x7fffffff, height: 1 },
|
||||
after_dimensions: { width: 0x7fffffff, height: 1 },
|
||||
},
|
||||
},
|
||||
});
|
||||
};
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { mkdir } from "node:fs/promises";
|
||||
import { mkdir, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
@@ -9,20 +9,34 @@ import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
|
||||
const execute = promisify(execFile);
|
||||
const MARKER = "REA_TEST_RESOLVED ";
|
||||
// Dependencies that only one command uses; startup must not resolve them.
|
||||
const DEFERRED_PACKAGES = ["playwright-core", "isomorphic-git"] as const;
|
||||
// Dependencies and composition modules used by one command family; startup
|
||||
// must not resolve them just to register the complete command catalog.
|
||||
const DEFERRED_IMPORTS = [
|
||||
{
|
||||
name: "playwright-core",
|
||||
path: "/node_modules/playwright-core/",
|
||||
},
|
||||
{
|
||||
name: "isomorphic-git",
|
||||
path: "/node_modules/isomorphic-git/",
|
||||
},
|
||||
{
|
||||
name: "binary-composition",
|
||||
path: "/dist/composition/binary.js",
|
||||
},
|
||||
] as const;
|
||||
// Report each resolved deferred package on stderr without changing resolution.
|
||||
const RESOLUTION_HOOK = `data:text/javascript,${encodeURIComponent(`
|
||||
import { registerHooks } from "node:module";
|
||||
const packages = ${JSON.stringify(DEFERRED_PACKAGES)};
|
||||
const imports = ${JSON.stringify(DEFERRED_IMPORTS)};
|
||||
registerHooks({
|
||||
resolve(specifier, context, next) {
|
||||
const resolved = next(specifier, context);
|
||||
const name = packages.find((item) =>
|
||||
resolved.url.includes("/node_modules/" + item + "/"),
|
||||
const name = imports.find((item) =>
|
||||
resolved.url.includes(item.path),
|
||||
);
|
||||
if (name !== undefined)
|
||||
process.stderr.write(${JSON.stringify(MARKER)} + name + "\\n");
|
||||
process.stderr.write(${JSON.stringify(MARKER)} + name.name + "\\n");
|
||||
return resolved;
|
||||
},
|
||||
});
|
||||
@@ -68,9 +82,45 @@ describe("CLI startup imports", () => {
|
||||
).resolves.toContain("isomorphic-git");
|
||||
});
|
||||
|
||||
it("does not load command-specific packages to register commands", async () => {
|
||||
it("does not load command-specific code to register the command catalog", async () => {
|
||||
await expect(
|
||||
resolvedPackages(["scripts/rea.mjs", "--help"]),
|
||||
).resolves.toEqual(new Set());
|
||||
});
|
||||
|
||||
it("does not load native provider composition for JavaScript application analysis", async () => {
|
||||
const source = await createTestTempDirectory("rea-cli-js-startup-");
|
||||
await mkdir(join(source, "src"));
|
||||
await writeFile(
|
||||
join(source, "package.json"),
|
||||
JSON.stringify({
|
||||
name: "startup-fixture",
|
||||
version: "1.0.0",
|
||||
main: "src/index.js",
|
||||
}),
|
||||
);
|
||||
await writeFile(
|
||||
join(source, "src/index.js"),
|
||||
"export const answer = 42;\n",
|
||||
);
|
||||
|
||||
await expect(
|
||||
resolvedPackages([
|
||||
"scripts/rea.mjs",
|
||||
"analyze-javascript-application",
|
||||
source,
|
||||
"--json",
|
||||
]),
|
||||
).resolves.toEqual(new Set());
|
||||
});
|
||||
|
||||
it("loads native provider composition when a provider operation is called", async () => {
|
||||
await expect(
|
||||
resolvedPackages(
|
||||
evaluate(
|
||||
'const { createDirectAnalysis } = await import("./dist/composition/directAnalysis.js"); await createDirectAnalysis({}).runSessionStatus();',
|
||||
),
|
||||
),
|
||||
).resolves.toContain("binary-composition");
|
||||
});
|
||||
});
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user