Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9ec8267bee | ||
|
|
d3a39a9376 | ||
|
|
5ddf8766b8 |
@@ -25,7 +25,6 @@ node scripts/release-notes.mjs inventory \
|
||||
--manifest /tmp/rea-VERSION.contributions.json
|
||||
node scripts/release-notes.mjs record \
|
||||
--manifest /tmp/rea-VERSION.contributions.json \
|
||||
--summary \
|
||||
--output /tmp/rea-VERSION.record.md
|
||||
```
|
||||
|
||||
@@ -57,31 +56,25 @@ version/date/link heading. Use single-line Markdown list items under:
|
||||
PR/commit references required by the existing checkpoint validator.
|
||||
|
||||
Group related work by analyst workflow or user impact. Explain the resulting
|
||||
behavior in short, concrete entries. Highlights summarize the release; avoid
|
||||
repeating their full descriptions in Changes. Internal cleanup, tests, CI, and
|
||||
routine documentation generally belong only in the provenance file. Read ambiguous PRs/diffs before making
|
||||
behavior; internal cleanup, tests, CI, and routine documentation generally
|
||||
belong only in the contribution record. Read ambiguous PRs/diffs before making
|
||||
claims. Distinguish simulated-provider checks from real-provider verification.
|
||||
|
||||
Credit each represented contribution once in the public notes with its link
|
||||
and `Thanks @...` for its verified human author, known co-authors, and confirmed
|
||||
closing-issue reporters. Grouping preserves all those credits in one line.
|
||||
When Highlights, detailed entries or migrations reference the same work,
|
||||
put its thanks in one of those entries, not all of them. Include issue links.
|
||||
Direct-commit bullets carry the commit link and known human credit.
|
||||
Each bullet covering a PR carries its link and `Thanks @...` for its verified
|
||||
human author, known co-authors, and confirmed closing-issue reporters. Grouping
|
||||
must preserve **all** those credits in the same bullet. Include the issue
|
||||
links. Direct-commit bullets carry the commit link and known human credit.
|
||||
Exclude GitHub bots and known agent accounts. REA's observed Anthropic and
|
||||
Cursor co-author trailers resolve to `@claude` and `@cursoragent` with GitHub
|
||||
type `User`; the credit policy excludes them while retaining their metadata.
|
||||
At the REA maintainer's request, omit `@morluto` self-thanks while preserving
|
||||
authorship in the inventory. Other human contributors remain eligible.
|
||||
Do not copy OpenClaw's maintainer-specific exclusions.
|
||||
Use explicit GitHub links for cross-repository references to avoid ambiguous
|
||||
`#NNN` numbers. Unknown handles stay unknown; they are not fabricated credit.
|
||||
|
||||
Append the generated `record --summary` provenance link verbatim at the end
|
||||
of this release. The linked inventory accounts for every discovered in-range
|
||||
PR and direct contribution, including internal work omitted from the prose.
|
||||
Do not paste the full `record` output into public notes: it duplicates thanks
|
||||
and obscures the user-facing changes. Full records remain available for review.
|
||||
Do not hand-edit generated credits to make validation pass. Refresh the inventory
|
||||
Append the generated `### Complete contribution record` verbatim as the last
|
||||
section of this release. It accounts for every discovered in-range PR and
|
||||
direct contribution, including internal work omitted from the prose. Do not
|
||||
hand-edit generated credits to make validation pass. Refresh the inventory
|
||||
when product history changes; a prose-only correction can reuse the snapshot.
|
||||
|
||||
## Finalize with Release Please
|
||||
|
||||
@@ -135,16 +135,6 @@ session to a selected path. The analysis remains complete in the ledger; avoid
|
||||
repeating it merely to request the same oversized response. Complete CLI JSON
|
||||
output also streams without a single MCP frame.
|
||||
|
||||
When the retained parent ID is unknown, use `get_evidence_bundle` with
|
||||
`detail: "summary"` and exact filters for operation, target SHA-256,
|
||||
analysis-profile digest or canonical procedure address. Select its returned
|
||||
Evidence ID for the next view instead of requesting the complete bundle.
|
||||
`retention: "complete-record"` means the whole record is stored; native
|
||||
value-flow truncation and unknown metadata remain separate. For an exported
|
||||
portable bundle, use `rea inspect-evidence-bundle` with JSON containing `path`,
|
||||
`detail: "summary"` and the same `filters`. Reading that file still requires
|
||||
validating its full contents within the local memory budget.
|
||||
|
||||
A client can also truncate a successful tool result without a REA transport
|
||||
error. Treat that preview as incomplete; use the retained parent ID for smaller
|
||||
views or export when the full record is needed. Do not parse the preview as
|
||||
|
||||
@@ -1,5 +1,13 @@
|
||||
# JavaScript and Electron application artifacts
|
||||
|
||||
For captured or bundled JavaScript that needs readable source recovery, use
|
||||
`recover_javascript_sources` with an absolute `path` and absent
|
||||
`output_directory`. Supply related runtime/shared/lazy files in
|
||||
`additional_paths` when advertised, then analyze the returned `analysis_input`
|
||||
directory together. Each input retains its own snapshot, digest and byte-range
|
||||
provenance; inputs and recovered code are never executed. Recovery requires a
|
||||
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/javascript-recovery.md).
|
||||
|
||||
Use `analyze_javascript_application` directly on the operator-supplied ASAR or
|
||||
extracted tree. When size is unknown or large and the connected schema advertises
|
||||
`detail`, request `detail: "summary"`. This returns an `inspect_analysis_view`
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
{
|
||||
".": "6.4.0"
|
||||
".": "6.3.0"
|
||||
}
|
||||
|
||||
@@ -1,69 +1,5 @@
|
||||
# Changelog
|
||||
|
||||
## [6.4.0](https://github.com/morluto/rea/compare/rea-agents-6.3.0...rea-agents-6.4.0) (2026-10-11)
|
||||
|
||||
### Highlights
|
||||
|
||||
- Android: added device inspection, split-APK acquisition, logs, screenshots, file transfers and explicit app lifecycle actions through ADB, plus manifest and localized-string decoding with Apktool. ([#1552](https://github.com/morluto/rea/pull/1552), [#1556](https://github.com/morluto/rea/pull/1556)) Thanks @3QL.
|
||||
- JEB: browse project units and decompile selected items through a caller-started local JEB MCP server. ([#1536](https://github.com/morluto/rea/pull/1536)) Thanks @3QL.
|
||||
- Large analyses: discover retained Evidence without loading every payload, and inspect JavaScript graphs, binary layouts and function dossiers through summaries and selected views. ([#1682](https://github.com/morluto/rea/pull/1682), [#1568](https://github.com/morluto/rea/pull/1568), [#1570](https://github.com/morluto/rea/pull/1570), [#1345](https://github.com/morluto/rea/pull/1345), [#1507](https://github.com/morluto/rea/pull/1507), [#1247](https://github.com/morluto/rea/issues/1247)) Thanks @jhuang-tw, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- JavaScript: isolate source analysis in an owned process with configurable heap and timeout limits; retain completed facts and explicit unknowns when analysis runs out of resources. ([#1651](https://github.com/morluto/rea/pull/1651), [#1563](https://github.com/morluto/rea/issues/1563), [#1565](https://github.com/morluto/rea/issues/1565)) Thanks @N0zoM1z0, @peter-research.
|
||||
- Ghidra: configure import language/compiler specifications and seed functions, code and labels before auto-analysis. ([#1667](https://github.com/morluto/rea/pull/1667)) Thanks @sqmagellan.
|
||||
- RPC discovery: recognize supported tRPC calls, HTTP-link URLs and named GraphQL operations in JavaScript applications and web bundles. ([#1670](https://github.com/morluto/rea/pull/1670)) Thanks @pompiliodonatoapp.
|
||||
|
||||
### Changes
|
||||
|
||||
- Added Ghidra support for Windows PE DLLs and bounded MIPS ELF analysis, including carved MIPS inputs with incomplete section tables. Windows P0 remains read-only. ([#1303](https://github.com/morluto/rea/pull/1303), [#1362](https://github.com/morluto/rea/pull/1362), [#1629](https://github.com/morluto/rea/pull/1629), [#1290](https://github.com/morluto/rea/issues/1290), [#718](https://github.com/morluto/rea/issues/718), [#1619](https://github.com/morluto/rea/issues/1619)) Thanks @bqw371, @jaelliot, @kvalls, @N0zoM1z0.
|
||||
- Enabled `set_address_name` and `set_addresses_names` for Ghidra sessions with mutation authority, including namespace-aware data labels and per-address batch results. ([#1635](https://github.com/morluto/rea/pull/1635), [#1634](https://github.com/morluto/rea/issues/1634)) Thanks @giri256.
|
||||
- Recovered additional signed AArch64 relative jump-table patterns. ([#1592](https://github.com/morluto/rea/pull/1592)) Thanks @RishTheFish21.
|
||||
- Added PE resource inspection and read-only NativeAOT metadata parsing for supported x64 PE32+/.NET 8 RTR 9.1 inputs. The NativeAOT parser’s Windows-host Ghidra workflow remains unverified on a real Windows host. ([#1482](https://github.com/morluto/rea/pull/1482), [76d8f68](https://github.com/morluto/rea/commit/76d8f68cb918dfb43869f2b649a0b6c0c1cf266d), [#1254](https://github.com/morluto/rea/issues/1254)) Thanks @N0zoM1z0, @thangldw.
|
||||
- Added Pi, Hermes and Qwen Code setup, including Hermes active-profile support. ([#1376](https://github.com/morluto/rea/pull/1376), [#1403](https://github.com/morluto/rea/pull/1403), [#1415](https://github.com/morluto/rea/pull/1415), [#1409](https://github.com/morluto/rea/issues/1409)) Thanks @rasatpetabit, @yiliang114, @zaydiscold.
|
||||
- Added `REA_MCP_INPUT_SCHEMA_PROFILE=compact` for size-capped MCP clients such as Kimi/Moonshot, while retaining canonical runtime validation. ([#1514](https://github.com/morluto/rea/pull/1514), [#1580](https://github.com/morluto/rea/pull/1580), [#1484](https://github.com/morluto/rea/issues/1484), [#1581](https://github.com/morluto/rea/issues/1581)) Thanks @haythemsellami, @N0zoM1z0.
|
||||
- Added live process-capture progress and optional `finalization_ms` to capture shutdown output before timeout escalation; cancellation remains immediate. ([#1483](https://github.com/morluto/rea/pull/1483), [#1583](https://github.com/morluto/rea/pull/1583), [#1252](https://github.com/morluto/rea/issues/1252), [#1253](https://github.com/morluto/rea/issues/1253), [modelcontextprotocol/typescript-sdk#2997](https://github.com/modelcontextprotocol/typescript-sdk/issues/2997)) Thanks @N0zoM1z0, @nikolay-dementiev, @sathariels.
|
||||
- Improved Codex onboarding and recovery from observed client, PATH and provider blockers. ([#1613](https://github.com/morluto/rea/pull/1613)) Thanks @N0zoM1z0.
|
||||
- Reduced redirected JSON and terminal output size, and deferred heavy imports until their commands need them. ([#1413](https://github.com/morluto/rea/pull/1413), [#1450](https://github.com/morluto/rea/pull/1450), [#1439](https://github.com/morluto/rea/pull/1439), [21dbcf7](https://github.com/morluto/rea/commit/21dbcf7a1458219989ad291e44ce7d017378b727)) Thanks @kaoru0822-kitauji.
|
||||
|
||||
### Fixes
|
||||
|
||||
- Electron analysis: follow renamed and default-imported APIs and optional calls, ignore overwritten identities and media preload attributes, and resolve packaged `app.asar` pages and unpacked companions. ([#1391](https://github.com/morluto/rea/pull/1391), [#1393](https://github.com/morluto/rea/pull/1393), [#1400](https://github.com/morluto/rea/pull/1400), [#1664](https://github.com/morluto/rea/pull/1664), [#1673](https://github.com/morluto/rea/pull/1673)) Thanks @kaoru0822-kitauji, @marnixpluim, @N0zoM1z0, @suninrain086.
|
||||
- JavaScript data flow: preserve returned/yielded reference uncertainty, invalidate property-stored aliases correctly, and retain bundled-module runtime scope. ([#1437](https://github.com/morluto/rea/pull/1437), [#1445](https://github.com/morluto/rea/pull/1445), [#1471](https://github.com/morluto/rea/pull/1471), [#1577](https://github.com/morluto/rea/pull/1577), [#1428](https://github.com/morluto/rea/issues/1428), [#1442](https://github.com/morluto/rea/issues/1442), [#1449](https://github.com/morluto/rea/issues/1449), [#1576](https://github.com/morluto/rea/issues/1576)) Thanks @CassieFox23, @dakjdakd, @N0zoM1z0.
|
||||
- JavaScript packages: resolve exact exports subpaths, preserve trailing-slash directory requests and exact CommonJS keys, and respect shadowed `require` bindings. ([#1424](https://github.com/morluto/rea/pull/1424), [#1615](https://github.com/morluto/rea/pull/1615), [5485483](https://github.com/morluto/rea/commit/5485483f7da2f6cb66441bb784aaf2a1a5f8731a), [8b2f851](https://github.com/morluto/rea/commit/8b2f8514c50e469c3add8ce9ba7ebebef888b14f), [f998fcb](https://github.com/morluto/rea/commit/f998fcb1cca9994c3589fb70d6e0127fc215f76f), [1e51c73](https://github.com/morluto/rea/commit/1e51c732bbf08191688d6526bf58ab2650b601e3), [#1365](https://github.com/morluto/rea/issues/1365)) Thanks @akshitkhokhani, @AWUQAQ, @immanuwell, @rudycelekli.
|
||||
- Large JavaScript inputs: handle oversized inline source maps, deep returns and reassignment loops; avoid argument expansion limits and yield during large analysis passes. ([#1386](https://github.com/morluto/rea/pull/1386), [#1368](https://github.com/morluto/rea/pull/1368), [#1390](https://github.com/morluto/rea/pull/1390), [#1539](https://github.com/morluto/rea/pull/1539), [#1525](https://github.com/morluto/rea/pull/1525), [#1542](https://github.com/morluto/rea/pull/1542), [#1545](https://github.com/morluto/rea/pull/1545), [#1557](https://github.com/morluto/rea/pull/1557), [#1380](https://github.com/morluto/rea/issues/1380), [#1401](https://github.com/morluto/rea/issues/1401), [#1538](https://github.com/morluto/rea/issues/1538), [#1462](https://github.com/morluto/rea/issues/1462)) Thanks @cold-atom, @KadirJeager, @kaoru0822-kitauji, @marnixpluim, @N0zoM1z0, @rudycelekli.
|
||||
- JavaScript receivers: expand escaped methods once, index returned-object targets, and avoid replaying earlier writes or duplicating retained graph allocations. ([#1517](https://github.com/morluto/rea/pull/1517), [#1537](https://github.com/morluto/rea/pull/1537), [#1444](https://github.com/morluto/rea/pull/1444), [#1601](https://github.com/morluto/rea/pull/1601), [#1495](https://github.com/morluto/rea/issues/1495)) Thanks @CassieFox23, @geekwolf, @kaoru0822-kitauji, @N0zoM1z0, @zeynelacikgoez.
|
||||
- Browser captures: retain WebSocket URLs, empty connections and frame order; compare only common known payload coverage and preserve incomplete WebMCP replay coverage. ([db4e360](https://github.com/morluto/rea/commit/db4e3608465486466591b594cf77037ac150c988), [aa257ea](https://github.com/morluto/rea/commit/aa257ea4b16f2d2a35c12df63359d650b48b1154), [e52c0a7](https://github.com/morluto/rea/commit/e52c0a72c59c740315e89149881a27ce4d89a11c))
|
||||
- Browser scenarios: seed storage in the correct partition, normalize loopback attach identities, retain declared-secret redaction and require verified lineage before Windows Electron signals. ([#1604](https://github.com/morluto/rea/pull/1604), [#1590](https://github.com/morluto/rea/pull/1590), [#1588](https://github.com/morluto/rea/pull/1588), [8b4c3de](https://github.com/morluto/rea/commit/8b4c3de3b3a3b580adec266f94f29ccedf1da228), [#1497](https://github.com/morluto/rea/issues/1497), [#1591](https://github.com/morluto/rea/issues/1591), [#1589](https://github.com/morluto/rea/issues/1589)) Thanks @Adolanium, @akushonkamen, @dakjdakd.
|
||||
- Added Termux Android browser-scenario support. ([#1655](https://github.com/morluto/rea/pull/1655)) Thanks @tyloryang.
|
||||
- Web inputs: accept BOM-prefixed HAR, preserve the first duplicate DOM attribute and resolve SVG/XLink URLs in HTML and XHTML. ([#1509](https://github.com/morluto/rea/pull/1509), [#1516](https://github.com/morluto/rea/pull/1516), [#1602](https://github.com/morluto/rea/pull/1602), [#1683](https://github.com/morluto/rea/pull/1683), [#1496](https://github.com/morluto/rea/issues/1496), [#1586](https://github.com/morluto/rea/issues/1586)) Thanks @CassieFox23, @kaoru0822-kitauji, @sathariels, @taljeri.
|
||||
- Screenshot comparisons: validate PNG chunks, distinguish corruption from unsupported encodings and bound comparison memory. ([4e4b15b](https://github.com/morluto/rea/commit/4e4b15b8250b29e176d4191c9610a3931b693474), [b3e977e](https://github.com/morluto/rea/commit/b3e977e6d073ab1783cece8581279f87fc360411)) Thanks @rudycelekli.
|
||||
- Archives: preserve case-distinct names, report collisions at their diverging segment, detect nested formats from bytes and reject Windows alternate-stream extraction paths. ([#1480](https://github.com/morluto/rea/pull/1480), [#1505](https://github.com/morluto/rea/pull/1505), [#1553](https://github.com/morluto/rea/pull/1553), [573598c](https://github.com/morluto/rea/commit/573598c6324698cc48ceafd24ed3864670e08d00), [#1470](https://github.com/morluto/rea/issues/1470), [#1491](https://github.com/morluto/rea/issues/1491)) Thanks @CassieFox23, @kaoru0822-kitauji.
|
||||
- Cleanup: retain failed readers, ASAR handles, output trees and provider workers for retry without discarding completed output or partial observations. ([ffe0574](https://github.com/morluto/rea/commit/ffe0574e17b9d973860371083cb1a351be912131), [de0d341](https://github.com/morluto/rea/commit/de0d341524dc68250a45b41eb7e49fcecc1e32a9), [3f8c51e](https://github.com/morluto/rea/commit/3f8c51e9b506d88fc41aded7b1a22b9a09037c26), [d1235fa](https://github.com/morluto/rea/commit/d1235fadc901e1fb9ab3149d53063306c8545414), [026126e](https://github.com/morluto/rea/commit/026126eafe741e5a8b857433587fd615cdd44c19))
|
||||
- Providers: canonicalize IDA workspace roots before identity checks, respect IDA cancellation, serialize JEB acquisition and correct Ghidra no-return/imported-symbol recovery. ([#1481](https://github.com/morluto/rea/pull/1481), [#1419](https://github.com/morluto/rea/pull/1419), [5f30ea6](https://github.com/morluto/rea/commit/5f30ea6ff2dd89aab7af6da6e8dfa5e508ed459b), [f88c990](https://github.com/morluto/rea/commit/f88c99021b88ce67e513025901f57478d8107d0c), [49e6f00](https://github.com/morluto/rea/commit/49e6f00ed08b225abf9ddb7c762553607fdab115)) Thanks @AS9929, @JonahSimon, @Y0UW1SH.
|
||||
- Binary metadata: respect PE/.NET section extents, bound binary-plist expansion, scope keyed-archive traversal and decode app `Info.plist` files without `plutil`. ([#1678](https://github.com/morluto/rea/pull/1678), [#1515](https://github.com/morluto/rea/pull/1515), [#1378](https://github.com/morluto/rea/pull/1378), [#1476](https://github.com/morluto/rea/pull/1476), [975dff2](https://github.com/morluto/rea/commit/975dff2e25b14b634e14dd5e0db1537d0ca9815b), [76296c5](https://github.com/morluto/rea/commit/76296c5a3868ba3b54be6ed3896719283d6de5f8)) Thanks @jayhemnani9910, @kaoru0822-kitauji, @rudycelekli.
|
||||
- Windows runtimes: preserve drive-root paths when creating private provider directories. ([ee46323](https://github.com/morluto/rea/commit/ee46323b493994902b04824a7dc8aae2a8117903))
|
||||
- Compatibility: load Windows native controls under Node-API hosts with renamed executables, preserve dylib directory requirements, support Wakaru 1.x and reject directory-valued Hopper launchers. ([#1531](https://github.com/morluto/rea/pull/1531), [#1623](https://github.com/morluto/rea/pull/1623), [#1540](https://github.com/morluto/rea/pull/1540), [#1426](https://github.com/morluto/rea/pull/1426)) Thanks @evenafterallisaid, @Fazmin, @pionxzh, @xeropresence.
|
||||
- Reference imports: normalize internal symlink lookup consistently, preserve exact paths and exclusion provenance, report root access failures and reject FIFO replacements without blocking. ([#1535](https://github.com/morluto/rea/pull/1535), [2ca3ad2](https://github.com/morluto/rea/commit/2ca3ad238608adfca80a7ff1aaadea6e59a58225), [6636045](https://github.com/morluto/rea/commit/6636045c9c5b472bba51afee9d434394861e93a4), [3967cfa](https://github.com/morluto/rea/commit/3967cfa43e786c65a57e89c265343983d19ef9d0), [f1c58cc](https://github.com/morluto/rea/commit/f1c58cc53fbd83b968e11b3e2eccebca51299c54)) Thanks @anujsingh-cse.
|
||||
- Setup: honor active configuration and skill roots, preserve configured MCP environments, validate Hermes/OMP profiles before writing, recognize Omarchy and retain recovery backups. ([#1504](https://github.com/morluto/rea/pull/1504), [#1607](https://github.com/morluto/rea/pull/1607), [#1528](https://github.com/morluto/rea/pull/1528), [#1562](https://github.com/morluto/rea/pull/1562), [#1662](https://github.com/morluto/rea/pull/1662), [a011485](https://github.com/morluto/rea/commit/a01148553e2a37c72ae78405418ab648876a96a6), [#1606](https://github.com/morluto/rea/issues/1606), [#1521](https://github.com/morluto/rea/issues/1521), [#1561](https://github.com/morluto/rea/issues/1561)) Thanks @Adkid-Zephyr, @CassieFox23, @LuisTaffelli, @N0zoM1z0, @peter-research.
|
||||
- Process capture: retain descendant lineage after launcher exit, preserve identity uncertainty before signaling and release terminals after observation failures. ([80896a1](https://github.com/morluto/rea/commit/80896a1d64b216ef893907249cd4252f86215d49), [c916850](https://github.com/morluto/rea/commit/c9168508527302ac57bc86385ccf0d824697b5dc)) Thanks @Y0UW1SH.
|
||||
- CLI/MCP errors: accept JSON files for web comparisons, preserve native option help and union correction details, remove invalid aliases and validate doctor against its discovered output contract. ([#1397](https://github.com/morluto/rea/pull/1397), [#1395](https://github.com/morluto/rea/pull/1395), [#1425](https://github.com/morluto/rea/pull/1425), [#1461](https://github.com/morluto/rea/pull/1461), [8b4c96e](https://github.com/morluto/rea/commit/8b4c96e6c6b3ed6806e4cd72b556b5db6114304a), [#1421](https://github.com/morluto/rea/issues/1421), [#1454](https://github.com/morluto/rea/issues/1454)) Thanks @Fazmin, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
- WebSocket results: use `network.websocket_connections` instead of `network.websocket_events`. Connections contain `request_id`, `url` and `events`; allow `payload_bytes: null` when size is unknown. No legacy alias remains. ([db4e360](https://github.com/morluto/rea/commit/db4e3608465486466591b594cf77037ac150c988))
|
||||
- Direct resource APIs: pass a caller-owned `ArtifactResourceScope` to inventory/extraction, JavaScript analysis, reference imports and web exports; retain it through cleanup/shutdown. For `readJavaScriptArtifactFiles`, scope is argument three and abort signal is argument four. Keep each root inventory source’s single cleanup owner. Use `OwnedFileHandle` owners, injected ASAR `fd`/`read` handles instead of `createReadStream`, VCS `Result` values, and `WebScriptExportError`’s options object. CLI/MCP inputs for the scoped operations are unchanged. ([ffe0574](https://github.com/morluto/rea/commit/ffe0574e17b9d973860371083cb1a351be912131), [53e7fad](https://github.com/morluto/rea/commit/53e7fadbd078422ba3d55b5499a70d237c5f4908), [684199b](https://github.com/morluto/rea/commit/684199ba0be7822266d57ddb8a40e95f087fdc42), [be75ed5](https://github.com/morluto/rea/commit/be75ed5e4b4bfb146af497f8fa5d172a1ea19462), [be6272f](https://github.com/morluto/rea/commit/be6272f26115bd8e94f2ab0df4cf092004587a18), [e0e49e9](https://github.com/morluto/rea/commit/e0e49e9dd53d9d1f1d2330b23f7f880dc1c81774), [95a77b6](https://github.com/morluto/rea/commit/95a77b6e25420b3879bd0e32bdbb14d452b38776))
|
||||
- Saved reference graphs: exclusions require their matching `pattern` and distinguish `project-ignored` from `default-ignored`. Re-import graphs missing that provenance. ([6585293](https://github.com/morluto/rea/commit/6585293c562853b95c3a70df02c12ad8cc17afc4))
|
||||
- Terminal captures: keep initial and resized buffers within 1,000,000 cells using `columns × (rows + scrollback)`; oversized scenarios are rejected before launch. ([f9ea9a9](https://github.com/morluto/rea/commit/f9ea9a9268c854d31048da9f8a50cfa37703fe5a))
|
||||
- Optional agent-verification reports: accept schema version 4, nullable `expectedFirstTool` and `targetClarificationPassed`. Production CLI/MCP contracts are unchanged. ([#1613](https://github.com/morluto/rea/pull/1613))
|
||||
- Capture comparisons: recapture both sides with matching `literal-replacements-v1` semantics when custom normalization patterns are used. Legacy unversioned pattern captures remain readable but cannot be compared with new pattern captures; captures without patterns are unaffected. ([#1456](https://github.com/morluto/rea/pull/1456), [2b6f14f](https://github.com/morluto/rea/commit/2b6f14f528297cf12c776c960af0859f5cfbefbf)) Thanks @Madhavan006, @N0zoM1z0.
|
||||
- Artifact graph edges: read child artifact IDs and paths from the occurrence selected by `occurrence_id`, not edge fields `child_artifact_id`/`logical_path`. Earlier inventory Evidence no longer validates against the new commitment. ([#1493](https://github.com/morluto/rea/pull/1493)) Thanks @kaoru0822-kitauji.
|
||||
- Semantic graph schema: nodes and relations no longer include `identifier_strategy`, and relation IDs no longer hash it. Earlier Evidence no longer validates against this schema. ([#1474](https://github.com/morluto/rea/pull/1474)) Thanks @kaoru0822-kitauji.
|
||||
- Semantic identities: select IDs from the parent graph; read exact event names from `event_name`, endpoints from `endpoint` and literal values from `properties.value`. New analyses use digest-based IDs for long/reserved event and endpoint names and `value-sha256` literal role keys, with shorter labels; string literal labels now read `string literal`. Previously stored graphs retain their original IDs and remain valid workflow inputs. ([#1427](https://github.com/morluto/rea/pull/1427), [#1423](https://github.com/morluto/rea/pull/1423), [#1407](https://github.com/morluto/rea/pull/1407)) Thanks @N0zoM1z0.
|
||||
|
||||
### Contributors
|
||||
|
||||
- Thanks also to @BaaRai, @commit-regrets, @coygeek, @Franky100-pig, @inii-sayaa, @jopemachine, @JosephDemarest, @keemsisi, @Totopo27 for contributions included in this release.
|
||||
|
||||
Full contribution record: [336 commits and 163 merged PRs](https://github.com/morluto/rea/blob/rea-agents-6.4.0/docs/releases/6.4.0.contributions.json).
|
||||
|
||||
## [6.3.0](https://github.com/morluto/rea/compare/rea-agents-6.2.0...rea-agents-6.3.0) (2026-10-09)
|
||||
|
||||
|
||||
|
||||
@@ -9,7 +9,6 @@
|
||||
**See a feature you like. Understand how it works, down to the binary level.**
|
||||
|
||||
[](https://www.npmjs.com/package/rea-agents)
|
||||
[](https://github.com/morluto/rea/actions/workflows/ci.yml)
|
||||
[](docs/mcp-contracts.md#generated-catalog)
|
||||
[](#what-you-can-analyze)
|
||||
[](https://skills.sh/morluto/rea/reverse-engineer-anything)
|
||||
|
||||
@@ -24,29 +24,6 @@ REA can attach to a user-owned Chrome-family browser through the Chrome DevTools
|
||||
|
||||
Target discovery returns the complete in-scope target array in one result.
|
||||
|
||||
## Screenshot comparison memory budget
|
||||
|
||||
`compare_web_screenshots` admits both PNGs against a 256 MiB estimated
|
||||
working-memory budget before inflating either image. This is a processing
|
||||
buffer budget for this operation, not a PNG dimension limit, CDP transport
|
||||
limit, or process RSS guarantee. Header-valid dimensions that exceed the
|
||||
budget return `resource_constraint` with `details.resource: "memory"` and the
|
||||
estimated and maximum working-set bytes. Invalid PNG structure remains
|
||||
malformed input; encodings outside supported pixel-comparison coverage remain
|
||||
unsupported-target errors.
|
||||
|
||||
The conservative estimate includes the two decoder-owned decoded input
|
||||
buffers, the current image's concatenated IDAT buffer, Node zlib's default
|
||||
16 KiB output chunk backing rounded up for retained chunks, the concatenated
|
||||
inflate result while its output chunks are retained, one possible extra output
|
||||
chunk before zlib checks `maxOutputLength`, unfiltered scanline storage,
|
||||
RGB-to-RGBA expansion, and the first retained RGBA image while the second is
|
||||
decoded. RGBA output aliases its unfiltered buffer and is counted once.
|
||||
Admission uses checked integer arithmetic before image-sized allocation.
|
||||
Caller-owned JSON/base64 strings, input parsing and validation, transport
|
||||
buffers, garbage-collector behavior, and other runtime memory are outside this
|
||||
estimate; the budget does not promise a bound on total process memory.
|
||||
|
||||
Electron `file://` pages use a separate provider and target boundary; see [electron-observation.md](electron-observation.md).
|
||||
|
||||
## Provider authorities
|
||||
|
||||
@@ -880,7 +880,7 @@ For a client that requires manual configuration, use:
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents@6.4.0", "mcp"]
|
||||
"args": ["-y", "rea-agents@6.3.0", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# JavaScript source recovery
|
||||
|
||||
`recover_javascript_sources` / `recover-javascript-sources` derives readable
|
||||
modules from one explicitly selected UTF-8 JavaScript file or bundle. Use it on
|
||||
modules from explicitly selected UTF-8 JavaScript files or bundles. Use it on
|
||||
files returned by `export_web_scripts` or other local scripts. The operation
|
||||
never executes the selected or recovered application code.
|
||||
|
||||
@@ -52,35 +52,58 @@ Both give the same binary (SHA-256
|
||||
```sh
|
||||
rea recover-javascript-sources /tmp/web-scripts/app.js /tmp/recovered-app --json
|
||||
rea analyze-javascript-application /tmp/recovered-app/modules --json
|
||||
|
||||
# Recover related code-split chunks together; repeat --additional-path.
|
||||
rea recover-javascript-sources /tmp/web-scripts/main.js /tmp/recovered-chunks \
|
||||
--additional-path /tmp/web-scripts/runtime.js \
|
||||
--additional-path /tmp/web-scripts/shared.js \
|
||||
--additional-path /tmp/web-scripts/checkout.chunk.js --json
|
||||
```
|
||||
|
||||
MCP input:
|
||||
|
||||
```json
|
||||
{
|
||||
"path": "/tmp/web-scripts/app.js",
|
||||
"output_directory": "/tmp/recovered-app"
|
||||
"path": "/tmp/web-scripts/main.js",
|
||||
"additional_paths": [
|
||||
"/tmp/web-scripts/runtime.js",
|
||||
"/tmp/web-scripts/shared.js",
|
||||
"/tmp/web-scripts/checkout.chunk.js"
|
||||
],
|
||||
"output_directory": "/tmp/recovered-chunks"
|
||||
}
|
||||
```
|
||||
|
||||
The output directory must be absent, absolute, and have an existing parent.
|
||||
The source is snapshotted in a private workspace, and the engine writes only to
|
||||
Each source is snapshotted separately in a private workspace. Multiple inputs
|
||||
retain their relative directory layout and are passed together to one Wakaru
|
||||
invocation, allowing cross-chunk module IDs to resolve. Inputs must be distinct;
|
||||
`additional_paths` accepts up to 16 additional files. The engine writes only to
|
||||
its own staging directory. Reported module paths, input identities, UTF-8 byte
|
||||
ranges, regular files and emitted map representations are validated before
|
||||
publication. REA writes verified bytes to an exclusively owned output tree and
|
||||
rolls it back on failure. It also checks that the original source and engine
|
||||
bytes remain unchanged before committing. The existing output is never overwritten.
|
||||
|
||||
Multi-input relative filenames must be representable unchanged as artifact paths:
|
||||
NFC Unicode spelling and no literal backslashes. Unsupported layouts fail before
|
||||
invoking the engine rather than silently renaming inputs referenced by source maps.
|
||||
|
||||
Published contents:
|
||||
|
||||
- `inputs/bundle.js`: exact original snapshot; its digest binds the Evidence.
|
||||
- `inputs/bundle.js`: exact original snapshot for a single input. Multiple inputs
|
||||
retain paths relative to their common parent under `inputs/`.
|
||||
- `modules/`: derived modules and any upstream-emitted source maps.
|
||||
- `reports/stdout.json` and `reports/provenance.json`: exact producer reports.
|
||||
- `manifest.json`: identities, modules, transform choices, warnings and limitations.
|
||||
|
||||
The inline result includes complete file paths/digests/sizes, extraction ranges,
|
||||
warnings and `analysis_input`, directly usable as the arguments to
|
||||
`analyze_javascript_application`. Raw Evidence retains selected command arguments,
|
||||
`analyze_javascript_application`. `source` identifies the primary input and binds
|
||||
the Evidence subject; `additional_sources` records the other snapshots and their
|
||||
published copies when selected. Each module names its owning snapshot, original
|
||||
path and original digest; all extraction and context ranges refer to that input.
|
||||
Raw Evidence retains selected command arguments,
|
||||
reported input paths, exit status, stderr and reports; temporary paths are recorded
|
||||
as historical execution coordinates after their workspace is removed.
|
||||
|
||||
@@ -107,7 +130,8 @@ equivalence remain unknown. No network fetching occurs.
|
||||
## Resource policy and verification
|
||||
|
||||
Each operation uses one worker, a 1 GiB address-space ceiling and a 120-second
|
||||
deadline; it accepts up to 64 MiB of input and 128 MiB/10000 entries of staged
|
||||
deadline; it accepts up to 64 MiB of total input across all selected files and
|
||||
128 MiB/10000 entries of staged
|
||||
output, with 8 MiB per diagnostic/report stream. Exceeding a limit is an explicit
|
||||
failure with rollback. These limits apply per operation and do not claim aggregate
|
||||
host containment across concurrent callers.
|
||||
|
||||
@@ -308,23 +308,6 @@ that reference for a summary, one section/module, or a stable page, or call
|
||||
bundle exports stream canonical JSON into an atomically published file. A broad
|
||||
follow-up or `get_evidence_bundle` can also exceed the response budget; exporting
|
||||
preserves the complete session without sending it through a single MCP frame.
|
||||
To discover existing records without their large payloads, call
|
||||
`get_evidence_bundle` with `detail: "summary"`. Its `result.records` contain
|
||||
Evidence IDs, artifact/provider identities, analysis-profile digests and native
|
||||
procedure metadata. Optional `filters` match exact `evidence_id`, `operation`,
|
||||
`target_sha256`, `analysis_profile_digest` or `procedure_address`. Use a discovered
|
||||
ID with `inspect_analysis_view`; neither discovery nor retained selection starts
|
||||
a provider. `retention: "complete-record"` describes stored Evidence, while
|
||||
`native_dossier.value_flow` independently reports truncation, unavailable or
|
||||
unknown metadata. Missing omission counts remain null. Summary output carries
|
||||
every matching record; actual transport admission still applies. Omitted
|
||||
`detail` preserves complete canonical bundle delivery; filters require summary.
|
||||
|
||||
For a portable exported bundle, the CLI shares this projection:
|
||||
`rea inspect-evidence-bundle '{"path":"/tmp/evidence.json","detail":"summary","filters":{"operation":"analyze_function"}}' --format json`.
|
||||
The CLI validates and materializes that local bundle under its existing memory
|
||||
budget; summary delivery does not make arbitrary large files cheap to import.
|
||||
|
||||
Cancelling an export stops further serialization and removes its staging file.
|
||||
The destination changes only when a complete export is atomically published.
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
+1
-5
@@ -21,11 +21,7 @@ Use [the rea-changelog-update skill](https://github.com/morluto/rea/blob/main/.a
|
||||
to turn the bot draft into grouped Highlights, Changes, and Fixes with inline
|
||||
`Thanks @...` credit. `npm run release:notes -- inventory` collects the full
|
||||
selected Git range and paginated GitHub PR/author/closing-issue metadata;
|
||||
`record --summary` produces a compact provenance link for public notes, while
|
||||
`record` produces the full record for review. Credit a contribution once even
|
||||
when it appears in highlights, details and migrations; omit maintainer
|
||||
`@morluto` self-thanks. Keep complete authorship metadata in the inventory
|
||||
instead of duplicating a credit ledger in the release body. `check` validates credits
|
||||
`record` produces the complete contribution record, `check` validates credits
|
||||
and optional PR-body equality, and `render` writes the verified release section.
|
||||
Run a subcommand without its required options to see its usage. GitHub reads
|
||||
require an authenticated `gh`; record/check/render work offline from the saved
|
||||
|
||||
+4
-1
@@ -1036,7 +1036,10 @@ and `REA_JAVASCRIPT_FIXTURE_TOOLS` pointing to an isolated npm prefix containing
|
||||
esbuild 0.25.10 and webpack 5.101.3. No global installation is required.
|
||||
The lane compiles source-owned fixtures, exercises CLI and stdio MCP, verifies
|
||||
published bytes and provenance, feeds recovered modules into existing analysis,
|
||||
and compares a finite set of known fixture results. The provider also accepts
|
||||
and compares a finite set of known fixture results. A webpack fixture with separate
|
||||
runtime, shared, entry and lazy chunks verifies multi-input recovery through both
|
||||
surfaces, per-file provenance and resolved cross-chunk imports in the downstream
|
||||
application graph. The provider also accepts
|
||||
other ^1.13.0 releases and reports them as unverified; this lane proves the
|
||||
audited release. It does not establish
|
||||
arbitrary recovered-application equivalence. CI installs these prerequisites only
|
||||
|
||||
@@ -1,7 +1,5 @@
|
||||
#include "authority.hpp"
|
||||
|
||||
#include <delayimp.h>
|
||||
|
||||
namespace rea {
|
||||
|
||||
static const napi_type_tag RESOURCE_TAG = {0x45fcbf0b49e84fe3ULL, 0x9e3e5074130de843ULL};
|
||||
@@ -123,13 +121,3 @@ static napi_value initialize(napi_env env, napi_value exports) {
|
||||
} // namespace rea
|
||||
|
||||
NAPI_MODULE(NODE_GYP_MODULE_NAME, rea::initialize)
|
||||
|
||||
// Node-API symbols are delay-imported from "node.exe". Resolve them against the
|
||||
// host executable so Node.js, Bun, and other Node-API hosts load the addon
|
||||
// regardless of their executable file name.
|
||||
static FARPROC WINAPI reaHostHook(unsigned event, PDelayLoadInfo info) {
|
||||
if (event != dliNotePreLoadLibrary || _stricmp(info->szDll, "node.exe") != 0) return nullptr;
|
||||
return reinterpret_cast<FARPROC>(GetModuleHandleW(nullptr));
|
||||
}
|
||||
|
||||
decltype(__pfnDliNotifyHook2) __pfnDliNotifyHook2 = reaHostHook;
|
||||
|
||||
@@ -219,10 +219,9 @@ std::unique_ptr<Runtime> createRuntime(const std::wstring& parentPath, const std
|
||||
require(BCryptGenRandom(nullptr, random.data(), random.size(), BCRYPT_USE_SYSTEM_PREFERRED_RNG) == 0,
|
||||
"Generate runtime identity failed", parentPath, ERROR_GEN_FAILURE);
|
||||
auto result = std::make_unique<Runtime>();
|
||||
const auto name = prefix + hex(random.data(), random.size());
|
||||
result->path = parent->path + (parent->path.back() == L'\\' ? L"" : L"\\") + name;
|
||||
result->path = parent->path + L"\\" + prefix + hex(random.data(), random.size());
|
||||
Security security;
|
||||
result->directory = createRelative(parent->get(), name,
|
||||
result->directory = createRelative(parent->get(), prefix + hex(random.data(), random.size()),
|
||||
DELETE | FILE_LIST_DIRECTORY | FILE_READ_ATTRIBUTES | READ_CONTROL,
|
||||
true, security, result->path);
|
||||
try {
|
||||
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "rea-agents",
|
||||
"version": "6.4.0",
|
||||
"version": "6.3.0",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "rea-agents",
|
||||
"version": "6.4.0",
|
||||
"version": "6.3.0",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@babel/parser": "8.0.4",
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "rea-agents",
|
||||
"mcpName": "io.github.morluto/rea",
|
||||
"version": "6.4.0",
|
||||
"version": "6.3.0",
|
||||
"description": "Reverse engineer anything from your terminal or agent with one CLI and MCP server.",
|
||||
"license": "MIT",
|
||||
"repository": {
|
||||
|
||||
@@ -84,8 +84,7 @@ if (!(await exists(join(headers, "node_api.h")))) {
|
||||
}
|
||||
const artifact = "rea-windows-x64.node";
|
||||
// MSVC short-import libraries do not produce a correct Node import table with
|
||||
// GNU ld. Generate a GNU delay-import library for the exact stable Node-API
|
||||
// symbols; addon.cc binds it to the host executable, whatever its file name.
|
||||
// GNU ld. Generate a GNU import library for the exact stable Node-API symbols.
|
||||
const sources = ["addon.cc", "filesystem.cc", "process.cc"];
|
||||
const sourceText = (
|
||||
await Promise.all(
|
||||
@@ -107,7 +106,7 @@ await writeFile(
|
||||
definition,
|
||||
`LIBRARY node.exe\nEXPORTS\n${symbols.join("\n")}\n`,
|
||||
);
|
||||
run("x86_64-w64-mingw32-dlltool", ["-d", definition, "-y", library]);
|
||||
run("x86_64-w64-mingw32-dlltool", ["-d", definition, "-l", library]);
|
||||
run(compiler, [
|
||||
"-std=c++17",
|
||||
"-O2",
|
||||
|
||||
@@ -1,61 +0,0 @@
|
||||
/* Keep the configured launcher identity alive while its resource-limited
|
||||
* Python child runs. Both remain in REA's owned process group. */
|
||||
#define _POSIX_C_SOURCE 200809L
|
||||
#include <errno.h>
|
||||
#include <signal.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <sys/resource.h>
|
||||
#include <sys/types.h>
|
||||
#include <sys/wait.h>
|
||||
#include <unistd.h>
|
||||
|
||||
int main(int argc, char **argv) {
|
||||
char *python = getenv("REA_VERIFY_LIMITED_PYTHON");
|
||||
char *limit = getenv("REA_VERIFY_LIMIT_OPTION");
|
||||
if (python == NULL || python[0] != '/' || limit == NULL) {
|
||||
fputs("limited Python fixture requires an absolute Python and limit\n", stderr);
|
||||
return 127;
|
||||
}
|
||||
struct rlimit core = {0, 0};
|
||||
if (setrlimit(RLIMIT_CORE, &core) != 0) {
|
||||
perror("setrlimit");
|
||||
return 127;
|
||||
}
|
||||
char **arguments = calloc((size_t)argc + 4, sizeof(char *));
|
||||
if (arguments == NULL) return 127;
|
||||
arguments[0] = "/usr/bin/prlimit";
|
||||
arguments[1] = limit;
|
||||
arguments[2] = "--";
|
||||
arguments[3] = python;
|
||||
for (int index = 1; index < argc; index++) arguments[index + 3] = argv[index];
|
||||
pid_t child = fork();
|
||||
if (child == 0) {
|
||||
execv(arguments[0], arguments);
|
||||
perror("execv");
|
||||
_exit(127);
|
||||
}
|
||||
free(arguments);
|
||||
if (child < 0) {
|
||||
perror("fork");
|
||||
return 127;
|
||||
}
|
||||
int status;
|
||||
while (waitpid(child, &status, 0) < 0) {
|
||||
if (errno == EINTR) continue;
|
||||
perror("waitpid");
|
||||
return 127;
|
||||
}
|
||||
if (WIFEXITED(status)) return WEXITSTATUS(status);
|
||||
if (WIFSIGNALED(status)) {
|
||||
int signum = WTERMSIG(status);
|
||||
signal(signum, SIG_DFL);
|
||||
sigset_t signals;
|
||||
sigemptyset(&signals);
|
||||
sigaddset(&signals, signum);
|
||||
sigprocmask(SIG_UNBLOCK, &signals, NULL);
|
||||
raise(signum);
|
||||
return 128 + signum;
|
||||
}
|
||||
return 127;
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
import { createRequire } from "node:module";
|
||||
import { mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import { mkdir, readFile, readdir, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
@@ -20,23 +20,56 @@ export async function buildRecoveryFixtures(root, prefix) {
|
||||
const source = fileURLToPath(new URL("./entry.mjs", import.meta.url));
|
||||
const webpackOutput = join(root, "webpack");
|
||||
await mkdir(webpackOutput);
|
||||
await new Promise((resolve, reject) => {
|
||||
const compiler = webpack({
|
||||
mode: "production",
|
||||
entry: source,
|
||||
parallelism: 1,
|
||||
optimization: { concatenateModules: false },
|
||||
output: { path: webpackOutput, filename: "bundle.js" },
|
||||
});
|
||||
compiler.run((error, stats) => {
|
||||
compiler.close((closeError) => {
|
||||
if (error || closeError) return reject(error ?? closeError);
|
||||
if (!stats || stats.hasErrors())
|
||||
return reject(new Error(stats?.toString() ?? "No webpack stats"));
|
||||
resolve();
|
||||
const compileWebpack = async (options) =>
|
||||
new Promise((resolve, reject) => {
|
||||
const compiler = webpack(options);
|
||||
compiler.run((error, stats) => {
|
||||
compiler.close((closeError) => {
|
||||
if (error || closeError) return reject(error ?? closeError);
|
||||
if (!stats || stats.hasErrors())
|
||||
return reject(new Error(stats?.toString() ?? "No webpack stats"));
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
});
|
||||
await compileWebpack({
|
||||
mode: "production",
|
||||
entry: source,
|
||||
parallelism: 1,
|
||||
optimization: { concatenateModules: false },
|
||||
output: { path: webpackOutput, filename: "bundle.js" },
|
||||
});
|
||||
const splitOutput = join(root, "webpack-code-split");
|
||||
await mkdir(splitOutput);
|
||||
const splitSource = fileURLToPath(
|
||||
new URL("./code-split/entry.mjs", import.meta.url),
|
||||
);
|
||||
await compileWebpack({
|
||||
mode: "production",
|
||||
entry: splitSource,
|
||||
parallelism: 1,
|
||||
optimization: {
|
||||
concatenateModules: false,
|
||||
runtimeChunk: "single",
|
||||
splitChunks: {
|
||||
chunks: "all",
|
||||
minSize: 0,
|
||||
cacheGroups: {
|
||||
shared: { test: /shared\.mjs$/u, name: "shared", enforce: true },
|
||||
},
|
||||
},
|
||||
},
|
||||
output: {
|
||||
path: splitOutput,
|
||||
filename: "[name].js",
|
||||
chunkFilename: "[id].chunk.js",
|
||||
},
|
||||
});
|
||||
const splitFiles = (await readdir(splitOutput))
|
||||
.filter((name) => name !== "main.js")
|
||||
.sort();
|
||||
if (splitFiles.length !== 3)
|
||||
throw new Error("Expected runtime, shared and lazy chunks");
|
||||
const esbuildPath = join(root, "esbuild.js");
|
||||
await esbuild.build({
|
||||
entryPoints: [source],
|
||||
@@ -95,6 +128,13 @@ export async function buildRecoveryFixtures(root, prefix) {
|
||||
options: { extraction_mode: "inspection", rewrite_level: "minimal" },
|
||||
},
|
||||
],
|
||||
codeSplit: {
|
||||
name: "webpack-code-split",
|
||||
path: join(splitOutput, "main.js"),
|
||||
additional_paths: splitFiles.map((name) => join(splitOutput, name)),
|
||||
expectedFormat: "webpack5",
|
||||
expectedModules: 4,
|
||||
},
|
||||
toolchains: { esbuild: esbuild.version, webpack: webpackVersion },
|
||||
};
|
||||
}
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
import { Cart, formatPrice, TAX_RATE } from "./shared.mjs";
|
||||
export function checkout(prices) {
|
||||
const cart = new Cart();
|
||||
prices.forEach((price) => cart.add(price));
|
||||
return formatPrice(cart.total() * (1 + TAX_RATE));
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
import { formatPrice } from "./shared.mjs";
|
||||
console.log(formatPrice(1));
|
||||
import("./checkout.mjs").then(({ checkout }) => console.log(checkout([1, 2])));
|
||||
@@ -0,0 +1,14 @@
|
||||
export class Cart {
|
||||
constructor() {
|
||||
this.items = [];
|
||||
}
|
||||
add(price) {
|
||||
this.items.push(price);
|
||||
return this;
|
||||
}
|
||||
total() {
|
||||
return this.items.reduce((sum, price) => sum + price, 0);
|
||||
}
|
||||
}
|
||||
export const formatPrice = (value) => `$${value.toFixed(2)}`;
|
||||
export const TAX_RATE = 0.08;
|
||||
@@ -28,11 +28,6 @@ import {
|
||||
isolatedFixtureUrl,
|
||||
} from "./browser-runtime-world-e2e.mjs";
|
||||
|
||||
// Arming, progress delivery, isolated-world proof and the page action cross
|
||||
// separate transports. Leave time for their round trips on shared CI runners;
|
||||
// coverage can include a late action even after request collection has ended.
|
||||
const executionObservationMs = 2_000;
|
||||
|
||||
/** Exercise native browser runtime attribution through public CLI and stdio MCP, including arming. */
|
||||
export async function verifyBrowserRuntime(
|
||||
executable,
|
||||
@@ -215,7 +210,7 @@ async function executionMcp(
|
||||
const response = await client.callTool(
|
||||
{
|
||||
name: "observe_web_execution",
|
||||
arguments: { ...input, observation_ms: executionObservationMs },
|
||||
arguments: { ...input, observation_ms: 300 },
|
||||
},
|
||||
{
|
||||
timeout: 40_000,
|
||||
@@ -264,7 +259,7 @@ async function executionCli(entrypoint, input, env, action, isolatedContext) {
|
||||
input.cdp_endpoint,
|
||||
input.target_id,
|
||||
"--observation-ms",
|
||||
String(executionObservationMs),
|
||||
"300",
|
||||
"--json",
|
||||
],
|
||||
{ env, stdio: ["ignore", "pipe", "pipe"] },
|
||||
@@ -387,14 +382,7 @@ function assertExecution(evidence, site, requireBlock = true) {
|
||||
const request = result.requests.find((item) =>
|
||||
item.url.endsWith("/evidence?marker=chosen"),
|
||||
);
|
||||
assert.ok(
|
||||
request,
|
||||
`The selected request initiator must be retained: ${JSON.stringify({
|
||||
window: result.window,
|
||||
requests: result.requests.map((item) => item.url),
|
||||
excluded_requests: result.excluded_requests,
|
||||
})}`,
|
||||
);
|
||||
assert.ok(request, "The selected request initiator must be retained");
|
||||
assert.ok(
|
||||
request.callsites.some(
|
||||
(site) =>
|
||||
|
||||
@@ -101,13 +101,10 @@ export async function readReleaseHistory(run, base, target) {
|
||||
// REA history maps Anthropic and Cursor's agent trailers to ordinary User actors.
|
||||
// Keep those source identities in the inventory while excluding automation credit.
|
||||
const AGENT_ACCOUNTS = new Set(["claude", "cursoragent"]);
|
||||
// REA's maintainer requested external-contributor thanks without self-credit.
|
||||
const MAINTAINER_ACCOUNTS = new Set(["morluto"]);
|
||||
const human = (value) =>
|
||||
value?.__typename === "User" &&
|
||||
!value.login.endsWith("[bot]") &&
|
||||
!AGENT_ACCOUNTS.has(value.login.toLowerCase()) &&
|
||||
!MAINTAINER_ACCOUNTS.has(value.login.toLowerCase())
|
||||
!AGENT_ACCOUNTS.has(value.login.toLowerCase())
|
||||
? value.login
|
||||
: undefined;
|
||||
const unique = (values) =>
|
||||
@@ -166,12 +163,6 @@ export function contributionRecord(raw) {
|
||||
return `${lines.join("\n")}\n`;
|
||||
}
|
||||
|
||||
/** Keep the full authorship snapshot available without duplicating public thanks. */
|
||||
export function contributionSummary(raw) {
|
||||
const inventory = releaseInventorySchema.parse(raw);
|
||||
return `Full contribution record: [${inventory.commits.length} commits and ${inventory.pullRequests.length} merged PR${inventory.pullRequests.length === 1 ? "" : "s"}](https://github.com/${inventory.repository}/blob/rea-agents-${inventory.version}/docs/releases/${inventory.version}.contributions.json).\n`;
|
||||
}
|
||||
|
||||
export function selectedReleaseNotes(text, version) {
|
||||
const headings = [...text.matchAll(/^## \[([^\]]+)\].*$/gmu)];
|
||||
if (headings[0]?.[1] !== version)
|
||||
@@ -245,8 +236,7 @@ function requiredCredits(inventory, bullet, references) {
|
||||
if (matches[0].parents.length > 1) continue;
|
||||
for (const identity of matches[0].authors) {
|
||||
const handle = human(identity.user);
|
||||
if (handle)
|
||||
required.push({ handle, context: `commit ${matches[0].sha}` });
|
||||
if (handle) required.push({ handle, context: `commit ${match[2]}` });
|
||||
}
|
||||
}
|
||||
return required;
|
||||
@@ -274,8 +264,6 @@ function verifyEditorialCredits(inventory, prose) {
|
||||
]).map((handle) => handle.toLowerCase()),
|
||||
);
|
||||
const errors = [];
|
||||
const requiredByContext = new Map();
|
||||
const thankedByContext = new Map();
|
||||
for (const bullet of proseBullets(prose)) {
|
||||
const references = bulletPrs(bullet, inventory.repository);
|
||||
const start = bullet.indexOf("Thanks ");
|
||||
@@ -294,21 +282,13 @@ function verifyEditorialCredits(inventory, prose) {
|
||||
bullet,
|
||||
references,
|
||||
)) {
|
||||
const required = requiredByContext.get(context) ?? new Set();
|
||||
required.add(handle);
|
||||
requiredByContext.set(context, required);
|
||||
const credited = thankedByContext.get(context) ?? new Set();
|
||||
for (const login of thanked) credited.add(login);
|
||||
thankedByContext.set(context, credited);
|
||||
if (!thanked.has(handle.toLowerCase()))
|
||||
errors.push(`Missing Thanks @${handle} in bullet for ${context}`);
|
||||
}
|
||||
for (const handle of thanked)
|
||||
if (!eligible.has(handle))
|
||||
errors.push(`Unverified human credit @${handle}`);
|
||||
}
|
||||
for (const [context, required] of requiredByContext)
|
||||
for (const handle of required)
|
||||
if (!thankedByContext.get(context)?.has(handle.toLowerCase()))
|
||||
errors.push(`Missing Thanks @${handle} for ${context}`);
|
||||
return errors;
|
||||
}
|
||||
|
||||
@@ -325,15 +305,8 @@ export function verifyReleaseNotes(raw, text, body) {
|
||||
"Release notes exceed GitHub's body budget; shorten prose without dropping credit",
|
||||
);
|
||||
const record = contributionRecord(inventory).trim();
|
||||
const summary = contributionSummary(inventory).trim();
|
||||
const start = notes.endsWith(summary)
|
||||
? notes.length - summary.length
|
||||
: notes.indexOf("### Complete contribution record");
|
||||
if (
|
||||
start < 0 ||
|
||||
(notes.slice(start).trim() !== record &&
|
||||
notes.slice(start).trim() !== summary)
|
||||
)
|
||||
const start = notes.indexOf("### Complete contribution record");
|
||||
if (start < 0 || notes.slice(start).trim() !== record)
|
||||
throw new Error(
|
||||
"Contribution record differs from inventory; regenerate it with release-notes record",
|
||||
);
|
||||
|
||||
@@ -6,7 +6,6 @@ import { collectGitHubCredits } from "./lib/release-notes-github.mjs";
|
||||
import { verifyNotesPublication } from "./lib/release-notes-publication.mjs";
|
||||
import {
|
||||
contributionRecord,
|
||||
contributionSummary,
|
||||
readReleaseHistory,
|
||||
releaseInventorySchema,
|
||||
selectedReleaseNotes,
|
||||
@@ -74,7 +73,6 @@ async function main() {
|
||||
"pr-body": { type: "string" },
|
||||
source: { type: "string" },
|
||||
output: { type: "string" },
|
||||
summary: { type: "boolean", default: false },
|
||||
},
|
||||
});
|
||||
const [command] = positionals;
|
||||
@@ -94,7 +92,7 @@ async function main() {
|
||||
!values.manifest
|
||||
)
|
||||
throw new Error(
|
||||
"Usage: release-notes inventory|record|check|render --manifest FILE [--repository OWNER/REPO --base TAG --target SHA --version VERSION] [--notes FILE --pr-body FILE --source SHA --output FILE] [--summary for record]",
|
||||
"Usage: release-notes inventory|record|check|render --manifest FILE [--repository OWNER/REPO --base TAG --target SHA --version VERSION] [--notes FILE --pr-body FILE --source SHA --output FILE]",
|
||||
);
|
||||
if (command === "inventory") {
|
||||
await createInventory(values);
|
||||
@@ -105,14 +103,11 @@ async function main() {
|
||||
);
|
||||
await verifyInventoryHistory(run, inventory, values.source);
|
||||
if (command === "record") {
|
||||
const record = values.summary
|
||||
? contributionSummary(inventory)
|
||||
: contributionRecord(inventory);
|
||||
if (values.output)
|
||||
await writeFile(values.output, record, {
|
||||
await writeFile(values.output, contributionRecord(inventory), {
|
||||
flag: "wx",
|
||||
});
|
||||
else process.stdout.write(record);
|
||||
else process.stdout.write(contributionRecord(inventory));
|
||||
return;
|
||||
}
|
||||
const notes = await readFile(values.notes, "utf8");
|
||||
|
||||
@@ -15,10 +15,6 @@ import { promisify } from "node:util";
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { PrivateRuntimeRoot } from "../dist/process/PrivateRuntimeRoot.js";
|
||||
import {
|
||||
OwnedCommandFailure,
|
||||
runOwnedCommand,
|
||||
} from "../dist/process/OwnedCommand.js";
|
||||
import { parseEvidence } from "../dist/domain/evidence.js";
|
||||
import { mcpTextValue } from "./lib/mcp-verifier-results.mjs";
|
||||
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
|
||||
@@ -60,13 +56,6 @@ const root = await PrivateRuntimeRoot.create({
|
||||
const environment = Object.fromEntries(
|
||||
Object.entries(process.env).filter(([, value]) => typeof value === "string"),
|
||||
);
|
||||
const limitedPython = join(root.path, "limited-python");
|
||||
const limitedEnvironment = (option) => ({
|
||||
...environment,
|
||||
REA_PWNTOOLS_PYTHON: limitedPython,
|
||||
REA_VERIFY_LIMITED_PYTHON: python,
|
||||
REA_VERIFY_LIMIT_OPTION: option,
|
||||
});
|
||||
const client = new Client({ name: "binary-layout-verifier", version: "1" });
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
@@ -84,46 +73,6 @@ let cases = 0;
|
||||
let bootstrapCases = 0;
|
||||
const failures = [];
|
||||
try {
|
||||
await execute(
|
||||
"gcc",
|
||||
[
|
||||
"-std=c11",
|
||||
"-Wall",
|
||||
"-Wextra",
|
||||
"-Werror",
|
||||
fileURLToPath(
|
||||
new URL("./fixtures/binary-layout-limited-python.c", import.meta.url),
|
||||
),
|
||||
"-o",
|
||||
limitedPython,
|
||||
],
|
||||
{ timeout: 30_000, maxBuffer: 1024 * 1024 },
|
||||
);
|
||||
// A live limited child must retain its configured launcher's identity so
|
||||
// timeout cleanup can still verify and terminate the entire owned group.
|
||||
await assert.rejects(
|
||||
runOwnedCommand(
|
||||
{
|
||||
command: limitedPython,
|
||||
arguments: [
|
||||
"-I",
|
||||
"-c",
|
||||
"import time; print('limited child ready', flush=True); time.sleep(60)",
|
||||
],
|
||||
cwd: root.path,
|
||||
runId: `${run.run_id}-limited-python-timeout`,
|
||||
hostEnvironment: limitedEnvironment("--as=2147483648"),
|
||||
},
|
||||
{ timeoutMs: 2000, diagnosticBytes: 1024 },
|
||||
),
|
||||
(error) => {
|
||||
assert.ok(error instanceof OwnedCommandFailure);
|
||||
assert.equal(error.reason, "timeout");
|
||||
assert.equal(error.cleanupFailure, null);
|
||||
assert.ok(error.snapshot.stdout.text.includes("limited child ready"));
|
||||
return true;
|
||||
},
|
||||
);
|
||||
const bootstrapRoot = join(root.path, "bootstrap-boundary");
|
||||
await mkdir(bootstrapRoot);
|
||||
const bootstrapPath = join(bootstrapRoot, "layout.py");
|
||||
@@ -847,12 +796,19 @@ try {
|
||||
assert.deepEqual(await readFile(extendedPath), extendedObject);
|
||||
cases++;
|
||||
}
|
||||
for (const [option, expected] of [
|
||||
["--as=2147483648", '"address_space_bytes": 2147483648'],
|
||||
["--cpu=20", '"cpu_seconds": 20'],
|
||||
["--fsize=33554432", '"file_size_bytes": 33554432'],
|
||||
for (const [name, option, expected] of [
|
||||
["address-space", "--as=2147483648", '"address_space_bytes": 2147483648'],
|
||||
["cpu", "--cpu=20", '"cpu_seconds": 20'],
|
||||
["file-size", "--fsize=33554432", '"file_size_bytes": 33554432'],
|
||||
]) {
|
||||
const selectedEnvironment = limitedEnvironment(option);
|
||||
const wrapper = join(root.path, `python-limit-${name}`);
|
||||
const quotedPython = "'" + python.replaceAll("'", "'\"'\"'") + "'";
|
||||
await writeFile(
|
||||
wrapper,
|
||||
`#!/bin/sh\nexec /usr/bin/prlimit ${option} -- ${quotedPython} "$@"\n`,
|
||||
{ mode: 0o700 },
|
||||
);
|
||||
const limitedEnvironment = { ...environment, REA_PWNTOOLS_PYTHON: wrapper };
|
||||
const limitedClient = new Client({
|
||||
name: "limited-layout-verifier",
|
||||
version: "1",
|
||||
@@ -860,7 +816,7 @@ try {
|
||||
const limitedTransport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [entrypoint, "mcp"],
|
||||
env: selectedEnvironment,
|
||||
env: limitedEnvironment,
|
||||
stderr: "pipe",
|
||||
});
|
||||
try {
|
||||
@@ -870,7 +826,7 @@ try {
|
||||
mode,
|
||||
join(root.path, "protected"),
|
||||
undefined,
|
||||
selectedEnvironment,
|
||||
limitedEnvironment,
|
||||
limitedClient,
|
||||
);
|
||||
assert.ok(value.limitations.some((item) => item.includes(expected)));
|
||||
@@ -906,29 +862,17 @@ try {
|
||||
sectionHeavy.writeUInt16LE(sectionCount, 60);
|
||||
const sectionHeavyPath = join(root.path, "section-heavy.o");
|
||||
await writeFile(sectionHeavyPath, sectionHeavy);
|
||||
// pwntools materializes each PT_LOAD zero-filled tail during inspection.
|
||||
// One allocation larger than the whole address-space budget fails promptly;
|
||||
// a large section count instead races CPU work against the command deadline.
|
||||
const memoryHeavy = Buffer.from(await readFile(join(root.path, "protected")));
|
||||
const memoryPhOffset = Number(memoryHeavy.readBigUInt64LE(32));
|
||||
const memoryPhSize = memoryHeavy.readUInt16LE(54);
|
||||
const memoryPhCount = memoryHeavy.readUInt16LE(56);
|
||||
const lastLoadIndex = Array.from(
|
||||
{ length: memoryPhCount },
|
||||
(_, index) => index,
|
||||
).findLast(
|
||||
(index) =>
|
||||
memoryHeavy.readUInt32LE(memoryPhOffset + index * memoryPhSize) === 1,
|
||||
const memoryWrapper = join(root.path, "python-memory-constraint");
|
||||
const quotedPython = "'" + python.replaceAll("'", "'\"'\"'") + "'";
|
||||
await writeFile(
|
||||
memoryWrapper,
|
||||
`#!/bin/sh\nexec /usr/bin/prlimit --as=100663296 -- ${quotedPython} "$@"\n`,
|
||||
{ mode: 0o700 },
|
||||
);
|
||||
assert.notEqual(lastLoadIndex, undefined);
|
||||
const memorySizeOffset = memoryPhOffset + lastLoadIndex * memoryPhSize + 40;
|
||||
memoryHeavy.writeBigUInt64LE(
|
||||
memoryHeavy.readBigUInt64LE(memorySizeOffset) + 128n * 1024n * 1024n,
|
||||
memorySizeOffset,
|
||||
);
|
||||
const memoryHeavyPath = join(root.path, "memory-zero-tail");
|
||||
await writeFile(memoryHeavyPath, memoryHeavy);
|
||||
const memoryEnvironment = limitedEnvironment("--as=100663296");
|
||||
const memoryEnvironment = {
|
||||
...environment,
|
||||
REA_PWNTOOLS_PYTHON: memoryWrapper,
|
||||
};
|
||||
const memoryClient = new Client({
|
||||
name: "memory-layout-verifier",
|
||||
version: "1",
|
||||
@@ -944,7 +888,7 @@ try {
|
||||
for (const mode of ["cli", "mcp"]) {
|
||||
const error = await inspect(
|
||||
mode,
|
||||
memoryHeavyPath,
|
||||
sectionHeavyPath,
|
||||
"resource_constraint",
|
||||
memoryEnvironment,
|
||||
memoryClient,
|
||||
@@ -959,7 +903,7 @@ try {
|
||||
assert.ok(error.remediation.action.includes("memory"));
|
||||
cases++;
|
||||
}
|
||||
assert.deepEqual(await readFile(memoryHeavyPath), memoryHeavy);
|
||||
assert.deepEqual(await readFile(sectionHeavyPath), sectionHeavy);
|
||||
} finally {
|
||||
try {
|
||||
await memoryClient.close();
|
||||
@@ -980,7 +924,13 @@ try {
|
||||
cpuHeavy.writeBigUInt64LE(BigInt(cpuSectionCount), sectionOffset + 32);
|
||||
const cpuHeavyPath = join(root.path, "cpu-section-heavy.o");
|
||||
await writeFile(cpuHeavyPath, cpuHeavy);
|
||||
const cpuEnvironment = limitedEnvironment("--cpu=1:");
|
||||
const cpuWrapper = join(root.path, "python-cpu-constraint");
|
||||
await writeFile(
|
||||
cpuWrapper,
|
||||
`#!/bin/sh\nexec /usr/bin/prlimit --cpu=1: --core=0: -- ${quotedPython} "$@"\n`,
|
||||
{ mode: 0o700 },
|
||||
);
|
||||
const cpuEnvironment = { ...environment, REA_PWNTOOLS_PYTHON: cpuWrapper };
|
||||
const cpuClient = new Client({ name: "cpu-layout-verifier", version: "1" });
|
||||
const cpuTransport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
@@ -1013,7 +963,13 @@ try {
|
||||
}
|
||||
}
|
||||
for (const limit of [1024, 1]) {
|
||||
const fileEnvironment = limitedEnvironment(`--fsize=${limit}:`);
|
||||
const wrapper = join(root.path, `python-file-size-${limit}`);
|
||||
await writeFile(
|
||||
wrapper,
|
||||
`#!/bin/sh\nexec /usr/bin/prlimit --fsize=${limit}: --core=0: -- ${quotedPython} "$@"\n`,
|
||||
{ mode: 0o700 },
|
||||
);
|
||||
const fileEnvironment = { ...environment, REA_PWNTOOLS_PYTHON: wrapper };
|
||||
const fileClient = new Client({
|
||||
name: "file-size-layout-verifier",
|
||||
version: "1",
|
||||
|
||||
+97
-145
@@ -38,147 +38,104 @@ let update;
|
||||
let repositorySkill;
|
||||
let report;
|
||||
|
||||
const runStage = async (name, action) => {
|
||||
const started = performance.now();
|
||||
process.stderr.write(`[verify:package] ${name}: start\n`);
|
||||
const result = await action();
|
||||
process.stderr.write(
|
||||
`[verify:package] ${name}: complete (${Math.round(performance.now() - started)} ms)\n`,
|
||||
);
|
||||
return result;
|
||||
};
|
||||
|
||||
try {
|
||||
repositorySkill = await runStage("repository skill", () =>
|
||||
verifyRepositorySkill({ root, workspace }),
|
||||
);
|
||||
({ tarball } = await runStage("package archive", () =>
|
||||
verifyPackagePack({ root, workspace }),
|
||||
));
|
||||
const environmentData = await runStage("package environment", () =>
|
||||
verifyPackageEnvironment({
|
||||
root,
|
||||
workspace,
|
||||
evidenceRoot,
|
||||
referenceRoot,
|
||||
}),
|
||||
);
|
||||
const { cli, packageRunnerCli } = await runStage("package install", () =>
|
||||
verifyPackageInstall({
|
||||
tarball,
|
||||
prefix: environmentData.prefix,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
update = await runStage("package update", () =>
|
||||
verifyPackageUpdate({
|
||||
prefix: environmentData.prefix,
|
||||
packageRoot: join(
|
||||
environmentData.prefix,
|
||||
"lib",
|
||||
"node_modules",
|
||||
PRODUCT_IDENTITY.packageName,
|
||||
),
|
||||
tarball,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
mcpStartup = await runStage("MCP startup", () =>
|
||||
measureMcpStartup({
|
||||
command: cli,
|
||||
args: ["mcp"],
|
||||
environment: {
|
||||
...environmentData.environment,
|
||||
},
|
||||
policy: MCP_STARTUP_POLICY,
|
||||
}),
|
||||
);
|
||||
mcpModuleLoading = await runStage("MCP module loading", () =>
|
||||
profileMcpModuleLoading({
|
||||
command: cli,
|
||||
args: ["mcp"],
|
||||
environment: {
|
||||
...environmentData.environment,
|
||||
},
|
||||
policy: MCP_STARTUP_POLICY,
|
||||
packageName: PRODUCT_IDENTITY.packageName,
|
||||
}),
|
||||
);
|
||||
const { supportedSetupHost, hopperSetupSupported } = await runStage(
|
||||
"package discovery",
|
||||
() =>
|
||||
verifyPackageDiscovery({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
const { artifactArchive } = await runStage("artifact and Electron CLI", () =>
|
||||
verifyPackageArtifactAndElectron({
|
||||
cli,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
await runStage("managed analysis CLI", () =>
|
||||
verifyManaged({
|
||||
cli,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
await runStage("unknown provider CLI", () =>
|
||||
verifyUnknownProvider({
|
||||
repositorySkill = await verifyRepositorySkill({ root, workspace });
|
||||
({ tarball } = await verifyPackagePack({ root, workspace }));
|
||||
const environmentData = await verifyPackageEnvironment({
|
||||
root,
|
||||
workspace,
|
||||
evidenceRoot,
|
||||
referenceRoot,
|
||||
});
|
||||
const { cli, packageRunnerCli } = await verifyPackageInstall({
|
||||
tarball,
|
||||
prefix: environmentData.prefix,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
update = await verifyPackageUpdate({
|
||||
prefix: environmentData.prefix,
|
||||
packageRoot: join(
|
||||
environmentData.prefix,
|
||||
"lib",
|
||||
"node_modules",
|
||||
PRODUCT_IDENTITY.packageName,
|
||||
),
|
||||
tarball,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
mcpStartup = await measureMcpStartup({
|
||||
command: cli,
|
||||
args: ["mcp"],
|
||||
environment: {
|
||||
...environmentData.environment,
|
||||
},
|
||||
policy: MCP_STARTUP_POLICY,
|
||||
});
|
||||
mcpModuleLoading = await profileMcpModuleLoading({
|
||||
command: cli,
|
||||
args: ["mcp"],
|
||||
environment: {
|
||||
...environmentData.environment,
|
||||
},
|
||||
policy: MCP_STARTUP_POLICY,
|
||||
packageName: PRODUCT_IDENTITY.packageName,
|
||||
});
|
||||
const { supportedSetupHost, hopperSetupSupported } =
|
||||
await verifyPackageDiscovery({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
await runStage("platform CLI", () =>
|
||||
verifyPackagePlatform({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
await runStage("capabilities and search CLI", () =>
|
||||
verifyPackageCapabilitiesAndSearch({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
await runStage("evidence CLI", () =>
|
||||
verifyPackageEvidence({
|
||||
cli,
|
||||
evidenceRoot,
|
||||
referenceRoot,
|
||||
environment: environmentData.environment,
|
||||
}),
|
||||
);
|
||||
await runStage("setup and uninstall CLI", () =>
|
||||
verifyPackageSetup({
|
||||
cli,
|
||||
packageRunnerCli,
|
||||
environment: environmentData.environment,
|
||||
home: environmentData.home,
|
||||
npxLog: environmentData.npxLog,
|
||||
claudeConfig: environmentData.claudeConfig,
|
||||
codexConfig: environmentData.codexConfig,
|
||||
cursorConfig: environmentData.cursorConfig,
|
||||
codexTarget: environmentData.codexTarget,
|
||||
cursorTarget: environmentData.cursorTarget,
|
||||
supportedSetupHost,
|
||||
hopperSetupSupported,
|
||||
root,
|
||||
}),
|
||||
);
|
||||
await runStage("MCP tools and evidence", () =>
|
||||
verifyPackageMcp({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
evidenceRoot,
|
||||
artifactArchive,
|
||||
}),
|
||||
);
|
||||
});
|
||||
const { artifactArchive } = await verifyPackageArtifactAndElectron({
|
||||
cli,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
await verifyManaged({
|
||||
cli,
|
||||
workspace,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
await verifyUnknownProvider({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
await verifyPackagePlatform({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
await verifyPackageCapabilitiesAndSearch({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
await verifyPackageEvidence({
|
||||
cli,
|
||||
evidenceRoot,
|
||||
referenceRoot,
|
||||
environment: environmentData.environment,
|
||||
});
|
||||
await verifyPackageSetup({
|
||||
cli,
|
||||
packageRunnerCli,
|
||||
environment: environmentData.environment,
|
||||
home: environmentData.home,
|
||||
npxLog: environmentData.npxLog,
|
||||
claudeConfig: environmentData.claudeConfig,
|
||||
codexConfig: environmentData.codexConfig,
|
||||
cursorConfig: environmentData.cursorConfig,
|
||||
codexTarget: environmentData.codexTarget,
|
||||
cursorTarget: environmentData.cursorTarget,
|
||||
supportedSetupHost,
|
||||
hopperSetupSupported,
|
||||
root,
|
||||
});
|
||||
await verifyPackageMcp({
|
||||
cli,
|
||||
environment: environmentData.environment,
|
||||
evidenceRoot,
|
||||
artifactArchive,
|
||||
});
|
||||
report = {
|
||||
cli: true,
|
||||
analysisCli: true,
|
||||
@@ -216,13 +173,8 @@ try {
|
||||
mcpModuleLoading,
|
||||
};
|
||||
} finally {
|
||||
await runStage("workspace cleanup", () =>
|
||||
rm(workspace, { recursive: true, force: true }),
|
||||
);
|
||||
await rm(workspace, { recursive: true, force: true });
|
||||
}
|
||||
const verifierLineage = await runStage("verifier lineage", () =>
|
||||
completeVerifierRun(verifierRun),
|
||||
);
|
||||
process.stdout.write(
|
||||
`${JSON.stringify({ verifier_run: verifierLineage, ...report })}\n`,
|
||||
`${JSON.stringify({ verifier_run: await completeVerifierRun(verifierRun), ...report })}\n`,
|
||||
);
|
||||
|
||||
@@ -6,7 +6,6 @@ import { createHash } from "node:crypto";
|
||||
import { once } from "node:events";
|
||||
import {
|
||||
access,
|
||||
copyFile,
|
||||
mkdir,
|
||||
mkdtemp,
|
||||
readFile,
|
||||
@@ -17,7 +16,7 @@ import {
|
||||
open,
|
||||
} from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { dirname, join, parse, resolve } from "node:path";
|
||||
import { join, resolve } from "node:path";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { verifyWindowsProtocolInput } from "./lib/windows-protocol-input.mjs";
|
||||
|
||||
@@ -33,9 +32,6 @@ const native = requireWindowsNativeAuthority();
|
||||
const { WindowsPrivateRuntime, windowsPrivateRuntime } = await import(
|
||||
pathToFileURL(join(packageRoot, "dist/windows/WindowsPrivateRuntime.js"))
|
||||
);
|
||||
const { ghidraSessionRoot } = await import(
|
||||
pathToFileURL(join(packageRoot, "dist/ghidra/GhidraSessionRoot.js"))
|
||||
);
|
||||
const workspace = await mkdtemp(join(tmpdir(), "rea-native-conformance-"));
|
||||
const environment = Object.entries(process.env)
|
||||
.filter(([, value]) => value !== undefined)
|
||||
@@ -81,31 +77,6 @@ const wait = async (child) => {
|
||||
};
|
||||
const report = { ok: false, artifact: "verified", controls: {}, processes: {} };
|
||||
try {
|
||||
// Exercise the loader in a process whose image is not named node.exe. The
|
||||
// ordinary Windows lane otherwise cannot detect eager Node-API imports.
|
||||
const renamedHost = join(workspace, "rea-renamed-node.exe");
|
||||
const renamedHostFixture = join(workspace, "renamed-host.mjs");
|
||||
const loaderUrl = pathToFileURL(
|
||||
join(packageRoot, "dist/windows/WindowsNativeLoader.js"),
|
||||
).href;
|
||||
await copyFile(process.execPath, renamedHost);
|
||||
await writeFile(
|
||||
renamedHostFixture,
|
||||
`import {basename} from 'node:path';
|
||||
import {requireWindowsNativeAuthority} from ${JSON.stringify(loaderUrl)};
|
||||
const {inspection}=requireWindowsNativeAuthority();
|
||||
process.stdout.write(JSON.stringify({executable:basename(process.execPath),abiVersion:inspection.abiVersion}));\n`,
|
||||
);
|
||||
const renamedHostResult = execFileSync(renamedHost, [renamedHostFixture], {
|
||||
encoding: "utf8",
|
||||
timeout: 20_000,
|
||||
windowsHide: true,
|
||||
});
|
||||
assert.deepEqual(JSON.parse(renamedHostResult), {
|
||||
executable: "rea-renamed-node.exe",
|
||||
abiVersion: 1,
|
||||
});
|
||||
report.controls.renamedHostNodeApi = true;
|
||||
const sourceDirectory = join(workspace, "source");
|
||||
await mkdir(sourceDirectory);
|
||||
const source = join(sourceDirectory, "target.bin");
|
||||
@@ -139,47 +110,6 @@ process.stdout.write(JSON.stringify({executable:basename(process.execPath),abiVe
|
||||
])
|
||||
assert.throws(() => native.call("open", [rejected]));
|
||||
report.controls.ordinaryDriveSeparatorsAndRequestedIdentity = true;
|
||||
const driveRoot = parse(workspace).root;
|
||||
const runtimeParent = ghidraSessionRoot({
|
||||
base: join(driveRoot, ".Tmp", "scratch"),
|
||||
});
|
||||
assert.equal(runtimeParent, driveRoot);
|
||||
const driveRootRuntime = native.call("runtime_create", [
|
||||
runtimeParent,
|
||||
"rea-drive-root-",
|
||||
]);
|
||||
try {
|
||||
assert.equal(driveRootRuntime.path.startsWith(`${driveRoot}\\`), false);
|
||||
assert.equal(dirname(driveRootRuntime.path), driveRoot);
|
||||
const snapshot = await native.call("runtime_snapshot", [
|
||||
driveRootRuntime.handle,
|
||||
source,
|
||||
"probe.bin",
|
||||
]);
|
||||
const sourceBytes = await readFile(source);
|
||||
assert.equal(
|
||||
snapshot.sha256,
|
||||
createHash("sha256").update(sourceBytes).digest("hex"),
|
||||
);
|
||||
assert.equal(snapshot.source.requestedPath, source);
|
||||
const readback = native.call("runtime_open", [
|
||||
driveRootRuntime.handle,
|
||||
"probe.bin",
|
||||
]);
|
||||
try {
|
||||
assert.equal(readback.size, sourceBytes.byteLength);
|
||||
assert.deepEqual(
|
||||
native.call("read", [readback.handle, 0, 65536]),
|
||||
sourceBytes.subarray(0, 65536),
|
||||
);
|
||||
} finally {
|
||||
native.call("close", [readback.handle]);
|
||||
}
|
||||
} finally {
|
||||
native.call("runtime_close", [driveRootRuntime.handle]);
|
||||
}
|
||||
assert.equal(await exists(driveRootRuntime.path), false);
|
||||
report.controls.driveRootRuntimeParent = true;
|
||||
const snapshotOwner = WindowsPrivateRuntime.create(
|
||||
workspace.replace(/\\/u, "/"),
|
||||
"single-flight-",
|
||||
|
||||
@@ -56,7 +56,7 @@ try {
|
||||
process.env.REA_JAVASCRIPT_FIXTURE_TOOLS,
|
||||
);
|
||||
await client.connect(transport);
|
||||
for (const fixture of fixtures.files) {
|
||||
for (const fixture of [...fixtures.files, fixtures.codeSplit]) {
|
||||
for (const surface of ["cli", "stdio_mcp"]) {
|
||||
const output = join(root, `${fixture.name}-${surface}`);
|
||||
const flags =
|
||||
@@ -75,11 +75,18 @@ try {
|
||||
fixture.path,
|
||||
output,
|
||||
...flags,
|
||||
...(fixture.additional_paths ?? []).flatMap((path) => [
|
||||
"--additional-path",
|
||||
path,
|
||||
]),
|
||||
)
|
||||
: await call("recover_javascript_sources", {
|
||||
path: fixture.path,
|
||||
output_directory: output,
|
||||
...fixture.options,
|
||||
...(fixture.additional_paths
|
||||
? { additional_paths: fixture.additional_paths }
|
||||
: {}),
|
||||
});
|
||||
const result = javascriptRecoveryResultSchema.parse(
|
||||
evidence.normalized_result,
|
||||
@@ -97,6 +104,25 @@ try {
|
||||
analysis.normalized_result.statistics.parsed_javascript_files,
|
||||
result.modules.length,
|
||||
);
|
||||
const imports = analysis.normalized_result.graph.edges.filter(
|
||||
(edge) =>
|
||||
edge.relation === "imports" &&
|
||||
edge.properties.resolution_status === "resolved",
|
||||
);
|
||||
if (fixture.additional_paths) {
|
||||
assert.ok(
|
||||
imports.length >= 4,
|
||||
"Expected cross-chunk imports in the recovered application graph",
|
||||
);
|
||||
const recoveredPaths = new Set(
|
||||
result.modules.map((module) => module.reported_filename),
|
||||
);
|
||||
for (const edge of imports)
|
||||
assert.ok(
|
||||
recoveredPaths.has(edge.properties.resolved_path),
|
||||
`Expected an emitted import target: ${String(edge.properties.resolved_path)}`,
|
||||
);
|
||||
}
|
||||
const recoveredEntry =
|
||||
result.modules.find(
|
||||
(module) => module.reported_filename === "entry.js",
|
||||
@@ -104,7 +130,8 @@ try {
|
||||
assert.ok(recoveredEntry);
|
||||
const oracle =
|
||||
'await import((await import("node:url")).pathToFileURL(process.argv[1])); const f=globalThis.recoveryFixture; console.log(JSON.stringify([f.greet(null),f.greet("REA"),f.total([1,2,3])]));';
|
||||
for (const selected of fixture.options?.extraction_mode === "inspection"
|
||||
for (const selected of fixture.additional_paths ||
|
||||
fixture.options?.extraction_mode === "inspection"
|
||||
? []
|
||||
: [fixture.path, recoveredEntry.artifact.path]) {
|
||||
const execution = await run(
|
||||
@@ -125,7 +152,9 @@ try {
|
||||
maps: result.modules.filter((module) => module.source_map !== null)
|
||||
.length,
|
||||
finite_fixture_oracle:
|
||||
!fixture.additional_paths &&
|
||||
fixture.options?.extraction_mode !== "inspection",
|
||||
cross_chunk_imports: fixture.additional_paths ? imports.length : null,
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -157,9 +186,15 @@ async function assertRecoveredArtifacts(evidence, result, fixture) {
|
||||
assert.equal(result.status, "complete");
|
||||
assert.equal(result.reported_format, fixture.expectedFormat);
|
||||
assert.equal(result.modules.length, fixture.expectedModules);
|
||||
assert.equal(result.source.sha256, fingerprint(await readFile(fixture.path)));
|
||||
const sources = [result.source, ...(result.additional_sources ?? [])];
|
||||
assert.deepEqual(
|
||||
sources.map((source) => source.path),
|
||||
[fixture.path, ...(fixture.additional_paths ?? [])],
|
||||
);
|
||||
for (const source of sources)
|
||||
assert.equal(source.sha256, fingerprint(await readFile(source.path)));
|
||||
for (const file of [
|
||||
result.source.published_copy,
|
||||
...sources.map((source) => source.published_copy),
|
||||
result.report,
|
||||
result.provenance,
|
||||
result.manifest,
|
||||
@@ -173,16 +208,27 @@ async function assertRecoveredArtifacts(evidence, result, fixture) {
|
||||
assert.equal(fingerprint(bytes), file.sha256);
|
||||
}
|
||||
for (const module of result.modules) {
|
||||
assert.equal(module.provenance.original_sha256, result.source.sha256);
|
||||
assert.equal(module.provenance.reported_input, result.source.snapshot_path);
|
||||
for (const range of module.provenance.byte_ranges)
|
||||
assert.ok(range.start <= range.end && range.end <= result.source.bytes);
|
||||
const source = sources.find(
|
||||
(source) => source.snapshot_path === module.provenance.reported_input,
|
||||
);
|
||||
assert.ok(source, "Module must name a selected input snapshot");
|
||||
assert.equal(module.provenance.original_path, source.path);
|
||||
assert.equal(module.provenance.original_sha256, source.sha256);
|
||||
for (const range of [
|
||||
...module.provenance.byte_ranges,
|
||||
...module.provenance.context_byte_ranges,
|
||||
])
|
||||
assert.ok(range.start <= range.end && range.end <= source.bytes);
|
||||
if (module.source_map) {
|
||||
const map = JSON.parse(await readFile(module.source_map.path, "utf8"));
|
||||
for (const source of map.sources)
|
||||
assert.equal(
|
||||
resolve(dirname(module.source_map.path), source),
|
||||
result.source.published_copy.path,
|
||||
assert.ok(
|
||||
sources.some(
|
||||
(selected) =>
|
||||
selected.published_copy.path ===
|
||||
resolve(dirname(module.source_map.path), source),
|
||||
),
|
||||
"Map source must name a published input snapshot",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
+2
-2
@@ -8,13 +8,13 @@
|
||||
"url": "https://github.com/morluto/rea",
|
||||
"source": "github"
|
||||
},
|
||||
"version": "6.4.0",
|
||||
"version": "6.3.0",
|
||||
"packages": [
|
||||
{
|
||||
"registryType": "npm",
|
||||
"registryBaseUrl": "https://registry.npmjs.org",
|
||||
"identifier": "rea-agents",
|
||||
"version": "6.4.0",
|
||||
"version": "6.3.0",
|
||||
"runtimeHint": "npx",
|
||||
"transport": {
|
||||
"type": "stdio"
|
||||
|
||||
@@ -1,21 +1,17 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { lstat, opendir, readFile, realpath, stat } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
import { parseBinary } from "plist";
|
||||
import { z } from "zod";
|
||||
|
||||
import { preflightPropertyListDecode } from "../artifacts/apple/InterfaceBuilderDecodeBudget.js";
|
||||
import { BinaryTargetError } from "../domain/configurationErrors.js";
|
||||
import { isPathWithinRoot } from "../domain/localPath.js";
|
||||
import {
|
||||
omitPrototypeKeys,
|
||||
parseXmlPropertyList,
|
||||
} from "../domain/propertyListKeys.js";
|
||||
import { parseXmlPropertyList } from "../domain/propertyListKeys.js";
|
||||
import { decodeXmlPlistText } from "../domain/propertyListXmlText.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
|
||||
/** Decoded representation bound for one binary Info.plist. */
|
||||
const BINARY_PLIST_DECODE_BYTES = 16 * 1024 * 1024;
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
/** Where one app bundle layout keeps its Info.plist and program file. */
|
||||
interface BundleLayout {
|
||||
@@ -197,8 +193,9 @@ const resolveLayoutExecutable = async (
|
||||
try {
|
||||
name =
|
||||
plist.subarray(0, 6).toString("ascii") === "bplist"
|
||||
? await (fileSystem.decodeBinaryPlist?.(plistPath) ??
|
||||
parseBinaryPlistExecutable(plist))
|
||||
? await (fileSystem.decodeBinaryPlist ?? readBinaryPlistExecutable)(
|
||||
plistPath,
|
||||
)
|
||||
: parseXmlPlistExecutable(decodeXmlPlistText(plist));
|
||||
} catch (cause: unknown) {
|
||||
return err(
|
||||
@@ -251,10 +248,6 @@ const resolveLayoutExecutable = async (
|
||||
const parseXmlPlistExecutable = (plist: string): string => {
|
||||
// An unrelated `__proto__` entry must not make the bundle unreadable.
|
||||
const { value } = parseXmlPropertyList(plist);
|
||||
return executableName(value);
|
||||
};
|
||||
|
||||
const executableName = (value: unknown): string => {
|
||||
const executable = executableEntrySchema.safeParse(value);
|
||||
if (!executable.success) throw new Error("CFBundleExecutable is missing");
|
||||
return executable.data.CFBundleExecutable;
|
||||
@@ -262,13 +255,19 @@ const executableName = (value: unknown): string => {
|
||||
|
||||
const executableEntrySchema = z.looseObject({ CFBundleExecutable: z.string() });
|
||||
|
||||
/** Decode a binary plist in-process, so no host plist tool is required. */
|
||||
const parseBinaryPlistExecutable = (plist: Buffer): string => {
|
||||
preflightPropertyListDecode(plist, BINARY_PLIST_DECODE_BYTES);
|
||||
// `plist` assigns a `__proto__` key as the prototype; copying drops it, as
|
||||
// the XML path does.
|
||||
return executableName(omitPrototypeKeys(parseBinary(plist)).value);
|
||||
};
|
||||
const readBinaryPlistExecutable = async (plistPath: string): Promise<string> =>
|
||||
// `-n` strips only the newline plutil appends; it requires macOS 12+.
|
||||
(
|
||||
await execFileAsync("/usr/bin/plutil", [
|
||||
"-extract",
|
||||
"CFBundleExecutable",
|
||||
"raw",
|
||||
"-n",
|
||||
"-o",
|
||||
"-",
|
||||
plistPath,
|
||||
])
|
||||
).stdout;
|
||||
|
||||
const isSafeExecutableName = (name: string): boolean =>
|
||||
name.length > 0 &&
|
||||
|
||||
@@ -138,7 +138,7 @@ export const runManagedProviderExecution = async (
|
||||
signal?: AbortSignal,
|
||||
): Promise<Result<AnalysisExecution, AnalysisError>> =>
|
||||
withProcessCancellation(signal, async (operationSignal) => {
|
||||
const session = await dependencies.createManagedBinarySession();
|
||||
const session = dependencies.createManagedBinarySession();
|
||||
return withSessionCleanup(
|
||||
session,
|
||||
async () => {
|
||||
@@ -194,7 +194,7 @@ const runAnalysis = async (
|
||||
const { logger, signal, snapshotPath } = options;
|
||||
const config = dependencies.readConfiguration();
|
||||
if (!config.ok) return cliError(config.error);
|
||||
const session = await dependencies.createBinarySession(config.value, logger);
|
||||
const session = dependencies.createBinarySession(config.value, logger);
|
||||
return withSessionCleanup(
|
||||
session,
|
||||
async () => {
|
||||
|
||||
@@ -10,8 +10,6 @@ export interface DirectAnalysisDependencies {
|
||||
readonly createBinarySession: (
|
||||
config: AppConfig,
|
||||
logger: Logger,
|
||||
) => BinarySession | Promise<BinarySession>;
|
||||
readonly createManagedBinarySession: () =>
|
||||
| BinarySession
|
||||
| Promise<BinarySession>;
|
||||
) => BinarySession;
|
||||
readonly createManagedBinarySession: () => BinarySession;
|
||||
}
|
||||
|
||||
@@ -15,7 +15,7 @@ export const runSessionStatus = async (
|
||||
): Promise<JsonValue> => {
|
||||
const config = dependencies.readConfiguration();
|
||||
if (!config.ok) return { error: projectAnalysisError(config.error) };
|
||||
const session = await dependencies.createBinarySession(config.value, logger);
|
||||
const session = dependencies.createBinarySession(config.value, logger);
|
||||
try {
|
||||
return {
|
||||
...jsonObjectSchema.parse(session.status()),
|
||||
|
||||
@@ -1,35 +0,0 @@
|
||||
import type { EvidenceBundle } from "../../domain/evidenceBundle.js";
|
||||
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
|
||||
import { err, ok, type Result } from "../../domain/result.js";
|
||||
import {
|
||||
summarizeEvidenceBundle,
|
||||
type EvidenceBundleSummary,
|
||||
type EvidenceRecordFilters,
|
||||
} from "../../domain/evidenceRecordSummary.js";
|
||||
|
||||
/** Select complete bundle delivery or discovery metadata without analysis payloads. */
|
||||
export const inspectEvidenceBundle = (
|
||||
bundle: EvidenceBundle,
|
||||
options: {
|
||||
readonly detail: "complete" | "summary";
|
||||
readonly filters?: EvidenceRecordFilters | undefined;
|
||||
},
|
||||
): Result<EvidenceBundle | EvidenceBundleSummary, AnalysisInputError> => {
|
||||
if (options.detail === "complete" && options.filters !== undefined)
|
||||
return err(
|
||||
new AnalysisInputError("get_evidence_bundle", undefined, [
|
||||
{
|
||||
path: ["filters"],
|
||||
reason: "invalid_value",
|
||||
message:
|
||||
"Record filters require detail: summary; complete returns the canonical bundle.",
|
||||
expected: "detail: summary",
|
||||
},
|
||||
]),
|
||||
);
|
||||
return ok(
|
||||
options.detail === "summary"
|
||||
? summarizeEvidenceBundle(bundle, options.filters)
|
||||
: bundle,
|
||||
);
|
||||
};
|
||||
@@ -129,4 +129,50 @@ describe("JavaScript recovery workflow", () => {
|
||||
if (cancelled.ok) throw new Error("expected cancellation");
|
||||
expect(cancelled.error._tag).toBe("AnalysisCancelledError");
|
||||
});
|
||||
it("rejects additional_paths repeating path or each other", async () => {
|
||||
const service = new JavaScriptRecoveryService({
|
||||
recover: async () => {
|
||||
throw new Error("port must not execute");
|
||||
},
|
||||
});
|
||||
for (const additional_paths of [
|
||||
[artifact.path],
|
||||
["/tmp/dir/../bundle.js", "/tmp/bundle.js"],
|
||||
["/tmp/shared.js", "/tmp/shared.js"],
|
||||
]) {
|
||||
const result = await service.recover({
|
||||
...input,
|
||||
additional_paths,
|
||||
});
|
||||
if (result.ok) throw new Error("expected duplicate rejection");
|
||||
expect(result.error._tag).toBe("AnalysisInputError");
|
||||
expect(result.error).toMatchObject({
|
||||
issues: [{ path: ["additional_paths"], reason: "invalid_format" }],
|
||||
});
|
||||
}
|
||||
});
|
||||
it("rejects relative additional_paths before acquiring the port", async () => {
|
||||
let invoked = false;
|
||||
const service = new JavaScriptRecoveryService({
|
||||
recover: async () => {
|
||||
invoked = true;
|
||||
return err(
|
||||
new AnalysisCapabilityUnavailableError(
|
||||
"fixture",
|
||||
"recover_javascript_sources",
|
||||
"engine missing",
|
||||
),
|
||||
);
|
||||
},
|
||||
});
|
||||
const result = await service.recover({
|
||||
...input,
|
||||
additional_paths: ["relative-chunk.js"],
|
||||
});
|
||||
expect(invoked).toBe(false);
|
||||
if (result.ok) throw new Error("expected invalid path");
|
||||
expect(result.error).toMatchObject({
|
||||
issues: [{ path: ["additional_paths"], reason: "invalid_format" }],
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { isAbsolute } from "node:path";
|
||||
import { isAbsolute, resolve } from "node:path";
|
||||
|
||||
import {
|
||||
AnalysisCancelledError,
|
||||
@@ -11,7 +11,7 @@ import {
|
||||
javascriptRecoveryInputSchema,
|
||||
javascriptRecoveryResultSchema,
|
||||
} from "../../domain/javascript/javascriptRecovery.js";
|
||||
import { jsonObjectSchema } from "../../domain/jsonValue.js";
|
||||
import { jsonObjectSchema, jsonValueSchema } from "../../domain/jsonValue.js";
|
||||
import { analysisInputErrorFromIssues } from "../../domain/inputIssueProjection.js";
|
||||
import { err, ok, type Result } from "../../domain/result.js";
|
||||
import type { ExecutionOptions } from "../AnalysisProvider.js";
|
||||
@@ -44,17 +44,38 @@ export class JavaScriptRecoveryService {
|
||||
const invalidPaths = (["path", "output_directory"] as const).filter(
|
||||
(field) => !isAbsolute(parsed.data[field]),
|
||||
);
|
||||
if (invalidPaths.length > 0)
|
||||
const invalidAdditional = (parsed.data.additional_paths ?? []).filter(
|
||||
(additional) => !isAbsolute(additional),
|
||||
);
|
||||
if (invalidPaths.length > 0 || invalidAdditional.length > 0)
|
||||
return err(
|
||||
new AnalysisInputError(
|
||||
OPERATION,
|
||||
undefined,
|
||||
invalidPaths.map((field) => ({
|
||||
new AnalysisInputError(OPERATION, undefined, [
|
||||
...invalidPaths.map((field) => ({
|
||||
path: [field],
|
||||
reason: "invalid_format",
|
||||
reason: "invalid_format" as const,
|
||||
message: `${field} must be an absolute filesystem path on this host.`,
|
||||
})),
|
||||
),
|
||||
...invalidAdditional.map((additional) => ({
|
||||
path: ["additional_paths"],
|
||||
reason: "invalid_format" as const,
|
||||
message: `additional_paths entries must be absolute filesystem paths on this host: ${additional}`,
|
||||
})),
|
||||
]),
|
||||
);
|
||||
const selectedInputs = new Set(
|
||||
[parsed.data.path, ...(parsed.data.additional_paths ?? [])].map((path) =>
|
||||
resolve(path),
|
||||
),
|
||||
);
|
||||
if (selectedInputs.size !== 1 + (parsed.data.additional_paths ?? []).length)
|
||||
return err(
|
||||
new AnalysisInputError(OPERATION, undefined, [
|
||||
{
|
||||
path: ["additional_paths"],
|
||||
reason: "invalid_format",
|
||||
message: "additional_paths must not repeat path or each other.",
|
||||
},
|
||||
]),
|
||||
);
|
||||
const execution = await this.provider.recover(parsed.data, options);
|
||||
if (!execution.ok) return execution;
|
||||
@@ -70,6 +91,11 @@ export class JavaScriptRecoveryService {
|
||||
);
|
||||
if (
|
||||
result.data.source.path !== parsed.data.path ||
|
||||
(result.data.additional_sources ?? []).length !==
|
||||
(parsed.data.additional_paths ?? []).length ||
|
||||
(parsed.data.additional_paths ?? []).some(
|
||||
(path, index) => result.data.additional_sources?.[index]?.path !== path,
|
||||
) ||
|
||||
execution.value.subject.path !== result.data.source.path ||
|
||||
execution.value.subject.sha256 !== result.data.source.sha256 ||
|
||||
execution.value.provider.id !== result.data.engine.id ||
|
||||
@@ -85,7 +111,7 @@ export class JavaScriptRecoveryService {
|
||||
createEvidence(execution.value.subject, execution.value.provider, {
|
||||
operation: OPERATION,
|
||||
parameters: jsonObjectSchema.parse(parsed.data),
|
||||
result: result.data,
|
||||
result: jsonValueSchema.parse(result.data),
|
||||
rawResult: execution.value.rawResult,
|
||||
locations: execution.value.locations,
|
||||
limitations: execution.value.limitations,
|
||||
|
||||
@@ -43,7 +43,6 @@ import {
|
||||
type WebScreenshotDiff,
|
||||
} from "../domain/webScreenshot.js";
|
||||
import { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import { AnalysisInputError } from "../domain/analysisErrorCore.js";
|
||||
import { BrowserObservationError } from "../domain/browserObservationError.js";
|
||||
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
|
||||
import { type BrowserObservationOperation } from "../domain/browserObservationErrors.js";
|
||||
@@ -315,12 +314,6 @@ export class CdpBrowserProvider implements BrowserObservationPort {
|
||||
try {
|
||||
return ok(webScreenshotDiffSchema.parse(comparePngScreenshots(input)));
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof TypeError)
|
||||
return err(
|
||||
new AnalysisInputError("compare_web_screenshots", { cause }, [
|
||||
{ path: [], reason: "invalid_format", message: cause.message },
|
||||
]),
|
||||
);
|
||||
return err(providerError(cause, "compare_web_screenshots"));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -606,402 +606,3 @@ describe("SVG xlink:href context", () => {
|
||||
expect(dangling.urls).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
const SVG_NAMESPACE = "http://www.w3.org/2000/svg";
|
||||
const XLINK_NAMESPACE = "http://www.w3.org/1999/xlink";
|
||||
const XHTML_NAMESPACE = "http://www.w3.org/1999/xhtml";
|
||||
|
||||
const xhtmlRoot = (
|
||||
attributes: readonly (readonly [string, string])[] = [
|
||||
["xmlns", XHTML_NAMESPACE],
|
||||
["xmlns:svg", SVG_NAMESPACE],
|
||||
["xmlns:xlink", XLINK_NAMESPACE],
|
||||
],
|
||||
): SvgSnapshotNode => ({
|
||||
name: "html",
|
||||
parent: 0,
|
||||
attributes,
|
||||
});
|
||||
|
||||
const approvedHref = (
|
||||
nodeIndex: number,
|
||||
url: string,
|
||||
): {
|
||||
readonly node_index: number;
|
||||
readonly attribute: "href";
|
||||
readonly url: string;
|
||||
readonly destination_scope: "approved";
|
||||
} => ({
|
||||
node_index: nodeIndex,
|
||||
attribute: "href",
|
||||
url,
|
||||
destination_scope: "approved",
|
||||
});
|
||||
|
||||
describe("XML SVG roots and namespace scope restoration", () => {
|
||||
it.each(["svg", "document"])(
|
||||
"resolves namespaces under an unprefixed XML %s root",
|
||||
(name) => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
{
|
||||
name,
|
||||
parent: 0,
|
||||
attributes: [
|
||||
["xmlns", SVG_NAMESPACE],
|
||||
["xmlns:p", XLINK_NAMESPACE],
|
||||
],
|
||||
},
|
||||
{ name: "a", parent: 1, attributes: [["p:href", "/linked"]] },
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([approvedHref(2, `${origin}/linked`)]);
|
||||
},
|
||||
);
|
||||
|
||||
it.each([2, 3, 99])(
|
||||
"rejects self-parent, cyclic and dangling XML ancestry (%s)",
|
||||
(parent) => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot(),
|
||||
{
|
||||
name: "svg:a",
|
||||
parent,
|
||||
attributes: [
|
||||
["xmlns:svg", SVG_NAMESPACE],
|
||||
["xmlns:p", XLINK_NAMESPACE],
|
||||
["p:href", "/invalid"],
|
||||
],
|
||||
},
|
||||
{ name: "svg:g", parent: 2 },
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([]);
|
||||
},
|
||||
);
|
||||
|
||||
it("restores namespace declarations and undeclarations before visiting siblings", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot(),
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [
|
||||
["xmlns:xlink", ""],
|
||||
["xlink:href", "/undeclared"],
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [["xlink:href", "/inherited"]],
|
||||
},
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [
|
||||
["xmlns:svg", "other"],
|
||||
["xlink:href", "/rebound"],
|
||||
],
|
||||
},
|
||||
{ name: "svg:a", parent: 1, attributes: [["xlink:href", "/restored"]] },
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([
|
||||
approvedHref(3, `${origin}/inherited`),
|
||||
approvedHref(5, `${origin}/restored`),
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("XHTML and XML SVG namespace prefixes", () => {
|
||||
it("reports prefixed svg:a, svg:image, and an svg:a with no svg parent", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot(),
|
||||
{ name: "svg:svg", parent: 1 },
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 2,
|
||||
attributes: [
|
||||
["xlink:href", "/xl-pa1"],
|
||||
["xlink:href", "/later"],
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "svg:image",
|
||||
parent: 2,
|
||||
attributes: [["xlink:href", "guide"]],
|
||||
},
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [["xlink:href", "/xl-orphan"]],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(result.nodes[3]?.node_name).toBe("svg:a");
|
||||
expect(result.nodes[3]?.attribute_names).toEqual(["xlink:href"]);
|
||||
expect(result.urls).toEqual([
|
||||
approvedHref(3, `${origin}/xl-pa1`),
|
||||
approvedHref(4, `${origin}/screens/assets/guide`),
|
||||
approvedHref(5, `${origin}/xl-orphan`),
|
||||
]);
|
||||
});
|
||||
|
||||
it("reports svg:a under svg:foreignObject and skips XHTML anchors there", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot([
|
||||
["xmlns", XHTML_NAMESPACE],
|
||||
["xmlns:svg", SVG_NAMESPACE],
|
||||
["xmlns:xlink", XLINK_NAMESPACE],
|
||||
["xmlns:h", XHTML_NAMESPACE],
|
||||
]),
|
||||
{ name: "svg:svg", parent: 1 },
|
||||
{ name: "svg:foreignObject", parent: 2 },
|
||||
{
|
||||
name: "a",
|
||||
parent: 3,
|
||||
attributes: [["xlink:href", "/xl-html-fo"]],
|
||||
},
|
||||
{
|
||||
name: "h:a",
|
||||
parent: 3,
|
||||
attributes: [["xlink:href", "/xl-h"]],
|
||||
},
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 3,
|
||||
attributes: [["xlink:href", "/xl-fo-svg"]],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(urlFor(result, 4)).toBeUndefined();
|
||||
expect(urlFor(result, 5)).toBeUndefined();
|
||||
expect(result.urls).toEqual([approvedHref(6, `${origin}/xl-fo-svg`)]);
|
||||
});
|
||||
|
||||
it("reads XLink by namespace, including a prefix other than xlink", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot([["xmlns", XHTML_NAMESPACE]]),
|
||||
{
|
||||
name: "svg",
|
||||
parent: 1,
|
||||
attributes: [
|
||||
["xmlns", SVG_NAMESPACE],
|
||||
["xmlns:xlink", "http://example.com/not-xlink"],
|
||||
["xmlns:x", XLINK_NAMESPACE],
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "a",
|
||||
parent: 2,
|
||||
attributes: [
|
||||
["xlink:href", "/not-xlink"],
|
||||
["x:href", "/xl-custom"],
|
||||
["x:href", "/later"],
|
||||
],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(result.nodes[3]?.attribute_names).toEqual(["xlink:href", "x:href"]);
|
||||
expect(result.urls).toEqual([approvedHref(3, `${origin}/xl-custom`)]);
|
||||
});
|
||||
|
||||
it("keeps plain href ahead of a namespaced XLink href, including an empty href", () => {
|
||||
const baseUrl = `${origin}/screens/assets/`;
|
||||
const anchor = (
|
||||
parent: number,
|
||||
attributes: readonly (readonly [string, string])[],
|
||||
): SvgSnapshotNode => ({
|
||||
name: "svg:a",
|
||||
parent,
|
||||
attributes,
|
||||
});
|
||||
const result = captureSvg(
|
||||
svgDocument(
|
||||
[
|
||||
xhtmlRoot(),
|
||||
{ name: "svg:svg", parent: 1 },
|
||||
anchor(2, [
|
||||
["xlink:href", "/xlink-loses"],
|
||||
["href", "/href-wins"],
|
||||
]),
|
||||
anchor(2, [
|
||||
["href", "/href-wins"],
|
||||
["xlink:href", "/xlink-loses"],
|
||||
]),
|
||||
anchor(2, [
|
||||
["href", ""],
|
||||
["xlink:href", "/xlink-should-lose"],
|
||||
]),
|
||||
],
|
||||
baseUrl,
|
||||
),
|
||||
);
|
||||
expect(result.urls).toEqual([
|
||||
approvedHref(3, `${origin}/href-wins`),
|
||||
approvedHref(4, `${origin}/href-wins`),
|
||||
approvedHref(5, baseUrl),
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("XHTML SVG exclusions, HTML documents, and prefix scope", () => {
|
||||
it("classifies a prefixed SVG xlink:href outside the allowed origins as outside_policy", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot(),
|
||||
{ name: "svg:svg", parent: 1 },
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 2,
|
||||
attributes: [["xlink:href", "https://cdn.example.test/out"]],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([
|
||||
{
|
||||
node_index: 3,
|
||||
attribute: "href",
|
||||
url: null,
|
||||
destination_scope: "outside_policy",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("does not treat bogus:svg in another namespace as SVG", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
xhtmlRoot([
|
||||
["xmlns", XHTML_NAMESPACE],
|
||||
["xmlns:svg", SVG_NAMESPACE],
|
||||
["xmlns:xlink", XLINK_NAMESPACE],
|
||||
["xmlns:bogus", "http://example.com/not-svg"],
|
||||
]),
|
||||
{
|
||||
name: "bogus:svg",
|
||||
parent: 1,
|
||||
attributes: [["xlink:href", "/xl-bogus"]],
|
||||
},
|
||||
{
|
||||
name: "bogus:a",
|
||||
parent: 2,
|
||||
attributes: [["xlink:href", "/xl-bogus-a"]],
|
||||
},
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [["xlink:href", "/xl-kept"]],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(result.nodes[2]?.node_name).toBe("bogus:svg");
|
||||
expect(result.urls).toEqual([approvedHref(4, `${origin}/xl-kept`)]);
|
||||
});
|
||||
|
||||
it("leaves HTML documents on literal svg and xlink:href names", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
{ name: "HTML", parent: 0 },
|
||||
{
|
||||
name: "svg",
|
||||
parent: 1,
|
||||
attributes: [
|
||||
["xmlns", SVG_NAMESPACE],
|
||||
["xmlns:x", XLINK_NAMESPACE],
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "a",
|
||||
parent: 2,
|
||||
attributes: [["xlink:href", "/xl-literal"]],
|
||||
},
|
||||
{
|
||||
name: "a",
|
||||
parent: 2,
|
||||
attributes: [["x:href", "/xl-custom-html"]],
|
||||
},
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [
|
||||
["xmlns:svg", SVG_NAMESPACE],
|
||||
["xmlns:xlink", XLINK_NAMESPACE],
|
||||
["xlink:href", "/xl-prefixed-html"],
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "A",
|
||||
parent: 1,
|
||||
attributes: [["xlink:href", "/xl-html-anchor"]],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([approvedHref(3, `${origin}/xl-literal`)]);
|
||||
});
|
||||
|
||||
it("applies an element's own xmlns declarations to its name and href", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
{ name: "html", parent: 0, attributes: [["xmlns", XHTML_NAMESPACE]] },
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 1,
|
||||
attributes: [
|
||||
["xmlns:svg", SVG_NAMESPACE],
|
||||
["xmlns:xlink", XLINK_NAMESPACE],
|
||||
["xlink:href", "/xl-self"],
|
||||
],
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([approvedHref(2, `${origin}/xl-self`)]);
|
||||
});
|
||||
|
||||
it("resolves a prefix declared on a later ancestor", () => {
|
||||
const result = captureSvg(
|
||||
svgDocument([
|
||||
{
|
||||
name: "svg:a",
|
||||
parent: 2,
|
||||
attributes: [["xlink:href", "/forward"]],
|
||||
},
|
||||
{ name: "svg:svg", parent: 3 },
|
||||
xhtmlRoot(),
|
||||
]),
|
||||
);
|
||||
expect(result.urls).toEqual([approvedHref(1, `${origin}/forward`)]);
|
||||
});
|
||||
|
||||
it("resolves a deep chain with distinct namespace declarations without quadratic expansion", () => {
|
||||
const depth = 12_000;
|
||||
const elements: SvgSnapshotNode[] = [
|
||||
xhtmlRoot(),
|
||||
{ name: "svg:svg", parent: 1 },
|
||||
];
|
||||
let parent = 2;
|
||||
for (let index = 0; index < depth; index += 1) {
|
||||
elements.push({
|
||||
name: "svg:g",
|
||||
parent,
|
||||
attributes: [[`xmlns:p${index}`, `urn:prefix:${index}`]],
|
||||
});
|
||||
parent = elements.length;
|
||||
}
|
||||
elements.push({
|
||||
name: "svg:a",
|
||||
parent,
|
||||
attributes: [["xlink:href", "/deep"]],
|
||||
});
|
||||
const snapshot = svgDocument(elements);
|
||||
const result = captureSvg(snapshot);
|
||||
expect(result.urls).toEqual([
|
||||
approvedHref(elements.length, `${origin}/deep`),
|
||||
]);
|
||||
}, 30_000);
|
||||
});
|
||||
|
||||
@@ -11,7 +11,6 @@ import {
|
||||
} from "./CdpCaptureValues.js";
|
||||
import { exclusionReasonForUrl } from "./CdpCaptureEventHelpers.js";
|
||||
import type { CdpCaptureCompleteness } from "./CdpCaptureCompleteness.js";
|
||||
import { createDocumentSvgAncestry, domSvgLink } from "./CdpCaptureDomSvg.js";
|
||||
|
||||
export type CapturedResource = Omit<
|
||||
WebPageInspection["resources"][number],
|
||||
@@ -172,20 +171,24 @@ export const captureDom = (
|
||||
? documentNodes.attributes
|
||||
: [];
|
||||
const baseIndex = nodes.length;
|
||||
const ancestry = createDocumentSvgAncestry({
|
||||
nodeTypes,
|
||||
nodeNames,
|
||||
parents,
|
||||
strings,
|
||||
attributes,
|
||||
});
|
||||
const svgContexts = new Map<number, boolean>();
|
||||
total += nodeTypes.length;
|
||||
for (let index = 0; index < nodeTypes.length; index += 1) {
|
||||
const attributeIndexes = numberArray(attributes[index]);
|
||||
const parent = Math.trunc(parents[index] ?? -1);
|
||||
const nodeIndex = nodes.length;
|
||||
const nodeName = indexedString(strings, nodeNames[index]);
|
||||
const attributeNames = uniqueAttributeNames(attributeIndexes, strings);
|
||||
const rawAttributeNames = attributeIndexes
|
||||
.filter((_value, attributeIndex) => attributeIndex % 2 === 0)
|
||||
.map((value) => indexedString(strings, value));
|
||||
const seenAttributeNames = new Set<string>();
|
||||
const attributeNames: string[] = [];
|
||||
for (const name of rawAttributeNames) {
|
||||
if (!seenAttributeNames.has(name)) {
|
||||
seenAttributeNames.add(name);
|
||||
attributeNames.push(name);
|
||||
}
|
||||
}
|
||||
nodes.push({
|
||||
index: nodeIndex,
|
||||
parent_index: parent < 0 ? -1 : baseIndex + parent,
|
||||
@@ -203,7 +206,13 @@ export const captureDom = (
|
||||
nodeIndex,
|
||||
nodeName,
|
||||
allowedOrigins,
|
||||
...domSvgLink(ancestry, index),
|
||||
svgElement: elementInSvgContext(
|
||||
index,
|
||||
nodeNames,
|
||||
parents,
|
||||
strings,
|
||||
svgContexts,
|
||||
),
|
||||
});
|
||||
for (const url of metadata.urls) {
|
||||
urls.push(url);
|
||||
@@ -358,21 +367,6 @@ const indexedString = (strings: readonly string[], index: unknown): string => {
|
||||
return integer === undefined ? "" : (strings[Math.trunc(integer)] ?? "");
|
||||
};
|
||||
|
||||
const uniqueAttributeNames = (
|
||||
attributeIndexes: readonly number[],
|
||||
strings: readonly string[],
|
||||
): string[] => {
|
||||
const names: string[] = [];
|
||||
const seen = new Set<string>();
|
||||
for (let index = 0; index < attributeIndexes.length; index += 2) {
|
||||
const name = indexedString(strings, attributeIndexes[index]);
|
||||
if (seen.has(name)) continue;
|
||||
seen.add(name);
|
||||
names.push(name);
|
||||
}
|
||||
return names;
|
||||
};
|
||||
|
||||
const numberArray = (value: unknown): readonly number[] =>
|
||||
Array.isArray(value)
|
||||
? value.flatMap((item) => {
|
||||
@@ -390,22 +384,41 @@ interface DomMetadataOptions {
|
||||
readonly nodeName: string;
|
||||
readonly allowedOrigins: ReadonlySet<string>;
|
||||
readonly svgElement: boolean;
|
||||
readonly xmlDocument: boolean;
|
||||
readonly xlinkHref: string | undefined;
|
||||
}
|
||||
|
||||
// Chrome reads an SVG XLink href only when href itself is absent, and an
|
||||
// empty href still wins. HTML keeps the literal attribute name `xlink:href`.
|
||||
// XML uses any prefix bound to the XLink namespace. Both are stored as href.
|
||||
const hrefAttributeValue = (
|
||||
pairs: ReadonlyMap<string, string>,
|
||||
attribute: string,
|
||||
options: DomMetadataOptions,
|
||||
): string | undefined => {
|
||||
const declared = pairs.get(attribute);
|
||||
if (declared !== undefined) return declared;
|
||||
if (attribute !== "href" || !options.svgElement) return undefined;
|
||||
return options.xmlDocument ? options.xlinkHref : pairs.get("xlink:href");
|
||||
// DOMSnapshot carries no namespace URI. Chrome's HTML parser records SVG
|
||||
// local names (`svg`, `foreignObject`). Descendants of foreignObject are HTML
|
||||
// until a nested `svg`.
|
||||
const elementInSvgContext = (
|
||||
index: number,
|
||||
nodeNames: readonly number[],
|
||||
parents: readonly number[],
|
||||
strings: readonly string[],
|
||||
contexts: Map<number, boolean>,
|
||||
): boolean => {
|
||||
let current = index;
|
||||
const seen = new Set<number>();
|
||||
let inSvg = false;
|
||||
while (current >= 0 && current < nodeNames.length && !seen.has(current)) {
|
||||
const cached = contexts.get(current);
|
||||
if (cached !== undefined) {
|
||||
inSvg = cached;
|
||||
break;
|
||||
}
|
||||
seen.add(current);
|
||||
if (indexedString(strings, nodeNames[current]) === "svg") {
|
||||
inSvg = true;
|
||||
break;
|
||||
}
|
||||
const parent = Math.trunc(parents[current] ?? -1);
|
||||
if (parent < 0 || parent === current) break;
|
||||
if (indexedString(strings, nodeNames[parent]) === "foreignObject") break;
|
||||
current = parent;
|
||||
}
|
||||
// Cache the entire examined chain, including forward references and cycles,
|
||||
// so shared ancestors are examined once per document.
|
||||
for (const node of seen) contexts.set(node, inSvg);
|
||||
return inSvg;
|
||||
};
|
||||
|
||||
const domMetadata = (
|
||||
@@ -422,6 +435,7 @@ const domMetadata = (
|
||||
nodeIndex,
|
||||
nodeName,
|
||||
allowedOrigins,
|
||||
svgElement,
|
||||
} = options;
|
||||
const pairs = new Map<string, string>();
|
||||
for (let index = 0; index + 1 < attributes.length; index += 2) {
|
||||
@@ -433,7 +447,13 @@ const domMetadata = (
|
||||
}
|
||||
const urls: WebPageInspection["metadata"]["dom_urls"] = [];
|
||||
for (const attribute of domUrlAttributes) {
|
||||
const value = hrefAttributeValue(pairs, attribute, options);
|
||||
// Chrome reads SVG xlink:href only when href itself is absent, and an
|
||||
// empty href still wins. Report the fallback as href.
|
||||
const value =
|
||||
pairs.get(attribute) ??
|
||||
(attribute === "href" && svgElement
|
||||
? pairs.get("xlink:href")
|
||||
: undefined);
|
||||
if (value === undefined) continue;
|
||||
const tagName = nodeName.toLowerCase();
|
||||
// Chrome strips HTML whitespace for FORM.action, while submit controls
|
||||
|
||||
@@ -1,275 +0,0 @@
|
||||
import { numberValue } from "./CdpCaptureValues.js";
|
||||
|
||||
const SVG_NAMESPACE = "http://www.w3.org/2000/svg";
|
||||
const XLINK_NAMESPACE = "http://www.w3.org/1999/xlink";
|
||||
|
||||
interface DomSnapshotNodes {
|
||||
readonly nodeTypes: readonly number[];
|
||||
readonly nodeNames: readonly number[];
|
||||
readonly parents: readonly number[];
|
||||
readonly strings: readonly string[];
|
||||
readonly attributes: readonly unknown[];
|
||||
}
|
||||
|
||||
interface XmlSvgLink {
|
||||
readonly svgElement: boolean;
|
||||
readonly xlinkHref: string | undefined;
|
||||
}
|
||||
|
||||
interface DocumentSvgAncestry {
|
||||
readonly xml: boolean;
|
||||
readonly nodes: DomSnapshotNodes;
|
||||
readonly svgContexts: Map<number, boolean>;
|
||||
readonly xmlLinks: Map<number, XmlSvgLink>;
|
||||
}
|
||||
|
||||
interface DomSvgLink {
|
||||
readonly svgElement: boolean;
|
||||
readonly xmlDocument: boolean;
|
||||
readonly xlinkHref: string | undefined;
|
||||
}
|
||||
|
||||
const UNRESOLVED: XmlSvgLink = {
|
||||
svgElement: false,
|
||||
xlinkHref: undefined,
|
||||
};
|
||||
|
||||
const stringAt = (strings: readonly string[], index: unknown): string => {
|
||||
const integer = numberValue(index);
|
||||
return integer === undefined ? "" : (strings[Math.trunc(integer)] ?? "");
|
||||
};
|
||||
|
||||
const numberArray = (value: unknown): readonly number[] =>
|
||||
Array.isArray(value)
|
||||
? value.flatMap((item) => {
|
||||
const number = numberValue(item);
|
||||
return number === undefined ? [] : [number];
|
||||
})
|
||||
: [];
|
||||
|
||||
// DOMSnapshot has no namespace URI. HTML documents use uppercase `HTML` and
|
||||
// the parser's local names. XML documents preserve prefixed names and xmlns
|
||||
// declarations, including those on standalone SVG roots.
|
||||
const xmlDocument = (nodes: DomSnapshotNodes): boolean => {
|
||||
const documentIndex = nodes.nodeTypes.findIndex(
|
||||
(nodeType) => Math.trunc(nodeType) === 9,
|
||||
);
|
||||
if (documentIndex < 0) return false;
|
||||
for (let index = 0; index < nodes.nodeTypes.length; index += 1) {
|
||||
if (Math.trunc(nodes.nodeTypes[index] ?? 0) !== 1) continue;
|
||||
if (Math.trunc(nodes.parents[index] ?? -1) !== documentIndex) continue;
|
||||
const name = stringAt(nodes.strings, nodes.nodeNames[index]);
|
||||
if (name === "HTML") return false;
|
||||
if (name === "html" || name.includes(":")) return true;
|
||||
// Explicit namespaces also identify standalone SVG and other XML roots.
|
||||
// Namespace-free synthetic snapshots retain the legacy HTML behavior.
|
||||
const attributes = numberArray(nodes.attributes[index]);
|
||||
return attributes.some(
|
||||
(value, offset) =>
|
||||
offset % 2 === 0 &&
|
||||
xmlnsPrefix(stringAt(nodes.strings, value)) !== undefined,
|
||||
);
|
||||
}
|
||||
return false;
|
||||
};
|
||||
|
||||
export const createDocumentSvgAncestry = (
|
||||
nodes: DomSnapshotNodes,
|
||||
): DocumentSvgAncestry => {
|
||||
const xml = xmlDocument(nodes);
|
||||
return {
|
||||
xml,
|
||||
nodes,
|
||||
svgContexts: new Map(),
|
||||
xmlLinks: xml ? resolveXmlElements(nodes) : new Map(),
|
||||
};
|
||||
};
|
||||
|
||||
export const domSvgLink = (
|
||||
ancestry: DocumentSvgAncestry,
|
||||
index: number,
|
||||
): DomSvgLink => {
|
||||
if (!ancestry.xml) {
|
||||
return {
|
||||
svgElement: elementInSvgContext(
|
||||
index,
|
||||
ancestry.nodes,
|
||||
ancestry.svgContexts,
|
||||
),
|
||||
xmlDocument: false,
|
||||
xlinkHref: undefined,
|
||||
};
|
||||
}
|
||||
const resolved = ancestry.xmlLinks.get(index) ?? UNRESOLVED;
|
||||
return {
|
||||
svgElement: resolved.svgElement,
|
||||
xmlDocument: true,
|
||||
xlinkHref: resolved.xlinkHref,
|
||||
};
|
||||
};
|
||||
|
||||
// Chrome's HTML parser records SVG local names (`svg`, `foreignObject`).
|
||||
// Descendants of foreignObject are HTML until a nested `svg`.
|
||||
const elementInSvgContext = (
|
||||
index: number,
|
||||
nodes: DomSnapshotNodes,
|
||||
contexts: Map<number, boolean>,
|
||||
): boolean => {
|
||||
let current = index;
|
||||
const seen = new Set<number>();
|
||||
let inSvg = false;
|
||||
while (
|
||||
current >= 0 &&
|
||||
current < nodes.nodeNames.length &&
|
||||
!seen.has(current)
|
||||
) {
|
||||
const cached = contexts.get(current);
|
||||
if (cached !== undefined) {
|
||||
inSvg = cached;
|
||||
break;
|
||||
}
|
||||
seen.add(current);
|
||||
if (stringAt(nodes.strings, nodes.nodeNames[current]) === "svg") {
|
||||
inSvg = true;
|
||||
break;
|
||||
}
|
||||
const parent = Math.trunc(nodes.parents[current] ?? -1);
|
||||
if (parent < 0 || parent === current) break;
|
||||
if (stringAt(nodes.strings, nodes.nodeNames[parent]) === "foreignObject")
|
||||
break;
|
||||
current = parent;
|
||||
}
|
||||
// Cache the entire examined chain, including forward references and cycles,
|
||||
// so shared ancestors are examined once per document.
|
||||
for (const node of seen) contexts.set(node, inSvg);
|
||||
return inSvg;
|
||||
};
|
||||
|
||||
interface NamespaceRestore {
|
||||
readonly prefix: string;
|
||||
readonly previous: string | undefined;
|
||||
}
|
||||
|
||||
type XmlWalkStep =
|
||||
| { readonly node: number }
|
||||
| { readonly restore: readonly NamespaceRestore[] };
|
||||
|
||||
// Walk parent-before-child with one mutable namespace environment. Restore
|
||||
// declarations on exit so siblings retain their own scope. Only declarations
|
||||
// on the active path are retained, rather than cumulative maps at every node.
|
||||
const resolveXmlElements = (
|
||||
nodes: DomSnapshotNodes,
|
||||
): Map<number, XmlSvgLink> => {
|
||||
const children = new Map<number, number[]>();
|
||||
const pending: XmlWalkStep[] = [];
|
||||
for (let node = 0; node < nodes.nodeNames.length; node += 1) {
|
||||
const parent = Math.trunc(nodes.parents[node] ?? -1);
|
||||
if (parent < 0) pending.push({ node });
|
||||
else if (parent < nodes.nodeNames.length && parent !== node) {
|
||||
const siblings = children.get(parent);
|
||||
if (siblings === undefined) children.set(parent, [node]);
|
||||
else siblings.push(node);
|
||||
}
|
||||
}
|
||||
const bindings = new Map<string, string>();
|
||||
const resolved = new Map<number, XmlSvgLink>();
|
||||
while (pending.length > 0) {
|
||||
const step = pending.pop();
|
||||
if (step === undefined) break;
|
||||
if ("restore" in step) {
|
||||
for (const { prefix, previous } of step.restore) {
|
||||
if (previous === undefined) bindings.delete(prefix);
|
||||
else bindings.set(prefix, previous);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
const { node } = step;
|
||||
const attributes = numberArray(nodes.attributes[node]);
|
||||
const restore = applyDeclarations(bindings, attributes, nodes.strings);
|
||||
const svgElement =
|
||||
Math.trunc(nodes.nodeTypes[node] ?? 0) === 1 &&
|
||||
elementNamespace(
|
||||
stringAt(nodes.strings, nodes.nodeNames[node]),
|
||||
bindings,
|
||||
) === SVG_NAMESPACE;
|
||||
resolved.set(node, {
|
||||
svgElement,
|
||||
xlinkHref: svgElement
|
||||
? xlinkHrefValue(attributes, nodes.strings, bindings)
|
||||
: undefined,
|
||||
});
|
||||
pending.push({ restore });
|
||||
for (const child of children.get(node) ?? []) pending.push({ node: child });
|
||||
}
|
||||
// Cycles (including self-parents) and dangling chains cannot reach a root,
|
||||
// so remain unresolved instead of deriving URLs from an invalid ancestry.
|
||||
return resolved;
|
||||
};
|
||||
|
||||
const applyDeclarations = (
|
||||
bindings: Map<string, string>,
|
||||
attributes: readonly number[],
|
||||
strings: readonly string[],
|
||||
): readonly NamespaceRestore[] => {
|
||||
const restore: NamespaceRestore[] = [];
|
||||
const seen = new Set<string>();
|
||||
for (let index = 0; index + 1 < attributes.length; index += 2) {
|
||||
const prefix = xmlnsPrefix(stringAt(strings, attributes[index]));
|
||||
if (prefix === undefined || seen.has(prefix)) continue;
|
||||
seen.add(prefix);
|
||||
restore.push({ prefix, previous: bindings.get(prefix) });
|
||||
const value = stringAt(strings, attributes[index + 1]);
|
||||
if (value.length === 0) bindings.delete(prefix);
|
||||
else bindings.set(prefix, value);
|
||||
}
|
||||
return restore;
|
||||
};
|
||||
|
||||
const xmlnsPrefix = (name: string): string | undefined => {
|
||||
if (name === "xmlns") return "";
|
||||
if (!name.startsWith("xmlns:")) return undefined;
|
||||
const prefix = name.slice("xmlns:".length);
|
||||
if (prefix.length === 0 || prefix === "xmlns" || prefix.includes(":"))
|
||||
return undefined;
|
||||
return prefix;
|
||||
};
|
||||
|
||||
const elementNamespace = (
|
||||
nodeName: string,
|
||||
bindings: ReadonlyMap<string, string>,
|
||||
): string => {
|
||||
const colon = nodeName.indexOf(":");
|
||||
if (colon < 0) return bindings.get("") ?? "";
|
||||
const prefix = nodeName.slice(0, colon);
|
||||
const local = nodeName.slice(colon + 1);
|
||||
if (
|
||||
prefix.length === 0 ||
|
||||
local.length === 0 ||
|
||||
local.includes(":") ||
|
||||
prefix === "xmlns"
|
||||
)
|
||||
return "";
|
||||
return bindings.get(prefix) ?? "";
|
||||
};
|
||||
|
||||
// First prefixed href bound to the XLink namespace. Duplicate names keep the
|
||||
// earlier value. Plain href is applied by the caller and wins even when empty.
|
||||
const xlinkHrefValue = (
|
||||
attributes: readonly number[],
|
||||
strings: readonly string[],
|
||||
bindings: ReadonlyMap<string, string>,
|
||||
): string | undefined => {
|
||||
const seen = new Set<string>();
|
||||
for (let index = 0; index + 1 < attributes.length; index += 2) {
|
||||
const name = stringAt(strings, attributes[index]);
|
||||
if (seen.has(name)) continue;
|
||||
seen.add(name);
|
||||
const colon = name.indexOf(":");
|
||||
if (colon <= 0) continue;
|
||||
const prefix = name.slice(0, colon);
|
||||
const local = name.slice(colon + 1);
|
||||
if (local !== "href" || bindings.get(prefix) !== XLINK_NAMESPACE) continue;
|
||||
return stringAt(strings, attributes[index + 1]);
|
||||
}
|
||||
return undefined;
|
||||
};
|
||||
@@ -3,10 +3,6 @@ import { crc32, deflateSync } from "node:zlib";
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { comparePngScreenshots } from "./PngVisualDiff.js";
|
||||
import {
|
||||
AnalysisResourceConstraintError,
|
||||
AnalysisUnsupportedTargetError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import {
|
||||
compareWebScreenshotsInputSchema,
|
||||
createWebScreenshotArtifact,
|
||||
@@ -121,7 +117,7 @@ describe("PNG visual diff", () => {
|
||||
});
|
||||
|
||||
it("rejects malformed PNG dimensions after validating image data", () => {
|
||||
const image = artifact(2, 1, [0, 0, 0, 255]);
|
||||
const image = artifact(32_000_001, 1, [0, 0, 0, 255]);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
@@ -153,256 +149,6 @@ describe("PNG visual diff", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("PNG comparison memory budget", () => {
|
||||
it("classifies legal huge dimensions as a memory constraint before inflate", () => {
|
||||
const image = createWebScreenshotArtifact(headerOnlyPng(0x7fffffff, 1));
|
||||
const input = compareWebScreenshotsInputSchema.parse({
|
||||
before: image,
|
||||
after: image,
|
||||
});
|
||||
|
||||
try {
|
||||
comparePngScreenshots(input);
|
||||
throw new Error("expected PNG working-memory rejection");
|
||||
} catch (cause: unknown) {
|
||||
expect(cause).toBeInstanceOf(AnalysisResourceConstraintError);
|
||||
expect(cause).toMatchObject({
|
||||
resource: "memory",
|
||||
reportedLimits: {
|
||||
maximum_working_memory_bytes: 256 * 1024 * 1024,
|
||||
estimated_working_memory_bytes: "34359738499",
|
||||
before_dimensions: { width: 0x7fffffff, height: 1 },
|
||||
after_dimensions: { width: 0x7fffffff, height: 1 },
|
||||
},
|
||||
});
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("PNG chunk integrity", () => {
|
||||
it.each(["IHDR", "IDAT", "IEND"])(
|
||||
"rejects a corrupt %s chunk checksum",
|
||||
(type) => {
|
||||
const bytes = png(1, 1, [0, 0, 0, 255]);
|
||||
let offset = 8;
|
||||
while (offset < bytes.length) {
|
||||
const length = bytes.readUInt32BE(offset);
|
||||
if (bytes.subarray(offset + 4, offset + 8).toString("ascii") === type) {
|
||||
bytes[offset + 8 + length] = (bytes[offset + 8 + length] ?? 0) ^ 1;
|
||||
break;
|
||||
}
|
||||
offset += length + 12;
|
||||
}
|
||||
const image = createWebScreenshotArtifact(bytes);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: image,
|
||||
after: image,
|
||||
}),
|
||||
),
|
||||
).toThrow("Invalid PNG chunk checksum");
|
||||
},
|
||||
);
|
||||
|
||||
it("reports a CRC-valid but corrupt compressed stream as invalid PNG data", () => {
|
||||
const header = Buffer.from(png(1, 1, [0, 0, 0, 255]).subarray(16, 29));
|
||||
const image = createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]),
|
||||
chunk("IHDR", header),
|
||||
chunk("IDAT", Buffer.from("bad")),
|
||||
chunk("IEND", Buffer.alloc(0)),
|
||||
]),
|
||||
);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({ before: image, after: image }),
|
||||
),
|
||||
).toThrow("Invalid PNG image data");
|
||||
});
|
||||
|
||||
it("rejects high-bit bytes in a CRC-valid PNG chunk name", () => {
|
||||
const bytes = png(1, 1, [0, 0, 0, 255]);
|
||||
const headerChunk = Buffer.from(bytes.subarray(8, 33));
|
||||
headerChunk[4] = 0xc9;
|
||||
headerChunk.writeUInt32BE(crc32(headerChunk.subarray(4, 21)), 21);
|
||||
const malformed = createWebScreenshotArtifact(
|
||||
Buffer.concat([bytes.subarray(0, 8), headerChunk, bytes.subarray(33)]),
|
||||
);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: malformed,
|
||||
after: malformed,
|
||||
}),
|
||||
),
|
||||
).toThrow("Invalid PNG chunk type");
|
||||
});
|
||||
|
||||
it("rejects duplicate image headers and unknown critical chunks while preserving ancillary chunks", () => {
|
||||
const bytes = png(1, 1, [0, 0, 0, 255]);
|
||||
const duplicate = Buffer.concat([
|
||||
bytes.subarray(0, 33),
|
||||
bytes.subarray(8, 33),
|
||||
bytes.subarray(33),
|
||||
]);
|
||||
const unknown = Buffer.concat([
|
||||
bytes.subarray(0, 33),
|
||||
chunk("ABCD", Buffer.alloc(0)),
|
||||
bytes.subarray(33),
|
||||
]);
|
||||
for (const malformed of [duplicate, unknown]) {
|
||||
const image = createWebScreenshotArtifact(malformed);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: image,
|
||||
after: image,
|
||||
}),
|
||||
),
|
||||
).toThrow(/Invalid PNG header order|Unsupported critical PNG chunk/u);
|
||||
}
|
||||
const ancillary = createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
bytes.subarray(0, 33),
|
||||
chunk("tEXt", Buffer.from("Comment\0original capture")),
|
||||
bytes.subarray(33),
|
||||
]),
|
||||
);
|
||||
expect(
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: ancillary,
|
||||
after: createWebScreenshotArtifact(bytes),
|
||||
}),
|
||||
).status,
|
||||
).toBe("identical");
|
||||
});
|
||||
|
||||
it("rejects bytes after IEND and reports valid unsupported PNG encodings separately", () => {
|
||||
const bytes = png(1, 1, [0, 0, 0, 255]);
|
||||
const trailing = createWebScreenshotArtifact(
|
||||
Buffer.concat([bytes, Buffer.from([0])]),
|
||||
);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: trailing,
|
||||
after: trailing,
|
||||
}),
|
||||
),
|
||||
).toThrow("Invalid PNG end chunk");
|
||||
|
||||
const grayscaleHeader = Buffer.from(bytes.subarray(16, 29));
|
||||
grayscaleHeader[9] = 0;
|
||||
const grayscale = createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]),
|
||||
chunk("IHDR", grayscaleHeader),
|
||||
chunk("IDAT", deflateSync(Buffer.from([0, 30]))),
|
||||
chunk("IEND", Buffer.alloc(0)),
|
||||
]),
|
||||
);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: grayscale,
|
||||
after: grayscale,
|
||||
}),
|
||||
),
|
||||
).toThrow(AnalysisUnsupportedTargetError);
|
||||
});
|
||||
});
|
||||
|
||||
describe("PNG chunk ordering", () => {
|
||||
it("rejects malformed palettes, interrupted data and misplaced transparency while retaining valid chunks", () => {
|
||||
const header = Buffer.from(png(1, 1, [10, 20, 30, 255]).subarray(16, 29));
|
||||
header[9] = 2;
|
||||
const ihdr = chunk("IHDR", header);
|
||||
const compressed = deflateSync(Buffer.from([0, 10, 20, 30]));
|
||||
const idat = chunk("IDAT", compressed);
|
||||
const iend = chunk("IEND", Buffer.alloc(0));
|
||||
const palette = chunk("PLTE", Buffer.from([10, 20, 30]));
|
||||
const transparency = chunk("tRNS", Buffer.from([0, 10, 0, 20, 0, 30]));
|
||||
const ancillary = chunk("tEXt", Buffer.from("Comment\0retained"));
|
||||
const image = (...chunks: readonly Buffer[]) =>
|
||||
createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]),
|
||||
...chunks,
|
||||
]),
|
||||
);
|
||||
const original = image(ihdr, idat, iend);
|
||||
const cases = [
|
||||
[ihdr, chunk("PLTE", Buffer.alloc(1)), idat, iend],
|
||||
[ihdr, palette, palette, idat, iend],
|
||||
[ihdr, idat, palette, iend],
|
||||
[ihdr, transparency, palette, idat, iend],
|
||||
[ihdr, transparency, transparency, idat, iend],
|
||||
[ihdr, idat, transparency, iend],
|
||||
[
|
||||
ihdr,
|
||||
chunk("IDAT", compressed.subarray(0, 6)),
|
||||
ancillary,
|
||||
chunk("IDAT", compressed.subarray(6)),
|
||||
iend,
|
||||
],
|
||||
[ihdr, idat, chunk("IEND", Buffer.from([1]))],
|
||||
];
|
||||
for (const chunks of cases) {
|
||||
const malformed = image(...chunks);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: malformed,
|
||||
after: original,
|
||||
}),
|
||||
),
|
||||
).toThrow(
|
||||
/Invalid PNG palette|Invalid PNG transparency order|Nonconsecutive PNG image data|Invalid PNG end chunk/u,
|
||||
);
|
||||
}
|
||||
const indexedHeader = Buffer.from(header);
|
||||
indexedHeader[8] = 1;
|
||||
indexedHeader[9] = 3;
|
||||
const emptyTransparency = image(
|
||||
chunk("IHDR", indexedHeader),
|
||||
chunk("PLTE", Buffer.from([10, 20, 30])),
|
||||
chunk("tRNS", Buffer.alloc(0)),
|
||||
chunk("IDAT", deflateSync(Buffer.from([0, 0]))),
|
||||
iend,
|
||||
);
|
||||
expect(() =>
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: emptyTransparency,
|
||||
after: original,
|
||||
}),
|
||||
),
|
||||
).toThrow("Unsupported PNG transparency");
|
||||
for (const valid of [
|
||||
image(ihdr, palette, idat, iend),
|
||||
image(ihdr, ancillary, idat, iend),
|
||||
image(
|
||||
ihdr,
|
||||
chunk("IDAT", compressed.subarray(0, 6)),
|
||||
chunk("IDAT", compressed.subarray(6)),
|
||||
iend,
|
||||
),
|
||||
]) {
|
||||
expect(
|
||||
comparePngScreenshots(
|
||||
compareWebScreenshotsInputSchema.parse({
|
||||
before: valid,
|
||||
after: original,
|
||||
}),
|
||||
).status,
|
||||
).toBe("identical");
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
const artifact = (width: number, height: number, pixels: readonly number[]) =>
|
||||
createWebScreenshotArtifact(png(width, height, pixels));
|
||||
|
||||
@@ -427,19 +173,6 @@ const png = (
|
||||
]);
|
||||
};
|
||||
|
||||
const headerOnlyPng = (width: number, height: number): Buffer => {
|
||||
const header = Buffer.alloc(13);
|
||||
header.writeUInt32BE(width, 0);
|
||||
header.writeUInt32BE(height, 4);
|
||||
header.set([8, 6, 0, 0, 0], 8);
|
||||
return Buffer.concat([
|
||||
Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]),
|
||||
chunk("IHDR", header),
|
||||
chunk("IDAT", deflateSync(Buffer.from([0]))),
|
||||
chunk("IEND", Buffer.alloc(0)),
|
||||
]);
|
||||
};
|
||||
|
||||
const chunk = (type: string, data: Buffer): Buffer => {
|
||||
const result = Buffer.alloc(12 + data.byteLength);
|
||||
result.writeUInt32BE(data.byteLength, 0);
|
||||
|
||||
+32
-268
@@ -1,14 +1,10 @@
|
||||
import { constants as zlibConstants, crc32, inflateSync } from "node:zlib";
|
||||
import { inflateSync } from "node:zlib";
|
||||
|
||||
import type {
|
||||
CompareWebScreenshotsInput,
|
||||
WebScreenshotArtifact,
|
||||
WebScreenshotDiff,
|
||||
} from "../domain/webScreenshot.js";
|
||||
import {
|
||||
AnalysisResourceConstraintError,
|
||||
AnalysisUnsupportedTargetError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import { decodeCanonicalBase64 } from "../domain/webScreenshot.js";
|
||||
|
||||
interface DecodedPng {
|
||||
@@ -17,32 +13,12 @@ interface DecodedPng {
|
||||
readonly rgba: Buffer;
|
||||
}
|
||||
|
||||
interface ParsedPng {
|
||||
readonly bytes: Buffer;
|
||||
readonly width: number;
|
||||
readonly height: number;
|
||||
readonly channels: number;
|
||||
readonly compressed: readonly Buffer[];
|
||||
readonly transparentColor: readonly number[] | undefined;
|
||||
readonly expectedRawBytes: bigint;
|
||||
readonly unfilteredBytes: bigint;
|
||||
readonly rgbaBytes: bigint;
|
||||
}
|
||||
|
||||
/** Conservative allowance for the simultaneously live PNG comparison buffers. */
|
||||
const PNG_COMPARISON_WORKING_MEMORY_BYTES = 256 * 1024 * 1024;
|
||||
const PNG_INFLATE_CHUNK_BYTES = zlibConstants.Z_DEFAULT_CHUNK;
|
||||
|
||||
/** Compare two validated screenshots using deterministic RGBA metrics. */
|
||||
export const comparePngScreenshots = (
|
||||
input: CompareWebScreenshotsInput,
|
||||
): WebScreenshotDiff => {
|
||||
admitEncodedInputPair(input.before, input.after);
|
||||
const beforePlan = parsePng(input.before, "before");
|
||||
const afterPlan = parsePng(input.after, "after");
|
||||
admitPngPair(beforePlan, afterPlan);
|
||||
const before = decodePng(beforePlan);
|
||||
const after = decodePng(afterPlan);
|
||||
const before = decodePng(input.before);
|
||||
const after = decodePng(input.after);
|
||||
if (before.width !== after.width || before.height !== after.height)
|
||||
return {
|
||||
status: "dimension_mismatch",
|
||||
@@ -97,10 +73,7 @@ export const comparePngScreenshots = (
|
||||
};
|
||||
};
|
||||
|
||||
const parsePng = (
|
||||
artifact: WebScreenshotArtifact,
|
||||
field: "before" | "after",
|
||||
): ParsedPng => {
|
||||
const decodePng = (artifact: WebScreenshotArtifact): DecodedPng => {
|
||||
const bytes = decodeCanonicalBase64(artifact.data_base64);
|
||||
if (bytes === undefined || !bytes.subarray(0, 8).equals(PNG_SIGNATURE))
|
||||
throw new TypeError("Invalid PNG signature");
|
||||
@@ -108,277 +81,68 @@ const parsePng = (
|
||||
let header: ReturnType<typeof parseHeader> | undefined;
|
||||
const compressed: Buffer[] = [];
|
||||
let sawEnd = false;
|
||||
let sawPalette = false;
|
||||
let imageDataEnded = false;
|
||||
let paletteEntries = 0;
|
||||
let transparentColor: readonly number[] | undefined;
|
||||
let sawTransparency = false;
|
||||
let unsupportedReason: string | undefined;
|
||||
while (offset < bytes.length) {
|
||||
if (offset + 12 > bytes.length) throw new TypeError("Truncated PNG chunk");
|
||||
const length = bytes.readUInt32BE(offset);
|
||||
if (length > 0x7fffffff) throw new TypeError("Invalid PNG chunk length");
|
||||
const type = bytes.subarray(offset + 4, offset + 8).toString("latin1");
|
||||
const type = bytes.subarray(offset + 4, offset + 8).toString("ascii");
|
||||
const dataStart = offset + 8;
|
||||
const dataEnd = dataStart + length;
|
||||
if (dataEnd + 4 > bytes.length) throw new TypeError("Truncated PNG data");
|
||||
const data = bytes.subarray(dataStart, dataEnd);
|
||||
if (
|
||||
crc32(bytes.subarray(offset + 4, dataEnd)) !== bytes.readUInt32BE(dataEnd)
|
||||
)
|
||||
throw new TypeError("Invalid PNG chunk checksum");
|
||||
if (!/^[A-Za-z]{4}$/u.test(type) || type[2] !== type[2]?.toUpperCase())
|
||||
throw new TypeError("Invalid PNG chunk type");
|
||||
if (type !== "IDAT" && compressed.length > 0) imageDataEnded = true;
|
||||
if (type === "IHDR") {
|
||||
if (offset !== 8 || header !== undefined)
|
||||
throw new TypeError("Invalid PNG header order");
|
||||
header = parseHeader(data);
|
||||
} else if (header === undefined)
|
||||
throw new TypeError("Invalid PNG header order");
|
||||
if (type === "IHDR") header = parseHeader(data);
|
||||
else if (type === "tRNS") {
|
||||
if (sawTransparency || compressed.length > 0)
|
||||
throw new TypeError("Invalid PNG transparency order");
|
||||
sawTransparency = true;
|
||||
if (header.colorType === 2 && data.length === 6) {
|
||||
transparentColor = [
|
||||
data.readUInt16BE(0),
|
||||
data.readUInt16BE(2),
|
||||
data.readUInt16BE(4),
|
||||
];
|
||||
} else if (
|
||||
(header.colorType === 0 && data.length === 2) ||
|
||||
(header.colorType === 3 &&
|
||||
sawPalette &&
|
||||
data.length > 0 &&
|
||||
data.length <= paletteEntries)
|
||||
) {
|
||||
unsupportedReason ??=
|
||||
"PNG transparency for this color type is not supported.";
|
||||
} else {
|
||||
if (header?.channels !== 3 || data.length !== 6)
|
||||
throw new TypeError("Unsupported PNG transparency");
|
||||
}
|
||||
} else if (type === "PLTE") {
|
||||
if (
|
||||
sawPalette ||
|
||||
sawTransparency ||
|
||||
compressed.length > 0 ||
|
||||
length === 0 ||
|
||||
length > 768 ||
|
||||
length % 3 !== 0 ||
|
||||
(header.colorType === 3 && length / 3 > 2 ** header.bitDepth) ||
|
||||
(header.colorType !== 2 &&
|
||||
header.colorType !== 3 &&
|
||||
header.colorType !== 6)
|
||||
)
|
||||
throw new TypeError("Invalid PNG palette");
|
||||
sawPalette = true;
|
||||
paletteEntries = length / 3;
|
||||
} else if (type === "IDAT") {
|
||||
if (imageDataEnded) throw new TypeError("Nonconsecutive PNG image data");
|
||||
compressed.push(data);
|
||||
} else if (type === "IEND") {
|
||||
if (length !== 0 || dataEnd + 4 !== bytes.length)
|
||||
throw new TypeError("Invalid PNG end chunk");
|
||||
transparentColor = [
|
||||
data.readUInt16BE(0),
|
||||
data.readUInt16BE(2),
|
||||
data.readUInt16BE(4),
|
||||
];
|
||||
} else if (type === "IDAT") compressed.push(data);
|
||||
else if (type === "IEND") {
|
||||
sawEnd = true;
|
||||
break;
|
||||
} else if (/^[A-Z]/u.test(type)) {
|
||||
unsupportedReason ??= `Unsupported critical PNG chunk ${type}.`;
|
||||
}
|
||||
offset = dataEnd + 4;
|
||||
}
|
||||
if (header === undefined || !sawEnd || compressed.length === 0)
|
||||
throw new TypeError("Incomplete PNG");
|
||||
if (header.colorType === 3 && !sawPalette)
|
||||
throw new TypeError("Missing PNG palette");
|
||||
if (unsupportedReason !== undefined)
|
||||
throw new AnalysisUnsupportedTargetError(
|
||||
"compare_web_screenshots",
|
||||
field,
|
||||
unsupportedReason,
|
||||
);
|
||||
if (!header.supported)
|
||||
throw new AnalysisUnsupportedTargetError(
|
||||
"compare_web_screenshots",
|
||||
field,
|
||||
"PNG encoding is not supported for pixel comparison.",
|
||||
);
|
||||
const pixels = BigInt(header.width) * BigInt(header.height);
|
||||
const expectedRawBytes =
|
||||
(BigInt(header.width) * BigInt(header.channels) + 1n) *
|
||||
BigInt(header.height);
|
||||
return {
|
||||
bytes,
|
||||
width: header.width,
|
||||
height: header.height,
|
||||
channels: header.channels,
|
||||
compressed,
|
||||
transparentColor,
|
||||
expectedRawBytes,
|
||||
unfilteredBytes: pixels * BigInt(header.channels),
|
||||
rgbaBytes: pixels * 4n,
|
||||
};
|
||||
};
|
||||
|
||||
const decodePng = (plan: ParsedPng): DecodedPng => {
|
||||
const expected = Number(plan.expectedRawBytes);
|
||||
const raw = inflatePng(Buffer.concat(plan.compressed), expected + 1);
|
||||
const rowBytes = header.width * header.channels;
|
||||
const expected = (rowBytes + 1) * header.height;
|
||||
const raw = inflateSync(Buffer.concat(compressed), {
|
||||
maxOutputLength: expected + 1,
|
||||
});
|
||||
if (raw.byteLength !== expected) throw new TypeError("Unexpected PNG size");
|
||||
return {
|
||||
width: plan.width,
|
||||
height: plan.height,
|
||||
width: header.width,
|
||||
height: header.height,
|
||||
rgba: expandRgba(
|
||||
unfilter(raw, plan.width, plan.height, plan.channels),
|
||||
plan.channels,
|
||||
plan.transparentColor,
|
||||
unfilter(raw, header.width, header.height, header.channels),
|
||||
header.channels,
|
||||
transparentColor,
|
||||
),
|
||||
};
|
||||
};
|
||||
|
||||
const admitEncodedInputPair = (
|
||||
before: WebScreenshotArtifact,
|
||||
after: WebScreenshotArtifact,
|
||||
): void => {
|
||||
const sourceBytes = BigInt(before.bytes) + BigInt(after.bytes);
|
||||
if (sourceBytes > BigInt(PNG_COMPARISON_WORKING_MEMORY_BYTES))
|
||||
throw pngWorkingMemoryError(sourceBytes, null, null);
|
||||
};
|
||||
|
||||
const admitPngPair = (before: ParsedPng, after: ParsedPng): void => {
|
||||
// Input buffers remain live for both plans. For each decode, count the
|
||||
// concatenated IDAT, rounded zlib output chunks, any concatenated output and
|
||||
// unfiltered rows, plus RGB-to-RGBA output. The first completed RGBA image
|
||||
// remains live while the second is decoded.
|
||||
const inputBytes = BigInt(before.bytes.byteLength + after.bytes.byteLength);
|
||||
const estimate = (current: ParsedPng, retained: ParsedPng | undefined) => {
|
||||
const compressedBytes = current.compressed.reduce(
|
||||
(total, chunk) => total + BigInt(chunk.byteLength),
|
||||
0n,
|
||||
);
|
||||
const rgbaExpansion = current.channels === 3 ? current.rgbaBytes : 0n;
|
||||
return (
|
||||
inputBytes +
|
||||
(retained?.rgbaBytes ?? 0n) +
|
||||
compressedBytes +
|
||||
inflateBufferAllowance(current.expectedRawBytes) +
|
||||
current.unfilteredBytes +
|
||||
rgbaExpansion
|
||||
);
|
||||
};
|
||||
const estimateBytes = [
|
||||
estimate(before, undefined),
|
||||
estimate(after, before),
|
||||
].reduce((maximum, value) => (value > maximum ? value : maximum), 0n);
|
||||
if (estimateBytes > BigInt(PNG_COMPARISON_WORKING_MEMORY_BYTES))
|
||||
throw pngWorkingMemoryError(estimateBytes, before, after);
|
||||
};
|
||||
|
||||
const inflateBufferAllowance = (expectedRawBytes: bigint): bigint => {
|
||||
const chunkBytes = BigInt(PNG_INFLATE_CHUNK_BYTES);
|
||||
const maximumOutputBytes = expectedRawBytes + 1n;
|
||||
const roundUpToChunk = (bytes: bigint): bigint =>
|
||||
((bytes + chunkBytes - 1n) / chunkBytes) * chunkBytes;
|
||||
|
||||
// Node checks maxOutputLength after writing a chunk, so a rejected stream can
|
||||
// retain one chunk's worth of output past the limit before it throws.
|
||||
const rejectedOutputChunks = roundUpToChunk(
|
||||
maximumOutputBytes + chunkBytes - 1n,
|
||||
);
|
||||
|
||||
// Sync zlib retains each output chunk while Buffer.concat creates the final
|
||||
// result. A full final chunk also allocates the next scratch chunk first.
|
||||
const returnedOutputChunks = roundUpToChunk(maximumOutputBytes);
|
||||
const scratchChunk =
|
||||
expectedRawBytes % chunkBytes === 0n ||
|
||||
maximumOutputBytes % chunkBytes === 0n
|
||||
? chunkBytes
|
||||
: 0n;
|
||||
const concatenatedOutput =
|
||||
maximumOutputBytes > chunkBytes ? maximumOutputBytes : 0n;
|
||||
const returnedOutput =
|
||||
returnedOutputChunks + scratchChunk + concatenatedOutput;
|
||||
|
||||
return rejectedOutputChunks > returnedOutput
|
||||
? rejectedOutputChunks
|
||||
: returnedOutput;
|
||||
};
|
||||
|
||||
const pngWorkingMemoryError = (
|
||||
estimateBytes: bigint,
|
||||
before: Pick<ParsedPng, "width" | "height"> | null,
|
||||
after: Pick<ParsedPng, "width" | "height"> | null,
|
||||
) =>
|
||||
new AnalysisResourceConstraintError(
|
||||
"compare_web_screenshots",
|
||||
"memory",
|
||||
"Screenshot comparison exceeds REA's 256 MiB estimated working-memory budget.",
|
||||
{
|
||||
boundary: "png-screenshot-comparison",
|
||||
estimated_working_memory_bytes: estimateBytes.toString(),
|
||||
maximum_working_memory_bytes: PNG_COMPARISON_WORKING_MEMORY_BYTES,
|
||||
before_dimensions:
|
||||
before === null ? null : { width: before.width, height: before.height },
|
||||
after_dimensions:
|
||||
after === null ? null : { width: after.width, height: after.height },
|
||||
},
|
||||
{
|
||||
remediationAction:
|
||||
"Compare smaller screenshots or reduce their viewport dimensions before capture.",
|
||||
},
|
||||
);
|
||||
|
||||
const inflatePng = (compressed: Buffer, maxOutputLength: number): Buffer => {
|
||||
try {
|
||||
return inflateSync(compressed, {
|
||||
maxOutputLength,
|
||||
chunkSize: PNG_INFLATE_CHUNK_BYTES,
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
if (
|
||||
cause instanceof Error &&
|
||||
"code" in cause &&
|
||||
(cause.code === "Z_DATA_ERROR" ||
|
||||
cause.code === "Z_BUF_ERROR" ||
|
||||
cause.code === "ERR_BUFFER_TOO_LARGE")
|
||||
)
|
||||
throw new TypeError("Invalid PNG image data", { cause });
|
||||
throw cause;
|
||||
}
|
||||
};
|
||||
|
||||
const parseHeader = (data: Buffer) => {
|
||||
if (data.byteLength !== 13) throw new TypeError("Invalid PNG header");
|
||||
const width = data.readUInt32BE(0);
|
||||
const height = data.readUInt32BE(4);
|
||||
const bitDepth = data[8] ?? 0;
|
||||
const colorType = data[9] ?? -1;
|
||||
const validDepth =
|
||||
(colorType === 0 && [1, 2, 4, 8, 16].includes(bitDepth)) ||
|
||||
(colorType === 2 && [8, 16].includes(bitDepth)) ||
|
||||
(colorType === 3 && [1, 2, 4, 8].includes(bitDepth)) ||
|
||||
(colorType === 4 && [8, 16].includes(bitDepth)) ||
|
||||
(colorType === 6 && [8, 16].includes(bitDepth));
|
||||
const bitDepth = data[8];
|
||||
const colorType = data[9];
|
||||
if (
|
||||
width === 0 ||
|
||||
height === 0 ||
|
||||
width > 0x7fffffff ||
|
||||
height > 0x7fffffff ||
|
||||
!validDepth ||
|
||||
!Number.isSafeInteger(width * height) ||
|
||||
bitDepth !== 8 ||
|
||||
(colorType !== 2 && colorType !== 6) ||
|
||||
data[10] !== 0 ||
|
||||
data[11] !== 0 ||
|
||||
(data[12] !== 0 && data[12] !== 1)
|
||||
data[12] !== 0
|
||||
)
|
||||
throw new TypeError("Invalid PNG header");
|
||||
return {
|
||||
width,
|
||||
height,
|
||||
colorType,
|
||||
bitDepth,
|
||||
interlace: data[12],
|
||||
channels: colorType === 6 ? 4 : 3,
|
||||
supported:
|
||||
bitDepth === 8 && (colorType === 2 || colorType === 6) && data[12] === 0,
|
||||
};
|
||||
throw new TypeError("Unsupported PNG");
|
||||
return { width, height, channels: colorType === 6 ? 4 : 3 };
|
||||
};
|
||||
|
||||
const unfilter = (
|
||||
|
||||
@@ -1,12 +1,6 @@
|
||||
import { z } from "incur";
|
||||
|
||||
import { inspectAnalysisViewValidated } from "../application/analysisView/AnalysisViewService.js";
|
||||
import { readEvidenceBundle } from "../application/EvidenceBundleFiles.js";
|
||||
import { inspectEvidenceBundle } from "../application/investigation/InspectEvidenceBundle.js";
|
||||
import {
|
||||
getEvidenceBundleInputSchema,
|
||||
importEvidenceBundleInputSchema,
|
||||
} from "../contracts/sessionToolSchemas.js";
|
||||
import { CLI_COMMANDS } from "../cliCommandNames.js";
|
||||
import { parseCliJsonInput } from "../cliJsonInput.js";
|
||||
import { logCliCommand } from "../cliLogging.js";
|
||||
@@ -22,53 +16,6 @@ export const registerAnalysisViewCommands = (
|
||||
cli: CliInstance,
|
||||
logger: Logger,
|
||||
): void => {
|
||||
cli.command(CLI_COMMANDS.inspectEvidenceBundle, {
|
||||
description:
|
||||
"Inspect complete or selected discovery metadata from a local Evidence bundle",
|
||||
args: z.object({
|
||||
inputJson: z
|
||||
.string()
|
||||
.describe(
|
||||
"JSON or JSON file with absolute path, detail and optional record filters",
|
||||
),
|
||||
}),
|
||||
run: ({ args }) =>
|
||||
logCliCommand(logger, CLI_COMMANDS.inspectEvidenceBundle, async () => {
|
||||
const input = await parseCliJsonInput(
|
||||
args.inputJson,
|
||||
CLI_COMMANDS.inspectEvidenceBundle,
|
||||
);
|
||||
if (!input.ok) return input.error;
|
||||
const schema = getEvidenceBundleInputSchema.safeExtend({
|
||||
path: importEvidenceBundleInputSchema.shape.path,
|
||||
});
|
||||
const parsed = schema.safeParse(input.value);
|
||||
if (!parsed.success)
|
||||
return {
|
||||
error: "Evidence bundle inspection failed",
|
||||
...projectAnalysisError(
|
||||
analysisInputErrorFromIssues(
|
||||
CLI_COMMANDS.inspectEvidenceBundle,
|
||||
parsed.error.issues,
|
||||
input.value,
|
||||
),
|
||||
),
|
||||
};
|
||||
const loaded = await readEvidenceBundle(parsed.data.path);
|
||||
if (!loaded.ok)
|
||||
return {
|
||||
error: "Evidence bundle inspection failed",
|
||||
...projectAnalysisError(loaded.error),
|
||||
};
|
||||
const result = inspectEvidenceBundle(loaded.value, parsed.data);
|
||||
return result.ok
|
||||
? result.value
|
||||
: {
|
||||
error: "Evidence bundle inspection failed",
|
||||
...projectAnalysisError(result.error),
|
||||
};
|
||||
}),
|
||||
});
|
||||
cli.command(CLI_COMMANDS.inspectAnalysisView, {
|
||||
description:
|
||||
"Project a selected view of completed layout, JavaScript application or native function Evidence JSON",
|
||||
|
||||
@@ -27,6 +27,12 @@ export const registerJavaScriptRecoveryCommands = (
|
||||
outputDirectory: z.string().describe("Absolute absent output directory"),
|
||||
}),
|
||||
options: z.object({
|
||||
additionalPath: z
|
||||
.array(z.string())
|
||||
.optional()
|
||||
.describe(
|
||||
"Related absolute UTF-8 JavaScript file; repeat for multiple chunks",
|
||||
),
|
||||
extractionMode: z
|
||||
.enum(["structural", "heuristic", "inspection"])
|
||||
.optional()
|
||||
@@ -51,6 +57,9 @@ export const registerJavaScriptRecoveryCommands = (
|
||||
{
|
||||
path: args.path,
|
||||
output_directory: args.outputDirectory,
|
||||
...(options.additionalPath === undefined
|
||||
? {}
|
||||
: { additional_paths: options.additionalPath }),
|
||||
...(options.extractionMode === undefined
|
||||
? {}
|
||||
: { extraction_mode: options.extractionMode }),
|
||||
|
||||
@@ -43,7 +43,6 @@ export const CLI_COMMANDS = Object.freeze({
|
||||
inspectBinaryLayout: "inspect-binary-layout",
|
||||
inspectPeResources: "inspect-pe-resources",
|
||||
inspectAnalysisView: "inspect-analysis-view",
|
||||
inspectEvidenceBundle: "inspect-evidence-bundle",
|
||||
inspectFirmwareRegions: "inspect-firmware-regions",
|
||||
extractFirmware: "extract-firmware",
|
||||
inspectAndroidPackage: "inspect-android-package",
|
||||
|
||||
@@ -7,6 +7,7 @@ import {
|
||||
} from "../application/DirectAnalysis.js";
|
||||
import { runSessionStatus as executeSessionStatus } from "../application/DirectAnalysisStatus.js";
|
||||
import type { DirectAnalysisDependencies } from "../application/DirectAnalysisDependencies.js";
|
||||
import { createBinarySession, createManagedBinarySession } from "./binary.js";
|
||||
|
||||
/** Bind one-shot analysis and status commands to one caller-selected environment. */
|
||||
export const createDirectAnalysis = (
|
||||
@@ -15,14 +16,9 @@ export const createDirectAnalysis = (
|
||||
const environment = snapshotEnvironment(selectedEnvironment);
|
||||
const dependencies: DirectAnalysisDependencies = {
|
||||
readConfiguration: () => parseConfig(environment),
|
||||
createBinarySession: async (config, logger) => {
|
||||
const { createBinarySession } = await import("./binary.js");
|
||||
return createBinarySession(config, logger, environment);
|
||||
},
|
||||
createManagedBinarySession: async () => {
|
||||
const { createManagedBinarySession } = await import("./binary.js");
|
||||
return createManagedBinarySession();
|
||||
},
|
||||
createBinarySession: (config, logger) =>
|
||||
createBinarySession(config, logger, environment),
|
||||
createManagedBinarySession,
|
||||
};
|
||||
return {
|
||||
runDirectAnalysis: executeDirectAnalysis.bind(undefined, dependencies),
|
||||
@@ -38,9 +34,6 @@ export type DirectAnalysis = ReturnType<typeof createDirectAnalysis>;
|
||||
export const runManagedProviderExecution = executeManagedProvider.bind(
|
||||
undefined,
|
||||
{
|
||||
createManagedBinarySession: async () => {
|
||||
const { createManagedBinarySession } = await import("./binary.js");
|
||||
return createManagedBinarySession();
|
||||
},
|
||||
createManagedBinarySession,
|
||||
},
|
||||
);
|
||||
|
||||
@@ -13,7 +13,7 @@ export const JAVASCRIPT_RECOVERY_TOOL_CONTRACTS = [
|
||||
...toolContractMetadata("recover_javascript_sources"),
|
||||
kind: "application",
|
||||
description:
|
||||
"Recover readable modules from one local UTF-8 JavaScript script/bundle into an absent directory, using caller-supplied Wakaru ^1.13.0 (verified with 1.14.0) on Linux x64. Returns original/derived digests, extraction byte ranges, emitted source maps, warnings and analysis_input inline. Structural splitting may fall back to one module; original names and runtime equivalence remain unknown. No application code execution. Requires util-linux prlimit; one worker, 1 GiB address-space ceiling, 120-second deadline, 64 MiB input, 128 MiB output, 10000 entries and 8 MiB per diagnostic/report stream.",
|
||||
"Recover readable modules from local UTF-8 JavaScript scripts/bundles into an absent directory, using caller-supplied Wakaru ^1.13.0 (verified with 1.14.0) on Linux x64. Use additional_paths for related chunks; each input retains its own snapshot and provenance. Returns original/derived digests, extraction byte ranges, emitted source maps, warnings and analysis_input inline. Structural splitting may fall back to one module; original names and runtime equivalence remain unknown. No application code execution. Requires util-linux prlimit; one worker, 1 GiB address-space ceiling, 120-second deadline, 64 MiB total input, 128 MiB output, 10000 entries and 8 MiB per diagnostic/report stream.",
|
||||
inputSchema: javascriptRecoveryInputSchema,
|
||||
outputSchema: evidenceResultOf(javascriptRecoveryResultSchema),
|
||||
examples: [
|
||||
@@ -24,6 +24,18 @@ export const JAVASCRIPT_RECOVERY_TOOL_CONTRACTS = [
|
||||
output_directory: "/tmp/recovered-app",
|
||||
},
|
||||
},
|
||||
{
|
||||
title: "Recover a code-split bundle together",
|
||||
input: {
|
||||
path: "/tmp/web-scripts/main.js",
|
||||
additional_paths: [
|
||||
"/tmp/web-scripts/runtime.js",
|
||||
"/tmp/web-scripts/shared.js",
|
||||
"/tmp/web-scripts/checkout.chunk.js",
|
||||
],
|
||||
output_directory: "/tmp/recovered-chunks",
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
] as const satisfies readonly ToolContract[];
|
||||
|
||||
@@ -159,7 +159,7 @@ export const SESSION_TOOL_CONTRACTS = [
|
||||
),
|
||||
session(
|
||||
"get_evidence_bundle",
|
||||
"Return every retained Evidence record and residual unknown as a complete canonical bundle by default. detail=summary discovers retained records without copying or transferring their payloads; optionally filter by exact Evidence ID, operation, target SHA-256, analysis-profile digest or canonical native procedure address. Metadata distinguishes complete record retention from native value-flow truncation or unavailable/unknown coverage. Use a discovered evidence_id with inspect_analysis_view without restarting a provider. Filters apply only to summary delivery.",
|
||||
"Return every Evidence record and residual unknown currently retained by this session as one inline bundle for direct inspection or follow-up workflows.",
|
||||
getEvidenceBundleInputSchema,
|
||||
),
|
||||
session(
|
||||
|
||||
@@ -6,13 +6,9 @@ import { localPathStringSchema } from "../domain/localPath.js";
|
||||
import { processTraceSpecificationSchema } from "../domain/process/processTraceSpecification.js";
|
||||
import { evidenceSchema } from "../domain/evidence.js";
|
||||
import { prefixedDigestSchema } from "./../domain/digests.js";
|
||||
import { evidenceRecordFiltersSchema } from "../domain/evidenceRecordSummary.js";
|
||||
|
||||
/** Complete retained Evidence, or selected discovery metadata without payloads. */
|
||||
export const getEvidenceBundleInputSchema = z.strictObject({
|
||||
detail: z.enum(["complete", "summary"]).default("complete"),
|
||||
filters: evidenceRecordFiltersSchema.optional(),
|
||||
});
|
||||
/** Return the current canonical Evidence bundle inline. */
|
||||
export const getEvidenceBundleInputSchema = z.strictObject({});
|
||||
|
||||
/** Optional document selection for volatile navigation context. */
|
||||
export const navigationContextInputSchema = z.strictObject({
|
||||
|
||||
@@ -53,10 +53,6 @@ export const TOOL_EXAMPLE_OVERRIDES: Readonly<
|
||||
trace_native_ui_action: { action: "buildTapped:" },
|
||||
open_binary: { path: "/tmp/fixture" },
|
||||
export_evidence_bundle: { path: "/tmp/evidence.json" },
|
||||
get_evidence_bundle: {
|
||||
detail: "summary",
|
||||
filters: { operation: "analyze_function", procedure_address: "0x1000" },
|
||||
},
|
||||
inspect_address_context: { address: "0x1000" },
|
||||
import_evidence_bundle: { path: "/tmp/evidence.json" },
|
||||
capture_process_scenario: {
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { z } from "zod";
|
||||
import { evidenceBundleSummarySchema } from "../domain/evidenceRecordSummary.js";
|
||||
import { BINARY_ARCHITECTURES } from "../domain/binaryTargetTypes.js";
|
||||
import { nativeFunctionAnnotationsSchema } from "../domain/native/nativeFunctionAnnotations.js";
|
||||
import { nativeLoadImageSchema } from "../domain/native/nativeLoadImage.js";
|
||||
@@ -471,9 +470,7 @@ export const sessionOutputSchemas = {
|
||||
unknowns: z.number().int().min(0),
|
||||
}),
|
||||
),
|
||||
get_evidence_bundle: lifecycleResultOf(
|
||||
z.union([evidenceBundleSchema, evidenceBundleSummarySchema]),
|
||||
),
|
||||
get_evidence_bundle: lifecycleResultOf(evidenceBundleSchema),
|
||||
get_navigation_context: lifecycleResultOf(
|
||||
z.object({
|
||||
document: z.string(),
|
||||
|
||||
@@ -47,7 +47,7 @@ export const analysisErrorRemediationAction = (
|
||||
return (
|
||||
error.remediationAction ??
|
||||
(error.resource === "transport"
|
||||
? "Discover retained records with get_evidence_bundle and detail: summary, then inspect compatible records through inspect_analysis_view using an exact Evidence reference. Export complete retained session evidence through export_evidence_bundle to a caller-selected path, or use complete CLI JSON output. The connection remains usable."
|
||||
? "Export retained session evidence through export_evidence_bundle to a caller-selected path, or use complete CLI JSON output. Tools that accept retained-evidence references can inspect compatible records. The connection remains usable."
|
||||
: error.resource === "cpu"
|
||||
? "Review the reported worker CPU limits and observed signal. Retry with sufficient CPU time or a smaller artifact; REA retains tighter inherited limits."
|
||||
: error.resource === "file-size"
|
||||
|
||||
@@ -21,12 +21,6 @@ describe("cooperative canonical JSON digest", () => {
|
||||
),
|
||||
{ payload: `${"x".repeat(8191)}𐀀${"\n\ud800".repeat(20_000)}` },
|
||||
{ [`${"x".repeat(8191)}𐀀${"\n".repeat(20_000)}`]: "value" },
|
||||
Object.fromEntries(
|
||||
Array.from({ length: 512 }, (_, index) => [
|
||||
`escaped\n\"\\𐀀${"x".repeat(32)}${index}`,
|
||||
{ repeated: index },
|
||||
]),
|
||||
),
|
||||
];
|
||||
it.each(values.map((value) => ({ value })))(
|
||||
"preserves complete canonical meaning (%#)",
|
||||
@@ -61,18 +55,6 @@ describe("cooperative canonical JSON digest", () => {
|
||||
expect(steps.next().done).toBe(true);
|
||||
});
|
||||
|
||||
it("continues object hashing when a pending property is omitted", () => {
|
||||
const value: { [key: string]: JsonValue } = {
|
||||
a: "x".repeat(32_768),
|
||||
b: 1,
|
||||
c: { retained: true },
|
||||
};
|
||||
const steps = canonicalJsonDigestSteps(value);
|
||||
expect(steps.next().done).toBe(false);
|
||||
delete value.b;
|
||||
expect(complete(steps)).toBe(digestCanonicalValue(value));
|
||||
});
|
||||
|
||||
it("rejects sparse arrays, nonfinite numbers and cycles", () => {
|
||||
const sparse: JsonValue[] = [];
|
||||
sparse.length = 2;
|
||||
|
||||
@@ -26,9 +26,6 @@ export function* canonicalJsonDigestSteps(
|
||||
const hash = createHash("sha256");
|
||||
const ancestors = new Set<object>();
|
||||
const stack: DigestFrame[] = [{ kind: "value", value }];
|
||||
const encodedKeys = new Map<string, string>();
|
||||
// Keep reusable key encodings within one existing digest-buffer quantum.
|
||||
let remainingKeyCodeUnits = 8192;
|
||||
let buffered = "";
|
||||
let flushed = false;
|
||||
const emit = (part: string): void => {
|
||||
@@ -72,22 +69,22 @@ export function* canonicalJsonDigestSteps(
|
||||
stack.push(frame);
|
||||
continue;
|
||||
}
|
||||
let item: JsonValue;
|
||||
if (frame.kind === "array") {
|
||||
if (frame.index === frame.length) {
|
||||
emit("]");
|
||||
ancestors.delete(frame.value);
|
||||
continue;
|
||||
}
|
||||
const element = frame.value[frame.index];
|
||||
if (element === undefined)
|
||||
const item = frame.value[frame.index];
|
||||
if (item === undefined)
|
||||
throw new TypeError(
|
||||
"Canonical JSON digest requires dense JSON arrays",
|
||||
);
|
||||
if (frame.index++ > 0) emit(",");
|
||||
stack.push(frame);
|
||||
item = element;
|
||||
} else if (frame.kind === "object") {
|
||||
stack.push(frame, { kind: "value", value: item });
|
||||
continue;
|
||||
}
|
||||
if (frame.kind === "object") {
|
||||
if (frame.index === frame.keys.length) {
|
||||
emit("}");
|
||||
ancestors.delete(frame.value);
|
||||
@@ -96,42 +93,25 @@ export function* canonicalJsonDigestSteps(
|
||||
const key = frame.keys[frame.index++];
|
||||
if (key === undefined)
|
||||
throw new TypeError("Canonical JSON object changed during hashing");
|
||||
const property = frame.value[key];
|
||||
if (property === undefined) {
|
||||
stack.push(frame);
|
||||
continue;
|
||||
}
|
||||
const item = frame.value[key];
|
||||
if (item === undefined) continue;
|
||||
if (frame.written++ > 0) emit(",");
|
||||
if (key.length > 8192) {
|
||||
emit('"');
|
||||
stack.push(
|
||||
frame,
|
||||
{ kind: "value", value: property },
|
||||
{ kind: "value", value: item },
|
||||
{ kind: "separator" },
|
||||
{ kind: "string", value: key, index: 0 },
|
||||
);
|
||||
continue;
|
||||
}
|
||||
let encodedKey = encodedKeys.get(key);
|
||||
if (encodedKey === undefined) {
|
||||
encodedKey = JSON.stringify(key);
|
||||
if (encodedKey.length <= remainingKeyCodeUnits) {
|
||||
encodedKeys.set(key, encodedKey);
|
||||
remainingKeyCodeUnits -= encodedKey.length;
|
||||
}
|
||||
}
|
||||
emit(encodedKey);
|
||||
emit(JSON.stringify(key));
|
||||
emit(":");
|
||||
stack.push(frame);
|
||||
item = property;
|
||||
} else {
|
||||
item = frame.value;
|
||||
}
|
||||
// Resume containers directly; scalar members need no temporary value frame.
|
||||
if (flushed) {
|
||||
flushed = false;
|
||||
yield;
|
||||
stack.push(frame, { kind: "value", value: item });
|
||||
continue;
|
||||
}
|
||||
const item = frame.value;
|
||||
if (typeof item === "string" && item.length > 8192) {
|
||||
emit('"');
|
||||
stack.push({ kind: "string", value: item, index: 0 });
|
||||
@@ -171,7 +151,6 @@ export function* canonicalJsonDigestSteps(
|
||||
} finally {
|
||||
stack.length = 0;
|
||||
ancestors.clear();
|
||||
encodedKeys.clear();
|
||||
buffered = "";
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,55 +0,0 @@
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import { createEvidence } from "./evidence.js";
|
||||
import { createEvidenceBundle } from "./evidenceBundle.js";
|
||||
import { summarizeEvidenceBundle } from "./evidenceRecordSummary.js";
|
||||
|
||||
it("preserves native identity independently of missing or incompatible value-flow metadata", () => {
|
||||
const records = [
|
||||
undefined,
|
||||
{ available: true, truncated: true },
|
||||
{ unexpected: "legacy" },
|
||||
{ available: false, reason: "Decompiler unavailable" },
|
||||
].map((flow) =>
|
||||
createEvidence(
|
||||
{ path: "/fixtures/native.exe", format: "pe", sha256: "a".repeat(64) },
|
||||
{ id: "provider", name: "Provider", version: "1" },
|
||||
{
|
||||
operation: "analyze_function",
|
||||
parameters: {},
|
||||
result: {
|
||||
procedure: { address: "space:0x10", name: "entry" },
|
||||
...(flow === undefined ? {} : { native_value_flow: flow }),
|
||||
},
|
||||
},
|
||||
),
|
||||
);
|
||||
const summary = summarizeEvidenceBundle(createEvidenceBundle(records), {
|
||||
procedure_address: "space:0x10",
|
||||
});
|
||||
expect(summary.matching_records).toBe(4);
|
||||
const native = summary.records.map((r) => r.native_dossier);
|
||||
expect(
|
||||
native.every((r) => r.available && r.procedure_address === "space:0x10"),
|
||||
).toBe(true);
|
||||
expect(native).toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.objectContaining({ value_flow: { status: "not-recorded" } }),
|
||||
expect.objectContaining({
|
||||
value_flow: {
|
||||
status: "observed",
|
||||
truncated: true,
|
||||
omitted_operations_lower_bound: null,
|
||||
known_omitted_inputs: null,
|
||||
known_omitted_edges: null,
|
||||
},
|
||||
}),
|
||||
expect.objectContaining({
|
||||
value_flow: { status: "unknown", reason: expect.any(String) },
|
||||
}),
|
||||
expect.objectContaining({
|
||||
value_flow: { status: "unavailable", reason: "Decompiler unavailable" },
|
||||
}),
|
||||
]),
|
||||
);
|
||||
});
|
||||
@@ -1,166 +0,0 @@
|
||||
import { z } from "zod";
|
||||
|
||||
import { digestSchema, prefixedDigestSchema } from "./digests.js";
|
||||
import { evidenceSchema, type Evidence } from "./evidence.js";
|
||||
import type { EvidenceBundle } from "./evidenceBundle.js";
|
||||
|
||||
/** Exact retained-record selectors; selection never resolves a provider address. */
|
||||
export const evidenceRecordFiltersSchema = z.strictObject({
|
||||
evidence_id: prefixedDigestSchema("ev").optional(),
|
||||
operation: z.string().min(1).optional(),
|
||||
target_sha256: digestSchema.optional(),
|
||||
analysis_profile_digest: digestSchema.optional(),
|
||||
procedure_address: z.string().min(1).optional(),
|
||||
});
|
||||
export type EvidenceRecordFilters = z.infer<typeof evidenceRecordFiltersSchema>;
|
||||
|
||||
const countSchema = z.number().int().nonnegative();
|
||||
const nativeMetadataSchema = z.object({
|
||||
procedure: z.object({ address: z.string(), name: z.string().nullish() }),
|
||||
native_value_flow: z.unknown().optional(),
|
||||
});
|
||||
const valueFlowMetadataSchema = z.union([
|
||||
z.object({
|
||||
available: z.literal(true),
|
||||
truncated: z.boolean(),
|
||||
omitted_operations_lower_bound: countSchema.optional(),
|
||||
known_omitted_inputs: countSchema.optional(),
|
||||
known_omitted_edges: countSchema.optional(),
|
||||
}),
|
||||
z.object({ available: z.literal(false), reason: z.string() }),
|
||||
]);
|
||||
|
||||
const nativeSummarySchema = z.discriminatedUnion("available", [
|
||||
z.strictObject({
|
||||
available: z.literal(true),
|
||||
procedure_address: z.string(),
|
||||
procedure_name: z.string().nullable(),
|
||||
value_flow: z.discriminatedUnion("status", [
|
||||
z.strictObject({
|
||||
status: z.literal("observed"),
|
||||
truncated: z.boolean(),
|
||||
omitted_operations_lower_bound: countSchema.nullable(),
|
||||
known_omitted_inputs: countSchema.nullable(),
|
||||
known_omitted_edges: countSchema.nullable(),
|
||||
}),
|
||||
z.strictObject({ status: z.literal("unavailable"), reason: z.string() }),
|
||||
z.strictObject({ status: z.literal("not-recorded") }),
|
||||
z.strictObject({ status: z.literal("unknown"), reason: z.string() }),
|
||||
]),
|
||||
}),
|
||||
z.strictObject({
|
||||
available: z.literal(false),
|
||||
reason: z.enum(["different-operation", "not-recorded-or-incompatible"]),
|
||||
}),
|
||||
]);
|
||||
|
||||
/** Metadata for complete retained records, independent of analysis completeness. */
|
||||
export const evidenceBundleSummarySchema = z.strictObject({
|
||||
kind: z.literal("evidence-bundle-summary"),
|
||||
filters: z.record(z.string(), z.string()),
|
||||
total_retained_records: countSchema,
|
||||
matching_records: countSchema,
|
||||
unknown_revisions: countSchema,
|
||||
records: z.array(
|
||||
z.strictObject({
|
||||
evidence_id: evidenceSchema.shape.evidence_id,
|
||||
subject: evidenceSchema.shape.subject,
|
||||
provider: evidenceSchema.shape.provider,
|
||||
operation: evidenceSchema.shape.operation,
|
||||
confidence: evidenceSchema.shape.confidence,
|
||||
authority: evidenceSchema.shape.authority,
|
||||
analysis_profile_digest: digestSchema.nullable(),
|
||||
retention: z.literal("complete-record"),
|
||||
limitation_count: countSchema,
|
||||
native_dossier: nativeSummarySchema,
|
||||
}),
|
||||
),
|
||||
});
|
||||
export type EvidenceBundleSummary = z.infer<typeof evidenceBundleSummarySchema>;
|
||||
|
||||
const nativeSummary = (
|
||||
evidence: Evidence,
|
||||
): z.infer<typeof nativeSummarySchema> => {
|
||||
if (evidence.operation !== "analyze_function")
|
||||
return { available: false, reason: "different-operation" };
|
||||
// Read only producer metadata. Parsing the full dossier would allocate every
|
||||
// large collection merely to discover its already authenticated parent ID.
|
||||
const parsed = nativeMetadataSchema.safeParse(evidence.normalized_result);
|
||||
if (!parsed.success)
|
||||
return { available: false, reason: "not-recorded-or-incompatible" };
|
||||
const flow = valueFlowMetadataSchema.safeParse(parsed.data.native_value_flow);
|
||||
return {
|
||||
available: true,
|
||||
procedure_address: parsed.data.procedure.address,
|
||||
procedure_name: parsed.data.procedure.name ?? null,
|
||||
value_flow:
|
||||
parsed.data.native_value_flow == null
|
||||
? { status: "not-recorded" }
|
||||
: !flow.success
|
||||
? {
|
||||
status: "unknown",
|
||||
reason: "Value-flow metadata has an incompatible representation.",
|
||||
}
|
||||
: flow.data.available
|
||||
? {
|
||||
status: "observed",
|
||||
truncated: flow.data.truncated,
|
||||
omitted_operations_lower_bound:
|
||||
flow.data.omitted_operations_lower_bound ?? null,
|
||||
known_omitted_inputs: flow.data.known_omitted_inputs ?? null,
|
||||
known_omitted_edges: flow.data.known_omitted_edges ?? null,
|
||||
}
|
||||
: { status: "unavailable", reason: flow.data.reason },
|
||||
};
|
||||
};
|
||||
|
||||
/** Discover retained Evidence without reading or copying its complete payloads. */
|
||||
export const summarizeEvidenceBundle = (
|
||||
bundle: EvidenceBundle,
|
||||
filters: EvidenceRecordFilters = {},
|
||||
): EvidenceBundleSummary => {
|
||||
const selectedFilters: Record<string, string> = {};
|
||||
for (const [field, value] of Object.entries(filters))
|
||||
if (value !== undefined) selectedFilters[field] = value;
|
||||
const records: EvidenceBundleSummary["records"] = [];
|
||||
for (const evidence of bundle.records) {
|
||||
if (
|
||||
(filters.evidence_id !== undefined &&
|
||||
filters.evidence_id !== evidence.evidence_id) ||
|
||||
(filters.operation !== undefined &&
|
||||
filters.operation !== evidence.operation) ||
|
||||
(filters.target_sha256 !== undefined &&
|
||||
filters.target_sha256 !== evidence.subject?.digest.sha256) ||
|
||||
(filters.analysis_profile_digest !== undefined &&
|
||||
filters.analysis_profile_digest !== evidence.analysis_profile?.digest)
|
||||
)
|
||||
continue;
|
||||
const native = nativeSummary(evidence);
|
||||
if (
|
||||
filters.procedure_address !== undefined &&
|
||||
(!native.available ||
|
||||
filters.procedure_address !== native.procedure_address)
|
||||
)
|
||||
continue;
|
||||
records.push({
|
||||
evidence_id: evidence.evidence_id,
|
||||
subject: evidence.subject,
|
||||
provider: evidence.provider,
|
||||
operation: evidence.operation,
|
||||
confidence: evidence.confidence,
|
||||
authority: evidence.authority,
|
||||
analysis_profile_digest: evidence.analysis_profile?.digest ?? null,
|
||||
retention: "complete-record",
|
||||
limitation_count: evidence.limitations.length,
|
||||
native_dossier: native,
|
||||
});
|
||||
}
|
||||
return {
|
||||
kind: "evidence-bundle-summary",
|
||||
filters: selectedFilters,
|
||||
total_retained_records: bundle.records.length,
|
||||
matching_records: records.length,
|
||||
unknown_revisions: bundle.unknowns.length,
|
||||
records,
|
||||
};
|
||||
};
|
||||
@@ -3,11 +3,18 @@ import { digestSchema } from "../digests.js";
|
||||
import { analyzeJavaScriptApplicationInputSchema } from "./javascriptApplicationAnalysis.js";
|
||||
import { localPathStringSchema } from "../localPath.js";
|
||||
|
||||
/** Recover derived readable modules from one explicitly selected local script. */
|
||||
/** Recover derived readable modules from explicitly selected local scripts. */
|
||||
export const javascriptRecoveryInputSchema = z.strictObject({
|
||||
path: localPathStringSchema.describe(
|
||||
"Absolute path to a UTF-8 JavaScript file; never executed",
|
||||
),
|
||||
additional_paths: z
|
||||
.array(localPathStringSchema)
|
||||
.max(16)
|
||||
.optional()
|
||||
.describe(
|
||||
"Additional absolute UTF-8 JavaScript files recovered together with path; each is snapshotted separately so imports can resolve across related chunks. All inputs share a 64 MiB total budget",
|
||||
),
|
||||
output_directory: localPathStringSchema.describe(
|
||||
"Absolute absent output directory; parent must exist",
|
||||
),
|
||||
@@ -36,12 +43,15 @@ const rangeSchema = z.strictObject({
|
||||
end: z.number().int().nonnegative(),
|
||||
});
|
||||
|
||||
const sourceSchema = fileSchema.extend({
|
||||
snapshot_path: z.string().min(1),
|
||||
published_copy: fileSchema,
|
||||
});
|
||||
|
||||
/** Complete published artifacts and exact producer provenance for source recovery. */
|
||||
export const javascriptRecoveryResultSchema = z.strictObject({
|
||||
source: fileSchema.extend({
|
||||
snapshot_path: z.string().min(1),
|
||||
published_copy: fileSchema,
|
||||
}),
|
||||
source: sourceSchema,
|
||||
additional_sources: z.array(sourceSchema).optional(),
|
||||
engine: z.strictObject({
|
||||
id: z.string().min(1),
|
||||
version: z.string().min(1),
|
||||
|
||||
@@ -72,62 +72,6 @@ describe("read-only NativeAOT PE parser", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("does not discover an RTR directory in raw padding beyond VirtualSize", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const sectionTable = 0x98 + 0xf0;
|
||||
const dataSection = sectionTable + 40;
|
||||
bytes.writeUInt32LE(0x20, dataSection + 8);
|
||||
const parsed = parseNativeAotPe(
|
||||
bytes,
|
||||
nativeAotPeDigest(bytes),
|
||||
SUMMARY_BUDGET,
|
||||
);
|
||||
expect(parsed.summary).toMatchObject({
|
||||
status: "not_applicable",
|
||||
reason:
|
||||
"No supported ReadyToRun directory signature was found in initialized non-executable PE sections.",
|
||||
method_tables: 0,
|
||||
});
|
||||
});
|
||||
|
||||
it("uses raw size as the virtual extent when VirtualSize is zero", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const sectionTable = 0x98 + 0xf0;
|
||||
bytes.writeUInt32LE(0, sectionTable + 40 + 8);
|
||||
const parsed = parseNativeAotPe(
|
||||
bytes,
|
||||
nativeAotPeDigest(bytes),
|
||||
SUMMARY_BUDGET,
|
||||
);
|
||||
expect(parsed.summary).toMatchObject({
|
||||
status: "complete",
|
||||
method_tables: 3,
|
||||
derived_overlay: { address: "0x140002380", size_bytes: 1 },
|
||||
});
|
||||
});
|
||||
|
||||
it("accepts executable-section targets in the zero-filled virtual tail", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
const sectionTable = 0x98 + 0xf0;
|
||||
bytes.writeUInt32LE(0x300, sectionTable + 8);
|
||||
const slotOffset = 0x600 + (0x2280 - 0x2000) + 24;
|
||||
// RVA 0x1200 is executable-section zero fill: virtual size 0x300, raw size 0x200.
|
||||
bytes.writeBigUInt64LE(0x140001200n, slotOffset);
|
||||
const parsed = parseNativeAotPe(
|
||||
bytes,
|
||||
nativeAotPeDigest(bytes),
|
||||
SUMMARY_BUDGET,
|
||||
"0x140000000",
|
||||
);
|
||||
expect(parsed.summary).toMatchObject({
|
||||
status: "complete",
|
||||
method_tables: 3,
|
||||
});
|
||||
expect(parsed.readTypeDetail("0x140002280", SUMMARY_BUDGET)).toMatchObject({
|
||||
virtual_slots: [{ slot: 0, target_address: "0x140001200" }],
|
||||
});
|
||||
});
|
||||
|
||||
it("does not claim an image mapping when Ghidra rebased the image", () => {
|
||||
const bytes = nativeAotPeFixture();
|
||||
expect(
|
||||
|
||||
@@ -13,7 +13,7 @@ export const NATIVE_AOT_MAX_WORK_UNITS = 64 * 1024 * 1024;
|
||||
export const NATIVE_AOT_MAX_WORKING_MEMORY_BYTES = 64 * 1024 * 1024;
|
||||
/** Version of the built-in read-only PE metadata interpretation in profiles. */
|
||||
export const NATIVE_AOT_PROFILE_CONTRACT_REVISION =
|
||||
"rtr-9.1-x64-pe-read-only-v3";
|
||||
"rtr-9.1-x64-pe-read-only-v1";
|
||||
|
||||
export type NativeAotParserLimits = {
|
||||
readonly sourceBytes: number;
|
||||
@@ -81,12 +81,6 @@ type Section = {
|
||||
readonly rawSize: number;
|
||||
readonly characteristics: number;
|
||||
};
|
||||
// Compatible PE loaders use raw size when VirtualSize is zero. Otherwise,
|
||||
// RVA containment follows the declared VirtualSize, independent of file size.
|
||||
const virtualExtent = (section: Section): number =>
|
||||
section.virtualSize === 0 ? section.rawSize : section.virtualSize;
|
||||
const fileBackedVirtualExtent = (section: Section): number =>
|
||||
Math.min(virtualExtent(section), section.rawSize);
|
||||
type RtrSection = {
|
||||
readonly type: number;
|
||||
readonly flags: number;
|
||||
@@ -423,7 +417,10 @@ class PeImage {
|
||||
});
|
||||
for (const section of this.sections) {
|
||||
this.#range(section.rawOffset, section.rawSize);
|
||||
if (section.rva + virtualExtent(section) > this.sizeOfImage)
|
||||
if (
|
||||
section.rva + Math.max(section.virtualSize, section.rawSize) >
|
||||
this.sizeOfImage
|
||||
)
|
||||
throw new Error("PE section exceeds the declared image extent");
|
||||
}
|
||||
}
|
||||
@@ -459,7 +456,7 @@ class PeImage {
|
||||
for (const section of this.sections) {
|
||||
if (this.#budget !== undefined && !this.#budget.consumeWork(1))
|
||||
return null;
|
||||
const extent = virtualExtent(section);
|
||||
const extent = Math.max(section.virtualSize, section.rawSize);
|
||||
if (relative < section.rva || relative - section.rva > extent - size)
|
||||
continue;
|
||||
const delta = relative - section.rva;
|
||||
@@ -469,7 +466,6 @@ class PeImage {
|
||||
}
|
||||
return null;
|
||||
}
|
||||
// This checks executable section membership, not file-backed instruction bytes.
|
||||
isExecutable(address: bigint): boolean {
|
||||
if (address < this.imageBase) return false;
|
||||
const rva = address - this.imageBase;
|
||||
@@ -479,7 +475,7 @@ class PeImage {
|
||||
if (
|
||||
(section.characteristics & 0x20000000) !== 0 &&
|
||||
rva >= BigInt(section.rva) &&
|
||||
rva < BigInt(section.rva + virtualExtent(section))
|
||||
rva < BigInt(section.rva + section.rawSize)
|
||||
)
|
||||
return true;
|
||||
}
|
||||
@@ -496,7 +492,10 @@ class PeImage {
|
||||
if (this.#budget !== undefined && !this.#budget.consumeWork(1)) return 0;
|
||||
if (
|
||||
rva >= BigInt(candidate.rva) &&
|
||||
rva < BigInt(candidate.rva + virtualExtent(candidate))
|
||||
rva <
|
||||
BigInt(
|
||||
candidate.rva + Math.max(candidate.virtualSize, candidate.rawSize),
|
||||
)
|
||||
) {
|
||||
section = candidate;
|
||||
break;
|
||||
@@ -504,7 +503,9 @@ class PeImage {
|
||||
}
|
||||
return section === undefined
|
||||
? 0
|
||||
: section.rva + virtualExtent(section) - Number(rva);
|
||||
: section.rva +
|
||||
Math.max(section.virtualSize, section.rawSize) -
|
||||
Number(rva);
|
||||
}
|
||||
#range(offset: number, size: number): void {
|
||||
if (
|
||||
@@ -601,7 +602,7 @@ const findDirectory = (
|
||||
)
|
||||
continue;
|
||||
const begin = section.rawOffset;
|
||||
const end = begin + fileBackedVirtualExtent(section);
|
||||
const end = begin + section.rawSize;
|
||||
const first = (begin + 7) & ~7;
|
||||
const candidateBytes =
|
||||
first <= end - 16 ? (Math.floor((end - 16 - first) / 8) + 1) * 8 : 0;
|
||||
@@ -969,11 +970,10 @@ const discoverTables = (
|
||||
(section.characteristics & 0x20000000) !== 0
|
||||
)
|
||||
continue;
|
||||
const extent = fileBackedVirtualExtent(section);
|
||||
if (extent > 0) {
|
||||
if (section.rawSize > 0) {
|
||||
ranges.push({
|
||||
start: image.addressForRva(section.rva),
|
||||
end: image.addressForRva(section.rva + extent),
|
||||
end: image.addressForRva(section.rva + section.rawSize),
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -984,7 +984,9 @@ const discoverTables = (
|
||||
(candidate.characteristics & 0x20000000) === 0 &&
|
||||
hydrated.base >= image.addressForRva(candidate.rva) &&
|
||||
hydrated.base + BigInt(hydrated.sizeBytes) <=
|
||||
image.addressForRva(candidate.rva + virtualExtent(candidate)),
|
||||
image.addressForRva(
|
||||
candidate.rva + Math.max(candidate.virtualSize, candidate.rawSize),
|
||||
),
|
||||
);
|
||||
if (section !== undefined)
|
||||
ranges.push({
|
||||
@@ -1590,10 +1592,11 @@ export const parseNativeAotPe = (
|
||||
? "raw-image-pointer-scan"
|
||||
: "raw-image-and-rehydrated-pointer-scan",
|
||||
},
|
||||
diagnostics:
|
||||
hydrationError === null
|
||||
diagnostics: [
|
||||
...(hydrationError === null
|
||||
? []
|
||||
: [`Read-only hydration stopped: ${hydrationError}`],
|
||||
: [`Read-only hydration stopped: ${hydrationError}`]),
|
||||
],
|
||||
limitations: reportLimitations,
|
||||
};
|
||||
if (Buffer.byteLength(JSON.stringify(summary)) > maximumSummaryBytes)
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
/** Generated from package.json and package-lock.json; do not edit. */
|
||||
export const PACKAGE_METADATA = {
|
||||
name: "rea-agents",
|
||||
version: "6.4.0", // x-release-please-version
|
||||
version: "6.3.0", // x-release-please-version
|
||||
serverSdkVersion: "2.3.1",
|
||||
clientSdkVersion: "2.3.1",
|
||||
coreSdkVersion: "2.3.1",
|
||||
|
||||
@@ -226,7 +226,7 @@ describe("Ghidra platform support", () => {
|
||||
executable_role: target.executableRole,
|
||||
managed: target.managed,
|
||||
native_aot_metadata: {
|
||||
contract_revision: "rtr-9.1-x64-pe-read-only-v3",
|
||||
contract_revision: "rtr-9.1-x64-pe-read-only-v1",
|
||||
source_bytes: 128 * 1024 * 1024,
|
||||
work_units: 64 * 1024 * 1024,
|
||||
working_memory_bytes: 64 * 1024 * 1024,
|
||||
|
||||
@@ -44,15 +44,6 @@ describe("ghidraSessionRoot", () => {
|
||||
).toBe(posixBase("Users", "operator"));
|
||||
});
|
||||
|
||||
it("can select a filesystem root when the inherited temp starts with a dot-prefixed element", () => {
|
||||
expect(
|
||||
ghidraSessionRoot({
|
||||
base: posixBase(".Tmp", "scratch"),
|
||||
platform: "win32",
|
||||
}),
|
||||
).toBe(resolve(sep));
|
||||
});
|
||||
|
||||
it.skipIf(!POSIX)("prefers the platform temp directory by default", () => {
|
||||
expect(
|
||||
ghidraSessionRoot({ base: posixBase("home", "operator", ".cache") }),
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { constants } from "node:fs";
|
||||
import { lstat, open, opendir, writeFile } from "node:fs/promises";
|
||||
import { isAbsolute, join } from "node:path";
|
||||
import { lstat, mkdir, open, opendir, writeFile } from "node:fs/promises";
|
||||
import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
|
||||
import { TextDecoder } from "node:util";
|
||||
import { normalizeArtifactPath } from "../../artifacts/ArtifactPaths.js";
|
||||
import {
|
||||
AnalysisCancelledError,
|
||||
AnalysisInputError,
|
||||
@@ -89,12 +90,22 @@ export const readRecoveryFile = async (
|
||||
return Buffer.concat(chunks, result.bytes);
|
||||
};
|
||||
|
||||
/** Snapshot one caller-selected UTF-8 script before invoking the engine. */
|
||||
export const snapshotRecoveryInput = async (
|
||||
export type RecoveryInputSnapshot = {
|
||||
path: string;
|
||||
snapshot_path: string;
|
||||
relative_path: string;
|
||||
sha256: string;
|
||||
bytes: number;
|
||||
};
|
||||
|
||||
/** Snapshot one script within the remaining operation-wide input budget. */
|
||||
const snapshotRecoveryInput = async (
|
||||
path: string,
|
||||
snapshotPath: string,
|
||||
relativePath: string,
|
||||
maximum: number,
|
||||
signal?: AbortSignal,
|
||||
) => {
|
||||
): Promise<RecoveryInputSnapshot> => {
|
||||
if (!isAbsolute(path))
|
||||
throw recoveryInputFailure(
|
||||
"path",
|
||||
@@ -102,7 +113,7 @@ export const snapshotRecoveryInput = async (
|
||||
);
|
||||
let bytes: Buffer;
|
||||
try {
|
||||
bytes = await readRecoveryFile(path, RECOVERY_LIMITS.inputBytes, signal);
|
||||
bytes = await readRecoveryFile(path, maximum, signal);
|
||||
new TextDecoder("utf-8", { fatal: true }).decode(bytes);
|
||||
} catch (cause: unknown) {
|
||||
if (cause instanceof AnalysisCancelledError) throw cause;
|
||||
@@ -112,15 +123,78 @@ export const snapshotRecoveryInput = async (
|
||||
cause,
|
||||
);
|
||||
}
|
||||
await mkdir(dirname(snapshotPath), { recursive: true, mode: 0o700 });
|
||||
await writeFile(snapshotPath, bytes, { flag: "wx", mode: 0o600 });
|
||||
return {
|
||||
path,
|
||||
snapshot_path: snapshotPath,
|
||||
relative_path: relativePath,
|
||||
sha256: recoveryDigest(bytes),
|
||||
bytes: bytes.length,
|
||||
};
|
||||
};
|
||||
|
||||
/** Preserve each selected file and its relative input layout without concatenation. */
|
||||
export const snapshotRecoveryInputs = async (
|
||||
paths: readonly [string, ...string[]],
|
||||
directory: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<readonly [RecoveryInputSnapshot, ...RecoveryInputSnapshot[]]> => {
|
||||
let commonDirectory = dirname(resolve(paths[0]));
|
||||
for (const path of paths) {
|
||||
while (true) {
|
||||
const fromCommon = relative(commonDirectory, resolve(path));
|
||||
if (
|
||||
fromCommon !== ".." &&
|
||||
!fromCommon.startsWith(`..${sep}`) &&
|
||||
!isAbsolute(fromCommon)
|
||||
)
|
||||
break;
|
||||
const parent = dirname(commonDirectory);
|
||||
if (parent === commonDirectory)
|
||||
throw recoveryInputFailure(
|
||||
"additional_paths",
|
||||
"Selected files must share a filesystem root",
|
||||
);
|
||||
commonDirectory = parent;
|
||||
}
|
||||
}
|
||||
const snapshots: RecoveryInputSnapshot[] = [];
|
||||
let remaining = RECOVERY_LIMITS.inputBytes;
|
||||
for (const path of paths) {
|
||||
const relativePath =
|
||||
paths.length === 1
|
||||
? "bundle.js"
|
||||
: relative(commonDirectory, resolve(path)).split(sep).join("/");
|
||||
try {
|
||||
if (normalizeArtifactPath(relativePath) !== relativePath)
|
||||
throw new TypeError(
|
||||
"Input names must already use NFC Unicode spelling",
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
throw recoveryInputFailure(
|
||||
"path",
|
||||
`Cannot preserve selected input path in published artifacts: ${path}: ${recoveryFailureMessage(cause)}`,
|
||||
cause,
|
||||
);
|
||||
}
|
||||
const snapshot = await snapshotRecoveryInput(
|
||||
path,
|
||||
join(directory, relativePath),
|
||||
relativePath,
|
||||
remaining,
|
||||
signal,
|
||||
);
|
||||
snapshots.push(snapshot);
|
||||
remaining -= snapshot.bytes;
|
||||
}
|
||||
// The selected-input tuple guarantees a primary snapshot.
|
||||
const [primary, ...additional] = snapshots;
|
||||
if (primary === undefined)
|
||||
throw recoveryInputFailure("path", "Select a JavaScript input");
|
||||
return [primary, ...additional];
|
||||
};
|
||||
|
||||
/** Inventory staging without following symlinks or publishing special files. */
|
||||
export const inventoryRecoveryFiles = async (
|
||||
root: string,
|
||||
|
||||
@@ -14,8 +14,7 @@ import {
|
||||
fingerprintRecoveryFile,
|
||||
inventoryRecoveryFiles,
|
||||
readRecoveryFile,
|
||||
recoveryDigest,
|
||||
type snapshotRecoveryInput,
|
||||
type snapshotRecoveryInputs,
|
||||
} from "./RecoveryFiles.js";
|
||||
import {
|
||||
runWakaruCommand,
|
||||
@@ -38,7 +37,7 @@ type RecoveryWorkspace = {
|
||||
root: string;
|
||||
tree: SafeOutputTree;
|
||||
input: JavaScriptRecoveryInput;
|
||||
source: Awaited<ReturnType<typeof snapshotRecoveryInput>>;
|
||||
sources: Awaited<ReturnType<typeof snapshotRecoveryInputs>>;
|
||||
engine: Awaited<ReturnType<typeof resolveWakaruCommand>>;
|
||||
environment: Readonly<Record<string, string | undefined>>;
|
||||
launcher: WakaruLauncher | undefined;
|
||||
@@ -48,7 +47,7 @@ type RecoveryWorkspace = {
|
||||
|
||||
/** Consume the pinned producer representation in a private workspace. */
|
||||
const executeWakaruRecovery = async (context: RecoveryWorkspace) => {
|
||||
const { root, input, source, engine, environment, launcher, options } =
|
||||
const { root, input, sources, engine, environment, launcher, options } =
|
||||
context;
|
||||
const staging = join(root, "modules");
|
||||
const commandContext = {
|
||||
@@ -81,7 +80,7 @@ const executeWakaruRecovery = async (context: RecoveryWorkspace) => {
|
||||
...commandContext,
|
||||
staging,
|
||||
arguments: [
|
||||
source.snapshot_path,
|
||||
...sources.map((source) => source.snapshot_path),
|
||||
`--unpack=${MODES[input.extraction_mode]}`,
|
||||
`--level=${input.rewrite_level}`,
|
||||
"--provenance",
|
||||
@@ -92,16 +91,20 @@ const executeWakaruRecovery = async (context: RecoveryWorkspace) => {
|
||||
],
|
||||
});
|
||||
const files = await inventoryRecoveryFiles(staging, options?.signal);
|
||||
const snapshot = await readRecoveryFile(
|
||||
source.snapshot_path,
|
||||
RECOVERY_LIMITS.inputBytes,
|
||||
options?.signal,
|
||||
);
|
||||
const snapshots = new Map<string, Buffer>();
|
||||
for (const source of sources)
|
||||
snapshots.set(
|
||||
source.snapshot_path,
|
||||
await readRecoveryFile(
|
||||
source.snapshot_path,
|
||||
source.bytes,
|
||||
options?.signal,
|
||||
),
|
||||
);
|
||||
const parsed = await parseWakaruReports({
|
||||
execution: run,
|
||||
provenancePath: join(staging, "provenance.json"),
|
||||
snapshotPath: source.snapshot_path,
|
||||
sourceBytes: snapshot,
|
||||
snapshots,
|
||||
files,
|
||||
});
|
||||
return { version, admitted, run, parsed };
|
||||
@@ -109,7 +112,8 @@ const executeWakaruRecovery = async (context: RecoveryWorkspace) => {
|
||||
|
||||
/** Publish validated recovery and retain exact source and engine bindings. */
|
||||
export const prepareWakaruExecution = async (context: RecoveryWorkspace) => {
|
||||
const { root, tree, input, source, engine, options } = context;
|
||||
const { root, tree, input, sources, engine, options } = context;
|
||||
const [source] = sources;
|
||||
const staging = join(root, "modules");
|
||||
const { version, admitted, run, parsed } =
|
||||
await executeWakaruRecovery(context);
|
||||
@@ -117,26 +121,25 @@ export const prepareWakaruExecution = async (context: RecoveryWorkspace) => {
|
||||
tree,
|
||||
staging,
|
||||
input,
|
||||
source,
|
||||
sources,
|
||||
engine,
|
||||
admitted,
|
||||
parsed,
|
||||
execution: run,
|
||||
...(options?.signal === undefined ? {} : { signal: options.signal }),
|
||||
});
|
||||
if (
|
||||
recoveryDigest(
|
||||
await readRecoveryFile(
|
||||
input.path,
|
||||
RECOVERY_LIMITS.inputBytes,
|
||||
options?.signal,
|
||||
),
|
||||
) !== source.sha256
|
||||
)
|
||||
throw new AnalysisOutputError(
|
||||
OPERATION,
|
||||
`Original input changed during recovery: ${input.path}`,
|
||||
for (const selected of sources) {
|
||||
const current = await fingerprintRecoveryFile(
|
||||
selected.path,
|
||||
RECOVERY_LIMITS.inputBytes,
|
||||
options?.signal,
|
||||
);
|
||||
if (current.sha256 !== selected.sha256 || current.bytes !== selected.bytes)
|
||||
throw new AnalysisOutputError(
|
||||
OPERATION,
|
||||
`Original input changed during recovery: ${selected.path}`,
|
||||
);
|
||||
}
|
||||
if (
|
||||
(
|
||||
await fingerprintRecoveryFile(
|
||||
@@ -166,7 +169,10 @@ export const prepareWakaruExecution = async (context: RecoveryWorkspace) => {
|
||||
format: "javascript",
|
||||
},
|
||||
locations: [
|
||||
{ kind: "artifact-path", path: source.path },
|
||||
...sources.map((selected) => ({
|
||||
kind: "artifact-path" as const,
|
||||
path: selected.path,
|
||||
})),
|
||||
...result.modules.map((module) => ({
|
||||
kind: "artifact-path" as const,
|
||||
path: module.artifact.path,
|
||||
|
||||
@@ -22,7 +22,7 @@ import {
|
||||
checkRecoveryDeadline,
|
||||
recoveryFailureMessage,
|
||||
recoveryInputFailure,
|
||||
snapshotRecoveryInput,
|
||||
snapshotRecoveryInputs,
|
||||
} from "./RecoveryFiles.js";
|
||||
import {
|
||||
resolveWakaruCommand,
|
||||
@@ -55,7 +55,7 @@ export class WakaruProvider implements JavaScriptRecoveryPort {
|
||||
readonly launcher?: WakaruLauncher,
|
||||
) {}
|
||||
|
||||
/** Recover one script using validated upstream reports and verified output bytes. */
|
||||
/** Recover selected scripts using validated reports and verified output bytes. */
|
||||
recover(
|
||||
input: JavaScriptRecoveryInput,
|
||||
options?: ExecutionOptions,
|
||||
@@ -109,9 +109,9 @@ export class WakaruProvider implements JavaScriptRecoveryPort {
|
||||
const staging = join(root, "modules");
|
||||
await mkdir(inputs, { mode: 0o700 });
|
||||
await mkdir(staging, { mode: 0o700 });
|
||||
const source = await snapshotRecoveryInput(
|
||||
input.path,
|
||||
join(inputs, "bundle.js"),
|
||||
const sources = await snapshotRecoveryInputs(
|
||||
[input.path, ...(input.additional_paths ?? [])],
|
||||
inputs,
|
||||
options?.signal,
|
||||
);
|
||||
try {
|
||||
@@ -128,7 +128,7 @@ export class WakaruProvider implements JavaScriptRecoveryPort {
|
||||
root,
|
||||
tree,
|
||||
input,
|
||||
source,
|
||||
sources,
|
||||
engine,
|
||||
environment: this.environment,
|
||||
launcher: this.launcher,
|
||||
|
||||
@@ -7,7 +7,11 @@ import {
|
||||
type JavaScriptRecoveryInput,
|
||||
type JavaScriptRecoveryResult,
|
||||
} from "../../domain/javascript/javascriptRecovery.js";
|
||||
import { readRecoveryFile, recoveryDigest } from "./RecoveryFiles.js";
|
||||
import {
|
||||
readRecoveryFile,
|
||||
recoveryDigest,
|
||||
type RecoveryInputSnapshot,
|
||||
} from "./RecoveryFiles.js";
|
||||
import {
|
||||
recoveryRanges,
|
||||
validateRecoverySourceMap,
|
||||
@@ -28,12 +32,7 @@ type PublicationContext = {
|
||||
tree: SafeOutputTree;
|
||||
staging: string;
|
||||
input: JavaScriptRecoveryInput;
|
||||
source: {
|
||||
path: string;
|
||||
snapshot_path: string;
|
||||
sha256: string;
|
||||
bytes: number;
|
||||
};
|
||||
sources: readonly [RecoveryInputSnapshot, ...RecoveryInputSnapshot[]];
|
||||
engine: Awaited<ReturnType<typeof resolveWakaruCommand>>;
|
||||
admitted: Exclude<WakaruVersionAdmission, { status: "unsupported" }>;
|
||||
parsed: Awaited<ReturnType<typeof parseWakaruReports>>;
|
||||
@@ -47,7 +46,7 @@ type RecoveryWriter = (path: string, bytes: Buffer) => Promise<PublishedFile>;
|
||||
export const publishWakaruArtifacts = async (
|
||||
context: PublicationContext,
|
||||
): Promise<JavaScriptRecoveryResult> => {
|
||||
const { tree, source, parsed, staging, signal } = context;
|
||||
const { tree, sources, parsed, staging, signal } = context;
|
||||
const write = async (path: string, bytes: Buffer) => {
|
||||
const sha256 = recoveryDigest(bytes);
|
||||
const file = await tree.write(
|
||||
@@ -62,17 +61,25 @@ export const publishWakaruArtifacts = async (
|
||||
bytes: file.bytesWritten,
|
||||
};
|
||||
};
|
||||
const snapshot = await readRecoveryFile(
|
||||
source.snapshot_path,
|
||||
RECOVERY_LIMITS.inputBytes,
|
||||
signal,
|
||||
);
|
||||
if (recoveryDigest(snapshot) !== source.sha256)
|
||||
throw new AnalysisOutputError(
|
||||
"recover_javascript_sources",
|
||||
"The input snapshot changed during recovery",
|
||||
const publishedSources: JavaScriptRecoveryResult["source"][] = [];
|
||||
for (const selected of sources) {
|
||||
const { relative_path, ...source } = selected;
|
||||
const snapshot = await readRecoveryFile(
|
||||
source.snapshot_path,
|
||||
source.bytes,
|
||||
signal,
|
||||
);
|
||||
const publishedCopy = await write("inputs/bundle.js", snapshot);
|
||||
if (recoveryDigest(snapshot) !== source.sha256)
|
||||
throw new AnalysisOutputError(
|
||||
"recover_javascript_sources",
|
||||
`The input snapshot changed during recovery: ${source.path}`,
|
||||
);
|
||||
publishedSources.push({
|
||||
...source,
|
||||
published_copy: await write(`inputs/${relative_path}`, snapshot),
|
||||
});
|
||||
}
|
||||
const [source, ...additionalSources] = publishedSources;
|
||||
const modules: JavaScriptRecoveryResult["modules"] = [];
|
||||
for (const module of parsed.report.modules) {
|
||||
modules.push(await publishModule(context, module, write));
|
||||
@@ -90,7 +97,10 @@ export const publishWakaruArtifacts = async (
|
||||
),
|
||||
);
|
||||
const details = {
|
||||
source: { ...source, published_copy: publishedCopy },
|
||||
source,
|
||||
...(additionalSources.length === 0
|
||||
? {}
|
||||
: { additional_sources: additionalSources }),
|
||||
engine: {
|
||||
id: "wakaru",
|
||||
version: context.admitted.version,
|
||||
@@ -151,7 +161,7 @@ const publishModule = async (
|
||||
module: PublicationContext["parsed"]["report"]["modules"][number],
|
||||
write: RecoveryWriter,
|
||||
): Promise<JavaScriptRecoveryResult["modules"][number]> => {
|
||||
const { staging, source, parsed, signal } = context;
|
||||
const { staging, sources, parsed, signal } = context;
|
||||
const artifact = await write(
|
||||
`modules/${module.filename}`,
|
||||
await readRecoveryFile(
|
||||
@@ -182,6 +192,14 @@ const publishModule = async (
|
||||
"recover_javascript_sources",
|
||||
`Missing module provenance: ${module.filename}`,
|
||||
);
|
||||
const origin = sources.find(
|
||||
(source) => source.snapshot_path === provenance.input,
|
||||
);
|
||||
if (origin === undefined)
|
||||
throw new AnalysisOutputError(
|
||||
"recover_javascript_sources",
|
||||
`Unselected module provenance input: ${provenance.input}`,
|
||||
);
|
||||
return {
|
||||
reported_filename: module.filename,
|
||||
reported_status: module.status,
|
||||
@@ -189,8 +207,8 @@ const publishModule = async (
|
||||
source_map: sourceMap,
|
||||
provenance: {
|
||||
reported_input: provenance.input,
|
||||
original_path: source.path,
|
||||
original_sha256: source.sha256,
|
||||
original_path: origin.path,
|
||||
original_sha256: origin.sha256,
|
||||
extraction: provenance.extraction,
|
||||
byte_ranges: recoveryRanges(provenance.ranges),
|
||||
context_byte_ranges: recoveryRanges(provenance.context_ranges ?? []),
|
||||
|
||||
@@ -36,5 +36,5 @@ export const RECOVERY_LIMITATIONS = [
|
||||
"Extraction ranges refer to half-open UTF-8 byte offsets in the original input, not rewritten line positions or whole-bundle coverage.",
|
||||
"Emitted source maps are preserved as reported by the engine; their mapping accuracy is not independently established.",
|
||||
"Inspection mode may produce non-executable regions. No input or recovered application code is executed.",
|
||||
"Linux adapter: one worker, 1 GiB address-space ceiling, 120-second deadline, 64 MiB input, 128 MiB output, 10000 output entries and 8 MiB per diagnostic/report stream. Resource limits are per operation, not aggregate host containment.",
|
||||
"Linux adapter: one worker, 1 GiB address-space ceiling, 120-second deadline, 64 MiB total input, 128 MiB output, 10000 output entries and 8 MiB per diagnostic/report stream. Resource limits are per operation, not aggregate host containment.",
|
||||
] as const;
|
||||
|
||||
@@ -54,8 +54,7 @@ const OPERATION = "recover_javascript_sources";
|
||||
type ReportContext = {
|
||||
execution: Awaited<ReturnType<typeof runWakaruCommand>>;
|
||||
provenancePath: string;
|
||||
snapshotPath: string;
|
||||
sourceBytes: Buffer;
|
||||
snapshots: ReadonlyMap<string, Buffer>;
|
||||
files: readonly { relativePath: string }[];
|
||||
};
|
||||
|
||||
@@ -121,7 +120,7 @@ const validateModuleSet = (
|
||||
provenance: z.output<typeof provenanceSchema>,
|
||||
context: ReportContext,
|
||||
): void => {
|
||||
const { files, snapshotPath, sourceBytes } = context;
|
||||
const { files, snapshots } = context;
|
||||
const registry = new ArtifactPathRegistry();
|
||||
const expected = new Set(["provenance.json"]);
|
||||
const observed = new Set(files.map((file) => file.relativePath));
|
||||
@@ -135,7 +134,9 @@ const validateModuleSet = (
|
||||
expected.add(path);
|
||||
if (observed.has(`${path}.map`)) expected.add(`${path}.map`);
|
||||
const entry = provenance.modules[path];
|
||||
if (entry === undefined || entry.input !== snapshotPath)
|
||||
const sourceBytes =
|
||||
entry === undefined ? undefined : snapshots.get(entry.input);
|
||||
if (entry === undefined || sourceBytes === undefined)
|
||||
throw new TypeError(
|
||||
`Module provenance does not bind the selected input snapshot: ${path}`,
|
||||
);
|
||||
|
||||
@@ -28,9 +28,8 @@ interface Section {
|
||||
readonly size: number;
|
||||
}
|
||||
|
||||
// RVA ownership follows the declared virtual extent. Raw size describes
|
||||
// initialized file bytes and can include file-alignment padding; loaders may
|
||||
// map page-rounded tails, but those bytes are outside the declared section.
|
||||
// The image loader maps VirtualSize bytes, using SizeOfRawData only when
|
||||
// VirtualSize is zero. Raw bytes past that extent are file-alignment padding.
|
||||
const virtualExtent = (section: Section): number =>
|
||||
section.virtualSize === 0 ? section.size : section.virtualSize;
|
||||
|
||||
|
||||
@@ -339,61 +339,11 @@ interface OwnedCleanupPlan {
|
||||
readonly unverified: readonly OwnershipSweepUnverifiedProcess[];
|
||||
}
|
||||
|
||||
// A process-table read can race exit and retain the old state with an altered
|
||||
// command. Only a fresh table proving that PID is no longer live can settle
|
||||
// that mismatch; surviving group members still require normal token checks.
|
||||
const settleLauncherExit = async (
|
||||
const createOwnedCleanupPlan = async (
|
||||
ownership: OwnedProcessGroup,
|
||||
processes: readonly ProcessTableEntry[],
|
||||
host: ProcessOwnershipHost,
|
||||
): Promise<
|
||||
| readonly ProcessTableEntry[]
|
||||
| Extract<ProcessCleanupResult, { readonly cleaned: false }>
|
||||
> => {
|
||||
const launcher = processes.find(({ pid }) => pid === ownership.leaderPid);
|
||||
if (launcher === undefined) return processes;
|
||||
const reason = launcherIdentityFailure(launcher, ownership);
|
||||
if (reason === null) return processes;
|
||||
const refreshed = await readLiveProcessTable(host);
|
||||
if (
|
||||
refreshed.available &&
|
||||
!refreshed.processes.some(({ pid }) => pid === launcher.pid)
|
||||
) {
|
||||
// Refreshing must not discard a surviving detached descendant's lineage,
|
||||
// including a child that changed groups while the launcher exited.
|
||||
const descendants = descendantsOf([launcher.pid], processes);
|
||||
const descendantPids = new Set(descendants.map(({ pid }) => pid));
|
||||
const descendantGroupIds = new Set(
|
||||
descendants.map(({ processGroupId }) => processGroupId),
|
||||
);
|
||||
const refreshedDescendantPids = new Set(
|
||||
descendantsOf(
|
||||
descendants.map(({ pid }) => pid),
|
||||
refreshed.processes,
|
||||
).map(({ pid }) => pid),
|
||||
);
|
||||
if (
|
||||
refreshed.processes.some(
|
||||
(process) =>
|
||||
process.processGroupId !== ownership.processGroupId &&
|
||||
(refreshedDescendantPids.has(process.pid) ||
|
||||
descendantPids.has(process.pid) ||
|
||||
descendantGroupIds.has(process.processGroupId)),
|
||||
)
|
||||
)
|
||||
return { cleaned: false, reason };
|
||||
return refreshed.processes;
|
||||
}
|
||||
return { cleaned: false, reason };
|
||||
};
|
||||
|
||||
const createOwnedCleanupPlan = async (
|
||||
ownership: OwnedProcessGroup,
|
||||
initialProcesses: readonly ProcessTableEntry[],
|
||||
host: ProcessOwnershipHost,
|
||||
): Promise<OwnedCleanupPlan | ProcessCleanupResult> => {
|
||||
const processes = await settleLauncherExit(ownership, initialProcesses, host);
|
||||
if ("cleaned" in processes) return processes;
|
||||
const sampledProcessGroupIds = new Set(
|
||||
(ownership.sampledProcessGroupIds ?? []).filter(
|
||||
(processGroupId) =>
|
||||
@@ -447,7 +397,10 @@ const createOwnedCleanupPlan = async (
|
||||
};
|
||||
let descendants: readonly ProcessTableEntry[] = [];
|
||||
if (launcher !== undefined) {
|
||||
descendants = descendantsOf([launcher.pid], processes);
|
||||
const identityFailure = launcherIdentityFailure(launcher, ownership);
|
||||
if (identityFailure !== null)
|
||||
return { cleaned: false, reason: identityFailure };
|
||||
descendants = descendantsOf(launcher.pid, processes);
|
||||
}
|
||||
const descendantPids = new Set(descendants.map(({ pid }) => pid));
|
||||
const processGroupIds = new Set<number>([ownership.processGroupId]);
|
||||
@@ -543,13 +496,7 @@ const revalidateOwnedProcessGroup = async (
|
||||
): Promise<{ readonly empty: boolean } | ProcessCleanupResult> => {
|
||||
let members: readonly ProcessTableEntry[];
|
||||
try {
|
||||
const processes = await settleLauncherExit(
|
||||
ownership,
|
||||
liveProcesses(await host.listProcesses()),
|
||||
host,
|
||||
);
|
||||
if ("cleaned" in processes) return processes;
|
||||
members = processes.filter(
|
||||
members = liveProcesses(await host.listProcesses()).filter(
|
||||
({ processGroupId: observedGroupId }) =>
|
||||
observedGroupId === processGroupId,
|
||||
);
|
||||
@@ -566,6 +513,14 @@ const revalidateOwnedProcessGroup = async (
|
||||
expectedIdentities,
|
||||
);
|
||||
if (failures.length > 0) return cleanupValidationFailure(failures);
|
||||
if (processGroupId === ownership.processGroupId) {
|
||||
const launcher = members.find(({ pid }) => pid === ownership.leaderPid);
|
||||
if (launcher !== undefined) {
|
||||
const identityFailure = launcherIdentityFailure(launcher, ownership);
|
||||
if (identityFailure !== null)
|
||||
return { cleaned: false, reason: identityFailure };
|
||||
}
|
||||
}
|
||||
if (
|
||||
processGroupId !== ownership.processGroupId &&
|
||||
members.length > 0 &&
|
||||
@@ -895,7 +850,7 @@ const scanTokenOwnedProcesses = async (
|
||||
const relatedGroupIds = new Set<number>();
|
||||
if (relation !== undefined) {
|
||||
relatedPids.add(relation.leaderPid);
|
||||
for (const descendant of descendantsOf([relation.leaderPid], processes))
|
||||
for (const descendant of descendantsOf(relation.leaderPid, processes))
|
||||
relatedPids.add(descendant.pid);
|
||||
relatedGroupIds.add(relation.processGroupId);
|
||||
for (const processGroupId of relation.sampledProcessGroupIds ?? [])
|
||||
|
||||
@@ -3,7 +3,6 @@ import {
|
||||
cleanupOwnedProcessGroup,
|
||||
cleanupWindowsProcessTree,
|
||||
type ProcessOwnershipHost,
|
||||
type ProcessTableEntry,
|
||||
type WindowsProcessTreeHost,
|
||||
} from "./ProcessOwnership.js";
|
||||
import {
|
||||
@@ -12,20 +11,18 @@ import {
|
||||
} from "./ProcessOwnershipObservation.js";
|
||||
import { host, ownership } from "./ProcessOwnership.fixture.js";
|
||||
|
||||
const launcher: ProcessTableEntry = {
|
||||
pid: 100,
|
||||
parentPid: 1,
|
||||
processGroupId: 100,
|
||||
state: "S",
|
||||
command: "fixture",
|
||||
};
|
||||
|
||||
describe("owned process-group cleanup validation: ownership and lineage", () => {
|
||||
it("fails closed when a descendant in another process group lacks the token", async () => {
|
||||
const adapter: ProcessOwnershipHost = {
|
||||
listProcesses: () =>
|
||||
Promise.resolve([
|
||||
launcher,
|
||||
{
|
||||
pid: 100,
|
||||
parentPid: 1,
|
||||
processGroupId: 100,
|
||||
state: "S",
|
||||
command: "fixture",
|
||||
},
|
||||
{
|
||||
pid: 101,
|
||||
parentPid: 100,
|
||||
@@ -214,9 +211,16 @@ describe("owned process-group cleanup liveness rechecks", () => {
|
||||
);
|
||||
|
||||
it("accepts a member that exits during ownership revalidation", async () => {
|
||||
const liveLauncher = {
|
||||
pid: 100,
|
||||
parentPid: 1,
|
||||
processGroupId: 100,
|
||||
state: "S",
|
||||
command: "fixture",
|
||||
};
|
||||
const listProcesses = vi
|
||||
.fn<ProcessOwnershipHost["listProcesses"]>()
|
||||
.mockResolvedValueOnce([launcher])
|
||||
.mockResolvedValueOnce([liveLauncher])
|
||||
.mockResolvedValue([]);
|
||||
const signalGroup = vi.fn();
|
||||
const adapter: ProcessOwnershipHost = {
|
||||
@@ -235,181 +239,6 @@ describe("owned process-group cleanup liveness rechecks", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("owned process-group cleanup validation: refreshed descendant lineage", () => {
|
||||
it.each(["planning", "signaling"])(
|
||||
"retains cleanup uncertainty for newly detached descendants during %s",
|
||||
async (phase) => {
|
||||
const child = { ...launcher, pid: 101, parentPid: 100 };
|
||||
const stale = { ...launcher, command: "[MainThread]" };
|
||||
const signalGroup = vi.fn();
|
||||
let reads = 0;
|
||||
const adapter: ProcessOwnershipHost = {
|
||||
listProcesses: () => {
|
||||
reads += 1;
|
||||
if (phase === "signaling" && reads === 1)
|
||||
return Promise.resolve([launcher, child]);
|
||||
if (reads === (phase === "planning" ? 1 : 2))
|
||||
return Promise.resolve([stale, child]);
|
||||
return Promise.resolve([
|
||||
{ ...child, parentPid: 1 },
|
||||
{ ...child, pid: 102, parentPid: 101 },
|
||||
{ ...child, pid: 103, parentPid: 102, processGroupId: 103 },
|
||||
]);
|
||||
},
|
||||
environment: () =>
|
||||
Promise.resolve({ REA_PROCESS_RUN_ID: ownership.runId }),
|
||||
signalGroup,
|
||||
};
|
||||
await expect(
|
||||
cleanupOwnedProcessGroup(
|
||||
{ ...ownership, expectedCommand: "fixture" },
|
||||
adapter,
|
||||
),
|
||||
).resolves.toMatchObject({ cleaned: false });
|
||||
expect(signalGroup).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("owned process-group cleanup validation: launcher exit races", () => {
|
||||
it.each([101, 999])(
|
||||
"retains cleanup uncertainty for a detached descendant in group %s after launcher exit",
|
||||
async (processGroupId) => {
|
||||
const child = {
|
||||
...launcher,
|
||||
pid: 101,
|
||||
parentPid: 100,
|
||||
processGroupId: 101,
|
||||
};
|
||||
const signalGroup = vi.fn();
|
||||
const listProcesses = vi
|
||||
.fn<ProcessOwnershipHost["listProcesses"]>()
|
||||
.mockResolvedValueOnce([
|
||||
{ ...launcher, command: "[MainThread]" },
|
||||
child,
|
||||
])
|
||||
.mockResolvedValue([{ ...child, parentPid: 1, processGroupId }]);
|
||||
await expect(
|
||||
cleanupOwnedProcessGroup(
|
||||
{
|
||||
...ownership,
|
||||
expectedCommand: "fixture",
|
||||
},
|
||||
{
|
||||
listProcesses,
|
||||
environment: () =>
|
||||
Promise.resolve({ REA_PROCESS_RUN_ID: "run-token" }),
|
||||
signalGroup,
|
||||
},
|
||||
),
|
||||
).resolves.toMatchObject({ cleaned: false });
|
||||
expect(signalGroup).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
it.each(["planning", "signaling"])(
|
||||
"settles a stale launcher command during %s without skipping surviving group ownership",
|
||||
async (phase) => {
|
||||
for (const remaining of ["absent", "zombie", "owned", "foreign"]) {
|
||||
const stale = { ...launcher, command: "[MainThread]" };
|
||||
const child = { ...launcher, pid: 101, command: "child" };
|
||||
let reads = 0;
|
||||
const signalGroup = vi.fn();
|
||||
const adapter: ProcessOwnershipHost = {
|
||||
listProcesses: () => {
|
||||
reads += 1;
|
||||
if (phase === "signaling" && reads === 1)
|
||||
return Promise.resolve([launcher]);
|
||||
if (reads === (phase === "planning" ? 1 : 2))
|
||||
return Promise.resolve([stale]);
|
||||
return Promise.resolve(
|
||||
remaining === "absent"
|
||||
? []
|
||||
: remaining === "zombie"
|
||||
? [{ ...stale, state: "Z" }]
|
||||
: [child],
|
||||
);
|
||||
},
|
||||
environment: () =>
|
||||
Promise.resolve({
|
||||
REA_PROCESS_RUN_ID:
|
||||
remaining === "foreign" && reads > 1
|
||||
? "other-run"
|
||||
: "run-token",
|
||||
}),
|
||||
signalGroup,
|
||||
};
|
||||
const result = await cleanupOwnedProcessGroup(
|
||||
{ ...ownership, expectedCommand: "fixture" },
|
||||
adapter,
|
||||
);
|
||||
if (remaining === "foreign")
|
||||
expect(result).toMatchObject({
|
||||
cleaned: false,
|
||||
reason: "process tree contains an unowned or PID-reused process",
|
||||
});
|
||||
else
|
||||
expect(result).toEqual({
|
||||
cleaned: true,
|
||||
signaled: remaining === "owned",
|
||||
});
|
||||
if (remaining === "owned")
|
||||
expect(signalGroup).toHaveBeenCalledWith(100, "SIGKILL");
|
||||
else expect(signalGroup).not.toHaveBeenCalled();
|
||||
}
|
||||
},
|
||||
);
|
||||
it("fails closed when a mismatched launcher's exit cannot be inspected", async () => {
|
||||
const { adapter, signalGroup } = host({
|
||||
100: { REA_PROCESS_RUN_ID: "run-token" },
|
||||
});
|
||||
const listProcesses = vi
|
||||
.fn<ProcessOwnershipHost["listProcesses"]>()
|
||||
.mockResolvedValueOnce(await adapter.listProcesses())
|
||||
.mockRejectedValue(new Error("process table unavailable"));
|
||||
await expect(
|
||||
cleanupOwnedProcessGroup(
|
||||
{ ...ownership, expectedCommand: "/owned/hopper" },
|
||||
{ ...adapter, listProcesses },
|
||||
),
|
||||
).resolves.toMatchObject({
|
||||
cleaned: false,
|
||||
reason: expect.stringContaining("command identity did not match"),
|
||||
});
|
||||
expect(signalGroup).not.toHaveBeenCalled();
|
||||
});
|
||||
it.each(["planning", "signaling"])(
|
||||
"rejects a surviving launcher PID in another group during %s",
|
||||
async (phase) => {
|
||||
let reads = 0;
|
||||
const signalGroup = vi.fn();
|
||||
const adapter: ProcessOwnershipHost = {
|
||||
listProcesses: () => {
|
||||
reads += 1;
|
||||
return Promise.resolve([
|
||||
phase === "signaling" && reads === 1
|
||||
? launcher
|
||||
: reads === (phase === "planning" ? 1 : 2)
|
||||
? { ...launcher, command: "[MainThread]" }
|
||||
: { ...launcher, processGroupId: 999 },
|
||||
]);
|
||||
},
|
||||
environment: () => Promise.resolve({ REA_PROCESS_RUN_ID: "run-token" }),
|
||||
signalGroup,
|
||||
};
|
||||
await expect(
|
||||
cleanupOwnedProcessGroup(
|
||||
{ ...ownership, expectedCommand: "fixture" },
|
||||
adapter,
|
||||
),
|
||||
).resolves.toMatchObject({
|
||||
cleaned: false,
|
||||
reason: expect.stringContaining("command identity did not match"),
|
||||
});
|
||||
expect(signalGroup).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("owned process-group cleanup validation: exited members", () => {
|
||||
it("preserves cancellation during process-table observation instead of reporting unverifiable ownership", async () => {
|
||||
const controller = new AbortController();
|
||||
@@ -464,7 +293,13 @@ describe("owned process-group cleanup validation: exited members", () => {
|
||||
const adapter: ProcessOwnershipHost = {
|
||||
listProcesses: () =>
|
||||
Promise.resolve([
|
||||
launcher,
|
||||
{
|
||||
pid: 100,
|
||||
parentPid: 1,
|
||||
processGroupId: 100,
|
||||
state: "S",
|
||||
command: "fixture",
|
||||
},
|
||||
{
|
||||
pid: 101,
|
||||
parentPid: 100,
|
||||
|
||||
@@ -116,7 +116,7 @@ export const observeOwnedProcessLineage = async (
|
||||
const identityFailure = launcherIdentityFailure(launcher, ownership);
|
||||
if (identityFailure !== null)
|
||||
return unavailableLineage(ownership, identityFailure);
|
||||
const descendants = descendantsOf([launcher.pid], processes);
|
||||
const descendants = descendantsOf(launcher.pid, processes);
|
||||
const verifiedDescendants: ProcessTableEntry[] = [];
|
||||
for await (const { process: member, observation } of readProcessRunTokens(
|
||||
host,
|
||||
|
||||
@@ -6,9 +6,9 @@ export const liveProcesses = (
|
||||
): readonly ProcessTableEntry[] =>
|
||||
members.filter(({ state }) => !state.startsWith("Z"));
|
||||
|
||||
/** Return every live-table descendant of the selected roots in breadth-first order. */
|
||||
/** Return every live-table descendant of a launcher in breadth-first order. */
|
||||
export const descendantsOf = (
|
||||
rootPids: readonly number[],
|
||||
launcherPid: number,
|
||||
processes: readonly ProcessTableEntry[],
|
||||
): ProcessTableEntry[] => {
|
||||
const childrenByParent = new Map<number, ProcessTableEntry[]>();
|
||||
@@ -18,8 +18,8 @@ export const descendantsOf = (
|
||||
childrenByParent.set(process.parentPid, siblings);
|
||||
}
|
||||
const descendants: ProcessTableEntry[] = [];
|
||||
const pending = rootPids.flatMap((pid) => childrenByParent.get(pid) ?? []);
|
||||
const visited = new Set<number>(rootPids);
|
||||
const pending = [...(childrenByParent.get(launcherPid) ?? [])];
|
||||
const visited = new Set<number>([launcherPid]);
|
||||
for (const process of pending) {
|
||||
if (visited.has(process.pid)) continue;
|
||||
visited.add(process.pid);
|
||||
|
||||
@@ -8,7 +8,6 @@ import {
|
||||
import { toolContract } from "../contracts/toolContracts.js";
|
||||
import type { EvidenceBundle } from "../domain/evidenceBundle.js";
|
||||
import { ok } from "../domain/result.js";
|
||||
import { inspectEvidenceBundle } from "../application/investigation/InspectEvidenceBundle.js";
|
||||
import { toolRegistrationOptions } from "./toolRegistrationOptions.js";
|
||||
|
||||
interface EvidenceToolRegistration {
|
||||
@@ -73,9 +72,9 @@ const registerSnapshotEvidenceTool = ({
|
||||
server.registerTool(
|
||||
snapshotContract.name,
|
||||
toolRegistrationOptions(snapshotContract),
|
||||
(input) =>
|
||||
() =>
|
||||
server.delivery.toCallToolResult(
|
||||
inspectEvidenceBundle(bundleForSerialization(session), input),
|
||||
ok(bundleForSerialization(session)),
|
||||
snapshotContract,
|
||||
),
|
||||
);
|
||||
|
||||
@@ -6,7 +6,7 @@ import { promisify } from "node:util";
|
||||
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { afterAll, describe, expect, it } from "vitest";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { z } from "zod";
|
||||
|
||||
import { javascriptApplicationAnalysisResultSchema } from "../../../src/domain/javascript/javascriptApplicationAnalysis.js";
|
||||
@@ -18,27 +18,6 @@ const requireFixture = createRequire(import.meta.url);
|
||||
const comparisonEnvelope = z.object({
|
||||
normalized_result: javaScriptExportShapeComparisonResultSchema,
|
||||
});
|
||||
let mcpComparisonConnection:
|
||||
| {
|
||||
readonly client: Client;
|
||||
readonly transport: StdioClientTransport;
|
||||
readonly ready: Promise<void>;
|
||||
}
|
||||
| undefined;
|
||||
|
||||
afterAll(async () => {
|
||||
if (mcpComparisonConnection === undefined) return;
|
||||
try {
|
||||
await mcpComparisonConnection.ready;
|
||||
} catch {
|
||||
// Keep the failed connection owned until both resources are closed below.
|
||||
}
|
||||
try {
|
||||
await mcpComparisonConnection.client.close();
|
||||
} finally {
|
||||
await mcpComparisonConnection.transport.close();
|
||||
}
|
||||
});
|
||||
|
||||
describe("exported function binding resolution through the CLI", () => {
|
||||
it.each([
|
||||
@@ -310,113 +289,6 @@ describe("CommonJS assignment-chain receiver identity through the CLI", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("CommonJS export identity and scope regressions", () => {
|
||||
it("does not export a named write through the original exports alias after replacement", async () => {
|
||||
const fixture = await analyzeVersions(
|
||||
(count) =>
|
||||
`const VERSION = ${String(count)};\nexports.parse = (module.exports = () => ({ count: VERSION }));`,
|
||||
"parse",
|
||||
"app.cjs",
|
||||
);
|
||||
for (const path of [fixture.leftPath, fixture.rightPath]) {
|
||||
const value = requireFixture(path);
|
||||
expect(typeof value).toBe("function");
|
||||
expect(Reflect.ownKeys(value)).toEqual(["length", "name"]);
|
||||
}
|
||||
const cli = await compareThroughCli(fixture);
|
||||
expect(cli).toMatchObject({
|
||||
left: { status: "missing" },
|
||||
right: { status: "missing" },
|
||||
summary: { added: 0, removed: 0, changed: 0 },
|
||||
});
|
||||
expect(await compareThroughMcp(fixture.input)).toEqual(cli);
|
||||
});
|
||||
|
||||
it("keeps a dead var shadow of the alias-chain RHS unavailable", async () => {
|
||||
const fixture = await analyzeVersions(
|
||||
(count) =>
|
||||
`function parse() { return { count: ${String(count)} }; }\nfunction configure() { if (false) { var parse = () => ({ count: 7 }); } module.exports = exports = parse; }\nconfigure();`,
|
||||
"default",
|
||||
"app.cjs",
|
||||
);
|
||||
expect(requireFixture(fixture.leftPath)).toBeUndefined();
|
||||
expect(requireFixture(fixture.rightPath)).toBeUndefined();
|
||||
const cli = await compareThroughCli(fixture);
|
||||
expectUncertainComparison(cli);
|
||||
expect(await compareThroughMcp(fixture.input)).toEqual(cli);
|
||||
});
|
||||
|
||||
it.each(["+=", "||=", "??="] as const)(
|
||||
"does not treat module.exports %s as a definite function export",
|
||||
async (operator) => {
|
||||
const fixture = await analyzeVersions(
|
||||
(count) =>
|
||||
`module.exports ${operator} (() => ({ kind: "result", count: ${String(count)} }));`,
|
||||
"default",
|
||||
"app.cjs",
|
||||
);
|
||||
const left = requireFixture(fixture.leftPath);
|
||||
const right = requireFixture(fixture.rightPath);
|
||||
if (operator === "+=") {
|
||||
expect(typeof left).toBe("string");
|
||||
expect(typeof right).toBe("string");
|
||||
} else {
|
||||
expect(left).toEqual({});
|
||||
expect(right).toEqual({});
|
||||
}
|
||||
const cli = await compareThroughCli(fixture);
|
||||
expectUncertainComparison(cli);
|
||||
expect(await compareThroughMcp(fixture.input)).toEqual(cli);
|
||||
},
|
||||
);
|
||||
|
||||
it("keeps a parameter-default mutation unknown in a CommonJS return shape", async () => {
|
||||
const fixture = await analyzeVersions(
|
||||
(count) =>
|
||||
`const source = { value: 0 }; module.exports = exports = function current(arg = null, set = (arg = source)) { var arg; arg.value = ${String(count)}; return { kind: "result", count: source.value }; };`,
|
||||
"default",
|
||||
"app.cjs",
|
||||
);
|
||||
expect(requireFixture(fixture.leftPath)()).toEqual({
|
||||
kind: "result",
|
||||
count: 1,
|
||||
});
|
||||
expect(requireFixture(fixture.rightPath)()).toEqual({
|
||||
kind: "result",
|
||||
count: 2,
|
||||
});
|
||||
for (const evidence of [fixture.input.left, fixture.input.right]) {
|
||||
const analysis = z
|
||||
.object({
|
||||
normalized_result: javascriptApplicationAnalysisResultSchema,
|
||||
})
|
||||
.parse(evidence).normalized_result;
|
||||
const shapes = analysis.graph.nodes
|
||||
.flatMap(({ observations }) => observations)
|
||||
.find(
|
||||
({ properties }) =>
|
||||
properties.semantic_role === "export-return-shapes" &&
|
||||
properties.exported_name === "default",
|
||||
)?.properties.static_return_shapes;
|
||||
expect(shapes).toEqual([
|
||||
expect.objectContaining({
|
||||
fields: expect.arrayContaining([
|
||||
expect.objectContaining({ path: "/count", state: "unknown" }),
|
||||
]),
|
||||
}),
|
||||
]);
|
||||
}
|
||||
const cli = await compareThroughCli(fixture);
|
||||
expect(cli).toMatchObject({
|
||||
left: { status: "selected" },
|
||||
right: { status: "selected" },
|
||||
summary: { added: 0, removed: 0, changed: 0 },
|
||||
changes: [],
|
||||
});
|
||||
expect(await compareThroughMcp(fixture.input)).toEqual(cli);
|
||||
});
|
||||
});
|
||||
|
||||
describe("stable exported function bindings through the CLI", () => {
|
||||
it.each([
|
||||
{
|
||||
@@ -926,37 +798,34 @@ const compareThroughCli = async (
|
||||
};
|
||||
|
||||
const compareThroughMcp = async (input: Record<string, unknown>) => {
|
||||
const client = await getMcpComparisonClient();
|
||||
const response = await client.callTool({
|
||||
name: "compare_javascript_export_shapes",
|
||||
arguments: input,
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [resolve("scripts/rea.mjs"), "mcp"],
|
||||
cwd: process.cwd(),
|
||||
env: { PATH: process.env.PATH ?? "", REA_LOG_LEVEL: "silent" },
|
||||
stderr: "pipe",
|
||||
});
|
||||
expect(response.isError).not.toBe(true);
|
||||
await client.ping();
|
||||
return comparisonEnvelope.parse(response.structuredContent).normalized_result;
|
||||
};
|
||||
|
||||
const getMcpComparisonClient = async (): Promise<Client> => {
|
||||
if (mcpComparisonConnection === undefined) {
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [resolve("scripts/rea.mjs"), "mcp"],
|
||||
cwd: process.cwd(),
|
||||
env: { PATH: process.env.PATH ?? "", REA_LOG_LEVEL: "silent" },
|
||||
stderr: "pipe",
|
||||
const client = new Client({
|
||||
name: "exported-function-bindings",
|
||||
version: "1",
|
||||
});
|
||||
try {
|
||||
await client.connect(transport);
|
||||
const response = await client.callTool({
|
||||
name: "compare_javascript_export_shapes",
|
||||
arguments: input,
|
||||
});
|
||||
const client = new Client({
|
||||
name: "exported-function-bindings",
|
||||
version: "1",
|
||||
});
|
||||
mcpComparisonConnection = {
|
||||
client,
|
||||
transport,
|
||||
ready: client.connect(transport),
|
||||
};
|
||||
expect(response.isError).not.toBe(true);
|
||||
await client.ping();
|
||||
return comparisonEnvelope.parse(response.structuredContent)
|
||||
.normalized_result;
|
||||
} finally {
|
||||
try {
|
||||
await client.close();
|
||||
} finally {
|
||||
await transport.close();
|
||||
}
|
||||
}
|
||||
await mcpComparisonConnection.ready;
|
||||
return mcpComparisonConnection.client;
|
||||
};
|
||||
|
||||
const runtimeExport = async (
|
||||
|
||||
@@ -39,57 +39,6 @@ const expectPeInputFailureAcrossAdapters = async (input: {
|
||||
expect(parseMcpToolError(response)).toEqual({ error: result.json });
|
||||
};
|
||||
|
||||
const expectPePaddingFailureAcrossAdapters = async (input: {
|
||||
readonly cli: TestCli;
|
||||
readonly client: Client;
|
||||
readonly toolName: string;
|
||||
readonly path: string;
|
||||
}) => {
|
||||
const { cli, client, toolName, path } = input;
|
||||
const fixture = peResourceFixture();
|
||||
const virtualSize = fixture.bytes.readUInt32LE(fixture.sectionAt + 8);
|
||||
fixture.bytes.writeUInt32LE(
|
||||
0x2000 + virtualSize,
|
||||
fixture.dataOffsets[0] ?? 0,
|
||||
);
|
||||
fixture.bytes.writeUInt32LE(4, (fixture.dataOffsets[0] ?? 0) + 4);
|
||||
await writeFile(path, fixture.bytes);
|
||||
const cliResult = await cli.run({
|
||||
arguments: ["inspect-pe-resources", path, "--json"],
|
||||
});
|
||||
expect(cliResult.exitCode).not.toBe(0);
|
||||
const mcpResult = await client.callTool({
|
||||
name: toolName,
|
||||
arguments: { path },
|
||||
});
|
||||
expect(mcpResult.isError).toBe(true);
|
||||
expect(peResourcesSchema.safeParse(mcpResult.structuredContent).success).toBe(
|
||||
false,
|
||||
);
|
||||
};
|
||||
|
||||
const expectPeInputFailuresAcrossAdapters = async (input: {
|
||||
readonly cli: TestCli;
|
||||
readonly client: Client;
|
||||
readonly toolName: string;
|
||||
readonly root: string;
|
||||
}) => {
|
||||
const { cli, client, toolName, root } = input;
|
||||
const malformedPath = join(root, "malformed.exe");
|
||||
await writeFile(malformedPath, Buffer.from("not a PE image"));
|
||||
for (const [path, reason] of [
|
||||
[join(root, "missing.exe"), "invalid_value"],
|
||||
[malformedPath, "invalid_format"],
|
||||
] as const)
|
||||
await expectPeInputFailureAcrossAdapters({
|
||||
cli,
|
||||
client,
|
||||
toolName,
|
||||
path,
|
||||
reason,
|
||||
});
|
||||
};
|
||||
|
||||
cliTest(
|
||||
"inspects PE32/PE32+ resources through compiled CLI and real stdio MCP without an active target",
|
||||
async ({ cli }) => {
|
||||
@@ -172,18 +121,20 @@ cliTest(
|
||||
).toBe(evidence.evidence_id);
|
||||
expect(await readFile(path)).toEqual(fixture.bytes);
|
||||
}
|
||||
await expectPePaddingFailureAcrossAdapters({
|
||||
cli,
|
||||
client,
|
||||
toolName: contract.name,
|
||||
path: join(root, "raw-padding.exe"),
|
||||
});
|
||||
await expectPeInputFailuresAcrossAdapters({
|
||||
cli,
|
||||
client,
|
||||
toolName: contract.name,
|
||||
root,
|
||||
});
|
||||
const malformedPath = join(root, "malformed.exe");
|
||||
await writeFile(malformedPath, Buffer.from("not a PE image"));
|
||||
for (const [path, reason] of [
|
||||
[join(root, "missing.exe"), "invalid_value"],
|
||||
[malformedPath, "invalid_format"],
|
||||
] as const) {
|
||||
await expectPeInputFailureAcrossAdapters({
|
||||
cli,
|
||||
client,
|
||||
toolName: contract.name,
|
||||
path,
|
||||
reason,
|
||||
});
|
||||
}
|
||||
const absent = peResourceFixture();
|
||||
absent.bytes.fill(0, absent.directoryAt, absent.directoryAt + 8);
|
||||
const path = join(root, "absent.exe");
|
||||
|
||||
@@ -30,9 +30,8 @@ const chromeExecutable = [
|
||||
);
|
||||
|
||||
// Issue 1586's page, plus the HTML, image, use, and foreign-origin cases the
|
||||
// real snapshot must distinguish. The XHTML page is the prefixed-svg gap
|
||||
// reported on PR 1602. The tests are skipped when no Chrome-family executable
|
||||
// is installed.
|
||||
// real snapshot must distinguish. The test is skipped when no Chrome-family
|
||||
// executable is installed.
|
||||
const pageHtml = `<!doctype html><title>x</title>
|
||||
<svg width="10" height="10">
|
||||
<a id="s2" xlink:href="/xl-s2h"><text>s2</text></a>
|
||||
@@ -160,161 +159,6 @@ it.skipIf(chromeExecutable === undefined)(
|
||||
},
|
||||
);
|
||||
|
||||
const xhtmlPage = `<?xml version="1.0" encoding="UTF-8"?>
|
||||
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:svg="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
|
||||
<head><title>x</title></head>
|
||||
<body>
|
||||
<svg:svg width="10" height="10">
|
||||
<svg:a id="pa1" xlink:href="/xl-pa1"><svg:text>p</svg:text></svg:a>
|
||||
<svg:image id="img" xlink:href="/xl-image"/>
|
||||
<svg:foreignObject width="10" height="10">
|
||||
<a id="fo-a" xlink:href="/xl-html-fo">no</a>
|
||||
<h:a xmlns:h="http://www.w3.org/1999/xhtml" id="fo-h" xlink:href="/xl-h">no</h:a>
|
||||
<svg:a id="fo-svg" xlink:href="/xl-fo-svg"><svg:text>y</svg:text></svg:a>
|
||||
</svg:foreignObject>
|
||||
</svg:svg>
|
||||
<svg:a id="orphan" xlink:href="/xl-orphan"><svg:text>o</svg:text></svg:a>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" xmlns:x="http://www.w3.org/1999/xlink" width="10" height="10">
|
||||
<a id="custom" x:href="/xl-custom"><text>c</text></a>
|
||||
<a id="both" href="/href-wins" x:href="/xlink-loses"><text>b</text></a>
|
||||
<a id="empty" href="" x:href="/xlink-empty"><text>e</text></a>
|
||||
</svg>
|
||||
<svg:a id="out" xlink:href="https://cdn.example.test/out"><svg:text>out</svg:text></svg:a>
|
||||
<bogus:svg xmlns:bogus="http://example.com/not-svg" id="bogus" xlink:href="/xl-bogus"/>
|
||||
<a id="xhtml-a" xlink:href="/xl-xhtml-a">no</a>
|
||||
</body>
|
||||
</html>
|
||||
`;
|
||||
|
||||
for (const { format, page, contentType, nodeNames, link, destinations } of [
|
||||
{
|
||||
format: "XHTML",
|
||||
page: xhtmlPage,
|
||||
contentType: "application/xhtml+xml",
|
||||
nodeNames: ["html", "svg:a", "bogus:svg"],
|
||||
link: { name: "svg:a", attribute: "xlink:href" },
|
||||
destinations: [
|
||||
"/xl-pa1",
|
||||
"/xl-image",
|
||||
"/xl-fo-svg",
|
||||
"/xl-orphan",
|
||||
"/xl-custom",
|
||||
"/href-wins",
|
||||
"",
|
||||
null,
|
||||
],
|
||||
},
|
||||
{
|
||||
format: "SVG",
|
||||
page: `<svg xmlns="http://www.w3.org/2000/svg" xmlns:p="http://www.w3.org/1999/xlink"><a p:href="/linked"><text>link</text></a></svg>`,
|
||||
contentType: "image/svg+xml",
|
||||
nodeNames: ["svg", "a"],
|
||||
link: { name: "a", attribute: "p:href" },
|
||||
destinations: ["/linked"],
|
||||
},
|
||||
]) {
|
||||
it.skipIf(chromeExecutable === undefined)(
|
||||
`reports namespaced SVG links from a standalone ${format} document through headless Chrome`,
|
||||
async ({ processes }) => {
|
||||
const executable = chromeExecutable;
|
||||
if (executable === undefined) return;
|
||||
const profile = await createTestTempDirectory("rea-svg-xhtml-");
|
||||
const server = createServer((_request, response) => {
|
||||
response.writeHead(200, {
|
||||
"content-type": `${contentType}; charset=utf-8`,
|
||||
});
|
||||
response.end(page);
|
||||
});
|
||||
await new Promise<void>((resolve) => {
|
||||
server.listen(0, "127.0.0.1", resolve);
|
||||
});
|
||||
const address = server.address();
|
||||
if (address === null || typeof address === "string")
|
||||
throw new Error("XHTML fixture server did not bind a TCP port");
|
||||
const origin = `http://127.0.0.1:${String(address.port)}`;
|
||||
const pageUrl = `${origin}/page.xhtml`;
|
||||
let stderr = "";
|
||||
const child = processes.spawn(executable, [
|
||||
"--headless=new",
|
||||
...(process.env.REA_BROWSER_NO_SANDBOX === "true"
|
||||
? ["--no-sandbox"]
|
||||
: []),
|
||||
"--remote-debugging-address=127.0.0.1",
|
||||
"--remote-debugging-port=0",
|
||||
`--user-data-dir=${profile}`,
|
||||
"--no-first-run",
|
||||
"--no-default-browser-check",
|
||||
"--disable-background-networking",
|
||||
"--disable-component-update",
|
||||
"--disable-dev-shm-usage",
|
||||
"--disable-sync",
|
||||
"--metrics-recording-only",
|
||||
pageUrl,
|
||||
]);
|
||||
child.stdout.on("data", () => undefined);
|
||||
child.stderr.on("data", (chunk: Buffer) => {
|
||||
if (stderr.length < 8_192) stderr += chunk.toString("utf8");
|
||||
});
|
||||
try {
|
||||
const port = await waitForBrowserDevtoolsPort({
|
||||
child,
|
||||
executable,
|
||||
activePortPath: join(profile, "DevToolsActivePort"),
|
||||
stderr: () => stderr,
|
||||
timeoutMs: 20_000,
|
||||
});
|
||||
const endpoint = `http://127.0.0.1:${String(port)}`;
|
||||
const provider = new CdpBrowserProvider();
|
||||
const targetId = await pageTarget(provider, endpoint, origin, pageUrl);
|
||||
const result = await provider.inspectPage(
|
||||
inspectWebPageInputSchema.parse({
|
||||
cdp_endpoint: endpoint,
|
||||
allowed_origins: [origin],
|
||||
target_id: targetId,
|
||||
observation_ms: 0,
|
||||
}),
|
||||
);
|
||||
if (!result.ok) throw result.error;
|
||||
const urls = result.value.metadata.dom_urls;
|
||||
for (const name of nodeNames) {
|
||||
expect(
|
||||
result.value.dom.nodes.some((node) => node.node_name === name),
|
||||
).toBe(true);
|
||||
}
|
||||
expect(
|
||||
result.value.dom.nodes.some(
|
||||
(node) =>
|
||||
node.node_name === link.name &&
|
||||
node.attribute_names.includes(link.attribute),
|
||||
),
|
||||
).toBe(true);
|
||||
expect(urls).toEqual(
|
||||
destinations.map((destination) =>
|
||||
expect.objectContaining({
|
||||
attribute: "href",
|
||||
url:
|
||||
destination === null
|
||||
? null
|
||||
: destination === ""
|
||||
? pageUrl
|
||||
: `${origin}${destination}`,
|
||||
destination_scope:
|
||||
destination === null ? "outside_policy" : "approved",
|
||||
}),
|
||||
),
|
||||
);
|
||||
} finally {
|
||||
child.kill("SIGTERM");
|
||||
if (!(await waitForExit(child, 2_000))) child.kill("SIGKILL");
|
||||
await waitForExit(child, 2_000);
|
||||
await new Promise<void>((resolve) => {
|
||||
server.close(() => resolve());
|
||||
});
|
||||
}
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
const pageTarget = async (
|
||||
provider: CdpBrowserProvider,
|
||||
endpoint: string,
|
||||
|
||||
@@ -9,53 +9,6 @@ import { cliTest } from "../../support/cli/cliFixture.js";
|
||||
import { analysisViewLayoutEvidence } from "../../fixtures/analysisView.js";
|
||||
import { createEvidence } from "../../../src/domain/evidence.js";
|
||||
import { ghidraFunctionDossier } from "../../../src/domain/ghidraValues.fixture.js";
|
||||
import { createEvidenceBundle } from "../../../src/domain/evidenceBundle.js";
|
||||
import { inspectEvidenceBundle } from "../../../src/application/investigation/InspectEvidenceBundle.js";
|
||||
|
||||
cliTest(
|
||||
"discovers native records from a complete portable bundle using the shared workflow",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-evidence-discovery-cli-");
|
||||
const parent = createEvidence(
|
||||
{ path: "/fixtures/native.exe", format: "pe", sha256: "a".repeat(64) },
|
||||
{ id: "ghidra", name: "Ghidra", version: "12.1.4" },
|
||||
{
|
||||
operation: "analyze_function",
|
||||
parameters: {},
|
||||
result: ghidraFunctionDossier(),
|
||||
},
|
||||
);
|
||||
const bundle = createEvidenceBundle([parent]);
|
||||
const path = join(root, "evidence.json");
|
||||
await writeFile(path, JSON.stringify(bundle));
|
||||
const options = {
|
||||
detail: "summary" as const,
|
||||
filters: { evidence_id: parent.evidence_id },
|
||||
};
|
||||
const result = await cli.run({
|
||||
arguments: [
|
||||
"inspect-evidence-bundle",
|
||||
JSON.stringify({ path, ...options }),
|
||||
"--json",
|
||||
],
|
||||
environment: { HOME: root, XDG_CONFIG_HOME: root, XDG_CACHE_HOME: root },
|
||||
});
|
||||
expect(result.exitCode).toBe(0);
|
||||
const expected = inspectEvidenceBundle(bundle, options);
|
||||
if (!expected.ok) throw expected.error;
|
||||
expect(result.json).toEqual(expected.value);
|
||||
const complete = await cli.run({
|
||||
arguments: [
|
||||
"inspect-evidence-bundle",
|
||||
JSON.stringify({ path }),
|
||||
"--json",
|
||||
],
|
||||
environment: { HOME: root, XDG_CONFIG_HOME: root, XDG_CACHE_HOME: root },
|
||||
});
|
||||
expect(complete.exitCode).toBe(0);
|
||||
expect(complete.json).toEqual(bundle);
|
||||
},
|
||||
);
|
||||
cliTest(
|
||||
"projects one section from a JSON file of inline layout Evidence",
|
||||
async ({ cli }) => {
|
||||
|
||||
@@ -97,14 +97,6 @@ describe("browser comparison JSON file inputs", () => {
|
||||
INTEGRATION_TEST_TIMEOUT_MS,
|
||||
);
|
||||
|
||||
it(
|
||||
"reports malformed PNG chunks as invalid input and valid unsupported PNGs as unsupported targets",
|
||||
async () => {
|
||||
await assertPngInputErrorProjections();
|
||||
},
|
||||
INTEGRATION_TEST_TIMEOUT_MS,
|
||||
);
|
||||
|
||||
it(
|
||||
"identifies a missing capture file instead of reporting malformed JSON",
|
||||
async () => {
|
||||
@@ -131,105 +123,3 @@ describe("browser comparison JSON file inputs", () => {
|
||||
INTEGRATION_TEST_TIMEOUT_MS,
|
||||
);
|
||||
});
|
||||
|
||||
const assertPngInputErrorProjections = async (): Promise<void> => {
|
||||
const damagedBytes = noisePng(1, 1);
|
||||
damagedBytes[29] = (damagedBytes[29] ?? 0) ^ 1;
|
||||
const damaged = JSON.stringify(createWebScreenshotArtifact(damagedBytes));
|
||||
expect(
|
||||
await runCli(["compare-web-screenshots", damaged, damaged, "--json"]),
|
||||
).toMatchObject({
|
||||
code: "invalid_request",
|
||||
details: {
|
||||
operation: "compare_web_screenshots",
|
||||
issues: [
|
||||
{ path: [], reason: "invalid_format", message: expect.any(String) },
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const header = Buffer.alloc(13);
|
||||
header.writeUInt32BE(1, 0);
|
||||
header.writeUInt32BE(1, 4);
|
||||
header.set([8, 2, 0, 0, 0], 8);
|
||||
const malformedDeflate = JSON.stringify(
|
||||
createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
|
||||
pngChunk("IHDR", header),
|
||||
pngChunk("IDAT", Buffer.from("bad")),
|
||||
pngChunk("IEND", Buffer.alloc(0)),
|
||||
]),
|
||||
),
|
||||
);
|
||||
expect(
|
||||
await runCli([
|
||||
"compare-web-screenshots",
|
||||
malformedDeflate,
|
||||
malformedDeflate,
|
||||
"--json",
|
||||
]),
|
||||
).toMatchObject({
|
||||
code: "invalid_request",
|
||||
details: {
|
||||
operation: "compare_web_screenshots",
|
||||
issues: [{ path: [], reason: "invalid_format" }],
|
||||
},
|
||||
});
|
||||
|
||||
const grayscaleHeader = Buffer.alloc(13);
|
||||
grayscaleHeader.writeUInt32BE(1, 0);
|
||||
grayscaleHeader.writeUInt32BE(1, 4);
|
||||
grayscaleHeader.set([8, 0, 0, 0, 0], 8);
|
||||
const grayscale = JSON.stringify(
|
||||
createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
|
||||
pngChunk("IHDR", grayscaleHeader),
|
||||
pngChunk("IDAT", deflateSync(Buffer.from([0, 30]))),
|
||||
pngChunk("IEND", Buffer.alloc(0)),
|
||||
]),
|
||||
),
|
||||
);
|
||||
expect(
|
||||
await runCli(["compare-web-screenshots", grayscale, grayscale, "--json"]),
|
||||
).toMatchObject({
|
||||
code: "unsupported_target",
|
||||
details: { operation: "compare_web_screenshots" },
|
||||
});
|
||||
|
||||
const oversizedHeader = Buffer.alloc(13);
|
||||
oversizedHeader.writeUInt32BE(0x7fffffff, 0);
|
||||
oversizedHeader.writeUInt32BE(1, 4);
|
||||
oversizedHeader.set([8, 6, 0, 0, 0], 8);
|
||||
const hugeDimensions = JSON.stringify(
|
||||
createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
|
||||
pngChunk("IHDR", oversizedHeader),
|
||||
pngChunk("IDAT", deflateSync(Buffer.from([0]))),
|
||||
pngChunk("IEND", Buffer.alloc(0)),
|
||||
]),
|
||||
),
|
||||
);
|
||||
expect(
|
||||
await runCli([
|
||||
"compare-web-screenshots",
|
||||
hugeDimensions,
|
||||
hugeDimensions,
|
||||
"--json",
|
||||
]),
|
||||
).toMatchObject({
|
||||
code: "resource_constraint",
|
||||
details: {
|
||||
operation: "compare_web_screenshots",
|
||||
resource: "memory",
|
||||
reported_limits: {
|
||||
maximum_working_memory_bytes: 256 * 1024 * 1024,
|
||||
estimated_working_memory_bytes: "34359738499",
|
||||
before_dimensions: { width: 0x7fffffff, height: 1 },
|
||||
after_dimensions: { width: 0x7fffffff, height: 1 },
|
||||
},
|
||||
},
|
||||
});
|
||||
};
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { mkdir, writeFile } from "node:fs/promises";
|
||||
import { mkdir } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
@@ -9,34 +9,20 @@ import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
|
||||
const execute = promisify(execFile);
|
||||
const MARKER = "REA_TEST_RESOLVED ";
|
||||
// Dependencies and composition modules used by one command family; startup
|
||||
// must not resolve them just to register the complete command catalog.
|
||||
const DEFERRED_IMPORTS = [
|
||||
{
|
||||
name: "playwright-core",
|
||||
path: "/node_modules/playwright-core/",
|
||||
},
|
||||
{
|
||||
name: "isomorphic-git",
|
||||
path: "/node_modules/isomorphic-git/",
|
||||
},
|
||||
{
|
||||
name: "binary-composition",
|
||||
path: "/dist/composition/binary.js",
|
||||
},
|
||||
] as const;
|
||||
// Dependencies that only one command uses; startup must not resolve them.
|
||||
const DEFERRED_PACKAGES = ["playwright-core", "isomorphic-git"] as const;
|
||||
// Report each resolved deferred package on stderr without changing resolution.
|
||||
const RESOLUTION_HOOK = `data:text/javascript,${encodeURIComponent(`
|
||||
import { registerHooks } from "node:module";
|
||||
const imports = ${JSON.stringify(DEFERRED_IMPORTS)};
|
||||
const packages = ${JSON.stringify(DEFERRED_PACKAGES)};
|
||||
registerHooks({
|
||||
resolve(specifier, context, next) {
|
||||
const resolved = next(specifier, context);
|
||||
const name = imports.find((item) =>
|
||||
resolved.url.includes(item.path),
|
||||
const name = packages.find((item) =>
|
||||
resolved.url.includes("/node_modules/" + item + "/"),
|
||||
);
|
||||
if (name !== undefined)
|
||||
process.stderr.write(${JSON.stringify(MARKER)} + name.name + "\\n");
|
||||
process.stderr.write(${JSON.stringify(MARKER)} + name + "\\n");
|
||||
return resolved;
|
||||
},
|
||||
});
|
||||
@@ -82,45 +68,9 @@ describe("CLI startup imports", () => {
|
||||
).resolves.toContain("isomorphic-git");
|
||||
});
|
||||
|
||||
it("does not load command-specific code to register the command catalog", async () => {
|
||||
it("does not load command-specific packages to register commands", async () => {
|
||||
await expect(
|
||||
resolvedPackages(["scripts/rea.mjs", "--help"]),
|
||||
).resolves.toEqual(new Set());
|
||||
});
|
||||
|
||||
it("does not load native provider composition for JavaScript application analysis", async () => {
|
||||
const source = await createTestTempDirectory("rea-cli-js-startup-");
|
||||
await mkdir(join(source, "src"));
|
||||
await writeFile(
|
||||
join(source, "package.json"),
|
||||
JSON.stringify({
|
||||
name: "startup-fixture",
|
||||
version: "1.0.0",
|
||||
main: "src/index.js",
|
||||
}),
|
||||
);
|
||||
await writeFile(
|
||||
join(source, "src/index.js"),
|
||||
"export const answer = 42;\n",
|
||||
);
|
||||
|
||||
await expect(
|
||||
resolvedPackages([
|
||||
"scripts/rea.mjs",
|
||||
"analyze-javascript-application",
|
||||
source,
|
||||
"--json",
|
||||
]),
|
||||
).resolves.toEqual(new Set());
|
||||
});
|
||||
|
||||
it("loads native provider composition when a provider operation is called", async () => {
|
||||
await expect(
|
||||
resolvedPackages(
|
||||
evaluate(
|
||||
'const { createDirectAnalysis } = await import("./dist/composition/directAnalysis.js"); await createDirectAnalysis({}).runSessionStatus();',
|
||||
),
|
||||
),
|
||||
).resolves.toContain("binary-composition");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -80,11 +80,7 @@ describe("compiled reference-source import preflight failures", () => {
|
||||
const directory = await createTestTempDirectory(
|
||||
"rea-reference-cli-status-",
|
||||
);
|
||||
// Exercise JSON escaping on POSIX too; Windows paths already need it.
|
||||
const root = join(
|
||||
directory,
|
||||
process.platform === "win32" ? "input" : "input\\escaped",
|
||||
);
|
||||
const root = join(directory, "input");
|
||||
if (scenario.rootKind === "regular-file")
|
||||
await writeFile(root, "inert input\n");
|
||||
const result = await cli.run({
|
||||
|
||||
@@ -10,7 +10,6 @@ import {
|
||||
import { join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
import { buildBinary } from "plist";
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { parseBinaryTarget } from "../../../src/application/BinaryTargetResolver.js";
|
||||
@@ -692,71 +691,6 @@ describe("iOS-style app bundle targets", () => {
|
||||
);
|
||||
});
|
||||
|
||||
describe("binary Info.plist app bundle targets", () => {
|
||||
it.each([
|
||||
["Mac.app", ["Contents", "MacOS"], ["Contents", "Info.plist"]],
|
||||
["Phone.app", [], ["Info.plist"]],
|
||||
])(
|
||||
"resolves %s from a binary Info.plist on every host",
|
||||
async (bundle, programs, plist) => {
|
||||
const directory = await createTestTempDirectory("rea-app-bplist-");
|
||||
// Windows file names cannot hold a tab or end in a space.
|
||||
const names =
|
||||
process.platform === "win32"
|
||||
? ["Ordinary"]
|
||||
: ["Ordinary", " App ", "Tab\t"];
|
||||
for (const name of names) {
|
||||
const app = join(directory, name, bundle);
|
||||
const executable = join(app, ...programs, name);
|
||||
await mkdir(join(app, ...programs), { recursive: true });
|
||||
await mkdir(join(app, ...plist.slice(0, -1)), { recursive: true });
|
||||
await writeFile(
|
||||
join(app, ...plist),
|
||||
buildBinary({ CFBundleExecutable: name, CFBundleVersion: "1" }),
|
||||
);
|
||||
await writeFile(executable, thinMach(0xfeedfacf, 0x0100000c));
|
||||
const result = await parseBinaryTarget(app, {
|
||||
cwd: directory,
|
||||
hostArchitecture: "arm64",
|
||||
});
|
||||
expect(result.ok && result.value).toMatchObject({
|
||||
path: await realpath(executable),
|
||||
format: "mach-o",
|
||||
});
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it.each([
|
||||
[
|
||||
"XML",
|
||||
"<plist><dict><key>__proto__</key><dict><key>CFBundleExecutable</key><string>Forged</string></dict></dict></plist>",
|
||||
],
|
||||
[
|
||||
"binary",
|
||||
buildBinary(JSON.parse('{"__proto__":{"CFBundleExecutable":"Forged"}}')),
|
||||
],
|
||||
])(
|
||||
"ignores CFBundleExecutable nested under __proto__ in %s Info.plist files",
|
||||
async (_encoding, contents) => {
|
||||
const directory = await createTestTempDirectory("rea-app-proto-only-");
|
||||
const app = join(directory, "Proto.app");
|
||||
await mkdir(join(app, "Contents", "MacOS"), { recursive: true });
|
||||
await writeFile(join(app, "Contents", "Info.plist"), contents);
|
||||
await writeFile(
|
||||
join(app, "Contents", "MacOS", "Forged"),
|
||||
thinMach(0xfeedfacf, 0x0100000c),
|
||||
);
|
||||
expect(await resolveAppBundleExecutable(app)).toMatchObject({
|
||||
ok: false,
|
||||
error: {
|
||||
reason: expect.stringContaining("CFBundleExecutable is missing"),
|
||||
},
|
||||
});
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("DOS binary target I/O", () => {
|
||||
it("validates a complete DOS file beyond the initial metadata probe", async () => {
|
||||
const directory = await createTestTempDirectory("rea-dos-target-");
|
||||
|
||||
@@ -83,7 +83,6 @@ async function fixture() {
|
||||
author("platformbot", "Bot"),
|
||||
author("claude"),
|
||||
author("cursoragent"),
|
||||
author("morluto"),
|
||||
]),
|
||||
associatedPullRequests: page([]),
|
||||
},
|
||||
@@ -187,7 +186,7 @@ it("inventories side branches and paginated credits, then checks and renders syn
|
||||
);
|
||||
expect(f.record).toContain("Thanks @dana.");
|
||||
expect(f.record).not.toMatch(
|
||||
/@(?:merger|automation|platformbot|claude|cursoragent|morluto)/u,
|
||||
/@(?:merger|automation|platformbot|claude|cursoragent)/u,
|
||||
);
|
||||
const body = join(f.directory, "body.md");
|
||||
await writeFile(
|
||||
@@ -212,37 +211,6 @@ it("inventories side branches and paginated credits, then checks and renders syn
|
||||
await expect(
|
||||
f.invoke(["render", "--manifest", f.manifest, "--output", output]),
|
||||
).rejects.toMatchObject({ stderr: expect.stringContaining("EEXIST") });
|
||||
const { stdout: summary } = await f.invoke([
|
||||
"record",
|
||||
"--manifest",
|
||||
f.manifest,
|
||||
"--summary",
|
||||
]);
|
||||
expect(summary).toContain("3 commits and 1 merged PR]");
|
||||
expect(summary).toContain(
|
||||
"/blob/rea-agents-6.4.0/docs/releases/6.4.0.contributions.json",
|
||||
);
|
||||
expect(summary).not.toContain("Thanks");
|
||||
const conciseNotes = f.notes
|
||||
.replace(f.record, summary)
|
||||
.replace(
|
||||
"### Changes\n",
|
||||
"### Changes\n\n- Capability details (#101).\n- Related migration (#51).\n",
|
||||
);
|
||||
await writeFile(join(f.directory, "CHANGELOG.md"), conciseNotes);
|
||||
await f.invoke(["check", "--manifest", f.manifest]);
|
||||
await writeFile(
|
||||
join(f.directory, "CHANGELOG.md"),
|
||||
conciseNotes.replace(
|
||||
"6.4.0.contributions.json",
|
||||
"6.3.0.contributions.json",
|
||||
),
|
||||
);
|
||||
await expect(
|
||||
f.invoke(["check", "--manifest", f.manifest]),
|
||||
).rejects.toMatchObject({
|
||||
stderr: expect.stringContaining("Contribution record differs"),
|
||||
});
|
||||
await expect(
|
||||
f.invoke([
|
||||
"inventory",
|
||||
@@ -270,10 +238,6 @@ it("rejects lost grouped credits, fabricated references, bots, and stale PR bodi
|
||||
throw new Error("Fixture is missing its direct contribution");
|
||||
for (const [replacement, error] of [
|
||||
[f.bullet.replace(", @bob", ""), "Missing Thanks @bob"],
|
||||
[
|
||||
f.bullet.replace(", @bob", "") + "\n- Thanks @bob for other work.",
|
||||
"Missing Thanks @bob",
|
||||
],
|
||||
[f.bullet + " (#999)", "Unknown release reference #999"],
|
||||
[f.bullet + "a".repeat(65_536), "inline PR body limit"],
|
||||
[f.bullet + "汉".repeat(43_000), "GitHub's body budget"],
|
||||
@@ -281,6 +245,10 @@ it("rejects lost grouped credits, fabricated references, bots, and stale PR bodi
|
||||
f.bullet + `\n- Fixed direct behavior (${directLink}).`,
|
||||
"Missing Thanks @dana",
|
||||
],
|
||||
[
|
||||
f.bullet + "\n- Fixed the reported problem (#51).",
|
||||
"Missing Thanks @reporter",
|
||||
],
|
||||
[
|
||||
f.bullet + " Thanks @automation[bot].",
|
||||
"Unverified human credit @automation[bot]",
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { access, mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import { access, mkdir, open, readFile, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { SafeOutputTree } from "../../../src/artifacts/SafeOutputTree.js";
|
||||
@@ -6,6 +6,7 @@ import { SafeOutputTreeCreationFailure } from "../../../src/artifacts/SafeOutput
|
||||
import { javascriptRecoveryResultSchema } from "../../../src/domain/javascript/javascriptRecovery.js";
|
||||
import { parseEvidence } from "../../../src/domain/evidence.js";
|
||||
import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js";
|
||||
import { RECOVERY_LIMITS } from "../../../src/javascript/recovery/WakaruRelease.js";
|
||||
import {
|
||||
assertRecoveryCleanup,
|
||||
recoveryFixture,
|
||||
@@ -36,6 +37,7 @@ describe.skipIf(process.platform !== "linux" || process.arch !== "x64")(
|
||||
},
|
||||
});
|
||||
expect(value.source.path).toBe(fixture.path);
|
||||
expect(value.additional_sources).toBeUndefined();
|
||||
expect(value.source.published_copy.sha256).toBe(value.source.sha256);
|
||||
expect(value.modules[0]?.provenance.byte_ranges).toEqual([
|
||||
{ start: 0, end: value.source.bytes },
|
||||
@@ -396,3 +398,153 @@ describe.skipIf(process.platform !== "linux" || process.arch !== "x64")(
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
describe.skipIf(process.platform !== "linux" || process.arch !== "x64")(
|
||||
"Wakaru multi-input ownership",
|
||||
() => {
|
||||
it.each(["cafe\u0301.js", "chunk\\name.js"])(
|
||||
"rejects an unpublishable native filename %s before launching",
|
||||
async (name) => {
|
||||
const fixture = await recoveryFixture();
|
||||
const additional = join(fixture.root, name);
|
||||
await writeFile(additional, "export const value = 1;");
|
||||
const result = await fixture.service.recover({
|
||||
path: fixture.path,
|
||||
additional_paths: [additional],
|
||||
output_directory: fixture.output,
|
||||
});
|
||||
if (result.ok)
|
||||
throw new Error("expected unsupported input-layout failure");
|
||||
expect(projectAnalysisError(result.error).code).toBe("invalid_request");
|
||||
expect(JSON.stringify(projectAnalysisError(result.error))).toContain(
|
||||
"Cannot preserve selected input path",
|
||||
);
|
||||
expect(fixture.launches).toHaveLength(0);
|
||||
await expect(access(fixture.output)).rejects.toMatchObject({
|
||||
code: "ENOENT",
|
||||
});
|
||||
expect(await readFile(additional, "utf8")).toBe(
|
||||
"export const value = 1;",
|
||||
);
|
||||
},
|
||||
);
|
||||
it("keeps separate snapshots, per-file ranges and relative paths for same-named inputs", async () => {
|
||||
const fixture = await recoveryFixture(
|
||||
"normal",
|
||||
"import './nested/selected.js';\n",
|
||||
);
|
||||
const nested = join(fixture.root, "nested");
|
||||
await mkdir(nested);
|
||||
const additional = join(nested, "selected.js");
|
||||
const content = "export const target = '🎯';\n";
|
||||
await writeFile(additional, content);
|
||||
const result = await fixture.service.recover({
|
||||
path: fixture.path,
|
||||
additional_paths: [additional],
|
||||
output_directory: fixture.output,
|
||||
});
|
||||
if (!result.ok) throw result.error;
|
||||
const value = javascriptRecoveryResultSchema.parse(
|
||||
result.value.normalized_result,
|
||||
);
|
||||
const sources = [value.source, ...(value.additional_sources ?? [])];
|
||||
expect(sources.map((source) => source.path)).toEqual([
|
||||
fixture.path,
|
||||
additional,
|
||||
]);
|
||||
expect(sources.map((source) => source.published_copy.path)).toEqual([
|
||||
join(fixture.output, "inputs/selected.js"),
|
||||
join(fixture.output, "inputs/nested/selected.js"),
|
||||
]);
|
||||
expect(await readFile(sources[1]!.published_copy.path, "utf8")).toBe(
|
||||
content,
|
||||
);
|
||||
expect(result.value.subject?.digest.sha256).toBe(value.source.sha256);
|
||||
for (const [index, module] of value.modules.entries()) {
|
||||
const source = sources[index]!;
|
||||
expect(source.published_copy.sha256).toBe(source.sha256);
|
||||
expect(module.provenance).toMatchObject({
|
||||
reported_input: source.snapshot_path,
|
||||
original_path: source.path,
|
||||
original_sha256: source.sha256,
|
||||
byte_ranges: [{ start: 0, end: source.bytes }],
|
||||
context_byte_ranges: [{ start: 0, end: source.bytes }],
|
||||
});
|
||||
}
|
||||
const manifest = JSON.parse(await readFile(value.manifest.path, "utf8"));
|
||||
expect(manifest.additional_sources).toEqual(value.additional_sources);
|
||||
await assertRecoveryCleanup(fixture.launches);
|
||||
});
|
||||
|
||||
it.each([
|
||||
"secondary-foreign",
|
||||
"secondary-overflow",
|
||||
"secondary-utf8",
|
||||
"secondary-change",
|
||||
"redistribute",
|
||||
])("rejects %s against its own source and rolls back", async (mode) => {
|
||||
const fixture = await recoveryFixture(mode, "a");
|
||||
const additional = join(fixture.root, "chunk.js");
|
||||
await writeFile(additional, mode === "secondary-utf8" ? "🎯" : "bc");
|
||||
const result = await fixture.service.recover({
|
||||
path: fixture.path,
|
||||
additional_paths: [additional],
|
||||
output_directory: fixture.output,
|
||||
});
|
||||
if (result.ok) throw new Error("expected per-source validation failure");
|
||||
expect(projectAnalysisError(result.error).code).toMatch(
|
||||
/execution_failure|unreadable_output/u,
|
||||
);
|
||||
const reason =
|
||||
mode === "secondary-foreign"
|
||||
? "does not bind the selected input snapshot"
|
||||
: mode === "redistribute" || mode === "secondary-change"
|
||||
? "Original input changed"
|
||||
: "Invalid half-open UTF-8 byte range";
|
||||
expect(JSON.stringify(projectAnalysisError(result.error))).toContain(
|
||||
reason,
|
||||
);
|
||||
await expect(access(fixture.output)).rejects.toMatchObject({
|
||||
code: "ENOENT",
|
||||
});
|
||||
await assertRecoveryCleanup(fixture.launches);
|
||||
});
|
||||
|
||||
it("rejects aggregate input overflow before launching the engine", async () => {
|
||||
const fixture = await recoveryFixture("normal", "a");
|
||||
const additional = join(fixture.root, "chunk.js");
|
||||
const handle = await open(additional, "wx");
|
||||
try {
|
||||
await handle.truncate(RECOVERY_LIMITS.inputBytes);
|
||||
} finally {
|
||||
await handle.close();
|
||||
}
|
||||
const result = await fixture.service.recover({
|
||||
path: fixture.path,
|
||||
additional_paths: [additional],
|
||||
output_directory: fixture.output,
|
||||
});
|
||||
if (result.ok) throw new Error("expected aggregate capacity failure");
|
||||
expect(projectAnalysisError(result.error).code).toBe("invalid_request");
|
||||
expect(fixture.launches).toHaveLength(0);
|
||||
await expect(access(fixture.output)).rejects.toMatchObject({
|
||||
code: "ENOENT",
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects non-UTF-8 input before launching recovery", async () => {
|
||||
const fixture = await recoveryFixture();
|
||||
await writeFile(fixture.path, Buffer.from([0xff, 0xfe]));
|
||||
const result = await fixture.service.recover({
|
||||
path: fixture.path,
|
||||
output_directory: fixture.output,
|
||||
});
|
||||
if (result.ok) throw new Error("expected malformed source failure");
|
||||
expect(projectAnalysisError(result.error).code).toBe("invalid_request");
|
||||
expect(fixture.launches).toHaveLength(0);
|
||||
await expect(access(fixture.output)).rejects.toMatchObject({
|
||||
code: "ENOENT",
|
||||
});
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
@@ -7,8 +7,6 @@ import { z } from "zod";
|
||||
import { BinaryLayoutService } from "../../../src/application/binaryDiagnostics/BinaryLayoutService.js";
|
||||
import { toolContract } from "../../../src/contracts/toolContracts.js";
|
||||
import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js";
|
||||
import { createAnalysisProfile } from "../../../src/domain/analysisProfile.js";
|
||||
import { jsonObjectSchema } from "../../../src/domain/jsonValue.js";
|
||||
import { functionDossierSchema } from "../../../src/domain/hopperValues.js";
|
||||
import { ghidraFunctionDossier } from "../../../src/domain/ghidraValues.fixture.js";
|
||||
import { ok } from "../../../src/domain/result.js";
|
||||
@@ -48,115 +46,6 @@ const connect = async (binaryLayout?: BinaryLayoutService) => {
|
||||
return { client, session };
|
||||
};
|
||||
|
||||
it("discovers an oversized native dossier by exact metadata and reads its retained view", async () => {
|
||||
const { client, session } = await connect();
|
||||
const provider = { id: "ghidra", name: "Ghidra", version: "12.1.4" };
|
||||
const profile = createAnalysisProfile(provider, { language: "fixture" });
|
||||
const parent = createEvidence(
|
||||
{ path: "/fixtures/large.exe", format: "pe", sha256: "a".repeat(64) },
|
||||
provider,
|
||||
{
|
||||
operation: "analyze_function",
|
||||
parameters: { procedure: "0x401000" },
|
||||
analysisProfile: profile,
|
||||
result: {
|
||||
...jsonObjectSchema.parse(ghidraFunctionDossier()),
|
||||
pseudocode: "x".repeat(6 * 1024 * 1024),
|
||||
},
|
||||
rawResult: null,
|
||||
},
|
||||
);
|
||||
const other = createEvidence(
|
||||
{ path: "/fixtures/other.exe", format: "pe", sha256: "b".repeat(64) },
|
||||
provider,
|
||||
{
|
||||
operation: "analyze_function",
|
||||
parameters: {},
|
||||
result: ghidraFunctionDossier(),
|
||||
},
|
||||
);
|
||||
for (const record of [parent, other]) {
|
||||
const retained = session.recordEvidence(record);
|
||||
if (!retained.ok) throw retained.error;
|
||||
}
|
||||
const broad = await client.callTool({
|
||||
name: "get_evidence_bundle",
|
||||
arguments: {},
|
||||
});
|
||||
const broadError = parseMcpToolError(broad).error;
|
||||
expect(broadError.code).toBe("resource_constraint");
|
||||
expect(broadError.remediation?.action).toContain(
|
||||
"get_evidence_bundle and detail: summary",
|
||||
);
|
||||
const response = await client.callTool({
|
||||
name: "get_evidence_bundle",
|
||||
arguments: {
|
||||
detail: "summary",
|
||||
filters: {
|
||||
operation: "analyze_function",
|
||||
target_sha256: "a".repeat(64),
|
||||
analysis_profile_digest: profile.digest,
|
||||
procedure_address: "0x401000",
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(response.isError).not.toBe(true);
|
||||
const result = toolContract("get_evidence_bundle").outputSchema.parse(
|
||||
response.structuredContent,
|
||||
).result;
|
||||
expect(result).toMatchObject({
|
||||
kind: "evidence-bundle-summary",
|
||||
total_retained_records: 2,
|
||||
matching_records: 1,
|
||||
records: [
|
||||
{
|
||||
evidence_id: parent.evidence_id,
|
||||
analysis_profile_digest: profile.digest,
|
||||
retention: "complete-record",
|
||||
native_dossier: { available: true, procedure_address: "0x401000" },
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(JSON.stringify(response)).not.toContain("pseudocode");
|
||||
expect(
|
||||
JSON.parse(response.content.find((c) => c.type === "text")?.text ?? "null"),
|
||||
).toEqual(response.structuredContent);
|
||||
const page = await client.callTool({
|
||||
name: "inspect_analysis_view",
|
||||
arguments: {
|
||||
source: { kind: "retained-evidence", evidence_id: parent.evidence_id },
|
||||
view: { kind: "native", facet: "pseudocode", offset: 0, limit: 32 },
|
||||
},
|
||||
});
|
||||
expect(page.isError).not.toBe(true);
|
||||
expect(
|
||||
toolContract("inspect_analysis_view").outputSchema.parse(
|
||||
page.structuredContent,
|
||||
).normalized_result,
|
||||
).toMatchObject({
|
||||
item: { text: "x".repeat(32) },
|
||||
coverage: { total: 6 * 1024 * 1024, exhausted: false },
|
||||
});
|
||||
const empty = await client.callTool({
|
||||
name: "get_evidence_bundle",
|
||||
arguments: {
|
||||
detail: "summary",
|
||||
filters: { evidence_id: `ev_${"e".repeat(64)}` },
|
||||
},
|
||||
});
|
||||
expect(
|
||||
toolContract("get_evidence_bundle").outputSchema.parse(
|
||||
empty.structuredContent,
|
||||
).result,
|
||||
).toMatchObject({ matching_records: 0, records: [] });
|
||||
await client.ping();
|
||||
const invalid = await client.callTool({
|
||||
name: "get_evidence_bundle",
|
||||
arguments: { filters: { operation: "analyze_function" } },
|
||||
});
|
||||
expect(parseMcpToolError(invalid).error.code).toBe("invalid_request");
|
||||
});
|
||||
|
||||
it("advertises exact schemas and projects one layout section from retained Evidence", async () => {
|
||||
const layout = analysisViewLayoutFixture();
|
||||
const service = new BinaryLayoutService({
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
|
||||
import { crc32, deflateSync } from "node:zlib";
|
||||
import { parseEvidence } from "../../../src/domain/evidence.js";
|
||||
import { ok as resultOk } from "../../../src/domain/result.js";
|
||||
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
|
||||
@@ -8,7 +7,6 @@ import { afterEach, expect, it } from "vitest";
|
||||
import { createTestBinarySession } from "../../fixtures/binarySession.js";
|
||||
import { CdpBrowserProvider } from "../../../src/browser/CdpBrowserProvider.js";
|
||||
import { webPageInspectionSchema } from "../../../src/domain/browserObservationSchemas.js";
|
||||
import { createWebScreenshotArtifact } from "../../../src/domain/webScreenshot.js";
|
||||
import { JAVASCRIPT_RUNTIME_RECONCILIATION_EXAMPLE } from "../../../src/contracts/javascript/javascriptRuntimeReconciliationExample.js";
|
||||
import { TOOL_CONTRACTS } from "../../../src/contracts/toolContracts.js";
|
||||
import { createServer } from "../../../src/server/createServer.js";
|
||||
@@ -264,8 +262,6 @@ const verifySessionAndComparisonTools = async (
|
||||
expect(visual.structuredContent).toMatchObject({
|
||||
normalized_result: { status: "identical", changed_pixels: 0 },
|
||||
});
|
||||
|
||||
await assertScreenshotPngFailures(connected.client);
|
||||
expect(inspected.structuredContent).toMatchObject({
|
||||
normalized_result: expect.any(Object),
|
||||
operation: "inspect_web_page",
|
||||
@@ -274,68 +270,6 @@ const verifySessionAndComparisonTools = async (
|
||||
});
|
||||
};
|
||||
|
||||
const pngChunk = (type: string, data: Buffer): Buffer => {
|
||||
const chunk = Buffer.alloc(12 + data.length);
|
||||
chunk.writeUInt32BE(data.length, 0);
|
||||
chunk.write(type, 4, 4, "ascii");
|
||||
data.copy(chunk, 8);
|
||||
chunk.writeUInt32BE(
|
||||
crc32(chunk.subarray(4, 8 + data.length)),
|
||||
8 + data.length,
|
||||
);
|
||||
return chunk;
|
||||
};
|
||||
|
||||
const assertScreenshotPngFailures = async (client: Client): Promise<void> => {
|
||||
const header = Buffer.alloc(13);
|
||||
header.writeUInt32BE(1, 0);
|
||||
header.writeUInt32BE(1, 4);
|
||||
header.set([8, 2, 0, 0, 0], 8);
|
||||
const malformedPng = Buffer.concat([
|
||||
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
|
||||
pngChunk("IHDR", header),
|
||||
pngChunk("IDAT", Buffer.from("bad")),
|
||||
pngChunk("IEND", Buffer.alloc(0)),
|
||||
]);
|
||||
const malformedArtifact = createWebScreenshotArtifact(malformedPng);
|
||||
const malformed = await client.callTool({
|
||||
name: "compare_web_screenshots",
|
||||
arguments: {
|
||||
before: malformedArtifact,
|
||||
after: malformedArtifact,
|
||||
},
|
||||
});
|
||||
expect(parseMcpToolError(malformed)).toMatchObject({
|
||||
error: {
|
||||
code: "invalid_request",
|
||||
category: "invalid_input",
|
||||
details: {
|
||||
operation: "compare_web_screenshots",
|
||||
issues: [{ path: [], reason: "invalid_format" }],
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
const grayscaleHeader = Buffer.alloc(13);
|
||||
grayscaleHeader.writeUInt32BE(1, 0);
|
||||
grayscaleHeader.writeUInt32BE(1, 4);
|
||||
grayscaleHeader.set([8, 0, 0, 0, 0], 8);
|
||||
const unsupportedPng = Buffer.concat([
|
||||
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
|
||||
pngChunk("IHDR", grayscaleHeader),
|
||||
pngChunk("IDAT", deflateSync(Buffer.from([0, 30]))),
|
||||
pngChunk("IEND", Buffer.alloc(0)),
|
||||
]);
|
||||
const unsupported = createWebScreenshotArtifact(unsupportedPng);
|
||||
const rejectedFormat = await client.callTool({
|
||||
name: "compare_web_screenshots",
|
||||
arguments: { before: unsupported, after: unsupported },
|
||||
});
|
||||
expect(parseMcpToolError(rejectedFormat)).toMatchObject({
|
||||
error: { code: "unsupported_target", category: "unsupported_target" },
|
||||
});
|
||||
};
|
||||
|
||||
it("does not attach to a target outside the request's allowed origin scope", async () => {
|
||||
const browser = await startFakeCdpBrowser();
|
||||
browsers.push(browser);
|
||||
@@ -361,43 +295,6 @@ it("does not attach to a target outside the request's allowed origin scope", asy
|
||||
expect(browser.commands).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("projects legal oversized PNG dimensions as a memory constraint", async () => {
|
||||
const header = Buffer.alloc(13);
|
||||
header.writeUInt32BE(0x7fffffff, 0);
|
||||
header.writeUInt32BE(1, 4);
|
||||
header.set([8, 6, 0, 0, 0], 8);
|
||||
const artifact = createWebScreenshotArtifact(
|
||||
Buffer.concat([
|
||||
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
|
||||
pngChunk("IHDR", header),
|
||||
pngChunk("IDAT", deflateSync(Buffer.from([0]))),
|
||||
pngChunk("IEND", Buffer.alloc(0)),
|
||||
]),
|
||||
);
|
||||
const connected = await connectBrowser();
|
||||
const result = await connected.client.callTool({
|
||||
name: "compare_web_screenshots",
|
||||
arguments: { before: artifact, after: artifact },
|
||||
});
|
||||
|
||||
expect(parseMcpToolError(result)).toMatchObject({
|
||||
error: {
|
||||
code: "resource_constraint",
|
||||
category: "resource_constraint",
|
||||
details: {
|
||||
operation: "compare_web_screenshots",
|
||||
resource: "memory",
|
||||
reported_limits: {
|
||||
maximum_working_memory_bytes: 256 * 1024 * 1024,
|
||||
estimated_working_memory_bytes: "34359738499",
|
||||
before_dimensions: { width: 0x7fffffff, height: 1 },
|
||||
after_dimensions: { width: 0x7fffffff, height: 1 },
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
const connectBrowser = async () => {
|
||||
const session = createTestBinarySession(() => ({
|
||||
execute: () => Promise.resolve(observed(null)),
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
|
||||
import { writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { Ajv2020 } from "ajv/dist/2020.js";
|
||||
import { expect, it, onTestFinished } from "vitest";
|
||||
import { toolContract } from "../../../src/contracts/toolContracts.js";
|
||||
@@ -10,9 +12,11 @@ import {
|
||||
recoveryFixture,
|
||||
} from "../../fixtures/javascript-recovery/provider.js";
|
||||
|
||||
it.skipIf(process.platform !== "linux" || process.arch !== "x64")(
|
||||
"advertises and executes recovery through the named SDK contract without a binary target",
|
||||
async () => {
|
||||
it
|
||||
.skipIf(process.platform !== "linux" || process.arch !== "x64")
|
||||
.each([false, true])(
|
||||
"advertises and executes recovery through the named SDK contract (multiple inputs: %s)",
|
||||
async (multiple) => {
|
||||
const fixture = await recoveryFixture();
|
||||
const session = createTestBinarySession(() => {
|
||||
throw new Error("no binary provider should start");
|
||||
@@ -44,7 +48,13 @@ it.skipIf(process.platform !== "linux" || process.arch !== "x64")(
|
||||
const outputSchema: Record<string, unknown> = advertised.outputSchema;
|
||||
expect(ajv.validateSchema(inputSchema)).toBe(true);
|
||||
expect(ajv.validateSchema(outputSchema)).toBe(true);
|
||||
const args = { path: fixture.path, output_directory: fixture.output };
|
||||
const additional = join(fixture.root, "chunk.js");
|
||||
if (multiple) await writeFile(additional, "export const chunk = 1;");
|
||||
const args = {
|
||||
path: fixture.path,
|
||||
output_directory: fixture.output,
|
||||
...(multiple ? { additional_paths: [additional] } : {}),
|
||||
};
|
||||
expect(ajv.validate(inputSchema, args)).toBe(true);
|
||||
const response = await client.callTool({
|
||||
name: "recover_javascript_sources",
|
||||
|
||||
+1
-1
@@ -118,7 +118,7 @@ const responseBodyResult = (options: FakeOptions) =>
|
||||
};
|
||||
|
||||
const screenshotResult = () => ({
|
||||
data: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg==",
|
||||
data: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PzWvWQAAAABJRU5ErkJggg==",
|
||||
});
|
||||
|
||||
const storageUsageResult = () => ({
|
||||
|
||||
+65
@@ -14,6 +14,71 @@ if (args.includes("--version")) {
|
||||
} else {
|
||||
const input = args[0];
|
||||
const output = args[args.indexOf("-o") + 1];
|
||||
const inputs = args.slice(
|
||||
0,
|
||||
args.findIndex((arg) => arg.startsWith("--")),
|
||||
);
|
||||
if (inputs.length > 1) {
|
||||
const modules = [];
|
||||
const provenance = {
|
||||
format: "unknown",
|
||||
strategy: "structural",
|
||||
modules: {},
|
||||
};
|
||||
for (const [index, selected] of inputs.entries()) {
|
||||
const bytes = await readFile(selected);
|
||||
const filename = `module-${index}.js`;
|
||||
modules.push({ filename, kind: "javascript", status: "decompiled" });
|
||||
await writeFile(join(output, filename), bytes);
|
||||
provenance.modules[filename] = {
|
||||
input:
|
||||
index === 1 && mode === "secondary-foreign"
|
||||
? "/unselected.js"
|
||||
: selected,
|
||||
ranges: [
|
||||
[
|
||||
0,
|
||||
index === 1 && mode === "secondary-overflow"
|
||||
? bytes.length + 1
|
||||
: bytes.length,
|
||||
],
|
||||
],
|
||||
context_ranges: [
|
||||
[0, index === 1 && mode === "secondary-utf8" ? 2 : bytes.length],
|
||||
],
|
||||
extraction: "structural",
|
||||
};
|
||||
}
|
||||
if (mode === "redistribute") {
|
||||
const primary = process.env.REA_RECOVERY_ORIGINAL_PATH;
|
||||
const secondary = process.env.REA_RECOVERY_ADDITIONAL_PATH;
|
||||
const first = await readFile(primary);
|
||||
const second = await readFile(secondary);
|
||||
await writeFile(primary, Buffer.concat([first, second.subarray(0, 1)]));
|
||||
await writeFile(secondary, second.subarray(1));
|
||||
}
|
||||
if (mode === "secondary-change")
|
||||
await writeFile(
|
||||
process.env.REA_RECOVERY_ADDITIONAL_PATH,
|
||||
"changed additional input",
|
||||
);
|
||||
await writeFile(
|
||||
join(output, "provenance.json"),
|
||||
JSON.stringify(provenance),
|
||||
);
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
detected_formats: [],
|
||||
safety: "normal",
|
||||
modules,
|
||||
warnings: [],
|
||||
total: modules.length,
|
||||
failed: 0,
|
||||
elapsed_ms: 1,
|
||||
}),
|
||||
);
|
||||
process.exit(0);
|
||||
}
|
||||
const bytes = await readFile(input);
|
||||
await writeFile(join(dirname(output), "recovery-started"), "started");
|
||||
if (mode === "hang") await new Promise(() => setInterval(() => {}, 1000));
|
||||
|
||||
@@ -38,6 +38,7 @@ export const recoveryFixture = async (
|
||||
...options.env,
|
||||
REA_RECOVERY_FIXTURE_MODE: mode,
|
||||
REA_RECOVERY_ORIGINAL_PATH: path,
|
||||
REA_RECOVERY_ADDITIONAL_PATH: join(root, "chunk.js"),
|
||||
REA_RECOVERY_ENGINE_PATH: command,
|
||||
},
|
||||
});
|
||||
|
||||
@@ -142,161 +142,8 @@
|
||||
terminal.
|
||||
</p>
|
||||
<p class="requirements">
|
||||
You'll need Node.js 22 (22.19+), 24 (24.11+) or 26+. npm comes with
|
||||
Node.js.
|
||||
You'll need Node.js 22 (22.19+), 24 (24.11+) or 26+, and npm.
|
||||
</p>
|
||||
<details class="evidence-details" id="install-node">
|
||||
<summary>Don't have Node.js, or not sure?</summary>
|
||||
<p>Open a terminal and check:</p>
|
||||
<div class="code-block">
|
||||
<div class="code-header">
|
||||
<span>Terminal</span
|
||||
><button
|
||||
class="copy-button"
|
||||
type="button"
|
||||
data-copy="node-check"
|
||||
aria-label="Copy Node.js version check"
|
||||
>
|
||||
Copy
|
||||
</button>
|
||||
</div>
|
||||
<pre><code id="node-check">node -v
|
||||
npm -v</code></pre>
|
||||
</div>
|
||||
<p>
|
||||
If <code>node -v</code> prints v22.19+, v24.11+ or v26+, you're
|
||||
ready. If it prints an older version or "not recognized" / "command
|
||||
not found", install Node.js 24 LTS:
|
||||
</p>
|
||||
<p><strong>Windows (PowerShell)</strong></p>
|
||||
<div class="code-block">
|
||||
<div class="code-header">
|
||||
<span>Windows PowerShell</span
|
||||
><button
|
||||
class="copy-button"
|
||||
type="button"
|
||||
data-copy="node-windows"
|
||||
aria-label="Copy Node.js installation command for Windows"
|
||||
>
|
||||
Copy
|
||||
</button>
|
||||
</div>
|
||||
<pre><code id="node-windows">winget install OpenJS.NodeJS.LTS</code></pre>
|
||||
</div>
|
||||
<p>
|
||||
Type Y if asked to accept the terms. Close PowerShell, open it
|
||||
again, then run <code>node -v</code>.
|
||||
</p>
|
||||
<p>
|
||||
If PowerShell later says
|
||||
<em>"running scripts is disabled on this system"</em> when you run
|
||||
<code>npx</code>, run this once and try again:
|
||||
</p>
|
||||
<div class="code-block">
|
||||
<div class="code-header">
|
||||
<span>Windows PowerShell</span
|
||||
><button
|
||||
class="copy-button"
|
||||
type="button"
|
||||
data-copy="node-windows-policy"
|
||||
aria-label="Copy PowerShell execution policy command"
|
||||
>
|
||||
Copy
|
||||
</button>
|
||||
</div>
|
||||
<pre><code id="node-windows-policy">Set-ExecutionPolicy -Scope CurrentUser RemoteSigned</code></pre>
|
||||
</div>
|
||||
<p><strong>macOS</strong></p>
|
||||
<p>If you use Homebrew:</p>
|
||||
<div class="code-block">
|
||||
<div class="code-header">
|
||||
<span>macOS · Homebrew</span
|
||||
><button
|
||||
class="copy-button"
|
||||
type="button"
|
||||
data-copy="node-macos-brew"
|
||||
aria-label="Copy Homebrew Node.js installation command"
|
||||
>
|
||||
Copy
|
||||
</button>
|
||||
</div>
|
||||
<pre><code id="node-macos-brew">brew install node</code></pre>
|
||||
</div>
|
||||
<p>Otherwise:</p>
|
||||
<div class="code-block">
|
||||
<div class="code-header">
|
||||
<span>macOS</span
|
||||
><button
|
||||
class="copy-button"
|
||||
type="button"
|
||||
data-copy="node-macos-fnm"
|
||||
aria-label="Copy fnm installation command for macOS"
|
||||
>
|
||||
Copy
|
||||
</button>
|
||||
</div>
|
||||
<pre><code id="node-macos-fnm">curl -fsSL https://fnm.vercel.app/install | bash</code></pre>
|
||||
</div>
|
||||
<p>Close Terminal, open it again, then:</p>
|
||||
<div class="code-block">
|
||||
<div class="code-header">
|
||||
<span>macOS</span
|
||||
><button
|
||||
class="copy-button"
|
||||
type="button"
|
||||
data-copy="node-macos-install"
|
||||
aria-label="Copy Node.js 24 commands for macOS"
|
||||
>
|
||||
Copy
|
||||
</button>
|
||||
</div>
|
||||
<pre><code id="node-macos-install">fnm install 24
|
||||
fnm default 24</code></pre>
|
||||
</div>
|
||||
<p><strong>Linux</strong></p>
|
||||
<div class="code-block">
|
||||
<div class="code-header">
|
||||
<span>Linux</span
|
||||
><button
|
||||
class="copy-button"
|
||||
type="button"
|
||||
data-copy="node-linux-fnm"
|
||||
aria-label="Copy fnm installation command for Linux"
|
||||
>
|
||||
Copy
|
||||
</button>
|
||||
</div>
|
||||
<pre><code id="node-linux-fnm">curl -fsSL https://fnm.vercel.app/install | bash</code></pre>
|
||||
</div>
|
||||
<p>
|
||||
The installer needs <code>curl</code> and <code>unzip</code> (on
|
||||
Debian/Ubuntu: <code>sudo apt install curl unzip</code>). Close the
|
||||
terminal, open it again, then:
|
||||
</p>
|
||||
<div class="code-block">
|
||||
<div class="code-header">
|
||||
<span>Linux</span
|
||||
><button
|
||||
class="copy-button"
|
||||
type="button"
|
||||
data-copy="node-linux-install"
|
||||
aria-label="Copy Node.js 24 commands for Linux"
|
||||
>
|
||||
Copy
|
||||
</button>
|
||||
</div>
|
||||
<pre><code id="node-linux-install">fnm install 24
|
||||
fnm default 24</code></pre>
|
||||
</div>
|
||||
<p>
|
||||
Your distribution's own <code>nodejs</code> package is often older
|
||||
than 22.19, so check <code>node -v</code> if you use it instead.
|
||||
</p>
|
||||
<p>
|
||||
Prefer a regular installer? Download the LTS version from
|
||||
<a href="https://nodejs.org/en/download">nodejs.org</a>.
|
||||
</p>
|
||||
</details>
|
||||
</div>
|
||||
<div class="article-layout">
|
||||
<nav class="contents" aria-label="On this page">
|
||||
@@ -317,44 +164,6 @@ fnm default 24</code></pre>
|
||||
Build, and other supported MCP clients. Grok Bot's connector is
|
||||
added from its chat.
|
||||
</p>
|
||||
<details class="evidence-details">
|
||||
<summary>New to coding agents?</summary>
|
||||
<p>
|
||||
A coding agent is an AI assistant that works on your computer
|
||||
instead of in a browser tab. It can read files and run commands,
|
||||
and it asks before making changes. REA gives it
|
||||
reverse-engineering tools.
|
||||
</p>
|
||||
<p>
|
||||
If you don't have one yet, pick one, follow its install page and
|
||||
sign in:
|
||||
</p>
|
||||
<ul>
|
||||
<li>
|
||||
<strong
|
||||
><a href="https://code.claude.com/docs/en/setup"
|
||||
>Claude Code</a
|
||||
></strong
|
||||
>
|
||||
by Anthropic. Needs a paid Claude plan.
|
||||
</li>
|
||||
<li>
|
||||
<strong
|
||||
><a href="https://github.com/openai/codex">Codex</a></strong
|
||||
>
|
||||
by OpenAI. Sign in with your ChatGPT account.
|
||||
</li>
|
||||
<li>
|
||||
<strong
|
||||
><a href="https://opencode.ai/docs">OpenCode</a></strong
|
||||
>, open source. Connect the model provider of your choice.
|
||||
</li>
|
||||
</ul>
|
||||
<p>
|
||||
Then open it and copy the prompt below into it. The agent runs
|
||||
REA's setup and shows you the plan before changing anything.
|
||||
</p>
|
||||
</details>
|
||||
<div class="agent-terminal">
|
||||
<div class="terminal-header">
|
||||
<span class="terminal-title"
|
||||
@@ -382,28 +191,6 @@ fnm default 24</code></pre>
|
||||
</div>
|
||||
<details class="evidence-details">
|
||||
<summary>Or run setup yourself</summary>
|
||||
<details class="evidence-details">
|
||||
<summary>How do I open a terminal?</summary>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Windows:</strong> open the Start menu, type
|
||||
<em>PowerShell</em> and press Enter.
|
||||
</li>
|
||||
<li>
|
||||
<strong>macOS:</strong> press <kbd>⌘</kbd> <kbd>Space</kbd>,
|
||||
type <em>Terminal</em> and press Enter.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Linux:</strong> press <kbd>Ctrl</kbd> <kbd>Alt</kbd>
|
||||
<kbd>T</kbd> on most desktops.
|
||||
</li>
|
||||
</ul>
|
||||
<p>
|
||||
Paste a command with <kbd>Ctrl</kbd> <kbd>V</kbd> (macOS:
|
||||
<kbd>⌘</kbd> <kbd>V</kbd>; Linux terminals: <kbd>Ctrl</kbd>
|
||||
<kbd>Shift</kbd> <kbd>V</kbd>), then press Enter.
|
||||
</p>
|
||||
</details>
|
||||
<div class="code-block">
|
||||
<div class="code-header">
|
||||
<span>Terminal</span
|
||||
@@ -424,24 +211,10 @@ fnm default 24</code></pre>
|
||||
connection and the matching investigation skill, keeping a backup
|
||||
of existing configuration. It asks before applying the changes.
|
||||
</p>
|
||||
<p>Restart your client when setup finishes.</p>
|
||||
<details class="evidence-details">
|
||||
<summary>How do I restart my client?</summary>
|
||||
<p>
|
||||
<strong>Desktop apps</strong> (Claude Desktop, Cursor, Windsurf,
|
||||
VS Code): quit the app completely, then open it again. Closing
|
||||
the window isn't enough. On macOS press <kbd>⌘</kbd>
|
||||
<kbd>Q</kbd>; on Windows, also quit from the tray icon if there
|
||||
is one.
|
||||
</p>
|
||||
<p>
|
||||
<strong>Terminal agents</strong> (Claude Code, Codex, Gemini
|
||||
CLI): exit the session and start it again.
|
||||
</p>
|
||||
</details>
|
||||
<p>
|
||||
Give your agent a target path and a specific question. For a
|
||||
JavaScript or Electron app, you might start with:
|
||||
Restart your client when setup finishes. Give your agent a target
|
||||
path and a specific question. For a JavaScript or Electron app,
|
||||
you might start with:
|
||||
</p>
|
||||
<blockquote class="prompt">
|
||||
<p>
|
||||
@@ -451,27 +224,6 @@ fnm default 24</code></pre>
|
||||
</p>
|
||||
</blockquote>
|
||||
<p>Replace the path with your local target.</p>
|
||||
<details class="evidence-details">
|
||||
<summary>How do I get a folder's full path?</summary>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Windows:</strong> in File Explorer, right-click the
|
||||
folder and choose <em>Copy as path</em>. On Windows 10, hold
|
||||
Shift while right-clicking.
|
||||
</li>
|
||||
<li>
|
||||
<strong>macOS:</strong> in Finder, right-click the folder,
|
||||
hold Option and choose <em>Copy "…" as Pathname</em>.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Any system:</strong> drag the folder into a terminal
|
||||
window and its path appears.
|
||||
</li>
|
||||
</ul>
|
||||
<p>
|
||||
In a terminal, put quotes around a path that contains spaces.
|
||||
</p>
|
||||
</details>
|
||||
</section>
|
||||
|
||||
<section class="article-section" id="terminal">
|
||||
@@ -500,37 +252,6 @@ fnm default 24</code></pre>
|
||||
<code>.asar</code> file. On Windows, a path such as
|
||||
<code>"D:/apps/example"</code> works here.
|
||||
</p>
|
||||
<details class="evidence-details">
|
||||
<summary>Where do I find an app's .asar file?</summary>
|
||||
<p>
|
||||
Electron apps usually keep it in a
|
||||
<code>resources</code> folder:
|
||||
</p>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Windows:</strong>
|
||||
<code
|
||||
>C:\Users\<you>\AppData\Local\Programs\<App>\resources\app.asar</code
|
||||
>
|
||||
</li>
|
||||
<li>
|
||||
<strong>macOS:</strong>
|
||||
<code
|
||||
>/Applications/<App>.app/Contents/Resources/app.asar</code
|
||||
>
|
||||
(right-click the app and choose
|
||||
<em>Show Package Contents</em>)
|
||||
</li>
|
||||
<li>
|
||||
<strong>Linux:</strong> often
|
||||
<code>/opt/<App>/resources/app.asar</code>
|
||||
</li>
|
||||
</ul>
|
||||
<p>
|
||||
Some apps install elsewhere. Search the app's folder for
|
||||
<code>app.asar</code>.
|
||||
</p>
|
||||
</details>
|
||||
<p>
|
||||
The result identifies the app, shows connections between modules
|
||||
and gives their source locations. Connections REA cannot resolve
|
||||
|
||||
@@ -143,14 +143,6 @@
|
||||
</p>
|
||||
</div>
|
||||
<div class="guide-index">
|
||||
<section class="guide-entry">
|
||||
<p class="eyebrow">First investigation</p>
|
||||
<div>
|
||||
<h2>Set up REA</h2>
|
||||
<p>Connect your coding agent or use REA from the terminal.</p>
|
||||
<a class="text-link" href="../get-started/">Get started with REA</a>
|
||||
</div>
|
||||
</section>
|
||||
<section class="guide-entry">
|
||||
<p class="eyebrow">Native binaries</p>
|
||||
<div>
|
||||
@@ -207,6 +199,14 @@
|
||||
<a class="text-link" href="browser/">Browser observation guide</a>
|
||||
</div>
|
||||
</section>
|
||||
<section class="guide-entry">
|
||||
<p class="eyebrow">First investigation</p>
|
||||
<div>
|
||||
<h2>Set up REA</h2>
|
||||
<p>Connect your coding agent or use REA from the terminal.</p>
|
||||
<a class="text-link" href="../get-started/">Get started with REA</a>
|
||||
</div>
|
||||
</section>
|
||||
</div>
|
||||
</main>
|
||||
<!-- website:footer:start -->
|
||||
|
||||
@@ -168,33 +168,6 @@
|
||||
returns its instructions and callers, which give the agent a way
|
||||
to investigate the missing arithmetic.
|
||||
</p>
|
||||
<details class="evidence-details">
|
||||
<summary>New to these terms?</summary>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Instructions:</strong> the steps the processor runs,
|
||||
shown in readable form such as <code>MOV</code> or
|
||||
<code>ADD</code>.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Decompiler result / pseudocode:</strong> an analysis
|
||||
tool's attempt to turn instructions back into C-like code. It
|
||||
can miss parts, as in this example.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Caller:</strong> a function that calls this one. It
|
||||
shows which values are passed in.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Address:</strong> a location in the program, written
|
||||
in hexadecimal, such as <code>0x00406400</code>.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Constants:</strong> fixed numbers stored in the
|
||||
program, read as raw bytes and decoded.
|
||||
</li>
|
||||
</ul>
|
||||
</details>
|
||||
<figure class="workflow-figure">
|
||||
<p class="figure-title">
|
||||
Three pieces of evidence, one calculation
|
||||
@@ -238,40 +211,9 @@
|
||||
<h2>Ask about the behavior</h2>
|
||||
<p>
|
||||
After
|
||||
<a href="../../get-started/#agent"
|
||||
>connecting REA to your agent</a
|
||||
>
|
||||
and
|
||||
<a href="../../get-started/#native"
|
||||
>setting up a native analysis tool</a
|
||||
<a href="../../get-started/#agent">connecting REA to your agent</a
|
||||
>, provide the local executable and a question:
|
||||
</p>
|
||||
<details class="evidence-details">
|
||||
<summary>Which analysis tool should I use?</summary>
|
||||
<p>REA needs one of these to read native programs:</p>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Hopper</strong> (macOS or Linux): REA's setup detects
|
||||
it or offers to install it. On macOS, open Hopper once and
|
||||
choose demo mode or activate a license before your first
|
||||
analysis.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Ghidra</strong> (macOS or Linux; experimental on
|
||||
Windows): free, but needs an existing Ghidra 12.1.x
|
||||
installation and a 64-bit JDK 21 or newer.
|
||||
</li>
|
||||
<li>
|
||||
<strong>IDA</strong> (Windows): for an existing IDA setup with
|
||||
an MCP registration.
|
||||
</li>
|
||||
</ul>
|
||||
<p>
|
||||
See
|
||||
<a href="../../get-started/#native">Set up native analysis</a>
|
||||
for the steps.
|
||||
</p>
|
||||
</details>
|
||||
<div class="agent-terminal">
|
||||
<div class="terminal-header">
|
||||
<span class="terminal-title"
|
||||
@@ -297,28 +239,8 @@
|
||||
</div>
|
||||
<p class="caption">
|
||||
Example prompt. Give your agent the path to your local DX-Ball
|
||||
1.07 executable. REA doesn't include DX-Ball, so use your own
|
||||
copy, or ask the same kind of question about another small Windows
|
||||
program you have.
|
||||
1.07 executable.
|
||||
</p>
|
||||
<details class="evidence-details">
|
||||
<summary>How do I get a file's full path?</summary>
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Windows:</strong> in File Explorer, right-click the
|
||||
file and choose <em>Copy as path</em>. On Windows 10, hold
|
||||
Shift while right-clicking.
|
||||
</li>
|
||||
<li>
|
||||
<strong>macOS:</strong> in Finder, right-click the file, hold
|
||||
Option and choose <em>Copy "…" as Pathname</em>.
|
||||
</li>
|
||||
<li>
|
||||
<strong>Any system:</strong> drag the file into a terminal
|
||||
window and its path appears.
|
||||
</li>
|
||||
</ul>
|
||||
</details>
|
||||
</section>
|
||||
<section class="article-section" id="follow">
|
||||
<h2>Read the REA results</h2>
|
||||
|
||||
Reference in New Issue
Block a user