Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
561877ad76 | ||
|
|
022c8b6f53 | ||
|
|
9defed9725 | ||
|
|
5f6bb7a5ba | ||
|
|
2596f8da14 | ||
|
|
b12d0c6361 | ||
|
|
cbcba2902d | ||
|
|
07d416505c | ||
|
|
3ea0233342 | ||
|
|
cf6957e796 | ||
|
|
ed200a2302 | ||
|
|
7df2050606 | ||
|
|
130ffdf6bc | ||
|
|
b09d02d93c | ||
|
|
1f0775684b | ||
|
|
7296f2b5bf | ||
|
|
3c7b176a3d | ||
|
|
aa23ec883f | ||
|
|
934ffd28d1 | ||
|
|
79f6756239 | ||
|
|
78e1308f49 | ||
|
|
4e3d335439 | ||
|
|
ea92328775 | ||
|
|
28d3cb545c | ||
|
|
b118f04492 | ||
|
|
ef96657d6b | ||
|
|
696fb834b8 | ||
|
|
2781cad201 | ||
|
|
96318b66a0 | ||
|
|
c89c8f5318 | ||
|
|
1ec490f361 | ||
|
|
66863b609f | ||
|
|
817efaca9a | ||
|
|
0d2082d36b | ||
|
|
f1231d4a2c | ||
|
|
a957f397ad | ||
|
|
e09cfe9b59 | ||
|
|
ea1dbda1c7 | ||
|
|
7633dd8d5d | ||
|
|
941dc425ca | ||
|
|
33b9b31a6f | ||
|
|
eeb83f6581 | ||
|
|
bb6fa1c6f5 | ||
|
|
ce16f7bc95 | ||
|
|
0a14ca41b4 | ||
|
|
dc5af6d68f | ||
|
|
0b9815b6e7 | ||
|
|
6375ccaf04 | ||
|
|
c29931ce81 | ||
|
|
f4a0c1ce95 | ||
|
|
a9ee2360fd | ||
|
|
4d95f2b2e7 | ||
|
|
28595276fa | ||
|
|
98022a8352 | ||
|
|
81cdafc132 | ||
|
|
4d8ef3f350 | ||
|
|
ca3131b36c | ||
|
|
de8fc67195 | ||
|
|
8550913cda | ||
|
|
1f0f728c63 | ||
|
|
7623e21ff0 | ||
|
|
499e2f54ef | ||
|
|
6875d4a64c | ||
|
|
2b6e9ec03f | ||
|
|
498cec042a | ||
|
|
5fc4d792ca | ||
|
|
e4a8e0b728 | ||
|
|
e17290cd2a | ||
|
|
40f632a155 | ||
|
|
7aa4d768eb | ||
|
|
74cf9e1401 | ||
|
|
bf58eac4ef | ||
|
|
565f2efa03 | ||
|
|
b8a8190b2f | ||
|
|
e10c934d5a | ||
|
|
08ec2bf82f | ||
|
|
be6863e9a1 |
+13
-6
@@ -2,7 +2,7 @@
|
|||||||
name: reverse-engineer-anything
|
name: reverse-engineer-anything
|
||||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||||
metadata:
|
metadata:
|
||||||
version: "34"
|
version: "35"
|
||||||
---
|
---
|
||||||
|
|
||||||
# REA
|
# REA
|
||||||
@@ -62,13 +62,20 @@ with a version-pinned `rea-agents` package or continue through the CLI.
|
|||||||
A concrete CLI fallback for an operator-supplied JavaScript tree or ASAR is:
|
A concrete CLI fallback for an operator-supplied JavaScript tree or ASAR is:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json
|
npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json > app-evidence.json
|
||||||
|
jq -c '{source: {kind: "inline", evidence: .}, view: {kind: "summary"}}' app-evidence.json > app-view.json
|
||||||
|
npx -y rea-agents@latest inspect-analysis-view app-view.json
|
||||||
```
|
```
|
||||||
|
|
||||||
No MCP registration or native engine is required. Read the returned Evidence,
|
No MCP registration or native engine is required. The CLI returns the complete
|
||||||
graph, limitations, and unknowns with the same care as an MCP result; the CLI
|
Evidence record directly, and a real application can produce hundreds of
|
||||||
returns the Evidence record directly. This fallback does not establish that
|
megabytes, so save it to a file rather than reading it into context. Start from
|
||||||
MCP is configured. Native CLI tasks still require their selected engine.
|
the summary view, then request a module page
|
||||||
|
(`{"kind": "page", "collection": "modules", "offset": 0, "limit": 32}`) or one
|
||||||
|
module the same way, or pass the saved Evidence inline to
|
||||||
|
`trace-application-feature`. Read graph, limitations, and unknowns with the
|
||||||
|
same care as an MCP result. This fallback does not establish that MCP is
|
||||||
|
configured. Native CLI tasks still require their selected engine.
|
||||||
|
|
||||||
## Route the target first
|
## Route the target first
|
||||||
|
|
||||||
+8
@@ -34,6 +34,14 @@ cross-references. Addresses and recovered pseudocode are analysis observations,
|
|||||||
not original source. Provider unavailability and unsupported metadata remain
|
not original source. Provider unavailability and unsupported metadata remain
|
||||||
unknown rather than false.
|
unknown rather than false.
|
||||||
|
|
||||||
|
For a completed `analyze_function` dossier, use `inspect_analysis_view` with
|
||||||
|
`view: {kind: "native", facet: "value_flow_operations", offset: 0, limit: 32}`
|
||||||
|
and the exact retained Evidence ID to read selected facts without reanalysis.
|
||||||
|
Native facets also include procedure, pseudocode, assembly, callers/callees,
|
||||||
|
references, and value-flow summaries. Pseudocode offsets and limits count UTF-16
|
||||||
|
code units; use the returned `next_offset` for continuation. Provider limitations
|
||||||
|
and unavailable facts remain explicit; the complete parent Evidence is retained.
|
||||||
|
|
||||||
To see which functions or Objective-C methods a Mach-O actually calls in one
|
To see which functions or Objective-C methods a Mach-O actually calls in one
|
||||||
run, use `observe_native_calls` with explicit breakpoints and a bounded
|
run, use `observe_native_calls` with explicit breakpoints and a bounded
|
||||||
`duration_ms`/`max_events`. It launches a new process under LLDB, so the target
|
`duration_ms`/`max_events`. It launches a new process under LLDB, so the target
|
||||||
@@ -28,7 +28,7 @@ intentionally not included. -->
|
|||||||
- Provider, bridge, target-format, or platform compatibility:
|
- Provider, bridge, target-format, or platform compatibility:
|
||||||
- Evidence, artifact, provenance, or reconstruction contract:
|
- Evidence, artifact, provenance, or reconstruction contract:
|
||||||
- Process execution, authorization, cleanup, or containment impact:
|
- Process execution, authorization, cleanup, or containment impact:
|
||||||
- Generated metadata (`docs/public/product-catalog.json`), package, or installation impact (CI generates and validates ignored catalogs and `skills/`; edit authored instructions in `skill-src/` and never commit build outputs):
|
- Generated metadata (`docs/public/product-catalog.json`), package, or installation impact (CI generates and validates ignored catalogs and `skills/`; edit authored instructions in `.agents/skills/reverse-engineer-anything/` and never commit build outputs):
|
||||||
|
|
||||||
## Evidence and regression coverage
|
## Evidence and regression coverage
|
||||||
|
|
||||||
|
|||||||
@@ -64,6 +64,14 @@ jobs:
|
|||||||
env:
|
env:
|
||||||
REA_ANALYSIS_PROVIDER: ghidra
|
REA_ANALYSIS_PROVIDER: ghidra
|
||||||
run: npm run verify:ghidra:windows -- --x86 | Tee-Object -FilePath windows-ghidra-x86-proof.log
|
run: npm run verify:ghidra:windows -- --x86 | Tee-Object -FilePath windows-ghidra-x86-proof.log
|
||||||
|
- name: Verify native x64 DLL Windows Ghidra operations
|
||||||
|
env:
|
||||||
|
REA_ANALYSIS_PROVIDER: ghidra
|
||||||
|
run: npm run verify:ghidra:windows -- --dll | Tee-Object -FilePath windows-ghidra-dll-proof.log
|
||||||
|
- name: Verify native x86 DLL Windows Ghidra operations
|
||||||
|
env:
|
||||||
|
REA_ANALYSIS_PROVIDER: ghidra
|
||||||
|
run: npm run verify:ghidra:windows -- --x86 --dll | Tee-Object -FilePath windows-ghidra-x86-dll-proof.log
|
||||||
- name: Verify ordinary-user CLI and MCP operations
|
- name: Verify ordinary-user CLI and MCP operations
|
||||||
env:
|
env:
|
||||||
REA_ANALYSIS_PROVIDER: ghidra
|
REA_ANALYSIS_PROVIDER: ghidra
|
||||||
@@ -72,6 +80,14 @@ jobs:
|
|||||||
env:
|
env:
|
||||||
REA_ANALYSIS_PROVIDER: ghidra
|
REA_ANALYSIS_PROVIDER: ghidra
|
||||||
run: npm run verify:ghidra:windows:package -- --x86 | Tee-Object -FilePath windows-ghidra-x86-package-proof.log
|
run: npm run verify:ghidra:windows:package -- --x86 | Tee-Object -FilePath windows-ghidra-x86-package-proof.log
|
||||||
|
- name: Verify ordinary-user x64 DLL CLI and MCP operations
|
||||||
|
env:
|
||||||
|
REA_ANALYSIS_PROVIDER: ghidra
|
||||||
|
run: npm run verify:ghidra:windows:package -- --dll | Tee-Object -FilePath windows-ghidra-dll-package-proof.log
|
||||||
|
- name: Verify ordinary-user x86 DLL CLI and MCP operations
|
||||||
|
env:
|
||||||
|
REA_ANALYSIS_PROVIDER: ghidra
|
||||||
|
run: npm run verify:ghidra:windows:package -- --x86 --dll | Tee-Object -FilePath windows-ghidra-x86-dll-package-proof.log
|
||||||
- uses: actions/upload-artifact@v4
|
- uses: actions/upload-artifact@v4
|
||||||
if: always()
|
if: always()
|
||||||
with:
|
with:
|
||||||
@@ -81,4 +97,8 @@ jobs:
|
|||||||
windows-ghidra-package-proof.log
|
windows-ghidra-package-proof.log
|
||||||
windows-ghidra-x86-proof.log
|
windows-ghidra-x86-proof.log
|
||||||
windows-ghidra-x86-package-proof.log
|
windows-ghidra-x86-package-proof.log
|
||||||
|
windows-ghidra-dll-proof.log
|
||||||
|
windows-ghidra-x86-dll-proof.log
|
||||||
|
windows-ghidra-dll-package-proof.log
|
||||||
|
windows-ghidra-x86-dll-package-proof.log
|
||||||
retention-days: 30
|
retention-days: 30
|
||||||
|
|||||||
@@ -35,7 +35,7 @@ Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain.
|
|||||||
|
|
||||||
Use relevant checks for the change. The complete local gate, `npm run check:pr`, is optional for broad changes or CI diagnosis. See [CONTRIBUTING.md](CONTRIBUTING.md) for development, generated-file ownership, and release conventions.
|
Use relevant checks for the change. The complete local gate, `npm run check:pr`, is optional for broad changes or CI diagnosis. See [CONTRIBUTING.md](CONTRIBUTING.md) for development, generated-file ownership, and release conventions.
|
||||||
|
|
||||||
`docs/public/product-catalog.json`, `docs/verification/managed-conformance-*.json`, and `skills/` are ignored outputs. Edit source contracts and `skill-src/`, then regenerate as needed. `.cache/mcp-tool-catalog.json` is test metadata; runtime builds and source checking do not consume it. Never commit binaries, provider project documents, credentials, `dist/`, `node_modules/`, or local planning artifacts.
|
`docs/public/product-catalog.json`, `docs/verification/managed-conformance-*.json`, and `skills/` are ignored outputs. Edit source contracts and `.agents/skills/reverse-engineer-anything/`, then regenerate as needed. `.cache/mcp-tool-catalog.json` is test metadata; runtime builds and source checking do not consume it. Never commit binaries, provider project documents, credentials, `dist/`, `node_modules/`, or local planning artifacts.
|
||||||
|
|
||||||
## Tool and Test Changes
|
## Tool and Test Changes
|
||||||
|
|
||||||
|
|||||||
@@ -66,6 +66,7 @@
|
|||||||
* **javascript:** resolve exported callables through lexical bindings ([#1351](https://github.com/morluto/rea/issues/1351)) ([a9f5dad](https://github.com/morluto/rea/commit/a9f5dad5a8f196c88f6305d8d958b355cc0b52c2))
|
* **javascript:** resolve exported callables through lexical bindings ([#1351](https://github.com/morluto/rea/issues/1351)) ([a9f5dad](https://github.com/morluto/rea/commit/a9f5dad5a8f196c88f6305d8d958b355cc0b52c2))
|
||||||
* **javascript:** resolve query-only HTML references to their document ([55a46f7](https://github.com/morluto/rea/commit/55a46f7c7176075c9edda4fe61ed995f2ceadf6a))
|
* **javascript:** resolve query-only HTML references to their document ([55a46f7](https://github.com/morluto/rea/commit/55a46f7c7176075c9edda4fe61ed995f2ceadf6a))
|
||||||
* **javascript:** retain array identity after length mutations ([4bcea1a](https://github.com/morluto/rea/commit/4bcea1aa76e3e7ca665197181739c5e27a62c40a))
|
* **javascript:** retain array identity after length mutations ([4bcea1a](https://github.com/morluto/rea/commit/4bcea1aa76e3e7ca665197181739c5e27a62c40a))
|
||||||
|
* **javascript:** retain conditional mutation ordering uncertainty ([b8a8190](https://github.com/morluto/rea/commit/b8a8190b2f30e377eee2d69d1ba31ea454fa70b9))
|
||||||
* **javascript:** retain distinct module provenance path components ([2bd7de6](https://github.com/morluto/rea/commit/2bd7de6f8ca62fc1d48334cfa7fdced461b7b153))
|
* **javascript:** retain distinct module provenance path components ([2bd7de6](https://github.com/morluto/rea/commit/2bd7de6f8ca62fc1d48334cfa7fdced461b7b153))
|
||||||
* **javascript:** retain unresolved value frontiers in provenance queries ([3e19875](https://github.com/morluto/rea/commit/3e19875c7fa9c61cff655bbe6046ddc4ab558533))
|
* **javascript:** retain unresolved value frontiers in provenance queries ([3e19875](https://github.com/morluto/rea/commit/3e19875c7fa9c61cff655bbe6046ddc4ab558533))
|
||||||
* **javascript:** store source map URLs once ([ed8fcae](https://github.com/morluto/rea/commit/ed8fcae94b699b62b44162f880faa077a36c455d))
|
* **javascript:** store source map URLs once ([ed8fcae](https://github.com/morluto/rea/commit/ed8fcae94b699b62b44162f880faa077a36c455d))
|
||||||
|
|||||||
+6
-2
@@ -95,8 +95,12 @@ Pre-commit formats and lints staged files; pre-push runs `check:fast`.
|
|||||||
`evidence:generate` prepares only the managed commitment and its runtime/skill
|
`evidence:generate` prepares only the managed commitment and its runtime/skill
|
||||||
dependencies. The ignored outputs are `docs/public/product-catalog.json`,
|
dependencies. The ignored outputs are `docs/public/product-catalog.json`,
|
||||||
`docs/verification/managed-conformance-*.json`, and `skills/`. Edit authored
|
`docs/verification/managed-conformance-*.json`, and `skills/`. Edit authored
|
||||||
skill instructions in `skill-src/`. CI validates generated outputs and retains
|
skill instructions in `.agents/skills/reverse-engineer-anything/`, where Skills
|
||||||
them as artifacts; do not commit them. Reviewed source metadata, including
|
can discover the public bundle directly from a clean checkout. `verify:package`
|
||||||
|
installs that bundle with Skills 1.7.2 using default repository-root discovery
|
||||||
|
and checks every installed file, alongside the packaged setup checks.
|
||||||
|
CI validates generated outputs and retains them as artifacts; do not commit
|
||||||
|
them. Reviewed source metadata, including
|
||||||
`src/generatedPackageMetadata.ts` and `docs/error-contract.schema.json`, remains
|
`src/generatedPackageMetadata.ts` and `docs/error-contract.schema.json`, remains
|
||||||
tracked and checked for freshness.
|
tracked and checked for freshness.
|
||||||
|
|
||||||
|
|||||||
@@ -307,7 +307,7 @@ Report vulnerabilities through [SECURITY.md](SECURITY.md).
|
|||||||
|
|
||||||
## Star history
|
## Star history
|
||||||
|
|
||||||
🎉 **40,000 GitHub stars — thank you!**
|
🎉 **50,000 GitHub stars — thank you!**
|
||||||
|
|
||||||
Thanks to everyone using REA, reporting bugs, requesting features, testing builds, and contributing fixes.
|
Thanks to everyone using REA, reporting bugs, requesting features, testing builds, and contributing fixes.
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -258,7 +258,7 @@ npx rea-agents@latest setup
|
|||||||
|
|
||||||
## سجل النجوم
|
## سجل النجوم
|
||||||
|
|
||||||
🎉 **40,000 نجمة على GitHub — شكرًا لكم!**
|
🎉 **50,000 نجمة على GitHub — شكرًا لكم!**
|
||||||
|
|
||||||
شكرًا لكل من يستخدم REA، ويبلّغ عن الأخطاء، ويقترح ميزات، ويختبر إصدارات البناء، ويساهم بالإصلاحات.
|
شكرًا لكل من يستخدم REA، ويبلّغ عن الأخطاء، ويقترح ميزات، ويختبر إصدارات البناء، ويساهم بالإصلاحات.
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -256,7 +256,7 @@ Melde Sicherheitslücken gemäß [SECURITY.md](SECURITY.md).
|
|||||||
|
|
||||||
## Sternverlauf
|
## Sternverlauf
|
||||||
|
|
||||||
🎉 **40.000 GitHub-Sterne – vielen Dank!**
|
🎉 **50.000 GitHub-Sterne – vielen Dank!**
|
||||||
|
|
||||||
Danke an alle, die REA nutzen, Fehler melden, Funktionswünsche äußern, Builds testen und Fehlerbehebungen beitragen.
|
Danke an alle, die REA nutzen, Fehler melden, Funktionswünsche äußern, Builds testen und Fehlerbehebungen beitragen.
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -256,7 +256,7 @@ Informa de vulnerabilidades siguiendo [SECURITY.md](SECURITY.md).
|
|||||||
|
|
||||||
## Historial de estrellas
|
## Historial de estrellas
|
||||||
|
|
||||||
🎉 **40.000 estrellas en GitHub: ¡gracias!**
|
🎉 **50.000 estrellas en GitHub: ¡gracias!**
|
||||||
|
|
||||||
Gracias a quienes usan REA, informan de errores, solicitan funciones, prueban compilaciones y contribuyen con correcciones.
|
Gracias a quienes usan REA, informan de errores, solicitan funciones, prueban compilaciones y contribuyen con correcciones.
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -241,7 +241,7 @@ npx rea-agents@latest setup
|
|||||||
|
|
||||||
## تاریخچهٔ ستارهها
|
## تاریخچهٔ ستارهها
|
||||||
|
|
||||||
🎉 **۴۰٬۰۰۰ ستارهٔ GitHub — سپاسگزاریم!**
|
🎉 **۵۰٬۰۰۰ ستارهٔ GitHub — سپاسگزاریم!**
|
||||||
|
|
||||||
از همهٔ کسانی که از REA استفاده میکنند، خطاها را گزارش میدهند، قابلیتهای جدید پیشنهاد میکنند، buildها را آزمایش میکنند و در بهبود پروژه مشارکت دارند سپاسگزاریم.
|
از همهٔ کسانی که از REA استفاده میکنند، خطاها را گزارش میدهند، قابلیتهای جدید پیشنهاد میکنند، buildها را آزمایش میکنند و در بهبود پروژه مشارکت دارند سپاسگزاریم.
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -257,7 +257,7 @@ Signalez les vulnérabilités via [SECURITY.md](SECURITY.md).
|
|||||||
|
|
||||||
## Historique des étoiles
|
## Historique des étoiles
|
||||||
|
|
||||||
🎉 **40 000 étoiles GitHub — merci !**
|
🎉 **50 000 étoiles GitHub — merci !**
|
||||||
|
|
||||||
Merci à toutes les personnes qui utilisent REA, signalent des bugs, proposent des fonctionnalités, testent les builds et contribuent des correctifs.
|
Merci à toutes les personnes qui utilisent REA, signalent des bugs, proposent des fonctionnalités, testent les builds et contribuent des correctifs.
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -258,7 +258,7 @@ npx rea-agents@latest setup
|
|||||||
|
|
||||||
## スター履歴
|
## スター履歴
|
||||||
|
|
||||||
🎉 **GitHub スター 40,000 件、ありがとうございます!**
|
🎉 **GitHub スター 50,000 件、ありがとうございます!**
|
||||||
|
|
||||||
REA の利用、バグ報告、機能の要望、ビルドのテスト、修正への貢献に感謝します。
|
REA の利用、バグ報告、機能の要望、ビルドのテスト、修正への貢献に感謝します。
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -258,7 +258,7 @@ npx rea-agents@latest setup
|
|||||||
|
|
||||||
## 스타 기록
|
## 스타 기록
|
||||||
|
|
||||||
🎉 **GitHub 스타 40,000개, 감사합니다!**
|
🎉 **GitHub 스타 50,000개, 감사합니다!**
|
||||||
|
|
||||||
REA를 사용하고, 버그를 보고하고, 기능을 요청하고, 빌드를 테스트하고, 수정에 기여해 주신 모든 분께 감사드립니다.
|
REA를 사용하고, 버그를 보고하고, 기능을 요청하고, 빌드를 테스트하고, 수정에 기여해 주신 모든 분께 감사드립니다.
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -256,7 +256,7 @@ Zgłaszaj podatności zgodnie z [SECURITY.md](SECURITY.md).
|
|||||||
|
|
||||||
## Historia gwiazdek
|
## Historia gwiazdek
|
||||||
|
|
||||||
🎉 **40 000 gwiazdek na GitHubie — dziękujemy!**
|
🎉 **50 000 gwiazdek na GitHubie — dziękujemy!**
|
||||||
|
|
||||||
Dziękujemy wszystkim, którzy używają REA, zgłaszają błędy, proponują nowe funkcje, testują kompilacje i przesyłają poprawki.
|
Dziękujemy wszystkim, którzy używają REA, zgłaszają błędy, proponują nowe funkcje, testują kompilacje i przesyłają poprawki.
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -256,7 +256,7 @@ Relate vulnerabilidades conforme [SECURITY.md](SECURITY.md).
|
|||||||
|
|
||||||
## Histórico de estrelas
|
## Histórico de estrelas
|
||||||
|
|
||||||
🎉 **40.000 estrelas no GitHub — muito obrigado!**
|
🎉 **50.000 estrelas no GitHub — muito obrigado!**
|
||||||
|
|
||||||
Obrigado a todos que usam o REA, relatam bugs, sugerem funcionalidades, testam builds e contribuem com correções.
|
Obrigado a todos que usam o REA, relatam bugs, sugerem funcionalidades, testam builds e contribuem com correções.
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -256,7 +256,7 @@ npx rea-agents@latest setup
|
|||||||
|
|
||||||
## История звёзд
|
## История звёзд
|
||||||
|
|
||||||
🎉 **40 000 звёзд на GitHub — спасибо!**
|
🎉 **50 000 звёзд на GitHub — спасибо!**
|
||||||
|
|
||||||
Спасибо всем, кто использует REA, сообщает об ошибках, предлагает новые функции, тестирует сборки и помогает с исправлениями.
|
Спасибо всем, кто использует REA, сообщает об ошибках, предлагает новые функции, тестирует сборки и помогает с исправлениями.
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -256,7 +256,7 @@ npx rea-agents@latest setup
|
|||||||
|
|
||||||
## ประวัติดาว
|
## ประวัติดาว
|
||||||
|
|
||||||
🎉 **40,000 ดาวบน GitHub — ขอบคุณทุกคน!**
|
🎉 **50,000 ดาวบน GitHub — ขอบคุณทุกคน!**
|
||||||
|
|
||||||
ขอบคุณทุกคนที่ใช้ REA รายงานบั๊ก เสนอฟีเจอร์ ทดสอบบิลด์ และช่วยแก้ไข
|
ขอบคุณทุกคนที่ใช้ REA รายงานบั๊ก เสนอฟีเจอร์ ทดสอบบิลด์ และช่วยแก้ไข
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -256,7 +256,7 @@ Güvenlik açıklarını [SECURITY.md](SECURITY.md) üzerinden bildirin.
|
|||||||
|
|
||||||
## Yıldız geçmişi
|
## Yıldız geçmişi
|
||||||
|
|
||||||
🎉 **GitHub'da 40.000 yıldız — teşekkürler!**
|
🎉 **GitHub'da 50.000 yıldız — teşekkürler!**
|
||||||
|
|
||||||
REA'yı kullanan, hata bildiren, özellik talep eden, derlemeleri test eden ve düzeltmelere katkıda bulunan herkese teşekkürler.
|
REA'yı kullanan, hata bildiren, özellik talep eden, derlemeleri test eden ve düzeltmelere katkıda bulunan herkese teşekkürler.
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -256,7 +256,7 @@ npx rea-agents@latest setup
|
|||||||
|
|
||||||
## Історія зірок
|
## Історія зірок
|
||||||
|
|
||||||
🎉 **40 000 зірок на GitHub — дякуємо!**
|
🎉 **50 000 зірок на GitHub — дякуємо!**
|
||||||
|
|
||||||
Дякуємо всім, хто використовує REA, повідомляє про помилки, пропонує нові функції, тестує збірки й допомагає з виправленнями.
|
Дякуємо всім, хто використовує REA, повідомляє про помилки, пропонує нові функції, тестує збірки й допомагає з виправленнями.
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -256,7 +256,7 @@ Báo cáo lỗ hổng theo [SECURITY.md](SECURITY.md).
|
|||||||
|
|
||||||
## Lịch sử sao
|
## Lịch sử sao
|
||||||
|
|
||||||
🎉 **40.000 sao trên GitHub — xin cảm ơn!**
|
🎉 **50.000 sao trên GitHub — xin cảm ơn!**
|
||||||
|
|
||||||
Cảm ơn mọi người đã sử dụng REA, báo lỗi, đề xuất tính năng, kiểm thử bản dựng và đóng góp bản sửa lỗi.
|
Cảm ơn mọi người đã sử dụng REA, báo lỗi, đề xuất tính năng, kiểm thử bản dựng và đóng góp bản sửa lỗi.
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -258,7 +258,7 @@ npx rea-agents@latest setup
|
|||||||
|
|
||||||
## Star 歷史
|
## Star 歷史
|
||||||
|
|
||||||
🎉 **GitHub Star 達到 40,000 個,感謝大家!**
|
🎉 **GitHub Star 達到 50,000 個,感謝大家!**
|
||||||
|
|
||||||
感謝每一位使用 REA、報告錯誤、提出功能需求、測試建置和貢獻修復的朋友。
|
感謝每一位使用 REA、報告錯誤、提出功能需求、測試建置和貢獻修復的朋友。
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -258,7 +258,7 @@ npx rea-agents@latest setup
|
|||||||
|
|
||||||
## Star 历史
|
## Star 历史
|
||||||
|
|
||||||
🎉 **GitHub Star 达到 40,000 个,感谢大家!**
|
🎉 **GitHub Star 达到 50,000 个,感谢大家!**
|
||||||
|
|
||||||
感谢每一位使用 REA、报告错误、提出功能需求、测试构建和贡献修复的朋友。
|
感谢每一位使用 REA、报告错误、提出功能需求、测试构建和贡献修复的朋友。
|
||||||
|
|
||||||
|
|||||||
@@ -126,8 +126,13 @@ also stops target hashing before provider admission.
|
|||||||
|
|
||||||
These operations do not execute the APK or provide split APK/AAB handling,
|
These operations do not execute the APK or provide split APK/AAB handling,
|
||||||
signature validation, full resource-table semantics, native-library analysis or
|
signature validation, full resource-table semantics, native-library analysis or
|
||||||
Android runtime capture. Existing artifact inventory/extraction tools can supply
|
Android runtime capture. An `.aab` suffix is a ZIP archive, not an APK, so
|
||||||
archive evidence. Project that Evidence with the execution-free inventory tool:
|
Android tools and the application graph do not treat it as a standalone APK.
|
||||||
|
Existing artifact inventory/extraction tools can supply archive evidence.
|
||||||
|
Inventory keeps ZIP names that differ only in case and records that a
|
||||||
|
case-insensitive destination cannot store both spellings. Extraction fails
|
||||||
|
only when that destination directory cannot keep both names. Project that
|
||||||
|
Evidence with the execution-free inventory tool:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
rea project-android-application-graph '{"inventory_evidence":[<inventory_artifact Evidence>]}'
|
rea project-android-application-graph '{"inventory_evidence":[<inventory_artifact Evidence>]}'
|
||||||
|
|||||||
@@ -18,6 +18,7 @@ REA can attach to a user-owned Chrome-family browser through the Chrome DevTools
|
|||||||
- `compare_web_captures` / `rea compare-web-captures` compares passive captures or exact-step-aligned browser scenarios. Scenario results commit reproducible literal normalization and expose alignment failures plus artifact-level action, screenshot, DOM, accessibility, URL, history, storage, and event differences. Missing or truncated evidence is never treated as equivalence.
|
- `compare_web_captures` / `rea compare-web-captures` compares passive captures or exact-step-aligned browser scenarios. Scenario results commit reproducible literal normalization and expose alignment failures plus artifact-level action, screenshot, DOM, accessibility, URL, history, storage, and event differences. Missing or truncated evidence is never treated as equivalence.
|
||||||
- `capture_web_screenshot` / `rea capture-web-screenshot` returns a content-addressed visible-viewport PNG.
|
- `capture_web_screenshot` / `rea capture-web-screenshot` returns a content-addressed visible-viewport PNG.
|
||||||
- `compare_web_screenshots` / `rea compare-web-screenshots` performs bounded local PNG pixel comparison without OCR or external services.
|
- `compare_web_screenshots` / `rea compare-web-screenshots` performs bounded local PNG pixel comparison without OCR or external services.
|
||||||
|
- The CLI comparison commands accept each capture, screenshot artifact or normalization policy as inline JSON or a JSON file path. Use files for screenshot artifacts and scenario captures, whose embedded PNG bytes can exceed the host's command-line length limit.
|
||||||
- Every surface has equivalent CLI and MCP contracts and returns Evidence provenance.
|
- Every surface has equivalent CLI and MCP contracts and returns Evidence provenance.
|
||||||
- MCP tools return complete results inline and include the Evidence ID. Session evidence can be exported with `export_evidence_bundle`.
|
- MCP tools return complete results inline and include the Evidence ID. Session evidence can be exported with `export_evidence_bundle`.
|
||||||
|
|
||||||
|
|||||||
+21
-1
@@ -20,6 +20,23 @@ files when neither `--provider` nor `--snapshot` is supplied. See
|
|||||||
[JavaScript artifact reconstruction](javascript-artifact-reconstruction.md)
|
[JavaScript artifact reconstruction](javascript-artifact-reconstruction.md)
|
||||||
for results, integrity checks and coverage.
|
for results, integrity checks and coverage.
|
||||||
|
|
||||||
|
The complete application Evidence can be hundreds of megabytes for a real
|
||||||
|
application. When an agent or a person reads the result, save it to a file and
|
||||||
|
project a summary, a module page or one module from the saved Evidence:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
rea analyze-javascript-application /absolute/path/to/app --json > app-evidence.json
|
||||||
|
jq -c '{source: {kind: "inline", evidence: .}, view: {kind: "summary"}}' app-evidence.json > app-view.json
|
||||||
|
rea inspect-analysis-view app-view.json
|
||||||
|
```
|
||||||
|
|
||||||
|
For example, a 334 MB Obsidian application Evidence file yields a summary view
|
||||||
|
of about 10 KB. Replace the view with
|
||||||
|
`{"kind": "page", "collection": "modules", "offset": 0, "limit": 32}` for a
|
||||||
|
module page. See
|
||||||
|
[JavaScript application workflows](javascript-application-workflows.md) for
|
||||||
|
views, feature traces and comparisons that take the same saved Evidence.
|
||||||
|
|
||||||
## Native analysis
|
## Native analysis
|
||||||
|
|
||||||
Configure [Hopper or Ghidra](installation.md#hopper), or the
|
Configure [Hopper or Ghidra](installation.md#hopper), or the
|
||||||
@@ -179,7 +196,10 @@ user permissions; consult the chosen guide for host requirements and effects.
|
|||||||
## Output and exit status
|
## Output and exit status
|
||||||
|
|
||||||
The default terminal format is TOON. Use `--json` when saving results for a
|
The default terminal format is TOON. Use `--json` when saving results for a
|
||||||
JSON consumer. Output selection and formatting do not change operation status.
|
JSON consumer. `--json` is indented on a terminal and compact when piped or
|
||||||
|
redirected, which keeps large results smaller for agents and scripts; pipe
|
||||||
|
through `jq .` for indented files. Output selection and formatting do not change
|
||||||
|
operation status.
|
||||||
|
|
||||||
| Status | Meaning |
|
| Status | Meaning |
|
||||||
| --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
|
| --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
|
|||||||
@@ -62,11 +62,14 @@ Mach-O target coverage. It requires `clang`, LLD, and `lld-link`; set
|
|||||||
these tools before compilation. Keeping this matrix separate lets Linux
|
these tools before compilation. Keeping this matrix separate lets Linux
|
||||||
host/provider acceptance run with only the host compiler.
|
host/provider acceptance run with only the host compiler.
|
||||||
|
|
||||||
`npm run verify:ghidra:windows` and its `-- --x86` variant check the source-owned native x86 and x86-64 PE
|
`npm run verify:ghidra:windows` checks source-owned native x86-64 PE applications;
|
||||||
fixture, all 25 read-only operations, target/snapshot/import SHA-256 linkage,
|
`-- --x86` selects x86 and `-- --dll` selects DLLs, and the flags can be combined.
|
||||||
|
These lanes check all 25 read-only operations, target/snapshot/import SHA-256 linkage,
|
||||||
and project, endpoint, process, and runtime cleanup. The independent native
|
and project, endpoint, process, and runtime cleanup. The independent native
|
||||||
lane checks DACLs, handle admission, cancellation, and Job Object lifecycle.
|
lane checks DACLs, handle admission, cancellation, and Job Object lifecycle.
|
||||||
`npm run verify:ghidra:windows:package` packs and installs REA into an isolated
|
`npm run verify:ghidra:windows:package` packs and installs REA into an isolated
|
||||||
prefix, then checks ordinary-user CLI/MCP operations against canonical contracts.
|
prefix, then checks ordinary-user CLI/MCP operations against canonical contracts.
|
||||||
|
It accepts the same architecture and DLL flags and verifies the selected PE role
|
||||||
|
in the analysis profile.
|
||||||
All lanes require matching native controls; package startup alone establishes
|
All lanes require matching native controls; package startup alone establishes
|
||||||
only package compatibility.
|
only package compatibility.
|
||||||
|
|||||||
+13
-6
@@ -95,7 +95,7 @@ npx skills add morluto/rea --skill reverse-engineer-anything
|
|||||||
|
|
||||||
This installs agent instructions and bundled references, not REA MCP
|
This installs agent instructions and bundled references, not REA MCP
|
||||||
registration or analysis engines. Follow the skill's
|
registration or analysis engines. Follow the skill's
|
||||||
[conditional connection guide](https://github.com/morluto/rea/blob/main/skill-src/reverse-engineer-anything/SKILL.md#connect-only-when-needed).
|
[conditional connection guide](https://github.com/morluto/rea/blob/main/.agents/skills/reverse-engineer-anything/SKILL.md#connect-only-when-needed).
|
||||||
Working tools can be used immediately. If tools are missing, inspect the current
|
Working tools can be used immediately. If tools are missing, inspect the current
|
||||||
client's registration with `doctor --client codex --json` (substitute its client
|
client's registration with `doctor --client codex --json` (substitute its client
|
||||||
ID), then plan repairs with `setup --client codex --dry-run --json`. Show and
|
ID), then plan repairs with `setup --client codex --dry-run --json`. Show and
|
||||||
@@ -135,10 +135,17 @@ listed after the table because its connector is not one of these files:
|
|||||||
| Antigravity | `antigravity` |
|
| Antigravity | `antigravity` |
|
||||||
| GitHub Copilot CLI | `copilot_cli` |
|
| GitHub Copilot CLI | `copilot_cli` |
|
||||||
| Command Code | `commandcode` |
|
| Command Code | `commandcode` |
|
||||||
|
| Qwen Code | `qwen_code` |
|
||||||
| VS Code | `vscode` |
|
| VS Code | `vscode` |
|
||||||
| Grok Build | `grok_build` |
|
| Grok Build | `grok_build` |
|
||||||
| OMP | `omp` |
|
| OMP | `omp` |
|
||||||
|
|
||||||
|
For Qwen Code, setup registers `rea` in the `mcpServers` table of the
|
||||||
|
user-level `~/.qwen/settings.json`, or `$QWEN_HOME/settings.json` when configured.
|
||||||
|
Setup and doctor follow Qwen Code's tilde and working-directory-relative home
|
||||||
|
overrides. Qwen Code also discovers personal skills in `~/.agents/skills`,
|
||||||
|
where setup installs the bundled workflow.
|
||||||
|
|
||||||
For OMP, setup writes a `type: "stdio"` entry to the user-level
|
For OMP, setup writes a `type: "stdio"` entry to the user-level
|
||||||
`~/.omp/agent/mcp.json`. It follows `PI_CONFIG_DIR`, an absolute
|
`~/.omp/agent/mcp.json`. It follows `PI_CONFIG_DIR`, an absolute
|
||||||
`PI_CODING_AGENT_DIR`, and the profile selected by `OMP_PROFILE` or
|
`PI_CODING_AGENT_DIR`, and the profile selected by `OMP_PROFILE` or
|
||||||
@@ -194,10 +201,10 @@ save verified paths for an existing Ghidra installation.
|
|||||||
|
|
||||||
The bundled skill is installed where each selected client discovers personal
|
The bundled skill is installed where each selected client discovers personal
|
||||||
skills: Claude Code uses `~/.claude/skills` (or
|
skills: Claude Code uses `~/.claude/skills` (or
|
||||||
`$CLAUDE_CONFIG_DIR/skills` when configured), while other supported clients
|
`$CLAUDE_CONFIG_DIR/skills` when configured), while other supported clients,
|
||||||
use the shared `~/.agents/skills` directory. A mixed selection plans both
|
including Qwen Code, use the shared `~/.agents/skills` directory. A mixed selection
|
||||||
paths. Selecting the skill without a client uses the shared directory. Setup
|
plans the relevant paths. Selecting the skill without a client uses the shared
|
||||||
leaves existing skill copies in other locations untouched.
|
directory. Setup leaves existing skill copies in other locations untouched.
|
||||||
|
|
||||||
`doctor --skill --json` verifies the selected copies against the bundled
|
`doctor --skill --json` verifies the selected copies against the bundled
|
||||||
instructions and references. Consumers should use `identity.skill.state`,
|
instructions and references. Consumers should use `identity.skill.state`,
|
||||||
@@ -489,7 +496,7 @@ entry comments atomically and returns refreshed analysis. These session metadata
|
|||||||
edits leave executable bytes unchanged and are discarded on close. GUI controls
|
edits leave executable bytes unchanged and are discarded on close. GUI controls
|
||||||
require Hopper; Windows P0 remains read-only.
|
require Hopper; Windows P0 remains read-only.
|
||||||
|
|
||||||
Windows P0 admits native x86 and x86-64 PE applications on fixed local NTFS volumes.
|
Windows P0 admits native x86 and x86-64 PE applications and DLLs on fixed local NTFS volumes.
|
||||||
The npm package bundles native Job Object ownership, protected private runtime
|
The npm package bundles native Job Object ownership, protected private runtime
|
||||||
DACLs, and handle-based path admission; no separate addon installation is needed.
|
DACLs, and handle-based path admission; no separate addon installation is needed.
|
||||||
See the [Windows Ghidra P0 guide](windows-ghidra-p0.md) for verified scope and
|
See the [Windows Ghidra P0 guide](windows-ghidra-p0.md) for verified scope and
|
||||||
|
|||||||
@@ -90,6 +90,44 @@ keep the result ambiguous. Runtime Evidence can later corroborate an exact
|
|||||||
mapped candidate, but structural reachability, semantic influence, runtime
|
mapped candidate, but structural reachability, semantic influence, runtime
|
||||||
observation, and causal proof remain separate claims.
|
observation, and causal proof remain separate claims.
|
||||||
|
|
||||||
|
Semantic literal nodes keep their complete value in `properties.value`; literal
|
||||||
|
queries match that field. String labels use `string literal` when that is
|
||||||
|
shorter than the complete JSON value. A literal's `identity.role_key` commits
|
||||||
|
its canonical JSON value as `value-sha256:<digest>` when the digest key is
|
||||||
|
shorter; smaller values keep their existing JSON key. This keeps large values
|
||||||
|
out of identity and display metadata without expanding short values or
|
||||||
|
truncating their Evidence.
|
||||||
|
|
||||||
|
New analyses therefore produce different IDs for hashed literals from the
|
||||||
|
earlier payload-bearing role keys. Use IDs returned by the selected parent
|
||||||
|
graph and read literal values from `properties.value`, rather than decoding
|
||||||
|
role keys or labels. Previously stored graphs and their original IDs remain
|
||||||
|
valid inputs.
|
||||||
|
|
||||||
|
Endpoint nodes keep the exact endpoint in each observation's `properties.value`;
|
||||||
|
semantic request and response nodes keep it in `properties.endpoint`. Endpoint
|
||||||
|
identity keys use `value-sha256:<digest>` over the canonical JSON string when
|
||||||
|
that is shorter, or when the original value starts with the reserved digest
|
||||||
|
prefix. Other short endpoint keys remain unchanged. Endpoint labels use the
|
||||||
|
shorter of the exact value and `<kind> endpoint`; semantic request/response
|
||||||
|
labels use the shorter of the endpoint and operation method. These labels are
|
||||||
|
display metadata, not endpoint lookup values.
|
||||||
|
|
||||||
|
New analyses therefore change hashed endpoint IDs and longer endpoint labels.
|
||||||
|
Use IDs from the selected graph and read the complete endpoint properties;
|
||||||
|
previously stored graphs still support their original IDs. Literal endpoint
|
||||||
|
search, semantic tracing, and version comparison use the preserved values.
|
||||||
|
|
||||||
|
Event and listener nodes likewise preserve the complete event name in
|
||||||
|
`properties.event_name`. Event role keys use the same reserved-prefix string
|
||||||
|
digest rule, scoped to the emitter; repeated registrations, removals, and
|
||||||
|
dispatches keep one event per exact emitter/name pair. Long event labels use
|
||||||
|
`event`, and listener labels use the shorter of `<method>:<name>` and
|
||||||
|
`<method>:listener`. Dynamic names remain `null` with explicit uncertainty. New analyses
|
||||||
|
change hashed event IDs and longer event/listener labels; query the preserved
|
||||||
|
name or select IDs from the parent graph. Previously stored graphs keep their
|
||||||
|
original IDs and remain valid trace inputs.
|
||||||
|
|
||||||
## Version comparison
|
## Version comparison
|
||||||
|
|
||||||
REA pairs entities only when a tier produces one unique candidate on each side.
|
REA pairs entities only when a tier produces one unique candidate on each side.
|
||||||
@@ -145,8 +183,10 @@ Direct return expressions, including expression-bodied arrows, are evaluated
|
|||||||
through an execution-free value lattice. Literal object fields and direct
|
through an execution-free value lattice. Literal object fields and direct
|
||||||
return sites are represented in the result. Calls, dynamic spreads, computed
|
return sites are represented in the result. Calls, dynamic spreads, computed
|
||||||
keys, and parser recovery remain partial or unknown.
|
keys, and parser recovery remain partial or unknown.
|
||||||
Objects and arrays passed to calls, constructors, or tagged templates, or used
|
Objects and arrays passed to calls, constructors, or tagged templates, used
|
||||||
as method receivers, are not assumed unchanged after the invocation. Aliases
|
as method receivers, or stored through property targets are not assumed
|
||||||
|
unchanged afterward. Property stores conservatively retain reference escape
|
||||||
|
uncertainty rather than pretending to execute later writes. Aliases
|
||||||
and shared children in spread and rest copies retain that uncertainty; copied
|
and shared children in spread and rest copies retain that uncertainty; copied
|
||||||
primitive slots and unrelated containing properties remain known. Object rest
|
primitive slots and unrelated containing properties remain known. Object rest
|
||||||
excludes consumed keys, array rest excludes the consumed prefix for known
|
excludes consumed keys, array rest excludes the consumed prefix for known
|
||||||
|
|||||||
@@ -82,6 +82,10 @@ x86-64 ELF and native Windows PE targets analyzed on Linux x64 with Ghidra
|
|||||||
See [NativeAOT recovery](ghidra-nativeaot.md) for build/configuration, recovered
|
See [NativeAOT recovery](ghidra-nativeaot.md) for build/configuration, recovered
|
||||||
metadata, derived-memory provenance, and the real verification lane.
|
metadata, derived-memory provenance, and the real verification lane.
|
||||||
|
|
||||||
|
REA's experimental Windows Ghidra boundary admits native x86 and x86-64 PE
|
||||||
|
applications and DLLs for static analysis. This does not establish Windows
|
||||||
|
NativeAOT metadata-recovery coverage.
|
||||||
|
|
||||||
For NativeAOT analysis, preserve the selected native artifact's path and digest
|
For NativeAOT analysis, preserve the selected native artifact's path and digest
|
||||||
and report recovered type data as provider observations. Linking those types
|
and report recovered type data as provider observations. Linking those types
|
||||||
to native function addresses requires verified
|
to native function addresses requires verified
|
||||||
|
|||||||
+37
-6
@@ -36,6 +36,13 @@ after SDK conversion and check advertised validation and actual calls.
|
|||||||
Individual model APIs can impose additional nesting limits; complete producer
|
Individual model APIs can impose additional nesting limits; complete producer
|
||||||
captures can exceed ten structural levels.
|
captures can exceed ten structural levels.
|
||||||
|
|
||||||
|
Self-contained output schemas advertise a content-bound `$id`, including their
|
||||||
|
declared dialect. SDK validators can reuse compiled schemas across complete
|
||||||
|
catalog refreshes and equivalent tool outputs; changing the schema changes its
|
||||||
|
identity. Explicit schema IDs and relative external reference bases are
|
||||||
|
preserved. This keeps every tool and validation rule in discovery and does not
|
||||||
|
change the SDK's catalog invalidation or availability checks.
|
||||||
|
|
||||||
`compare_web_captures` accepts exactly one of two input shapes:
|
`compare_web_captures` accepts exactly one of two input shapes:
|
||||||
|
|
||||||
- Passive: `before` and `after` each contain `inspection`, the complete
|
- Passive: `before` and `after` each contain `inspection`, the complete
|
||||||
@@ -186,10 +193,27 @@ the current client, so it is not a fresh-start recovery. On the same connection:
|
|||||||
|
|
||||||
This close/reopen flow was exercised on one real Linux stdio connection after a
|
This close/reopen flow was exercised on one real Linux stdio connection after a
|
||||||
controlled startup timeout, followed by a successful overview and function
|
controlled startup timeout, followed by a successful overview and function
|
||||||
analysis. It is not a Windows/macOS coverage claim. REA cleans only resources it
|
analysis. A separate Windows x64 stdio run with Ghidra 12.1.4, JDK 21 and Node
|
||||||
owns and never switches to another provider automatically. A provider timeout,
|
22.19.0 exercised the same connection and caller-selected native PE: the default
|
||||||
installation failure, or host permission denial needs its own reported recovery;
|
60-second client request timed out, close succeeded, and reopening before a
|
||||||
increasing a client deadline alone does not fix those failures.
|
240-second request completed the cold overview in about 163 seconds. The next
|
||||||
|
overview took about 19 ms. The cold and warm overviews reported two procedures
|
||||||
|
and two segments; the input digest was unchanged and no owned runtime root or
|
||||||
|
observed process-family member remained after final close. This was a small
|
||||||
|
1,024-byte fixture, so the longer request setting remains an example, not a
|
||||||
|
guarantee for larger targets. No macOS cold-start recovery was exercised.
|
||||||
|
|
||||||
|
With a supplied progress token, the Windows run received an operation-start
|
||||||
|
notification before timeout and start/completion notifications for the
|
||||||
|
successful retry. It received no intermediate import or auto-analysis progress.
|
||||||
|
These operation markers do not measure analysis work completed or extend the
|
||||||
|
client's deadline. Each new provider session still imports the target into a
|
||||||
|
fresh temporary project.
|
||||||
|
|
||||||
|
REA cleans only resources it owns and never switches to another provider
|
||||||
|
automatically. A provider timeout, installation failure, or host permission
|
||||||
|
denial needs its own reported recovery; increasing a client deadline alone does
|
||||||
|
not fix those failures.
|
||||||
|
|
||||||
## Tool results
|
## Tool results
|
||||||
|
|
||||||
@@ -198,7 +222,9 @@ Custom clients upgrading from 6.1 should follow the
|
|||||||
|
|
||||||
Evidence-producing tools return the complete canonical Evidence record in both
|
Evidence-producing tools return the complete canonical Evidence record in both
|
||||||
text and structured content. Read `structuredContent.normalized_result` for the
|
text and structured content. Read `structuredContent.normalized_result` for the
|
||||||
operation result and `structuredContent.evidence_id` for its identity. The same
|
operation result and `structuredContent.evidence_id` for its identity.
|
||||||
|
`raw_result` holds a provider representation distinct from that result, such
|
||||||
|
as an upstream tool's original report, and is `null` when there is none. The same
|
||||||
record is retained in the session bundle. `analysis_profile` is always present:
|
record is retained in the session bundle. `analysis_profile` is always present:
|
||||||
a concrete profile object or `null`; either value participates in semantic
|
a concrete profile object or `null`; either value participates in semantic
|
||||||
identity. Records omitting this field are rejected. Pass the returned Evidence
|
identity. Records omitting this field are rejected. Pass the returned Evidence
|
||||||
@@ -235,7 +261,12 @@ Ordinary responses keep their existing complete result contract.
|
|||||||
## Retained application Evidence inputs
|
## Retained application Evidence inputs
|
||||||
|
|
||||||
`inspect_analysis_view` projects a caller-selected view of already completed
|
`inspect_analysis_view` projects a caller-selected view of already completed
|
||||||
`inspect_binary_layout` or `analyze_javascript_application` Evidence. Source is
|
`inspect_binary_layout`, `analyze_javascript_application`, or `analyze_function`
|
||||||
|
Evidence. Native views select procedure, pseudocode, assembly, callers/callees,
|
||||||
|
references, and high-pcode facets without starting a provider. Native offset
|
||||||
|
and limit default to 0 and 64; pseudocode uses UTF-16 code units and never
|
||||||
|
splits surrogate pairs, while other pages count rows. Unavailable facts and
|
||||||
|
provider limitations remain explicit. Source is
|
||||||
an exact same-session retained reference or portable inline Evidence. Views are
|
an exact same-session retained reference or portable inline Evidence. Views are
|
||||||
a summary, a layout mitigations or linkage facet, one section/symbol/module, or
|
a summary, a layout mitigations or linkage facet, one section/symbol/module, or
|
||||||
a stable page with a caller-selected positive `limit`. Module pages include
|
a stable page with a caller-selected positive `limit`. Module pages include
|
||||||
|
|||||||
@@ -379,7 +379,7 @@ the tool fails with a `debugger-attach-denied` reason.
|
|||||||
Install a Ghidra 12.1.x release and the 64-bit full JDK it declares, then configure REA to
|
Install a Ghidra 12.1.x release and the 64-bit full JDK it declares, then configure REA to
|
||||||
use them. Ghidra analysis supports Linux x64/arm64 and macOS x64/arm64; each installation requires
|
use them. Ghidra analysis supports Linux x64/arm64 and macOS x64/arm64; each installation requires
|
||||||
the matching native decompiler. Experimental Windows x64 P0 admits native
|
the matching native decompiler. Experimental Windows x64 P0 admits native
|
||||||
x86-64 PE applications on local NTFS using bundled Job Object ownership,
|
x86 and x86-64 PE applications and DLLs on local NTFS using bundled Job Object ownership,
|
||||||
protected runtime DACLs, and handle-based path admission. See [Windows Ghidra P0](windows-ghidra-p0.md) and
|
protected runtime DACLs, and handle-based path admission. See [Windows Ghidra P0](windows-ghidra-p0.md) and
|
||||||
[issue #527](https://github.com/morluto/rea/issues/527).
|
[issue #527](https://github.com/morluto/rea/issues/527).
|
||||||
On Linux and macOS, `annotate_native_function` atomically edits a function name
|
On Linux and macOS, `annotate_native_function` atomically edits a function name
|
||||||
|
|||||||
+61
-2
@@ -110,6 +110,23 @@ while comparisons and trace assertions use the normalized text. Retaining the
|
|||||||
original adds at most `limits.output_bytes` of text, because it comes from the
|
original adds at most `limits.output_bytes` of text, because it comes from the
|
||||||
same admitted PTY chunks rather than a second unbounded stream.
|
same admitted PTY chunks rather than a second unbounded stream.
|
||||||
|
|
||||||
|
Rendered frame `lines` hold the visible rows after normalization, with trailing
|
||||||
|
U+0020 spaces removed. Blank rows and internal spaces remain. `columns` records
|
||||||
|
terminal cell width; normalization and Unicode can make string lengths differ
|
||||||
|
from that width. `serialized_state` holds terminal serialization after the
|
||||||
|
selected normalization. Original admitted PTY chunks are available through
|
||||||
|
`raw_data ?? data`, as described above. Captures commit this line format in their
|
||||||
|
comparison contract, so a capture whose lines were padded to full width does not
|
||||||
|
compare as the same contract.
|
||||||
|
|
||||||
|
Custom `normalization.patterns` replace literal text with the caller's literal
|
||||||
|
replacement, including dollar sequences such as `$&` and `$$`. An empty pattern
|
||||||
|
also matches the first and last string positions. Captures with custom patterns
|
||||||
|
commit `pattern_normalization_version: "literal-replacements-v1"`. Legacy captures
|
||||||
|
remain readable. Cross-version comparison is rejected; capture both sides with
|
||||||
|
the same replacement semantics. Captures without custom patterns keep their
|
||||||
|
existing comparison contract.
|
||||||
|
|
||||||
Port normalization recognizes explicit `port`, `tcp_port`, `udp_port`, and
|
Port normalization recognizes explicit `port`, `tcp_port`, `udp_port`, and
|
||||||
`listen` fields, URL authorities, IP endpoints, and `localhost` endpoints, with
|
`listen` fields, URL authorities, IP endpoints, and `localhost` endpoints, with
|
||||||
ports from 0 through 65535. A port may end a sentence, as in `port: 8080.`;
|
ports from 0 through 65535. A port may end a sentence, as in `port: 8080.`;
|
||||||
@@ -159,8 +176,11 @@ Trace assertions use coverage for the sources they select, so an assertion
|
|||||||
about complete raw terminal output need not fail because rendered snapshots
|
about complete raw terminal output need not fail because rendered snapshots
|
||||||
were omitted. Captures missing `truncation_details` are rejected on import;
|
were omitted. Captures missing `truncation_details` are rejected on import;
|
||||||
preserve them as historical files and recapture for current comparisons.
|
preserve them as historical files and recapture for current comparisons.
|
||||||
These diagnostics use the existing scenario
|
`limits.file_bytes` is already the cumulative whole-file hashing budget,
|
||||||
budgets; they do not introduce a separate file-hashing budget.
|
independent of `limits.output_bytes`, which bounds retained terminal content.
|
||||||
|
Each filesystem checkpoint starts with a fresh `file_bytes` budget across all
|
||||||
|
selected roots; it is not a per-file allowance. File contents are read in
|
||||||
|
64 KiB chunks for hashing and are not embedded in capture Evidence.
|
||||||
|
|
||||||
When the host withholds an unrelated process’s ownership token, REA leaves that
|
When the host withholds an unrelated process’s ownership token, REA leaves that
|
||||||
process untouched and records its PID and reason in `cleanup.unverified_processes`
|
process untouched and records its PID and reason in `cleanup.unverified_processes`
|
||||||
@@ -176,6 +196,45 @@ as well. A newly started unreadable process can prevent verified cleanup even
|
|||||||
after the selected command exits. REA preserves this uncertainty and leaves
|
after the selected command exits. REA preserves this uncertainty and leaves
|
||||||
that process untouched.
|
that process untouched.
|
||||||
|
|
||||||
|
## Hash a report independently of terminal output
|
||||||
|
|
||||||
|
Select the producer's report directory and give `file_bytes` enough room for
|
||||||
|
the complete files you need, including other files in those roots. For example,
|
||||||
|
save this scenario and run `rea capture-process ./scenario.json --json > capture.json`:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"executable": "node",
|
||||||
|
"arguments": [
|
||||||
|
"-e",
|
||||||
|
"const fs=require('node:fs');fs.mkdirSync('reports',{recursive:true});fs.writeFileSync('reports/final.json',JSON.stringify({status:'complete',data:'X'.repeat(131072)}));console.log('x'.repeat(8192));"
|
||||||
|
],
|
||||||
|
"working_directory": ".",
|
||||||
|
"filesystem_observation_paths": ["./reports"],
|
||||||
|
"limits": { "output_bytes": 1024, "file_bytes": 262144 }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The report exceeds the terminal budget but fits the independent 256 KiB hash
|
||||||
|
budget. Its complete SHA-256 appears in `normalized_result.files_after` at
|
||||||
|
`root_0:final.json`; check `truncation_details.filesystem_after` for omissions
|
||||||
|
and actual bytes hashed. The verbose terminal output is truncated without
|
||||||
|
preventing the report digest. The same scenario object is the MCP
|
||||||
|
`capture_process_scenario` input. REA never returns a prefix hash as a full-file
|
||||||
|
digest: an insufficient remaining budget produces a null digest and an
|
||||||
|
explicit omission reason.
|
||||||
|
|
||||||
|
A digest binds the complete bytes observed at that checkpoint, not the
|
||||||
|
producer's notion of a finished report. A stable periodic or running report
|
||||||
|
can also have a complete digest. Choose the final file and verify the
|
||||||
|
producer's documented completion marker, exit outcome and settlement
|
||||||
|
observations before calling it final. `after_settlement` names a filesystem
|
||||||
|
checkpoint, not a guarantee that an application finished successfully or
|
||||||
|
that an unobserved writer will never change the file. REA checks opened-file
|
||||||
|
identity/state around hashing and reports observed changes or unavailable
|
||||||
|
files; cancellation stops the hash. If an external sealer produces a manifest,
|
||||||
|
keep its completion claims and digest separate from REA's checkpoint facts.
|
||||||
|
|
||||||
## Compare two captures
|
## Compare two captures
|
||||||
|
|
||||||
Compare saved capture Evidence with:
|
Compare saved capture Evidence with:
|
||||||
|
|||||||
@@ -17,7 +17,7 @@ additional permission flag or degraded mode is required.
|
|||||||
- Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+;
|
- Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+;
|
||||||
- an operator-installed Ghidra 12.1.x distribution (verified with 12.1.4);
|
- an operator-installed Ghidra 12.1.x distribution (verified with 12.1.4);
|
||||||
- a 64-bit full JDK inside that installation's declared Java range (JDK 21 or newer, with no maximum, for current 12.1 releases);
|
- a 64-bit full JDK inside that installation's declared Java range (JDK 21 or newer, with no maximum, for current 12.1 releases);
|
||||||
- an explicit native, non-managed, non-DLL x86 or x86-64 PE application; and
|
- an explicit native, non-managed x86 or x86-64 PE application or DLL; and
|
||||||
- the 25 read-only Ghidra inventory, memory, and function-analysis operations.
|
- the 25 read-only Ghidra inventory, memory, and function-analysis operations.
|
||||||
|
|
||||||
Loaded memory reads and file offsets preserve Ghidra's observed source mappings.
|
Loaded memory reads and file offsets preserve Ghidra's observed source mappings.
|
||||||
@@ -38,6 +38,13 @@ REA selects its packaged bridge scripts by explicit path. Same-named files or
|
|||||||
directories in the caller's working directory do not select a different
|
directories in the caller's working directory do not select a different
|
||||||
script or require users to clean that directory before analysis.
|
script or require users to clean that directory before analysis.
|
||||||
|
|
||||||
|
Native DLL analysis preserves the PE shared-library role in the analysis profile.
|
||||||
|
REA imports an immutable copy for static analysis; it does not load the library
|
||||||
|
into the analyst process or execute its entry routine. The source-owned DLL
|
||||||
|
fixtures have no entry routine or imports and expose two exported functions.
|
||||||
|
The same native snapshot, path-admission and cleanup requirements apply to EXEs
|
||||||
|
and DLLs.
|
||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
```powershell
|
```powershell
|
||||||
@@ -100,7 +107,7 @@ and artifact SHA-256 before loading the package-owned addon. Native failure
|
|||||||
reasons remain distinct from a valid installation and an unsupported target.
|
reasons remain distinct from a valid installation and an unsupported target.
|
||||||
|
|
||||||
PE header classification and SHA-256 come from the same open file. The Windows
|
PE header classification and SHA-256 come from the same open file. The Windows
|
||||||
provider admits native x86 and x86-64 PE applications and rejects unsupported roles,
|
provider admits native x86 and x86-64 PE applications and DLLs and rejects unsupported roles,
|
||||||
architectures, managed images, and malformed headers. The original selected
|
architectures, managed images, and malformed headers. The original selected
|
||||||
source coordinate is preserved for native admission.
|
source coordinate is preserved for native admission.
|
||||||
|
|
||||||
@@ -159,7 +166,11 @@ artifact and manifest.
|
|||||||
npm run verify:windows-native
|
npm run verify:windows-native
|
||||||
npm run verify:ghidra:windows
|
npm run verify:ghidra:windows
|
||||||
npm run verify:ghidra:windows -- --x86
|
npm run verify:ghidra:windows -- --x86
|
||||||
|
npm run verify:ghidra:windows -- --dll
|
||||||
|
npm run verify:ghidra:windows -- --x86 --dll
|
||||||
npm run verify:ghidra:windows:package -- --x86
|
npm run verify:ghidra:windows:package -- --x86
|
||||||
|
npm run verify:ghidra:windows:package -- --dll
|
||||||
|
npm run verify:ghidra:windows:package -- --x86 --dll
|
||||||
npm run verify:ghidra:windows:package -- C:\fixtures\installed-rea C:\fixtures\sample.exe
|
npm run verify:ghidra:windows:package -- C:\fixtures\installed-rea C:\fixtures\sample.exe
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
@@ -3,68 +3,103 @@ import { dirname, resolve } from "node:path";
|
|||||||
import { fileURLToPath } from "node:url";
|
import { fileURLToPath } from "node:url";
|
||||||
|
|
||||||
const root = resolve(dirname(fileURLToPath(import.meta.url)), "..");
|
const root = resolve(dirname(fileURLToPath(import.meta.url)), "..");
|
||||||
|
// Source-owned static fixtures: no compiler, target loading or execution.
|
||||||
for (const architecture of ["x86_64", "x86"]) {
|
for (const architecture of ["x86_64", "x86"]) {
|
||||||
const x86 = architecture === "x86";
|
for (const dll of [false, true]) {
|
||||||
const output = resolve(
|
const x86 = architecture === "x86";
|
||||||
root,
|
const output = resolve(
|
||||||
"build",
|
root,
|
||||||
"fixtures",
|
"build",
|
||||||
x86 ? "rea-ghidra-windows-x86.exe" : "rea-ghidra-windows.exe",
|
"fixtures",
|
||||||
);
|
`rea-ghidra-windows${x86 ? "-x86" : ""}.${dll ? "dll" : "exe"}`,
|
||||||
const bytes = Buffer.alloc(1024);
|
);
|
||||||
const peOffset = 0x80;
|
const bytes = Buffer.alloc(1024);
|
||||||
const optionalHeader = peOffset + 24;
|
const peOffset = 0x80;
|
||||||
const optionalHeaderSize = x86 ? 0xe0 : 0xf0;
|
const optionalHeader = peOffset + 24;
|
||||||
const sectionTable = optionalHeader + optionalHeaderSize;
|
const optionalHeaderSize = x86 ? 0xe0 : 0xf0;
|
||||||
|
const sectionTable = optionalHeader + optionalHeaderSize;
|
||||||
|
const directories = optionalHeader + (x86 ? 96 : 112);
|
||||||
|
|
||||||
bytes.write("MZ", 0, "ascii");
|
bytes.write("MZ", 0, "ascii");
|
||||||
bytes.writeUInt32LE(peOffset, 0x3c);
|
bytes.writeUInt32LE(peOffset, 0x3c);
|
||||||
bytes.write("PE\0\0", peOffset, "binary");
|
bytes.write("PE\0\0", peOffset, "binary");
|
||||||
bytes.writeUInt16LE(x86 ? 0x14c : 0x8664, peOffset + 4);
|
bytes.writeUInt16LE(x86 ? 0x14c : 0x8664, peOffset + 4);
|
||||||
bytes.writeUInt16LE(1, peOffset + 6);
|
bytes.writeUInt16LE(1, peOffset + 6);
|
||||||
bytes.writeUInt16LE(optionalHeaderSize, peOffset + 20);
|
bytes.writeUInt16LE(optionalHeaderSize, peOffset + 20);
|
||||||
bytes.writeUInt16LE(x86 ? 0x102 : 0x22, peOffset + 22);
|
bytes.writeUInt16LE(
|
||||||
|
(x86 ? 0x102 : 0x22) | (dll ? 0x2000 : 0),
|
||||||
|
peOffset + 22,
|
||||||
|
);
|
||||||
|
|
||||||
bytes.writeUInt16LE(x86 ? 0x10b : 0x20b, optionalHeader);
|
bytes.writeUInt16LE(x86 ? 0x10b : 0x20b, optionalHeader);
|
||||||
bytes.writeUInt32LE(0x200, optionalHeader + 4);
|
bytes.writeUInt32LE(0x200, optionalHeader + 4);
|
||||||
bytes.writeUInt32LE(0x1000, optionalHeader + 16);
|
bytes.writeUInt32LE(dll ? 0 : 0x1000, optionalHeader + 16);
|
||||||
bytes.writeUInt32LE(0x1000, optionalHeader + 20);
|
bytes.writeUInt32LE(0x1000, optionalHeader + 20);
|
||||||
if (x86) bytes.writeUInt32LE(0x400000, optionalHeader + 28);
|
if (x86) bytes.writeUInt32LE(0x400000, optionalHeader + 28);
|
||||||
else bytes.writeBigUInt64LE(0x1_4000_0000n, optionalHeader + 24);
|
else bytes.writeBigUInt64LE(0x1_4000_0000n, optionalHeader + 24);
|
||||||
bytes.writeUInt32LE(0x1000, optionalHeader + 32);
|
bytes.writeUInt32LE(0x1000, optionalHeader + 32);
|
||||||
bytes.writeUInt32LE(0x200, optionalHeader + 36);
|
bytes.writeUInt32LE(0x200, optionalHeader + 36);
|
||||||
bytes.writeUInt16LE(6, optionalHeader + 40);
|
bytes.writeUInt16LE(6, optionalHeader + 40);
|
||||||
bytes.writeUInt16LE(6, optionalHeader + 48);
|
bytes.writeUInt16LE(6, optionalHeader + 48);
|
||||||
bytes.writeUInt32LE(0x2000, optionalHeader + 56);
|
bytes.writeUInt32LE(0x2000, optionalHeader + 56);
|
||||||
bytes.writeUInt32LE(0x200, optionalHeader + 60);
|
bytes.writeUInt32LE(0x200, optionalHeader + 60);
|
||||||
bytes.writeUInt16LE(3, optionalHeader + 68);
|
bytes.writeUInt16LE(3, optionalHeader + 68);
|
||||||
bytes.writeUInt16LE(x86 ? 0x8140 : 0x8160, optionalHeader + 70);
|
bytes.writeUInt16LE(x86 ? 0x8140 : 0x8160, optionalHeader + 70);
|
||||||
if (x86) {
|
if (x86) {
|
||||||
bytes.writeUInt32LE(0x10_0000, optionalHeader + 72);
|
bytes.writeUInt32LE(0x10_0000, optionalHeader + 72);
|
||||||
bytes.writeUInt32LE(0x1000, optionalHeader + 76);
|
bytes.writeUInt32LE(0x1000, optionalHeader + 76);
|
||||||
bytes.writeUInt32LE(0x10_0000, optionalHeader + 80);
|
bytes.writeUInt32LE(0x10_0000, optionalHeader + 80);
|
||||||
bytes.writeUInt32LE(0x1000, optionalHeader + 84);
|
bytes.writeUInt32LE(0x1000, optionalHeader + 84);
|
||||||
bytes.writeUInt32LE(16, optionalHeader + 92);
|
bytes.writeUInt32LE(16, optionalHeader + 92);
|
||||||
} else {
|
} else {
|
||||||
bytes.writeBigUInt64LE(0x10_0000n, optionalHeader + 72);
|
bytes.writeBigUInt64LE(0x10_0000n, optionalHeader + 72);
|
||||||
bytes.writeBigUInt64LE(0x1000n, optionalHeader + 80);
|
bytes.writeBigUInt64LE(0x1000n, optionalHeader + 80);
|
||||||
bytes.writeBigUInt64LE(0x10_0000n, optionalHeader + 88);
|
bytes.writeBigUInt64LE(0x10_0000n, optionalHeader + 88);
|
||||||
bytes.writeBigUInt64LE(0x1000n, optionalHeader + 96);
|
bytes.writeBigUInt64LE(0x1000n, optionalHeader + 96);
|
||||||
bytes.writeUInt32LE(16, optionalHeader + 108);
|
bytes.writeUInt32LE(16, optionalHeader + 108);
|
||||||
|
}
|
||||||
|
|
||||||
|
bytes.write(".text", sectionTable, "ascii");
|
||||||
|
bytes.writeUInt32LE(dll ? 0x180 : 0x16, sectionTable + 8);
|
||||||
|
bytes.writeUInt32LE(0x1000, sectionTable + 12);
|
||||||
|
bytes.writeUInt32LE(0x200, sectionTable + 16);
|
||||||
|
bytes.writeUInt32LE(0x200, sectionTable + 20);
|
||||||
|
bytes.writeUInt32LE(0x6000_0020, sectionTable + 36);
|
||||||
|
|
||||||
|
// EXE entry returns zero. DLL caller adds one to the callee's 42.
|
||||||
|
bytes.set(
|
||||||
|
dll
|
||||||
|
? [0xe8, 0x0b, 0, 0, 0, 0x83, 0xc0, 1, 0xc3]
|
||||||
|
: [0xe8, 0x0b, 0, 0, 0, 0x31, 0xc0, 0xc3],
|
||||||
|
0x200,
|
||||||
|
);
|
||||||
|
bytes.set([0xb8, 0x2a, 0, 0, 0, 0xc3], 0x210);
|
||||||
|
|
||||||
|
if (dll) {
|
||||||
|
// Export both functions; an entry-point-free library is discovered by exports.
|
||||||
|
bytes.writeUInt32LE(0x1040, directories);
|
||||||
|
bytes.writeUInt32LE(0x130, directories + 4);
|
||||||
|
const exports = 0x240;
|
||||||
|
bytes.writeUInt32LE(0x1100, exports + 12);
|
||||||
|
bytes.writeUInt32LE(1, exports + 16);
|
||||||
|
bytes.writeUInt32LE(2, exports + 20);
|
||||||
|
bytes.writeUInt32LE(2, exports + 24);
|
||||||
|
bytes.writeUInt32LE(0x1080, exports + 28);
|
||||||
|
bytes.writeUInt32LE(0x1088, exports + 32);
|
||||||
|
bytes.writeUInt32LE(0x1090, exports + 36);
|
||||||
|
bytes.writeUInt32LE(0x1000, 0x280);
|
||||||
|
bytes.writeUInt32LE(0x1010, 0x284);
|
||||||
|
bytes.writeUInt32LE(0x1120, 0x288);
|
||||||
|
bytes.writeUInt32LE(0x1140, 0x28c);
|
||||||
|
bytes.writeUInt16LE(0, 0x290);
|
||||||
|
bytes.writeUInt16LE(1, 0x292);
|
||||||
|
bytes.write("rea-fixture.dll\0", 0x300, "binary");
|
||||||
|
bytes.write("fixture_caller\0", 0x320, "binary");
|
||||||
|
bytes.write("fixture_return42\0", 0x340, "binary");
|
||||||
|
}
|
||||||
|
|
||||||
|
await mkdir(dirname(output), { recursive: true });
|
||||||
|
await writeFile(output, bytes);
|
||||||
|
if (!x86 && !dll) process.stdout.write(`${output}\n`);
|
||||||
}
|
}
|
||||||
|
|
||||||
bytes.write(".text", sectionTable, "ascii");
|
|
||||||
bytes.writeUInt32LE(0x16, sectionTable + 8);
|
|
||||||
bytes.writeUInt32LE(0x1000, sectionTable + 12);
|
|
||||||
bytes.writeUInt32LE(0x200, sectionTable + 16);
|
|
||||||
bytes.writeUInt32LE(0x200, sectionTable + 20);
|
|
||||||
bytes.writeUInt32LE(0x6000_0020, sectionTable + 36);
|
|
||||||
|
|
||||||
// entry calls the second function, returns zero; second returns 42.
|
|
||||||
bytes.set([0xe8, 0x0b, 0, 0, 0, 0x31, 0xc0, 0xc3], 0x200);
|
|
||||||
bytes.set([0xb8, 0x2a, 0, 0, 0, 0xc3], 0x210);
|
|
||||||
|
|
||||||
await mkdir(dirname(output), { recursive: true });
|
|
||||||
await writeFile(output, bytes);
|
|
||||||
if (!x86) process.stdout.write(`${output}\n`);
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -17,7 +17,7 @@ const lock = JSON.parse(
|
|||||||
await readFile(join(root, "package-lock.json"), "utf8"),
|
await readFile(join(root, "package-lock.json"), "utf8"),
|
||||||
);
|
);
|
||||||
const skill = await readFile(
|
const skill = await readFile(
|
||||||
join(root, "skill-src/reverse-engineer-anything/SKILL.md"),
|
join(root, ".agents/skills/reverse-engineer-anything/SKILL.md"),
|
||||||
"utf8",
|
"utf8",
|
||||||
);
|
);
|
||||||
const skillVersion = /^\s{2}version:\s*"([^"]+)"\s*$/mu.exec(skill)?.[1];
|
const skillVersion = /^\s{2}version:\s*"([^"]+)"\s*$/mu.exec(skill)?.[1];
|
||||||
|
|||||||
@@ -10,7 +10,7 @@ for (const argument of arguments_)
|
|||||||
throw new Error(`Unknown skill metadata option: ${argument}`);
|
throw new Error(`Unknown skill metadata option: ${argument}`);
|
||||||
|
|
||||||
const root = join(dirname(fileURLToPath(import.meta.url)), "..");
|
const root = join(dirname(fileURLToPath(import.meta.url)), "..");
|
||||||
const sourceRoot = join(root, "skill-src/reverse-engineer-anything");
|
const sourceRoot = join(root, ".agents/skills/reverse-engineer-anything");
|
||||||
const outputRoot = join(root, "skills/reverse-engineer-anything");
|
const outputRoot = join(root, "skills/reverse-engineer-anything");
|
||||||
const check = arguments_.has("--check");
|
const check = arguments_.has("--check");
|
||||||
const current = (
|
const current = (
|
||||||
@@ -25,7 +25,7 @@ const { PRODUCT_IDENTITY } = await import(
|
|||||||
);
|
);
|
||||||
if (/^\s{2}(?:tool_count|catalog_digest):/mu.test(current))
|
if (/^\s{2}(?:tool_count|catalog_digest):/mu.test(current))
|
||||||
throw new Error(
|
throw new Error(
|
||||||
"Catalog metadata belongs in the generated skill, not skill-src",
|
"Catalog metadata belongs in the generated skill, not the authored source",
|
||||||
);
|
);
|
||||||
const versionLine = /^ {2}version: "[^"\r\n]+"$/mu;
|
const versionLine = /^ {2}version: "[^"\r\n]+"$/mu;
|
||||||
if (!versionLine.test(current))
|
if (!versionLine.test(current))
|
||||||
|
|||||||
@@ -1283,7 +1283,7 @@ async function inspect(
|
|||||||
evidence.provider.version,
|
evidence.provider.version,
|
||||||
"pwntools@4.15.0;pyelftools@0.33;unicorn@2.1.2",
|
"pwntools@4.15.0;pyelftools@0.33;unicorn@2.1.2",
|
||||||
);
|
);
|
||||||
assert.deepEqual(evidence.normalized_result, evidence.raw_result);
|
assert.equal(evidence.raw_result, null);
|
||||||
assert.equal(evidence.normalized_result.diagnostics.truncated, false);
|
assert.equal(evidence.normalized_result.diagnostics.truncated, false);
|
||||||
return evidence.normalized_result;
|
return evidence.normalized_result;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -85,6 +85,7 @@ export async function verifyPackageEnvironment({
|
|||||||
PI_CODING_AGENT_DIR: undefined,
|
PI_CODING_AGENT_DIR: undefined,
|
||||||
PI_CONFIG_DIR: undefined,
|
PI_CONFIG_DIR: undefined,
|
||||||
PI_PROFILE: undefined,
|
PI_PROFILE: undefined,
|
||||||
|
QWEN_HOME: undefined,
|
||||||
OPENCODE_CONFIG: undefined,
|
OPENCODE_CONFIG: undefined,
|
||||||
XDG_CONFIG_HOME: join(home, ".config"),
|
XDG_CONFIG_HOME: join(home, ".config"),
|
||||||
PATH: `${fakeBin}:${process.env.PATH ?? ""}`,
|
PATH: `${fakeBin}:${process.env.PATH ?? ""}`,
|
||||||
|
|||||||
@@ -38,7 +38,7 @@ export async function verifyPackagePack({ root, workspace }) {
|
|||||||
throw new Error("package did not expose both rea command entry points");
|
throw new Error("package did not expose both rea command entry points");
|
||||||
if (packedManifest.mcpName !== "io.github.morluto/rea")
|
if (packedManifest.mcpName !== "io.github.morluto/rea")
|
||||||
throw new Error("package did not retain its MCP Registry ownership marker");
|
throw new Error("package did not retain its MCP Registry ownership marker");
|
||||||
if (packedFiles.some((path) => path.startsWith("package/skill-src/")))
|
if (packedFiles.some((path) => path.startsWith("package/.agents/")))
|
||||||
throw new Error(
|
throw new Error(
|
||||||
"package included authored skill sources instead of only the generated bundle",
|
"package included authored skill sources instead of only the generated bundle",
|
||||||
);
|
);
|
||||||
|
|||||||
@@ -17,6 +17,7 @@ import { verifyPackageMcp } from "./verify-package-mcp.mjs";
|
|||||||
import { verifyPackagePack } from "./verify-package-pack.mjs";
|
import { verifyPackagePack } from "./verify-package-pack.mjs";
|
||||||
import { verifyPackagePlatform } from "./verify-package-platform.mjs";
|
import { verifyPackagePlatform } from "./verify-package-platform.mjs";
|
||||||
import { verifyPackageSetup } from "./verify-package-setup.mjs";
|
import { verifyPackageSetup } from "./verify-package-setup.mjs";
|
||||||
|
import { verifyRepositorySkill } from "./verify-repository-skill.mjs";
|
||||||
import { verifyManaged } from "./verify-package-managed.mjs";
|
import { verifyManaged } from "./verify-package-managed.mjs";
|
||||||
import { verifyUnknownProvider } from "./verify-package-unknown-provider.mjs";
|
import { verifyUnknownProvider } from "./verify-package-unknown-provider.mjs";
|
||||||
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
|
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
|
||||||
@@ -34,9 +35,11 @@ let tarball;
|
|||||||
let mcpStartup;
|
let mcpStartup;
|
||||||
let mcpModuleLoading;
|
let mcpModuleLoading;
|
||||||
let update;
|
let update;
|
||||||
|
let repositorySkill;
|
||||||
let report;
|
let report;
|
||||||
|
|
||||||
try {
|
try {
|
||||||
|
repositorySkill = await verifyRepositorySkill({ root, workspace });
|
||||||
({ tarball } = await verifyPackagePack({ root, workspace }));
|
({ tarball } = await verifyPackagePack({ root, workspace }));
|
||||||
const environmentData = await verifyPackageEnvironment({
|
const environmentData = await verifyPackageEnvironment({
|
||||||
root,
|
root,
|
||||||
@@ -156,6 +159,7 @@ try {
|
|||||||
configSymlinkLifecycle: supportedSetupHost,
|
configSymlinkLifecycle: supportedSetupHost,
|
||||||
skill: supportedSetupHost,
|
skill: supportedSetupHost,
|
||||||
skillReferences: supportedSetupHost,
|
skillReferences: supportedSetupHost,
|
||||||
|
repositorySkill,
|
||||||
mcpTools: TOOL_CONTRACTS.length,
|
mcpTools: TOOL_CONTRACTS.length,
|
||||||
mcpPrompts: prompts.names.length,
|
mcpPrompts: prompts.names.length,
|
||||||
promptCompletion: true,
|
promptCompletion: true,
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ import {
|
|||||||
import { tmpdir } from "node:os";
|
import { tmpdir } from "node:os";
|
||||||
import { dirname, join, resolve } from "node:path";
|
import { dirname, join, resolve } from "node:path";
|
||||||
import { promisify } from "node:util";
|
import { promisify } from "node:util";
|
||||||
import { pathToFileURL } from "node:url";
|
import { fileURLToPath, pathToFileURL } from "node:url";
|
||||||
|
|
||||||
import { Client } from "@modelcontextprotocol/client";
|
import { Client } from "@modelcontextprotocol/client";
|
||||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||||
@@ -24,10 +24,50 @@ if (process.platform !== "win32" || process.arch !== "x64")
|
|||||||
throw new Error(
|
throw new Error(
|
||||||
"Packaged Windows Ghidra acceptance requires a Windows x64 host.",
|
"Packaged Windows Ghidra acceptance requires a Windows x64 host.",
|
||||||
);
|
);
|
||||||
const workspace = await mkdtemp(join(tmpdir(), "rea-packaged-ghidra-"));
|
|
||||||
const exec = promisify(execFile);
|
const exec = promisify(execFile);
|
||||||
const x86Fixture = process.argv[2] === "--x86";
|
const arguments_ = process.argv.slice(2);
|
||||||
const packageRootArgument = x86Fixture ? undefined : process.argv[2];
|
const internalWorkspace = arguments_[0] === "--internal-workspace";
|
||||||
|
const fixtureArguments = internalWorkspace ? arguments_.slice(2) : arguments_;
|
||||||
|
const x86Fixture = fixtureArguments.includes("--x86");
|
||||||
|
const dllFixture = fixtureArguments.includes("--dll");
|
||||||
|
const [packageRootArgument, targetArgument, reportArgument] =
|
||||||
|
fixtureArguments.filter(
|
||||||
|
(argument) => argument !== "--x86" && argument !== "--dll",
|
||||||
|
);
|
||||||
|
// Windows locks the loaded package-owned addon until the verifier exits.
|
||||||
|
// Keep installation cleanup in a parent that never loads that addon.
|
||||||
|
if (!internalWorkspace) {
|
||||||
|
const ownedWorkspace = await mkdtemp(join(tmpdir(), "rea-packaged-ghidra-"));
|
||||||
|
let result;
|
||||||
|
try {
|
||||||
|
result = await exec(
|
||||||
|
process.execPath,
|
||||||
|
[
|
||||||
|
fileURLToPath(import.meta.url),
|
||||||
|
"--internal-workspace",
|
||||||
|
ownedWorkspace,
|
||||||
|
...arguments_,
|
||||||
|
],
|
||||||
|
{ timeout: 900_000, maxBuffer: 8 * 1024 * 1024 },
|
||||||
|
);
|
||||||
|
} catch (cause) {
|
||||||
|
if (typeof cause.stdout === "string") process.stdout.write(cause.stdout);
|
||||||
|
if (typeof cause.stderr === "string") process.stderr.write(cause.stderr);
|
||||||
|
throw cause;
|
||||||
|
} finally {
|
||||||
|
await rm(ownedWorkspace, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
const report = JSON.parse(result.stdout);
|
||||||
|
report.workspaceCleanup = true;
|
||||||
|
report.ok = true;
|
||||||
|
if (reportArgument !== undefined)
|
||||||
|
await writeFile(reportArgument, `${JSON.stringify(report)}\n`);
|
||||||
|
process.stdout.write(`${JSON.stringify(report)}\n`);
|
||||||
|
process.stderr.write(result.stderr);
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
assert.ok(arguments_[1] !== undefined, "Missing verifier workspace.");
|
||||||
|
const workspace = resolve(arguments_[1]);
|
||||||
let packageRoot = resolve(packageRootArgument ?? ".");
|
let packageRoot = resolve(packageRootArgument ?? ".");
|
||||||
// The default lane verifies the npm artifact in an isolated prefix. Explicit
|
// The default lane verifies the npm artifact in an isolated prefix. Explicit
|
||||||
// package roots support an already-installed artifact without a second install.
|
// package roots support an already-installed artifact without a second install.
|
||||||
@@ -75,17 +115,17 @@ if (packageRootArgument === undefined) {
|
|||||||
);
|
);
|
||||||
packageRoot = join(prefix, "node_modules", "rea-agents");
|
packageRoot = join(prefix, "node_modules", "rea-agents");
|
||||||
} catch (cause) {
|
} catch (cause) {
|
||||||
await rm(workspace, { recursive: true, force: true });
|
throw new Error("Packaged Windows verifier installation failed.", {
|
||||||
throw cause;
|
cause,
|
||||||
|
});
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
const target = resolve(
|
const target = resolve(
|
||||||
process.argv[3] ??
|
targetArgument ??
|
||||||
(x86Fixture
|
`build/fixtures/rea-ghidra-windows${x86Fixture ? "-x86" : ""}.${dllFixture ? "dll" : "exe"}`,
|
||||||
? "build/fixtures/rea-ghidra-windows-x86.exe"
|
|
||||||
: "build/fixtures/rea-ghidra-windows.exe"),
|
|
||||||
);
|
);
|
||||||
let TOOL_CONTRACTS, native, token, sha256;
|
let TOOL_CONTRACTS, native, token, sha256;
|
||||||
|
let parsedTarget;
|
||||||
try {
|
try {
|
||||||
({ TOOL_CONTRACTS } = await import(
|
({ TOOL_CONTRACTS } = await import(
|
||||||
pathToFileURL(join(packageRoot, "dist/contracts/toolContracts.js"))
|
pathToFileURL(join(packageRoot, "dist/contracts/toolContracts.js"))
|
||||||
@@ -107,9 +147,25 @@ try {
|
|||||||
);
|
);
|
||||||
const digest = (bytes) => createHash("sha256").update(bytes).digest("hex");
|
const digest = (bytes) => createHash("sha256").update(bytes).digest("hex");
|
||||||
sha256 = digest(await readFile(target));
|
sha256 = digest(await readFile(target));
|
||||||
|
const { parseBinaryTarget } = await import(
|
||||||
|
pathToFileURL(join(packageRoot, "dist/application/BinaryTargetResolver.js"))
|
||||||
|
);
|
||||||
|
parsedTarget = await parseBinaryTarget(target);
|
||||||
|
assert.equal(parsedTarget.ok, true);
|
||||||
|
assert.equal(parsedTarget.value.format, "pe");
|
||||||
|
assert.equal(parsedTarget.value.managed, false);
|
||||||
|
if (targetArgument === undefined) {
|
||||||
|
assert.equal(
|
||||||
|
parsedTarget.value.architecture,
|
||||||
|
x86Fixture ? "x86" : "x86_64",
|
||||||
|
);
|
||||||
|
assert.equal(
|
||||||
|
parsedTarget.value.executableRole,
|
||||||
|
dllFixture ? "shared-library" : "application",
|
||||||
|
);
|
||||||
|
}
|
||||||
} catch (cause) {
|
} catch (cause) {
|
||||||
await rm(workspace, { recursive: true, force: true });
|
throw new Error("Packaged Windows verifier prerequisites failed.", { cause });
|
||||||
throw cause;
|
|
||||||
}
|
}
|
||||||
const digest = (bytes) => createHash("sha256").update(bytes).digest("hex");
|
const digest = (bytes) => createHash("sha256").update(bytes).digest("hex");
|
||||||
const runtimeParent = join(workspace, "runtime with spaces");
|
const runtimeParent = join(workspace, "runtime with spaces");
|
||||||
@@ -145,6 +201,8 @@ const report = {
|
|||||||
architecture: "x64",
|
architecture: "x64",
|
||||||
nonAdmin: true,
|
nonAdmin: true,
|
||||||
targetSha256: sha256,
|
targetSha256: sha256,
|
||||||
|
targetArchitecture: parsedTarget.value.architecture,
|
||||||
|
targetRole: parsedTarget.value.executableRole,
|
||||||
cli: {},
|
cli: {},
|
||||||
mcpOperations: [],
|
mcpOperations: [],
|
||||||
nativeControls: "available",
|
nativeControls: "available",
|
||||||
@@ -208,8 +266,8 @@ try {
|
|||||||
});
|
});
|
||||||
const call = async (name, arguments_) => {
|
const call = async (name, arguments_) => {
|
||||||
report.lastOperation = name;
|
report.lastOperation = name;
|
||||||
if (process.argv[4] !== undefined)
|
if (reportArgument !== undefined)
|
||||||
await writeFile(process.argv[4], `${JSON.stringify(report)}\n`);
|
await writeFile(reportArgument, `${JSON.stringify(report)}\n`);
|
||||||
const contract = TOOL_CONTRACTS.find((value) => value.name === name);
|
const contract = TOOL_CONTRACTS.find((value) => value.name === name);
|
||||||
assert.ok(contract !== undefined, `Missing canonical contract: ${name}`);
|
assert.ok(contract !== undefined, `Missing canonical contract: ${name}`);
|
||||||
const argumentsParsed = contract.inputSchema.parse(arguments_);
|
const argumentsParsed = contract.inputSchema.parse(arguments_);
|
||||||
@@ -230,6 +288,25 @@ try {
|
|||||||
await call("list_names", {});
|
await call("list_names", {});
|
||||||
const procedures = await call("list_procedures", {});
|
const procedures = await call("list_procedures", {});
|
||||||
assert.ok(procedures.length > 0);
|
assert.ok(procedures.length > 0);
|
||||||
|
if (targetArgument === undefined) {
|
||||||
|
const calleeAddress = x86Fixture ? "0x401010" : "0x140001010";
|
||||||
|
const calleeCode = await call("procedure_pseudo_code", {
|
||||||
|
procedure: calleeAddress,
|
||||||
|
});
|
||||||
|
assert.match(calleeCode, /\b(?:42|0x2a)\b/iu);
|
||||||
|
if (dllFixture) {
|
||||||
|
const callerAddress = x86Fixture ? "0x401000" : "0x140001000";
|
||||||
|
assert.ok(
|
||||||
|
(
|
||||||
|
await call("procedure_callees", { procedure: callerAddress })
|
||||||
|
).includes(calleeAddress),
|
||||||
|
);
|
||||||
|
assert.match(
|
||||||
|
await call("procedure_pseudo_code", { procedure: callerAddress }),
|
||||||
|
/\+\s*1\b/u,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
assert.ok((await call("list_segments", {})).length > 0);
|
assert.ok((await call("list_segments", {})).length > 0);
|
||||||
await call("list_strings", {});
|
await call("list_strings", {});
|
||||||
const procedure =
|
const procedure =
|
||||||
@@ -296,6 +373,12 @@ try {
|
|||||||
"Packaged verification omitted an admitted operation.",
|
"Packaged verification omitted an admitted operation.",
|
||||||
);
|
);
|
||||||
const session = await call("binary_session", {});
|
const session = await call("binary_session", {});
|
||||||
|
assert.equal(
|
||||||
|
session.analysis_provider_binding?.analysis_profile?.parameters
|
||||||
|
?.executable_role,
|
||||||
|
parsedTarget.value.executableRole,
|
||||||
|
"MCP profile must preserve the selected PE role.",
|
||||||
|
);
|
||||||
assert.equal(
|
assert.equal(
|
||||||
session.analysis_provider_binding?.selection_source,
|
session.analysis_provider_binding?.selection_source,
|
||||||
"auto-single-candidate",
|
"auto-single-candidate",
|
||||||
@@ -326,9 +409,9 @@ try {
|
|||||||
assert.deepEqual(await readdir(join(callerDirectory, name)), []);
|
assert.deepEqual(await readdir(join(callerDirectory, name)), []);
|
||||||
report.callerScriptCollisionsPreserved = true;
|
report.callerScriptCollisionsPreserved = true;
|
||||||
report.runtimeCleanup = true;
|
report.runtimeCleanup = true;
|
||||||
report.ok = true;
|
// Only the parent can attest success after removing the installation.
|
||||||
if (process.argv[4] !== undefined)
|
if (reportArgument !== undefined)
|
||||||
await writeFile(process.argv[4], `${JSON.stringify(report)}\n`);
|
await writeFile(reportArgument, `${JSON.stringify(report)}\n`);
|
||||||
process.stdout.write(`${JSON.stringify(report)}\n`);
|
process.stdout.write(`${JSON.stringify(report)}\n`);
|
||||||
} catch (cause) {
|
} catch (cause) {
|
||||||
report.failure = {
|
report.failure = {
|
||||||
@@ -338,9 +421,7 @@ try {
|
|||||||
.replaceAll(target, "<target>")
|
.replaceAll(target, "<target>")
|
||||||
.replaceAll(workspace, "<workspace>"),
|
.replaceAll(workspace, "<workspace>"),
|
||||||
};
|
};
|
||||||
if (process.argv[4] !== undefined)
|
if (reportArgument !== undefined)
|
||||||
await writeFile(process.argv[4], `${JSON.stringify(report)}\n`);
|
await writeFile(reportArgument, `${JSON.stringify(report)}\n`);
|
||||||
throw cause;
|
throw cause;
|
||||||
} finally {
|
|
||||||
await rm(workspace, { recursive: true, force: true });
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -62,21 +62,21 @@ if (
|
|||||||
);
|
);
|
||||||
|
|
||||||
const root = resolve(fileURLToPath(new URL("..", import.meta.url)));
|
const root = resolve(fileURLToPath(new URL("..", import.meta.url)));
|
||||||
const architecture = process.argv[2] === "--x86" ? "x86" : "x86_64";
|
const architecture = process.argv.includes("--x86") ? "x86" : "x86_64";
|
||||||
|
const dllFixture = process.argv.includes("--dll");
|
||||||
|
const role = dllFixture ? "shared-library" : "application";
|
||||||
const targetPath = resolve(
|
const targetPath = resolve(
|
||||||
root,
|
root,
|
||||||
"build",
|
"build",
|
||||||
"fixtures",
|
"fixtures",
|
||||||
architecture === "x86"
|
`rea-ghidra-windows${architecture === "x86" ? "-x86" : ""}.${dllFixture ? "dll" : "exe"}`,
|
||||||
? "rea-ghidra-windows-x86.exe"
|
|
||||||
: "rea-ghidra-windows.exe",
|
|
||||||
);
|
);
|
||||||
const target = await parseBinaryTarget(targetPath);
|
const target = await parseBinaryTarget(targetPath);
|
||||||
if (
|
if (
|
||||||
!target.ok ||
|
!target.ok ||
|
||||||
target.value.format !== "pe" ||
|
target.value.format !== "pe" ||
|
||||||
target.value.architecture !== architecture ||
|
target.value.architecture !== architecture ||
|
||||||
target.value.executableRole !== "application" ||
|
target.value.executableRole !== role ||
|
||||||
target.value.managed !== false
|
target.value.managed !== false
|
||||||
)
|
)
|
||||||
throw new Error(
|
throw new Error(
|
||||||
@@ -89,6 +89,8 @@ const profile = await resolveGhidraAnalysisProfile(
|
|||||||
);
|
);
|
||||||
if (!profile.ok || profile.value.profile === null)
|
if (!profile.ok || profile.value.profile === null)
|
||||||
throw new Error("Windows Ghidra profile could not be committed");
|
throw new Error("Windows Ghidra profile could not be committed");
|
||||||
|
if (profile.value.profile.parameters.executable_role !== role)
|
||||||
|
throw new Error("Windows Ghidra profile lost the observed PE role");
|
||||||
|
|
||||||
const client = new GhidraClient({
|
const client = new GhidraClient({
|
||||||
platform: installation.platform,
|
platform: installation.platform,
|
||||||
@@ -162,6 +164,19 @@ try {
|
|||||||
throw new Error(
|
throw new Error(
|
||||||
"Windows fixture decompilation lost its known return value",
|
"Windows fixture decompilation lost its known return value",
|
||||||
);
|
);
|
||||||
|
if (dllFixture) {
|
||||||
|
const callerAddress = architecture === "x86" ? "0x401000" : "0x140001000";
|
||||||
|
const callees = await functionOperation("procedure_callees", {
|
||||||
|
procedure: callerAddress,
|
||||||
|
});
|
||||||
|
if (!callees.includes(callee.address))
|
||||||
|
throw new Error("DLL export caller lost its controlled call reference");
|
||||||
|
const callerCode = await functionOperation("procedure_pseudo_code", {
|
||||||
|
procedure: callerAddress,
|
||||||
|
});
|
||||||
|
if (typeof callerCode !== "string" || !/\+\s*1\b/u.test(callerCode))
|
||||||
|
throw new Error("DLL export caller lost its addition of one");
|
||||||
|
}
|
||||||
|
|
||||||
await inventory("inspect_native_load_image", {});
|
await inventory("inspect_native_load_image", {});
|
||||||
const memory = await inventory("read_bytes", {
|
const memory = await inventory("read_bytes", {
|
||||||
@@ -254,7 +269,7 @@ try {
|
|||||||
limitations: [
|
limitations: [
|
||||||
"approved-non-sensitive-fixtures-only",
|
"approved-non-sensitive-fixtures-only",
|
||||||
"windows-x64-local-ntfs-only",
|
"windows-x64-local-ntfs-only",
|
||||||
"native-x86-and-x86-64-pe-applications-only",
|
"native-x86-and-x86-64-pe-applications-and-dlls-only",
|
||||||
"no-gui-or-mutation-authority",
|
"no-gui-or-mutation-authority",
|
||||||
],
|
],
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -0,0 +1,97 @@
|
|||||||
|
import { cp, mkdir, readFile, readdir } from "node:fs/promises";
|
||||||
|
import { dirname, join, relative } from "node:path";
|
||||||
|
|
||||||
|
import { exec, pathExists } from "./lib/verify-package-core.mjs";
|
||||||
|
|
||||||
|
/** Exercise Skills' default root discovery without ignored build outputs. */
|
||||||
|
export async function verifyRepositorySkill({ root, workspace }) {
|
||||||
|
const snapshot = join(workspace, "repository");
|
||||||
|
const consumer = join(workspace, "skill-consumer");
|
||||||
|
const home = join(workspace, "skill-consumer-home");
|
||||||
|
const tracked = (
|
||||||
|
await exec(
|
||||||
|
"git",
|
||||||
|
["ls-files", "--cached", "--format=%(objectmode) %(path)", "-z"],
|
||||||
|
{ cwd: root },
|
||||||
|
)
|
||||||
|
).stdout
|
||||||
|
.split("\0")
|
||||||
|
.filter(Boolean);
|
||||||
|
for (const entry of tracked) {
|
||||||
|
const mode = entry.slice(0, 6);
|
||||||
|
const path = entry.slice(7);
|
||||||
|
const destination = join(snapshot, path);
|
||||||
|
// A GitHub clone without submodule initialization has only these directories.
|
||||||
|
if (mode === "160000") {
|
||||||
|
await mkdir(destination, { recursive: true });
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
await mkdir(dirname(destination), { recursive: true });
|
||||||
|
await cp(join(root, path), destination, { verbatimSymlinks: true });
|
||||||
|
}
|
||||||
|
if (await pathExists(join(snapshot, "skills")))
|
||||||
|
throw new Error("repository skill check must run without generated skills");
|
||||||
|
await mkdir(consumer, { recursive: true });
|
||||||
|
await mkdir(home, { recursive: true });
|
||||||
|
const skillsVersion = "1.7.2";
|
||||||
|
await exec(
|
||||||
|
"npm",
|
||||||
|
[
|
||||||
|
"exec",
|
||||||
|
"--yes",
|
||||||
|
"--package",
|
||||||
|
`skills@${skillsVersion}`,
|
||||||
|
"--",
|
||||||
|
"skills",
|
||||||
|
"add",
|
||||||
|
snapshot,
|
||||||
|
"--skill",
|
||||||
|
"reverse-engineer-anything",
|
||||||
|
"--agent",
|
||||||
|
"codex",
|
||||||
|
"--yes",
|
||||||
|
],
|
||||||
|
{
|
||||||
|
cwd: consumer,
|
||||||
|
env: {
|
||||||
|
...process.env,
|
||||||
|
HOME: home,
|
||||||
|
USERPROFILE: home,
|
||||||
|
XDG_CONFIG_HOME: join(home, ".config"),
|
||||||
|
CODEX_HOME: join(home, ".codex"),
|
||||||
|
DO_NOT_TRACK: "1",
|
||||||
|
SKILLS_NO_TELEMETRY: "1",
|
||||||
|
CI: "1",
|
||||||
|
},
|
||||||
|
timeout: 120_000,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
const authoredRoot = join(
|
||||||
|
snapshot,
|
||||||
|
".agents/skills/reverse-engineer-anything",
|
||||||
|
);
|
||||||
|
const installedSkills = join(consumer, ".agents/skills");
|
||||||
|
const installedRoot = join(installedSkills, "reverse-engineer-anything");
|
||||||
|
const files = await filePaths(authoredRoot);
|
||||||
|
if (
|
||||||
|
JSON.stringify(await readdir(installedSkills)) !==
|
||||||
|
JSON.stringify(["reverse-engineer-anything"]) ||
|
||||||
|
JSON.stringify(await filePaths(installedRoot)) !== JSON.stringify(files)
|
||||||
|
)
|
||||||
|
throw new Error(
|
||||||
|
"Skills did not install exactly the selected public bundle",
|
||||||
|
);
|
||||||
|
for (const path of files) {
|
||||||
|
const authored = await readFile(join(authoredRoot, path));
|
||||||
|
if (!authored.equals(await readFile(join(installedRoot, path))))
|
||||||
|
throw new Error(`Skills changed authored skill content: ${path}`);
|
||||||
|
}
|
||||||
|
return { skillsVersion, files };
|
||||||
|
}
|
||||||
|
|
||||||
|
async function filePaths(root) {
|
||||||
|
return (await readdir(root, { recursive: true, withFileTypes: true }))
|
||||||
|
.filter((entry) => entry.isFile())
|
||||||
|
.map((entry) => relative(root, join(entry.parentPath, entry.name)))
|
||||||
|
.sort();
|
||||||
|
}
|
||||||
@@ -17,6 +17,7 @@ import {
|
|||||||
import { verifyDyldEnvironment } from "./dyld-environment.mjs";
|
import { verifyDyldEnvironment } from "./dyld-environment.mjs";
|
||||||
import { verifyDylibResolution } from "./macos-dylib-e2e.mjs";
|
import { verifyDylibResolution } from "./macos-dylib-e2e.mjs";
|
||||||
import { verifyMachOOccurrences } from "./macho-occurrences.mjs";
|
import { verifyMachOOccurrences } from "./macho-occurrences.mjs";
|
||||||
|
import { verifyNativeProvenance } from "./native-provenance.mjs";
|
||||||
|
|
||||||
const exec = promisify(execFile);
|
const exec = promisify(execFile);
|
||||||
|
|
||||||
@@ -69,6 +70,7 @@ const EXPECTED_BUNDLES = [
|
|||||||
let report;
|
let report;
|
||||||
try {
|
try {
|
||||||
const { app, executable } = await buildMacosBundleFixture(root);
|
const { app, executable } = await buildMacosBundleFixture(root);
|
||||||
|
const nativeProvenance = await verifyNativeProvenance(app, executable);
|
||||||
const machoOccurrences = await verifyMachOOccurrences(app, executable);
|
const machoOccurrences = await verifyMachOOccurrences(app, executable);
|
||||||
const zip = join(root, "MacFixture.zip");
|
const zip = join(root, "MacFixture.zip");
|
||||||
await exec("/usr/bin/ditto", ["-c", "-k", "--keepParent", app, zip]);
|
await exec("/usr/bin/ditto", ["-c", "-k", "--keepParent", app, zip]);
|
||||||
@@ -166,6 +168,7 @@ try {
|
|||||||
containers: containers.map(({ format }) => format),
|
containers: containers.map(({ format }) => format),
|
||||||
bundles: EXPECTED_BUNDLES.length,
|
bundles: EXPECTED_BUNDLES.length,
|
||||||
dmg_detached: true,
|
dmg_detached: true,
|
||||||
|
native_provenance: nativeProvenance,
|
||||||
macho_occurrences: machoOccurrences,
|
macho_occurrences: machoOccurrences,
|
||||||
dylib_resolution: dylibs,
|
dylib_resolution: dylibs,
|
||||||
dyld_environment: dyldEnvironment,
|
dyld_environment: dyldEnvironment,
|
||||||
|
|||||||
@@ -0,0 +1,71 @@
|
|||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { createHash } from "node:crypto";
|
||||||
|
import { readFile } from "node:fs/promises";
|
||||||
|
|
||||||
|
import { toolContract } from "../../../dist/contracts/toolContracts.js";
|
||||||
|
import {
|
||||||
|
artifactCliEvidence,
|
||||||
|
withArtifactMcp,
|
||||||
|
} from "../../lib/artifact-e2e.mjs";
|
||||||
|
|
||||||
|
/** Inspect the compiled bundle with real native utilities through both adapters. */
|
||||||
|
export async function verifyNativeProvenance(app, executable) {
|
||||||
|
const original = await readFile(executable);
|
||||||
|
const sha256 = createHash("sha256").update(original).digest("hex");
|
||||||
|
const symbols = ["$s4main3FooV", "plain_symbol"];
|
||||||
|
const operations = [
|
||||||
|
["inspect_macho", []],
|
||||||
|
["inspect_signature", []],
|
||||||
|
["inspect_plist", []],
|
||||||
|
["list_architectures", []],
|
||||||
|
["demangle_swift", symbols],
|
||||||
|
];
|
||||||
|
const observations = [];
|
||||||
|
await withArtifactMcp(app, async (client) => {
|
||||||
|
const definitions = new Map(
|
||||||
|
(await client.listTools()).tools.map((tool) => [tool.name, tool]),
|
||||||
|
);
|
||||||
|
for (const [operation, arguments_] of operations) {
|
||||||
|
const cli = toolContract(operation).outputSchema.parse(
|
||||||
|
await artifactCliEvidence(
|
||||||
|
operation.replaceAll("_", "-"),
|
||||||
|
app,
|
||||||
|
arguments_,
|
||||||
|
),
|
||||||
|
);
|
||||||
|
const response = await client.callTool(
|
||||||
|
{
|
||||||
|
name: operation,
|
||||||
|
arguments: operation === "demangle_swift" ? { symbols } : {},
|
||||||
|
},
|
||||||
|
{ toolDefinition: definitions.get(operation) },
|
||||||
|
);
|
||||||
|
assert.notEqual(response.isError, true, JSON.stringify(response));
|
||||||
|
const mcp = toolContract(operation).outputSchema.parse(
|
||||||
|
response.structuredContent,
|
||||||
|
);
|
||||||
|
assert.equal(cli.raw_result, null);
|
||||||
|
assert.equal(mcp.raw_result, null);
|
||||||
|
assert.ok(cli.normalized_result.provenance.length > 0);
|
||||||
|
assert.deepEqual(mcp.normalized_result, cli.normalized_result);
|
||||||
|
assert.deepEqual(
|
||||||
|
JSON.parse(response.content.find((item) => item.type === "text").text),
|
||||||
|
response.structuredContent,
|
||||||
|
);
|
||||||
|
await client.ping();
|
||||||
|
observations.push({
|
||||||
|
operation,
|
||||||
|
commands: cli.normalized_result.provenance.length,
|
||||||
|
cli_bytes: Buffer.byteLength(JSON.stringify(cli)),
|
||||||
|
mcp_bytes: Buffer.byteLength(JSON.stringify(response)),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
assert.deepEqual(await readFile(executable), original);
|
||||||
|
return {
|
||||||
|
real_native_utilities: true,
|
||||||
|
target_sha256: sha256,
|
||||||
|
bytes_unchanged: true,
|
||||||
|
observations,
|
||||||
|
};
|
||||||
|
}
|
||||||
@@ -54,15 +54,15 @@ try {
|
|||||||
data: { ...evidence, normalized_result: { payload: [marker] } },
|
data: { ...evidence, normalized_result: { payload: [marker] } },
|
||||||
meta: metadata,
|
meta: metadata,
|
||||||
};
|
};
|
||||||
const skeleton = JSON.stringify(template, null, format === "json" ? 2 : 0);
|
// The destination is a redirected file, so both formats are compact.
|
||||||
|
const skeleton = JSON.stringify(template);
|
||||||
const encodedMarker = JSON.stringify(marker);
|
const encodedMarker = JSON.stringify(marker);
|
||||||
const markerIndex = skeleton.indexOf(encodedMarker);
|
const markerIndex = skeleton.indexOf(encodedMarker);
|
||||||
assert.ok(markerIndex >= 0);
|
assert.ok(markerIndex >= 0);
|
||||||
assert.equal(markerIndex, skeleton.lastIndexOf(encodedMarker));
|
assert.equal(markerIndex, skeleton.lastIndexOf(encodedMarker));
|
||||||
const prefix = skeleton.slice(0, markerIndex);
|
const prefix = skeleton.slice(0, markerIndex);
|
||||||
const suffix = `${skeleton.slice(markerIndex + encodedMarker.length)}\n`;
|
const suffix = `${skeleton.slice(markerIndex + encodedMarker.length)}\n`;
|
||||||
const indentation = prefix.slice(prefix.lastIndexOf("\n") + 1);
|
const separator = ",";
|
||||||
const separator = format === "json" ? `,\n${indentation}` : ",";
|
|
||||||
const expectedHash = createHash("sha256");
|
const expectedHash = createHash("sha256");
|
||||||
expectedHash.update(prefix);
|
expectedHash.update(prefix);
|
||||||
for (let index = 0; index < count; index += 1) {
|
for (let index = 0; index < count; index += 1) {
|
||||||
|
|||||||
@@ -80,7 +80,7 @@ export async function connectRecordedCrash({ entrypoint, environment }) {
|
|||||||
evidence.confidence,
|
evidence.confidence,
|
||||||
debuggerContext ? "derived" : "observed",
|
debuggerContext ? "derived" : "observed",
|
||||||
);
|
);
|
||||||
assert.deepEqual(evidence.raw_result, report);
|
assert.equal(evidence.raw_result, null);
|
||||||
assert.equal(evidence.subject.local_path, path);
|
assert.equal(evidence.subject.local_path, path);
|
||||||
assert.equal(report.artifact.path, path);
|
assert.equal(report.artifact.path, path);
|
||||||
assert.equal(
|
assert.equal(
|
||||||
|
|||||||
@@ -0,0 +1,22 @@
|
|||||||
|
import { expect, it } from "vitest";
|
||||||
|
|
||||||
|
import { createAnalysisExecution } from "./AnalysisProvider.js";
|
||||||
|
|
||||||
|
const PROVIDER = { id: "fixture", name: "Fixture", version: "1" };
|
||||||
|
|
||||||
|
it("records no raw result unless the provider supplies a distinct one", () => {
|
||||||
|
const result = { entries: [{ value: "normalized" }] };
|
||||||
|
|
||||||
|
expect(createAnalysisExecution(result, PROVIDER)).toMatchObject({
|
||||||
|
result,
|
||||||
|
rawResult: null,
|
||||||
|
});
|
||||||
|
expect(
|
||||||
|
createAnalysisExecution(result, PROVIDER, {
|
||||||
|
rawResult: { stdout: "raw" },
|
||||||
|
}).rawResult,
|
||||||
|
).toEqual({ stdout: "raw" });
|
||||||
|
expect(
|
||||||
|
createAnalysisExecution(result, PROVIDER, { rawResult: null }).rawResult,
|
||||||
|
).toBeNull();
|
||||||
|
});
|
||||||
@@ -41,7 +41,13 @@ export interface AnalysisExecution {
|
|||||||
readonly subject: EvidenceSubjectTarget | null;
|
readonly subject: EvidenceSubjectTarget | null;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Build a validated atomic provider observation at an adapter boundary. */
|
/**
|
||||||
|
* Build a validated atomic provider observation at an adapter boundary.
|
||||||
|
*
|
||||||
|
* Pass `rawResult` only for a provider representation distinct from
|
||||||
|
* `result`. Omitting it records `null` rather than a second complete copy of
|
||||||
|
* the normalized result in every Evidence record and tool response.
|
||||||
|
*/
|
||||||
export const createAnalysisExecution = (
|
export const createAnalysisExecution = (
|
||||||
result: unknown,
|
result: unknown,
|
||||||
provider: ProviderIdentity,
|
provider: ProviderIdentity,
|
||||||
@@ -56,7 +62,7 @@ export const createAnalysisExecution = (
|
|||||||
result: jsonValueSchema.parse(result),
|
result: jsonValueSchema.parse(result),
|
||||||
rawResult:
|
rawResult:
|
||||||
options.rawResult === undefined
|
options.rawResult === undefined
|
||||||
? jsonValueSchema.parse(result)
|
? null
|
||||||
: jsonValueSchema.parse(options.rawResult),
|
: jsonValueSchema.parse(options.rawResult),
|
||||||
provider,
|
provider,
|
||||||
...(options.analysisProfile === undefined
|
...(options.analysisProfile === undefined
|
||||||
|
|||||||
@@ -91,6 +91,7 @@ export const readClientRegistrationStatuses = async (
|
|||||||
PI_CODING_AGENT_DIR: options.environment.PI_CODING_AGENT_DIR,
|
PI_CODING_AGENT_DIR: options.environment.PI_CODING_AGENT_DIR,
|
||||||
PI_CONFIG_DIR: options.environment.PI_CONFIG_DIR,
|
PI_CONFIG_DIR: options.environment.PI_CONFIG_DIR,
|
||||||
PI_PROFILE: options.environment.PI_PROFILE,
|
PI_PROFILE: options.environment.PI_PROFILE,
|
||||||
|
QWEN_HOME: options.environment.QWEN_HOME,
|
||||||
SAND_DATA_ROOT: options.environment.SAND_DATA_ROOT,
|
SAND_DATA_ROOT: options.environment.SAND_DATA_ROOT,
|
||||||
XDG_CONFIG_HOME: options.environment.XDG_CONFIG_HOME,
|
XDG_CONFIG_HOME: options.environment.XDG_CONFIG_HOME,
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -2,8 +2,6 @@ import fs from "node:fs";
|
|||||||
import { lstat, readFile } from "node:fs/promises";
|
import { lstat, readFile } from "node:fs/promises";
|
||||||
import { join, resolve } from "node:path";
|
import { join, resolve } from "node:path";
|
||||||
|
|
||||||
import { resolveRef } from "isomorphic-git";
|
|
||||||
|
|
||||||
export type ReferenceSourceVcsInfo =
|
export type ReferenceSourceVcsInfo =
|
||||||
| {
|
| {
|
||||||
readonly kind: "git";
|
readonly kind: "git";
|
||||||
@@ -42,6 +40,8 @@ export const readReferenceSourceVcs = async (
|
|||||||
};
|
};
|
||||||
|
|
||||||
const resolveSourceHead = async (root: string): Promise<string> => {
|
const resolveSourceHead = async (root: string): Promise<string> => {
|
||||||
|
// isomorphic-git is loaded on first use so CLI and MCP startup skip it.
|
||||||
|
const { resolveRef } = await import("isomorphic-git");
|
||||||
const dotgit = join(root, ".git");
|
const dotgit = join(root, ".git");
|
||||||
if (!(await lstat(dotgit)).isFile())
|
if (!(await lstat(dotgit)).isFile())
|
||||||
return resolveRef({ fs, dir: root, ref: "HEAD" });
|
return resolveRef({ fs, dir: root, ref: "HEAD" });
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
import { isAbsolute, join } from "node:path";
|
import { isAbsolute, join, resolve } from "node:path";
|
||||||
import { lstatSync } from "node:fs";
|
import { lstatSync } from "node:fs";
|
||||||
|
|
||||||
/** One supported client configuration location. */
|
/** One supported client configuration location. */
|
||||||
@@ -35,6 +35,7 @@ interface ClientPathContext {
|
|||||||
readonly PI_CODING_AGENT_DIR?: string | undefined;
|
readonly PI_CODING_AGENT_DIR?: string | undefined;
|
||||||
readonly PI_CONFIG_DIR?: string | undefined;
|
readonly PI_CONFIG_DIR?: string | undefined;
|
||||||
readonly PI_PROFILE?: string | undefined;
|
readonly PI_PROFILE?: string | undefined;
|
||||||
|
readonly QWEN_HOME?: string | undefined;
|
||||||
readonly SAND_DATA_ROOT?: string | undefined;
|
readonly SAND_DATA_ROOT?: string | undefined;
|
||||||
readonly XDG_CONFIG_HOME?: string | undefined;
|
readonly XDG_CONFIG_HOME?: string | undefined;
|
||||||
};
|
};
|
||||||
@@ -85,6 +86,25 @@ const claudeCodeMarkerDirectory = ({ home, env }: ClientPathContext): string =>
|
|||||||
const codexDirectory = ({ home, env }: ClientPathContext): string =>
|
const codexDirectory = ({ home, env }: ClientPathContext): string =>
|
||||||
env.CODEX_HOME ?? join(home, ".codex");
|
env.CODEX_HOME ?? join(home, ".codex");
|
||||||
|
|
||||||
|
/** Match Qwen Code's home override, including tilde and cwd-relative paths. */
|
||||||
|
const qwenCodeDirectory = ({ home, env }: ClientPathContext): string => {
|
||||||
|
const override = env.QWEN_HOME;
|
||||||
|
if (!override) return join(home, ".qwen");
|
||||||
|
const expanded =
|
||||||
|
override === "~"
|
||||||
|
? home
|
||||||
|
: override.startsWith("~/") || override.startsWith("~\\")
|
||||||
|
? join(
|
||||||
|
home,
|
||||||
|
...override
|
||||||
|
.slice(2)
|
||||||
|
.split(/[/\\]+/)
|
||||||
|
.filter(Boolean),
|
||||||
|
)
|
||||||
|
: override;
|
||||||
|
return isAbsolute(expanded) ? expanded : resolve(expanded);
|
||||||
|
};
|
||||||
|
|
||||||
const grokDirectory = ({ home, env }: ClientPathContext): string =>
|
const grokDirectory = ({ home, env }: ClientPathContext): string =>
|
||||||
env.GROK_HOME ?? join(home, ".grok");
|
env.GROK_HOME ?? join(home, ".grok");
|
||||||
|
|
||||||
@@ -272,6 +292,14 @@ export const SUPPORTED_CLIENT_DEFINITIONS = [
|
|||||||
markerPath: commandCodeDirectory,
|
markerPath: commandCodeDirectory,
|
||||||
format: "commandcode",
|
format: "commandcode",
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
name: "qwen_code",
|
||||||
|
displayName: "Qwen Code",
|
||||||
|
configPath: (context: ClientPathContext) =>
|
||||||
|
join(qwenCodeDirectory(context), "settings.json"),
|
||||||
|
markerPath: qwenCodeDirectory,
|
||||||
|
format: "json",
|
||||||
|
},
|
||||||
{
|
{
|
||||||
name: "vscode",
|
name: "vscode",
|
||||||
displayName: "VS Code",
|
displayName: "VS Code",
|
||||||
@@ -323,6 +351,7 @@ export const supportedClients = (
|
|||||||
PI_CODING_AGENT_DIR: process.env.PI_CODING_AGENT_DIR,
|
PI_CODING_AGENT_DIR: process.env.PI_CODING_AGENT_DIR,
|
||||||
PI_CONFIG_DIR: process.env.PI_CONFIG_DIR,
|
PI_CONFIG_DIR: process.env.PI_CONFIG_DIR,
|
||||||
PI_PROFILE: process.env.PI_PROFILE,
|
PI_PROFILE: process.env.PI_PROFILE,
|
||||||
|
QWEN_HOME: process.env.QWEN_HOME,
|
||||||
SAND_DATA_ROOT: process.env.SAND_DATA_ROOT,
|
SAND_DATA_ROOT: process.env.SAND_DATA_ROOT,
|
||||||
XDG_CONFIG_HOME: process.env.XDG_CONFIG_HOME,
|
XDG_CONFIG_HOME: process.env.XDG_CONFIG_HOME,
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -2,6 +2,8 @@ import { expect, it } from "vitest";
|
|||||||
|
|
||||||
import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "../../contracts/javascript/javascriptRuntimeReconciliationExample.js";
|
import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "../../contracts/javascript/javascriptRuntimeReconciliationExample.js";
|
||||||
import { createEvidence, parseEvidence } from "../../domain/evidence.js";
|
import { createEvidence, parseEvidence } from "../../domain/evidence.js";
|
||||||
|
import { functionDossierSchema } from "../../domain/hopperValues.js";
|
||||||
|
import { ghidraFunctionDossier } from "../../domain/ghidraValues.fixture.js";
|
||||||
import {
|
import {
|
||||||
analysisViewJavaScriptAnalysisWithSource,
|
analysisViewJavaScriptAnalysisWithSource,
|
||||||
analysisViewJavaScriptEvidence,
|
analysisViewJavaScriptEvidence,
|
||||||
@@ -9,6 +11,96 @@ import {
|
|||||||
} from "../../../tests/fixtures/analysisView.js";
|
} from "../../../tests/fixtures/analysisView.js";
|
||||||
import { inspectAnalysisView } from "./AnalysisViewService.js";
|
import { inspectAnalysisView } from "./AnalysisViewService.js";
|
||||||
|
|
||||||
|
it.each(["/fixtures/unknown.exe", ""])(
|
||||||
|
"preserves recorded empty native identity fields: path=%j",
|
||||||
|
(path) => {
|
||||||
|
const original = functionDossierSchema.parse(ghidraFunctionDossier());
|
||||||
|
const parent = createEvidence(
|
||||||
|
{ path, format: "pe", sha256: "c".repeat(64) },
|
||||||
|
{ id: "ghidra", name: "Ghidra", version: "12.1.4" },
|
||||||
|
{
|
||||||
|
operation: "analyze_function",
|
||||||
|
parameters: {},
|
||||||
|
result: {
|
||||||
|
...original,
|
||||||
|
procedure: {
|
||||||
|
...original.procedure,
|
||||||
|
address: "",
|
||||||
|
body: { available: false, reason: "Not recorded" },
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
);
|
||||||
|
const result = inspectAnalysisView({
|
||||||
|
source: { kind: "inline", evidence: parent },
|
||||||
|
view: {
|
||||||
|
kind: "native",
|
||||||
|
facet: "procedure",
|
||||||
|
offset: 0,
|
||||||
|
limit: 64,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
if (!result.ok) throw result.error;
|
||||||
|
expect(result.value.normalized_result).toMatchObject({
|
||||||
|
procedure_address: "",
|
||||||
|
artifact: { path },
|
||||||
|
item: { address: "" },
|
||||||
|
});
|
||||||
|
expect(result.value.locations).toEqual(
|
||||||
|
path.length > 0 ? [{ kind: "artifact-path", path }] : [],
|
||||||
|
);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
it("projects a small native view from an authenticated >10 MiB retained dossier", () => {
|
||||||
|
const original = functionDossierSchema.parse(ghidraFunctionDossier());
|
||||||
|
const parent = createEvidence(
|
||||||
|
{ path: "/fixtures/large.exe", format: "pe", sha256: "c".repeat(64) },
|
||||||
|
{ id: "ghidra", name: "Ghidra", version: "12.1.4" },
|
||||||
|
{
|
||||||
|
operation: "analyze_function",
|
||||||
|
parameters: { address: "0x401000" },
|
||||||
|
result: { ...original, pseudocode: "X".repeat(11 * 1024 * 1024) },
|
||||||
|
limitations: ["Synthetic fixture"],
|
||||||
|
},
|
||||||
|
);
|
||||||
|
const input = {
|
||||||
|
source: {
|
||||||
|
kind: "retained-evidence" as const,
|
||||||
|
evidence_id: parent.evidence_id,
|
||||||
|
},
|
||||||
|
view: {
|
||||||
|
kind: "native" as const,
|
||||||
|
facet: "pseudocode" as const,
|
||||||
|
offset: 0,
|
||||||
|
limit: 128,
|
||||||
|
},
|
||||||
|
};
|
||||||
|
const view = inspectAnalysisView(input, (id) =>
|
||||||
|
id === parent.evidence_id ? parent : undefined,
|
||||||
|
);
|
||||||
|
if (!view.ok) throw view.error;
|
||||||
|
expect(view.value.evidence_links).toEqual([parent.evidence_id]);
|
||||||
|
expect(view.value.normalized_result).toMatchObject({
|
||||||
|
kind: "native",
|
||||||
|
parent_operation: "analyze_function",
|
||||||
|
parent_evidence_id: parent.evidence_id,
|
||||||
|
procedure_address: "0x401000",
|
||||||
|
artifact: { path: "/fixtures/large.exe", sha256: "c".repeat(64) },
|
||||||
|
coverage: { total: 11 * 1024 * 1024, examined: 128, next_offset: 128 },
|
||||||
|
});
|
||||||
|
expect(view.value.locations).toEqual([
|
||||||
|
{ kind: "artifact-path", path: "/fixtures/large.exe" },
|
||||||
|
{ kind: "address", address: "0x401000" },
|
||||||
|
]);
|
||||||
|
expect(JSON.stringify(view.value).length).toBeLessThan(12000);
|
||||||
|
expect(JSON.stringify(parent).length).toBeGreaterThan(10 * 1024 * 1024);
|
||||||
|
expect(inspectAnalysisView(input, () => undefined)).toMatchObject({
|
||||||
|
ok: false,
|
||||||
|
error: { _tag: "EvidenceIntegrityError" },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
it("projects inline layout Evidence and retains a derived view record", () => {
|
it("projects inline layout Evidence and retains a derived view record", () => {
|
||||||
const parent = analysisViewLayoutEvidence();
|
const parent = analysisViewLayoutEvidence();
|
||||||
const result = inspectAnalysisView({
|
const result = inspectAnalysisView({
|
||||||
|
|||||||
@@ -53,6 +53,13 @@ export const inspectAnalysisViewValidated = (
|
|||||||
operation: parent.operation,
|
operation: parent.operation,
|
||||||
normalizedResult: parent.normalized_result,
|
normalizedResult: parent.normalized_result,
|
||||||
limitations: parent.limitations,
|
limitations: parent.limitations,
|
||||||
|
artifact:
|
||||||
|
parent.subject === null
|
||||||
|
? null
|
||||||
|
: {
|
||||||
|
path: parent.subject.local_path,
|
||||||
|
sha256: parent.subject.digest.sha256,
|
||||||
|
},
|
||||||
},
|
},
|
||||||
input.view,
|
input.view,
|
||||||
);
|
);
|
||||||
@@ -126,7 +133,16 @@ const createAnalysisViewEvidence = (
|
|||||||
confidence: "derived",
|
confidence: "derived",
|
||||||
authority: parent.authority,
|
authority: parent.authority,
|
||||||
limitations: result.limitations,
|
limitations: result.limitations,
|
||||||
locations: [{ kind: "artifact-path", path: result.artifact.path }],
|
locations: [
|
||||||
|
...(result.artifact.path.length > 0
|
||||||
|
? [{ kind: "artifact-path" as const, path: result.artifact.path }]
|
||||||
|
: []),
|
||||||
|
...(result.kind === "native" &&
|
||||||
|
result.procedure_address !== null &&
|
||||||
|
result.procedure_address.length > 0
|
||||||
|
? [{ kind: "address" as const, address: result.procedure_address }]
|
||||||
|
: []),
|
||||||
|
],
|
||||||
evidenceLinks: [parent.evidence_id],
|
evidenceLinks: [parent.evidence_id],
|
||||||
});
|
});
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import {
|
|||||||
BINARY_LAYOUT_TEST_PROVIDER,
|
BINARY_LAYOUT_TEST_PROVIDER,
|
||||||
} from "../../../tests/fixtures/binaryDiagnostics/layout.js";
|
} from "../../../tests/fixtures/binaryDiagnostics/layout.js";
|
||||||
import { AnalysisCapabilityUnavailableError } from "../../domain/analysisErrorCore.js";
|
import { AnalysisCapabilityUnavailableError } from "../../domain/analysisErrorCore.js";
|
||||||
|
import { parseEvidence } from "../../domain/evidence.js";
|
||||||
import { err, ok } from "../../domain/result.js";
|
import { err, ok } from "../../domain/result.js";
|
||||||
|
|
||||||
it.each([
|
it.each([
|
||||||
@@ -28,6 +29,19 @@ it.each([
|
|||||||
},
|
},
|
||||||
);
|
);
|
||||||
|
|
||||||
|
it("records the decoded layout once in its Evidence", async () => {
|
||||||
|
const layout = binaryLayoutFixture();
|
||||||
|
const service = new BinaryLayoutService({
|
||||||
|
identity: BINARY_LAYOUT_TEST_PROVIDER,
|
||||||
|
inspect: () => Promise.resolve(ok(layout)),
|
||||||
|
});
|
||||||
|
const result = await service.inspect({ path: layout.artifact.path });
|
||||||
|
if (!result.ok) throw result.error;
|
||||||
|
const evidence = parseEvidence(result.value);
|
||||||
|
expect(evidence.normalized_result).toEqual(layout);
|
||||||
|
expect(evidence.raw_result).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
it("preserves a provider's actionable unsupported reason", async () => {
|
it("preserves a provider's actionable unsupported reason", async () => {
|
||||||
const failure = new AnalysisCapabilityUnavailableError(
|
const failure = new AnalysisCapabilityUnavailableError(
|
||||||
"fixture-layout",
|
"fixture-layout",
|
||||||
|
|||||||
@@ -78,7 +78,8 @@ export class BinaryLayoutService {
|
|||||||
operation: OPERATION,
|
operation: OPERATION,
|
||||||
parameters: input.data,
|
parameters: input.data,
|
||||||
result: value,
|
result: value,
|
||||||
rawResult: value,
|
// The decoded report is the only representation; do not repeat it.
|
||||||
|
rawResult: null,
|
||||||
confidence: "observed",
|
confidence: "observed",
|
||||||
limitations: value.limitations,
|
limitations: value.limitations,
|
||||||
locations: [{ kind: "artifact-path", path: value.artifact.path }],
|
locations: [{ kind: "artifact-path", path: value.artifact.path }],
|
||||||
|
|||||||
@@ -24,6 +24,7 @@ it("labels an envelope containing debugger-derived maps as derived", async () =>
|
|||||||
const evidence = parseEvidence(result.value);
|
const evidence = parseEvidence(result.value);
|
||||||
expect(evidence.confidence).toBe("derived");
|
expect(evidence.confidence).toBe("derived");
|
||||||
expect(evidence.normalized_result).toEqual(fixture);
|
expect(evidence.normalized_result).toEqual(fixture);
|
||||||
|
expect(evidence.raw_result).toBeNull();
|
||||||
});
|
});
|
||||||
|
|
||||||
it.each([
|
it.each([
|
||||||
|
|||||||
@@ -83,7 +83,8 @@ export class RecordedCrashService {
|
|||||||
operation: OPERATION,
|
operation: OPERATION,
|
||||||
parameters: input.data,
|
parameters: input.data,
|
||||||
result: value,
|
result: value,
|
||||||
rawResult: value,
|
// The decoded report is the only representation; do not repeat it.
|
||||||
|
rawResult: null,
|
||||||
confidence:
|
confidence:
|
||||||
value.debugger.status === "available" ? "derived" : "observed",
|
value.debugger.status === "available" ? "derived" : "observed",
|
||||||
limitations: value.limitations,
|
limitations: value.limitations,
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
import { createHash } from "node:crypto";
|
import { createHash } from "node:crypto";
|
||||||
|
|
||||||
|
import { compactStringIdentityKey } from "../../domain/compactStringIdentity.js";
|
||||||
import type { ApplicationNode } from "../../domain/javascript/javascriptApplicationGraphSchemas.js";
|
import type { ApplicationNode } from "../../domain/javascript/javascriptApplicationGraphSchemas.js";
|
||||||
import type { JavaScriptStaticAnalysis } from "../../domain/javascript/javascriptStaticAnalysisTypes.js";
|
import type { JavaScriptStaticAnalysis } from "../../domain/javascript/javascriptStaticAnalysisTypes.js";
|
||||||
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
|
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
|
||||||
@@ -162,16 +163,15 @@ const addEndpoint = (
|
|||||||
const source =
|
const source =
|
||||||
sourceNodeFor(context, input.file.path, input.value.module_key) ??
|
sourceNodeFor(context, input.file.path, input.value.module_key) ??
|
||||||
input.asset;
|
input.asset;
|
||||||
|
const key = compactStringIdentityKey(input.value.value);
|
||||||
|
const label = `${input.value.kind} endpoint`;
|
||||||
const node = context.accumulator.addNode({
|
const node = context.accumulator.addNode({
|
||||||
kind: "endpoint",
|
kind: "endpoint",
|
||||||
identity: artifactLocalIdentity(
|
identity: artifactLocalIdentity(input.file.sha256, input.value.kind, key),
|
||||||
input.file.sha256,
|
|
||||||
input.value.kind,
|
|
||||||
input.value.value,
|
|
||||||
),
|
|
||||||
observations: [
|
observations: [
|
||||||
{
|
{
|
||||||
label: input.value.value,
|
label:
|
||||||
|
input.value.value.length <= label.length ? input.value.value : label,
|
||||||
properties: {
|
properties: {
|
||||||
endpoint_kind: input.value.kind,
|
endpoint_kind: input.value.kind,
|
||||||
value: input.value.value,
|
value: input.value.value,
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { compactStringIdentityKey } from "../../domain/compactStringIdentity.js";
|
||||||
import { createJavaScriptSemanticGraphUnknown } from "../../domain/javascript/javascriptSemanticGraph.js";
|
import { createJavaScriptSemanticGraphUnknown } from "../../domain/javascript/javascriptSemanticGraph.js";
|
||||||
import type { JavaScriptSemanticGraphNode } from "../../domain/javascript/javascriptSemanticGraphSchemas.js";
|
import type { JavaScriptSemanticGraphNode } from "../../domain/javascript/javascriptSemanticGraphSchemas.js";
|
||||||
import type {
|
import type {
|
||||||
@@ -21,7 +22,7 @@ export const projectSemanticEvents = (
|
|||||||
const eventKey =
|
const eventKey =
|
||||||
operation.eventName === null
|
operation.eventName === null
|
||||||
? operation.eventId
|
? operation.eventId
|
||||||
: `${operation.emitterKey}\u0000${operation.eventName}`;
|
: `${operation.emitterKey}\u0000${compactStringIdentityKey(operation.eventName)}`;
|
||||||
const event =
|
const event =
|
||||||
eventNodes.get(eventKey) ?? addEventNode(context, operation, eventKey);
|
eventNodes.get(eventKey) ?? addEventNode(context, operation, eventKey);
|
||||||
if (event === null) continue;
|
if (event === null) continue;
|
||||||
@@ -59,7 +60,11 @@ const addEventNode = (
|
|||||||
kind: "event",
|
kind: "event",
|
||||||
roleKey: `event:${eventKey}`,
|
roleKey: `event:${eventKey}`,
|
||||||
location: operation.eventName === null ? operation.location : null,
|
location: operation.eventName === null ? operation.location : null,
|
||||||
label: operation.eventName,
|
label:
|
||||||
|
operation.eventName === null ||
|
||||||
|
operation.eventName.length <= "event".length
|
||||||
|
? operation.eventName
|
||||||
|
: "event",
|
||||||
functionNodeId: null,
|
functionNodeId: null,
|
||||||
properties: {
|
properties: {
|
||||||
emitter_key: operation.emitterKey,
|
emitter_key: operation.emitterKey,
|
||||||
@@ -78,7 +83,9 @@ const addListenerNode = (
|
|||||||
label:
|
label:
|
||||||
operation.eventName === null
|
operation.eventName === null
|
||||||
? operation.method
|
? operation.method
|
||||||
: `${operation.method}:${operation.eventName}`,
|
: operation.eventName.length <= "listener".length
|
||||||
|
? `${operation.method}:${operation.eventName}`
|
||||||
|
: `${operation.method}:listener`,
|
||||||
functionNodeId:
|
functionNodeId:
|
||||||
operation.ownerCallableId === null
|
operation.ownerCallableId === null
|
||||||
? null
|
? null
|
||||||
|
|||||||
@@ -175,7 +175,21 @@ export const createJavaScriptSemanticGraphProjection =
|
|||||||
): Value => {
|
): Value => {
|
||||||
bindSemanticGraphApplicationNodes(state, applicationGraph);
|
bindSemanticGraphApplicationNodes(state, applicationGraph);
|
||||||
if (state.roots.size === 0) addFallbackRoot(rootArtifactSha256, state);
|
if (state.roots.size === 0) addFallbackRoot(rootArtifactSha256, state);
|
||||||
|
const relations = [...state.relations.values()];
|
||||||
const unknowns = [...state.unknowns.values()];
|
const unknowns = [...state.unknowns.values()];
|
||||||
|
type Family = (typeof JAVASCRIPT_SEMANTIC_RELATION_FAMILIES)[number];
|
||||||
|
const retainedByFamily = new Map<Family, number>();
|
||||||
|
const unknownIdsByFamily = new Map<Family, string[]>();
|
||||||
|
for (const relation of relations) {
|
||||||
|
const family = JAVASCRIPT_SEMANTIC_RELATION_FAMILY[relation.relation];
|
||||||
|
retainedByFamily.set(family, (retainedByFamily.get(family) ?? 0) + 1);
|
||||||
|
}
|
||||||
|
for (const unknown of unknowns) {
|
||||||
|
const identifiers = unknownIdsByFamily.get(unknown.family);
|
||||||
|
if (identifiers === undefined)
|
||||||
|
unknownIdsByFamily.set(unknown.family, [unknown.unknown_id]);
|
||||||
|
else identifiers.push(unknown.unknown_id);
|
||||||
|
}
|
||||||
const graph = factory({
|
const graph = factory({
|
||||||
schema: "JavaScriptSemanticRelationGraph",
|
schema: "JavaScriptSemanticRelationGraph",
|
||||||
root_artifact_sha256: rootArtifactSha256,
|
root_artifact_sha256: rootArtifactSha256,
|
||||||
@@ -183,7 +197,7 @@ export const createJavaScriptSemanticGraphProjection =
|
|||||||
root_node_ids: [...state.roots],
|
root_node_ids: [...state.roots],
|
||||||
evidence_contexts: state.evidenceContexts.contexts,
|
evidence_contexts: state.evidenceContexts.contexts,
|
||||||
nodes: [...state.nodes.values()],
|
nodes: [...state.nodes.values()],
|
||||||
relations: [...state.relations.values()],
|
relations,
|
||||||
fingerprints,
|
fingerprints,
|
||||||
unknowns,
|
unknowns,
|
||||||
coverage: {
|
coverage: {
|
||||||
@@ -204,15 +218,9 @@ export const createJavaScriptSemanticGraphProjection =
|
|||||||
families: JAVASCRIPT_SEMANTIC_RELATION_FAMILIES.map((family) => ({
|
families: JAVASCRIPT_SEMANTIC_RELATION_FAMILIES.map((family) => ({
|
||||||
family,
|
family,
|
||||||
status: semanticFamilyStatus(family),
|
status: semanticFamilyStatus(family),
|
||||||
retained_relations: [...state.relations.values()].filter(
|
retained_relations: retainedByFamily.get(family) ?? 0,
|
||||||
(relation) =>
|
|
||||||
JAVASCRIPT_SEMANTIC_RELATION_FAMILY[relation.relation] ===
|
|
||||||
family,
|
|
||||||
).length,
|
|
||||||
omitted_relations: truncatedFiles > 0 ? null : 0,
|
omitted_relations: truncatedFiles > 0 ? null : 0,
|
||||||
unknown_ids: unknowns
|
unknown_ids: unknownIdsByFamily.get(family) ?? [],
|
||||||
.filter((unknown) => unknown.family === family)
|
|
||||||
.map(({ unknown_id: identifier }) => identifier),
|
|
||||||
})),
|
})),
|
||||||
},
|
},
|
||||||
limitations: [
|
limitations: [
|
||||||
|
|||||||
@@ -137,7 +137,11 @@ const addRequestNode = (
|
|||||||
kind: operation.kind === "request" ? "request" : "response",
|
kind: operation.kind === "request" ? "request" : "response",
|
||||||
roleKey: operation.requestId,
|
roleKey: operation.requestId,
|
||||||
location: operation.location,
|
location: operation.location,
|
||||||
label: operation.endpoint ?? operation.method,
|
label:
|
||||||
|
operation.endpoint !== null &&
|
||||||
|
operation.endpoint.length <= operation.method.length
|
||||||
|
? operation.endpoint
|
||||||
|
: operation.method,
|
||||||
functionNodeId:
|
functionNodeId:
|
||||||
operation.ownerCallableId === null
|
operation.ownerCallableId === null
|
||||||
? null
|
? null
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ import {
|
|||||||
semanticContainer,
|
semanticContainer,
|
||||||
semanticPropertyPointer,
|
semanticPropertyPointer,
|
||||||
} from "../../domain/javascript/javascriptSemanticSlots.js";
|
} from "../../domain/javascript/javascriptSemanticSlots.js";
|
||||||
|
import { digestCanonicalValue } from "../../domain/canonicalDigest.js";
|
||||||
import type { JavaScriptSemanticBinding } from "../../domain/javascript/javascriptSemanticIr.js";
|
import type { JavaScriptSemanticBinding } from "../../domain/javascript/javascriptSemanticIr.js";
|
||||||
import type {
|
import type {
|
||||||
JavaScriptSemanticProperty,
|
JavaScriptSemanticProperty,
|
||||||
@@ -153,16 +154,27 @@ const addLiteralNode = (
|
|||||||
binding: JavaScriptSemanticBinding,
|
binding: JavaScriptSemanticBinding,
|
||||||
value: string | number | boolean | null,
|
value: string | number | boolean | null,
|
||||||
role: string,
|
role: string,
|
||||||
) =>
|
) => {
|
||||||
retainSemanticGraphNode(context.state, context.file, {
|
const encodedValue = JSON.stringify(value);
|
||||||
|
const valueDigest = `value-sha256:${digestCanonicalValue(value, "Semantic literal")}`;
|
||||||
|
const valueKey =
|
||||||
|
encodedValue.length <= valueDigest.length ? encodedValue : valueDigest;
|
||||||
|
const stringLabel = "string literal";
|
||||||
|
return retainSemanticGraphNode(context.state, context.file, {
|
||||||
kind: "literal",
|
kind: "literal",
|
||||||
roleKey: `literal:${binding.bindingId}:${role}:${JSON.stringify(value)}`,
|
// Identity needs an exact value commitment, not another payload copy.
|
||||||
|
// Keep the complete literal in properties for queries and Evidence export.
|
||||||
|
roleKey: `literal:${binding.bindingId}:${role}:${valueKey}`,
|
||||||
location: binding.definitions[0]?.location ?? null,
|
location: binding.definitions[0]?.location ?? null,
|
||||||
label: JSON.stringify(value),
|
label:
|
||||||
|
typeof value === "string" && encodedValue.length > stringLabel.length
|
||||||
|
? stringLabel
|
||||||
|
: encodedValue,
|
||||||
functionNodeId:
|
functionNodeId:
|
||||||
context.bindingNodes.get(binding.bindingId)?.function_node_id ?? null,
|
context.bindingNodes.get(binding.bindingId)?.function_node_id ?? null,
|
||||||
properties: { value },
|
properties: { value },
|
||||||
});
|
});
|
||||||
|
};
|
||||||
|
|
||||||
/** Create one canonical slot for a root binding and exact static property path. */
|
/** Create one canonical slot for a root binding and exact static property path. */
|
||||||
export const semanticPropertySlot = (
|
export const semanticPropertySlot = (
|
||||||
|
|||||||
+157
-11
@@ -1,9 +1,24 @@
|
|||||||
import { posix } from "node:path";
|
import { posix } from "node:path";
|
||||||
|
|
||||||
|
import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js";
|
||||||
import { ArtifactReaderFailure } from "./ArtifactReader.js";
|
import { ArtifactReaderFailure } from "./ArtifactReader.js";
|
||||||
|
|
||||||
const DRIVE_OR_UNC = /^(?:[A-Za-z]:|\\|\/\/)/u;
|
const DRIVE_OR_UNC = /^(?:[A-Za-z]:|\\|\/\/)/u;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* English Unicode case folding used only to describe logical names a
|
||||||
|
* case-insensitive destination might collapse. It is not a filesystem case table.
|
||||||
|
*/
|
||||||
|
const ENGLISH_UNICODE_CASE_NOTE =
|
||||||
|
"This comparison is not a filesystem observation.";
|
||||||
|
|
||||||
|
/** Inventory limitation when logical names collide under English Unicode case folding. */
|
||||||
|
export const ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION = `Logical archive names collide under English (en-US) Unicode case folding; a case-insensitive destination cannot store both spellings. ${ENGLISH_UNICODE_CASE_NOTE}`;
|
||||||
|
|
||||||
|
/** User-facing prefix for a destination that cannot keep two logical spellings. */
|
||||||
|
export const DESTINATION_CASE_COLLISION_PREFIX =
|
||||||
|
"Destination filesystem cannot store both ";
|
||||||
|
|
||||||
/** Normalize an untrusted logical archive path without touching filesystem. */
|
/** Normalize an untrusted logical archive path without touching filesystem. */
|
||||||
export const normalizeArtifactPath = (input: string): string => {
|
export const normalizeArtifactPath = (input: string): string => {
|
||||||
if (
|
if (
|
||||||
@@ -30,7 +45,7 @@ export const normalizeArtifactPath = (input: string): string => {
|
|||||||
return normalized;
|
return normalized;
|
||||||
};
|
};
|
||||||
|
|
||||||
/** Reject exact, Unicode-normalized, case-folded, and prefix collisions. */
|
/** Reject exact and same-spelling prefix collisions. Case-distinct names stay distinct. */
|
||||||
export class ArtifactPathRegistry {
|
export class ArtifactPathRegistry {
|
||||||
readonly #root: PathTrieNode = { kind: undefined, children: new Map() };
|
readonly #root: PathTrieNode = { kind: undefined, children: new Map() };
|
||||||
|
|
||||||
@@ -43,16 +58,10 @@ export class ArtifactPathRegistry {
|
|||||||
"path",
|
"path",
|
||||||
`Artifact prefix conflict: ${path}`,
|
`Artifact prefix conflict: ${path}`,
|
||||||
);
|
);
|
||||||
const foldedPart = part.toLocaleLowerCase("en-US");
|
let child = node.children.get(part);
|
||||||
let child = node.children.get(foldedPart);
|
|
||||||
if (child !== undefined && child.spelling !== part)
|
|
||||||
throw new ArtifactReaderFailure(
|
|
||||||
"path",
|
|
||||||
`Artifact path collision: ${path} differs only in case from ${[...parts.slice(0, index), child.spelling].join("/")}`,
|
|
||||||
);
|
|
||||||
if (child === undefined) {
|
if (child === undefined) {
|
||||||
child = { spelling: part, kind: undefined, children: new Map() };
|
child = { kind: undefined, children: new Map() };
|
||||||
node.children.set(foldedPart, child);
|
node.children.set(part, child);
|
||||||
}
|
}
|
||||||
node = child;
|
node = child;
|
||||||
if (index === parts.length - 1) {
|
if (index === parts.length - 1) {
|
||||||
@@ -73,7 +82,144 @@ export class ArtifactPathRegistry {
|
|||||||
}
|
}
|
||||||
|
|
||||||
interface PathTrieNode {
|
interface PathTrieNode {
|
||||||
readonly spelling?: string;
|
|
||||||
kind: "file" | "directory" | "symlink" | "slice" | undefined;
|
kind: "file" | "directory" | "symlink" | "slice" | undefined;
|
||||||
readonly children: Map<string, PathTrieNode>;
|
readonly children: Map<string, PathTrieNode>;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
interface CaseNode {
|
||||||
|
readonly bySpelling: Map<string, CaseNode>;
|
||||||
|
readonly spellingsByFold: Map<string, string[]>;
|
||||||
|
readonly terminals: string[];
|
||||||
|
}
|
||||||
|
|
||||||
|
const emptyCaseNode = (): CaseNode => ({
|
||||||
|
bySpelling: new Map(),
|
||||||
|
spellingsByFold: new Map(),
|
||||||
|
terminals: [],
|
||||||
|
});
|
||||||
|
|
||||||
|
const englishCaseFold = (part: string): string =>
|
||||||
|
part.toLocaleLowerCase("en-US");
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Record English (en-US) Unicode case collisions on each affected occurrence.
|
||||||
|
* Returns the inventory limitation when any logical names collide.
|
||||||
|
*/
|
||||||
|
export const annotateEnglishUnicodeCaseCollisions = (
|
||||||
|
occurrences: readonly {
|
||||||
|
readonly logical_path: string;
|
||||||
|
limitations: string[];
|
||||||
|
}[],
|
||||||
|
): string | undefined => {
|
||||||
|
const collisions = englishUnicodeCaseCollisions(
|
||||||
|
occurrences.map(({ logical_path }) => logical_path),
|
||||||
|
);
|
||||||
|
if (collisions.size === 0) return undefined;
|
||||||
|
for (const occurrence of occurrences) {
|
||||||
|
const others = collisions.get(occurrence.logical_path);
|
||||||
|
if (others === undefined || others.length === 0) continue;
|
||||||
|
occurrence.limitations.push(
|
||||||
|
`Logical path ${occurrence.logical_path} collides under English (en-US) Unicode case folding with ${others.join(", ")}; a case-insensitive destination cannot store both. ${ENGLISH_UNICODE_CASE_NOTE}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION;
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Name the directory-listing spelling that differs from the requested segment
|
||||||
|
* only by English (en-US) Unicode case. Exact matches are the same logical name.
|
||||||
|
*/
|
||||||
|
export const destinationCaseCollisionMessage = (
|
||||||
|
requestedPath: string,
|
||||||
|
existingNames: readonly string[],
|
||||||
|
): string | undefined => {
|
||||||
|
const slash = requestedPath.lastIndexOf("/");
|
||||||
|
const segment = slash < 0 ? requestedPath : requestedPath.slice(slash + 1);
|
||||||
|
if (existingNames.includes(segment)) return undefined;
|
||||||
|
const folded = englishCaseFold(segment);
|
||||||
|
const existing = existingNames
|
||||||
|
.filter((name) => englishCaseFold(name) === folded)
|
||||||
|
.sort(compareUnicodeCodePoints)[0];
|
||||||
|
if (existing === undefined) return undefined;
|
||||||
|
const parent = slash < 0 ? "" : requestedPath.slice(0, slash + 1);
|
||||||
|
return `${DESTINATION_CASE_COLLISION_PREFIX}${requestedPath} and ${parent}${existing}; inventory retains both logical names.`;
|
||||||
|
};
|
||||||
|
|
||||||
|
const englishUnicodeCaseCollisions = (
|
||||||
|
paths: readonly string[],
|
||||||
|
): ReadonlyMap<string, readonly string[]> => {
|
||||||
|
const root = emptyCaseNode();
|
||||||
|
for (const path of paths) {
|
||||||
|
if (path.length === 0 || path === ".") continue;
|
||||||
|
let node = root;
|
||||||
|
for (const part of path.split("/")) {
|
||||||
|
const folded = englishCaseFold(part);
|
||||||
|
const spellings = node.spellingsByFold.get(folded);
|
||||||
|
if (spellings === undefined) node.spellingsByFold.set(folded, [part]);
|
||||||
|
else if (!spellings.includes(part)) spellings.push(part);
|
||||||
|
let child = node.bySpelling.get(part);
|
||||||
|
if (child === undefined) {
|
||||||
|
child = emptyCaseNode();
|
||||||
|
node.bySpelling.set(part, child);
|
||||||
|
}
|
||||||
|
node = child;
|
||||||
|
}
|
||||||
|
node.terminals.push(path);
|
||||||
|
}
|
||||||
|
const collisions = new Map<string, Set<string>>();
|
||||||
|
recordCaseCollisions(root, collisions);
|
||||||
|
return new Map(
|
||||||
|
[...collisions.entries()].map(([path, others]) => [
|
||||||
|
path,
|
||||||
|
[...others].sort(compareUnicodeCodePoints),
|
||||||
|
]),
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
|
const recordCaseCollisions = (
|
||||||
|
node: CaseNode,
|
||||||
|
collisions: Map<string, Set<string>>,
|
||||||
|
): void => {
|
||||||
|
for (const spellings of node.spellingsByFold.values())
|
||||||
|
if (spellings.length > 1) noteFoldedSpellings(node, spellings, collisions);
|
||||||
|
for (const child of node.bySpelling.values())
|
||||||
|
recordCaseCollisions(child, collisions);
|
||||||
|
};
|
||||||
|
|
||||||
|
const noteFoldedSpellings = (
|
||||||
|
node: CaseNode,
|
||||||
|
spellings: readonly string[],
|
||||||
|
collisions: Map<string, Set<string>>,
|
||||||
|
): void => {
|
||||||
|
const groups = spellings.map((spelling) => {
|
||||||
|
const child = node.bySpelling.get(spelling);
|
||||||
|
return child === undefined ? [] : terminalsUnder(child);
|
||||||
|
});
|
||||||
|
for (const [index, group] of groups.entries())
|
||||||
|
noteCaseGroup(
|
||||||
|
group,
|
||||||
|
groups.flatMap((candidate, candidateIndex) =>
|
||||||
|
candidateIndex === index ? [] : candidate,
|
||||||
|
),
|
||||||
|
collisions,
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
|
const noteCaseGroup = (
|
||||||
|
group: readonly string[],
|
||||||
|
others: readonly string[],
|
||||||
|
collisions: Map<string, Set<string>>,
|
||||||
|
): void => {
|
||||||
|
for (const path of group) {
|
||||||
|
const set = collisions.get(path) ?? new Set<string>();
|
||||||
|
for (const other of others) if (other !== path) set.add(other);
|
||||||
|
collisions.set(path, set);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const terminalsUnder = (node: CaseNode): readonly string[] => {
|
||||||
|
const paths = [...node.terminals];
|
||||||
|
for (const child of node.bySpelling.values())
|
||||||
|
paths.push(...terminalsUnder(child));
|
||||||
|
return paths;
|
||||||
|
};
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ import {
|
|||||||
lstat,
|
lstat,
|
||||||
mkdir,
|
mkdir,
|
||||||
open,
|
open,
|
||||||
|
readdir,
|
||||||
realpath,
|
realpath,
|
||||||
rm,
|
rm,
|
||||||
type FileHandle,
|
type FileHandle,
|
||||||
@@ -14,6 +15,7 @@ import { streamChunkToBuffer } from "./StreamBytes.js";
|
|||||||
|
|
||||||
import {
|
import {
|
||||||
ArtifactPathRegistry,
|
ArtifactPathRegistry,
|
||||||
|
destinationCaseCollisionMessage,
|
||||||
normalizeArtifactPath,
|
normalizeArtifactPath,
|
||||||
} from "./ArtifactPaths.js";
|
} from "./ArtifactPaths.js";
|
||||||
import { ArtifactReaderFailure } from "./ArtifactReader.js";
|
import { ArtifactReaderFailure } from "./ArtifactReader.js";
|
||||||
@@ -134,7 +136,8 @@ export class SafeOutputTree {
|
|||||||
constants.O_WRONLY |
|
constants.O_WRONLY |
|
||||||
constants.O_NOFOLLOW,
|
constants.O_NOFOLLOW,
|
||||||
0o600,
|
0o600,
|
||||||
).catch((cause: unknown) => {
|
).catch(async (cause: unknown) => {
|
||||||
|
await throwIfDestinationCaseCollision(dirname(destination), path, cause);
|
||||||
throw new ArtifactReaderFailure(
|
throw new ArtifactReaderFailure(
|
||||||
"path",
|
"path",
|
||||||
`Could not exclusively create extraction path: ${path}`,
|
`Could not exclusively create extraction path: ${path}`,
|
||||||
@@ -245,17 +248,28 @@ export class SafeOutputTree {
|
|||||||
if (fileName === undefined)
|
if (fileName === undefined)
|
||||||
throw new ArtifactReaderFailure("path", "Invalid extraction path");
|
throw new ArtifactReaderFailure("path", "Invalid extraction path");
|
||||||
let current = this.#outputRoot;
|
let current = this.#outputRoot;
|
||||||
|
let logicalParent = "";
|
||||||
for (const part of parts) {
|
for (const part of parts) {
|
||||||
|
const parentDirectory = current;
|
||||||
|
const logicalPath =
|
||||||
|
logicalParent.length === 0 ? part : `${logicalParent}/${part}`;
|
||||||
current = join(current, part);
|
current = join(current, part);
|
||||||
|
let created = true;
|
||||||
await mkdir(current, { mode: 0o700 }).catch((cause: unknown) => {
|
await mkdir(current, { mode: 0o700 }).catch((cause: unknown) => {
|
||||||
if (!isAlreadyExists(cause)) throw cause;
|
if (!isAlreadyExists(cause)) throw cause;
|
||||||
|
created = false;
|
||||||
});
|
});
|
||||||
|
if (!created)
|
||||||
|
await throwIfDestinationCaseCollision(parentDirectory, logicalPath);
|
||||||
const metadata = await lstat(current);
|
const metadata = await lstat(current);
|
||||||
if (!metadata.isDirectory() || metadata.isSymbolicLink())
|
if (!metadata.isDirectory() || metadata.isSymbolicLink()) {
|
||||||
|
await throwIfDestinationCaseCollision(parentDirectory, logicalPath);
|
||||||
throw new ArtifactReaderFailure(
|
throw new ArtifactReaderFailure(
|
||||||
"path",
|
"path",
|
||||||
`Unsafe extraction parent: ${relativePath}`,
|
`Unsafe extraction parent: ${relativePath}`,
|
||||||
);
|
);
|
||||||
|
}
|
||||||
|
logicalParent = logicalPath;
|
||||||
}
|
}
|
||||||
return join(current, fileName);
|
return join(current, fileName);
|
||||||
}
|
}
|
||||||
@@ -330,6 +344,23 @@ const isNotFound = (cause: unknown): boolean =>
|
|||||||
const isAlreadyExists = (cause: unknown): boolean =>
|
const isAlreadyExists = (cause: unknown): boolean =>
|
||||||
cause instanceof Error && "code" in cause && cause.code === "EEXIST";
|
cause instanceof Error && "code" in cause && cause.code === "EEXIST";
|
||||||
|
|
||||||
|
/** Fail when this directory already holds another spelling of the requested segment. */
|
||||||
|
const throwIfDestinationCaseCollision = async (
|
||||||
|
parentDirectory: string,
|
||||||
|
logicalPath: string,
|
||||||
|
cause?: unknown,
|
||||||
|
): Promise<void> => {
|
||||||
|
const names = await readdir(parentDirectory).catch(() => undefined);
|
||||||
|
if (names === undefined) return;
|
||||||
|
const message = destinationCaseCollisionMessage(logicalPath, names);
|
||||||
|
if (message === undefined) return;
|
||||||
|
throw new ArtifactReaderFailure(
|
||||||
|
"path",
|
||||||
|
message,
|
||||||
|
cause === undefined ? undefined : { cause },
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
const abortIfNeeded = (signal?: AbortSignal): void => {
|
const abortIfNeeded = (signal?: AbortSignal): void => {
|
||||||
if (signal?.aborted === true)
|
if (signal?.aborted === true)
|
||||||
throw new ArtifactReaderFailure(
|
throw new ArtifactReaderFailure(
|
||||||
|
|||||||
@@ -73,12 +73,12 @@ export const scanCanonicalArtifactInventory = async (
|
|||||||
try {
|
try {
|
||||||
if (reader instanceof AsarArtifactReader)
|
if (reader instanceof AsarArtifactReader)
|
||||||
await reader.prepareContainer(rootDigest?.sha256, options.signal);
|
await reader.prepareContainer(rootDigest?.sha256, options.signal);
|
||||||
const { nodes, occurrences, pendingContradictions } = await scanReader(
|
const {
|
||||||
reader,
|
nodes,
|
||||||
ownedReaders,
|
occurrences,
|
||||||
options.signal,
|
pendingContradictions,
|
||||||
integrity,
|
caseCollisionLimitation,
|
||||||
);
|
} = await scanReader(reader, ownedReaders, options.signal, integrity);
|
||||||
outcome = {
|
outcome = {
|
||||||
kind: "completed",
|
kind: "completed",
|
||||||
snapshot: await buildInventorySnapshot({
|
snapshot: await buildInventorySnapshot({
|
||||||
@@ -91,6 +91,7 @@ export const scanCanonicalArtifactInventory = async (
|
|||||||
nodes,
|
nodes,
|
||||||
occurrences,
|
occurrences,
|
||||||
pendingContradictions,
|
pendingContradictions,
|
||||||
|
caseCollisionLimitation,
|
||||||
}),
|
}),
|
||||||
};
|
};
|
||||||
} catch (cause: unknown) {
|
} catch (cause: unknown) {
|
||||||
@@ -130,6 +131,7 @@ interface SnapshotBuildInput {
|
|||||||
readonly nodes: Map<string, ArtifactNode>;
|
readonly nodes: Map<string, ArtifactNode>;
|
||||||
readonly occurrences: MutableOccurrence[];
|
readonly occurrences: MutableOccurrence[];
|
||||||
readonly pendingContradictions: PendingIntegrityContradiction[];
|
readonly pendingContradictions: PendingIntegrityContradiction[];
|
||||||
|
readonly caseCollisionLimitation: string | undefined;
|
||||||
}
|
}
|
||||||
|
|
||||||
const buildInventorySnapshot = async (
|
const buildInventorySnapshot = async (
|
||||||
@@ -202,7 +204,12 @@ const buildInventorySnapshot = async (
|
|||||||
edges: orderedEdges,
|
edges: orderedEdges,
|
||||||
provenance: reader?.provenance() ?? [],
|
provenance: reader?.provenance() ?? [],
|
||||||
integrity_contradictions: integrityContradictions,
|
integrity_contradictions: integrityContradictions,
|
||||||
limitations: buildLimitations(rootFormat, reader, integrityContradictions),
|
limitations: buildLimitations(
|
||||||
|
rootFormat,
|
||||||
|
reader,
|
||||||
|
integrityContradictions,
|
||||||
|
input.caseCollisionLimitation,
|
||||||
|
),
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -310,8 +317,10 @@ const buildLimitations = (
|
|||||||
rootFormat: ArtifactOccurrence["artifact_format"],
|
rootFormat: ArtifactOccurrence["artifact_format"],
|
||||||
reader: ArtifactReader | undefined,
|
reader: ArtifactReader | undefined,
|
||||||
integrityContradictions: readonly IntegrityContradiction[],
|
integrityContradictions: readonly IntegrityContradiction[],
|
||||||
|
caseCollisionLimitation: string | undefined,
|
||||||
): string[] => [
|
): string[] => [
|
||||||
...inventoryLimitations(rootFormat, reader),
|
...inventoryLimitations(rootFormat, reader),
|
||||||
|
...(caseCollisionLimitation === undefined ? [] : [caseCollisionLimitation]),
|
||||||
...(integrityContradictions.length === 0
|
...(integrityContradictions.length === 0
|
||||||
? []
|
? []
|
||||||
: [
|
: [
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
import { isAbsolute } from "node:path";
|
import { isAbsolute } from "node:path";
|
||||||
|
|
||||||
import {
|
import {
|
||||||
|
annotateEnglishUnicodeCaseCollisions,
|
||||||
ArtifactPathRegistry,
|
ArtifactPathRegistry,
|
||||||
normalizeArtifactPath,
|
normalizeArtifactPath,
|
||||||
} from "../ArtifactPaths.js";
|
} from "../ArtifactPaths.js";
|
||||||
@@ -64,10 +65,16 @@ export const scanReader = async (
|
|||||||
readonly nodes: Map<string, ArtifactNode>;
|
readonly nodes: Map<string, ArtifactNode>;
|
||||||
readonly occurrences: MutableOccurrence[];
|
readonly occurrences: MutableOccurrence[];
|
||||||
readonly pendingContradictions: PendingIntegrityContradiction[];
|
readonly pendingContradictions: PendingIntegrityContradiction[];
|
||||||
|
readonly caseCollisionLimitation: string | undefined;
|
||||||
}> => {
|
}> => {
|
||||||
const { nodes, occurrences, pendingContradictions } = emptyScan();
|
const { nodes, occurrences, pendingContradictions } = emptyScan();
|
||||||
if (reader === undefined)
|
if (reader === undefined)
|
||||||
return { nodes, occurrences, pendingContradictions };
|
return {
|
||||||
|
nodes,
|
||||||
|
occurrences,
|
||||||
|
pendingContradictions,
|
||||||
|
caseCollisionLimitation: undefined,
|
||||||
|
};
|
||||||
const context: ScanContext = {
|
const context: ScanContext = {
|
||||||
reader,
|
reader,
|
||||||
signal,
|
signal,
|
||||||
@@ -90,7 +97,12 @@ export const scanReader = async (
|
|||||||
context.occurrenceByPath,
|
context.occurrenceByPath,
|
||||||
context.expandedContainerIds,
|
context.expandedContainerIds,
|
||||||
)?.occurrence_id ?? null;
|
)?.occurrence_id ?? null;
|
||||||
return { nodes, occurrences, pendingContradictions };
|
return {
|
||||||
|
nodes,
|
||||||
|
occurrences,
|
||||||
|
pendingContradictions,
|
||||||
|
caseCollisionLimitation: annotateEnglishUnicodeCaseCollisions(occurrences),
|
||||||
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
const visitArtifactEntries = async (
|
const visitArtifactEntries = async (
|
||||||
|
|||||||
@@ -25,6 +25,7 @@ export class LocalWebScriptArtifacts implements WebScriptArtifactPort {
|
|||||||
async load(input: SelectedWebScriptInput, options?: ExecutionOptions) {
|
async load(input: SelectedWebScriptInput, options?: ExecutionOptions) {
|
||||||
let field: readonly (string | number)[] = ["manifest_path"];
|
let field: readonly (string | number)[] = ["manifest_path"];
|
||||||
let targetPath = input.manifest_path;
|
let targetPath = input.manifest_path;
|
||||||
|
let callerSelected = true;
|
||||||
try {
|
try {
|
||||||
const manifestBytes = await readWebArtifactJson(
|
const manifestBytes = await readWebArtifactJson(
|
||||||
input.manifest_path,
|
input.manifest_path,
|
||||||
@@ -58,6 +59,7 @@ export class LocalWebScriptArtifacts implements WebScriptArtifactPort {
|
|||||||
const root = resolve(dirname(input.manifest_path));
|
const root = resolve(dirname(input.manifest_path));
|
||||||
const sourcePath = join(root, "files", portable);
|
const sourcePath = join(root, "files", portable);
|
||||||
targetPath = sourcePath;
|
targetPath = sourcePath;
|
||||||
|
callerSelected = false;
|
||||||
await assertContained(root, sourcePath);
|
await assertContained(root, sourcePath);
|
||||||
const sourceBytes = await readStableArtifact(
|
const sourceBytes = await readStableArtifact(
|
||||||
sourcePath,
|
sourcePath,
|
||||||
@@ -99,7 +101,7 @@ export class LocalWebScriptArtifacts implements WebScriptArtifactPort {
|
|||||||
return err(
|
return err(
|
||||||
webArtifactReadError(
|
webArtifactReadError(
|
||||||
cause,
|
cause,
|
||||||
{ operation: this.operation, field, targetPath },
|
{ operation: this.operation, field, targetPath, callerSelected },
|
||||||
options?.signal,
|
options?.signal,
|
||||||
),
|
),
|
||||||
);
|
);
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js";
|
|||||||
import { readStableArtifact } from "../../artifacts/readStableArtifact.js";
|
import { readStableArtifact } from "../../artifacts/readStableArtifact.js";
|
||||||
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
|
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||||
import {
|
import {
|
||||||
|
AnalysisAccessDeniedError,
|
||||||
AnalysisCancelledError,
|
AnalysisCancelledError,
|
||||||
AnalysisInputError,
|
AnalysisInputError,
|
||||||
} from "../../domain/analysisErrorCore.js";
|
} from "../../domain/analysisErrorCore.js";
|
||||||
@@ -15,6 +16,8 @@ export interface WebArtifactReadContext {
|
|||||||
readonly operation: "trace_web_module_imports" | "trace_web_source_location";
|
readonly operation: "trace_web_module_imports" | "trace_web_source_location";
|
||||||
readonly field: readonly (string | number)[];
|
readonly field: readonly (string | number)[];
|
||||||
readonly targetPath: string;
|
readonly targetPath: string;
|
||||||
|
/** Whether the caller named this file, rather than a manifest deriving it. */
|
||||||
|
readonly callerSelected: boolean;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** A malformed producer document, distinct from host I/O failures. */
|
/** A malformed producer document, distinct from host I/O failures. */
|
||||||
@@ -54,6 +57,10 @@ export const webArtifactReadError = (
|
|||||||
): AnalysisError => {
|
): AnalysisError => {
|
||||||
if (signal?.aborted === true)
|
if (signal?.aborted === true)
|
||||||
return new AnalysisCancelledError(context.operation);
|
return new AnalysisCancelledError(context.operation);
|
||||||
|
const selection = context.callerSelected
|
||||||
|
? selectedFileError(cause, context)
|
||||||
|
: undefined;
|
||||||
|
if (selection !== undefined) return selection;
|
||||||
if (cause instanceof ArtifactReaderFailure)
|
if (cause instanceof ArtifactReaderFailure)
|
||||||
return new ArtifactOperationError(
|
return new ArtifactOperationError(
|
||||||
context.operation,
|
context.operation,
|
||||||
@@ -91,3 +98,33 @@ export const webArtifactReadError = (
|
|||||||
},
|
},
|
||||||
});
|
});
|
||||||
};
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Report a missing, non-regular or unreadable caller-selected file as the
|
||||||
|
* caller's selection or a host permission denial, not an artifact fault.
|
||||||
|
*/
|
||||||
|
const selectedFileError = (
|
||||||
|
cause: unknown,
|
||||||
|
context: WebArtifactReadContext,
|
||||||
|
): AnalysisError | undefined => {
|
||||||
|
const selected = (message: string) =>
|
||||||
|
new AnalysisInputError(context.operation, { cause }, [
|
||||||
|
{ path: context.field, reason: "invalid_value", message },
|
||||||
|
]);
|
||||||
|
if (cause instanceof ArtifactReaderFailure)
|
||||||
|
return cause.reason === "path" ? selected(cause.message) : undefined;
|
||||||
|
const code =
|
||||||
|
cause instanceof Error && "code" in cause ? String(cause.code) : "";
|
||||||
|
if (code === "EACCES" || code === "EPERM")
|
||||||
|
return new AnalysisAccessDeniedError(
|
||||||
|
context.operation,
|
||||||
|
context.targetPath,
|
||||||
|
code,
|
||||||
|
{ cause },
|
||||||
|
);
|
||||||
|
if (code === "ENOENT" || code === "ENOTDIR")
|
||||||
|
return selected(
|
||||||
|
`Selected file could not be read (${code}): ${context.targetPath}`,
|
||||||
|
);
|
||||||
|
return undefined;
|
||||||
|
};
|
||||||
|
|||||||
@@ -48,6 +48,7 @@ export class LocalWebSourceLocationArtifacts implements WebSourceLocationArtifac
|
|||||||
operation: "trace_web_source_location",
|
operation: "trace_web_source_location",
|
||||||
field: ["source_map", "path"],
|
field: ["source_map", "path"],
|
||||||
targetPath: input.source_map.path,
|
targetPath: input.source_map.path,
|
||||||
|
callerSelected: true,
|
||||||
},
|
},
|
||||||
options?.signal,
|
options?.signal,
|
||||||
),
|
),
|
||||||
|
|||||||
@@ -52,6 +52,7 @@ export class LocalWebModuleArtifacts implements WebModuleArtifactPort {
|
|||||||
operation: "trace_web_module_imports",
|
operation: "trace_web_module_imports",
|
||||||
field: ["import_map", "path"],
|
field: ["import_map", "path"],
|
||||||
targetPath: selected.path,
|
targetPath: selected.path,
|
||||||
|
callerSelected: true,
|
||||||
},
|
},
|
||||||
options?.signal,
|
options?.signal,
|
||||||
),
|
),
|
||||||
|
|||||||
@@ -18,7 +18,7 @@ export const registerAnalysisViewCommands = (
|
|||||||
): void => {
|
): void => {
|
||||||
cli.command(CLI_COMMANDS.inspectAnalysisView, {
|
cli.command(CLI_COMMANDS.inspectAnalysisView, {
|
||||||
description:
|
description:
|
||||||
"Project a selected view of completed layout or JavaScript application Evidence JSON",
|
"Project a selected view of completed layout, JavaScript application or native function Evidence JSON",
|
||||||
args: z.object({
|
args: z.object({
|
||||||
inputJson: z.string().describe("Inline workflow JSON or JSON file path"),
|
inputJson: z.string().describe("Inline workflow JSON or JSON file path"),
|
||||||
}),
|
}),
|
||||||
|
|||||||
@@ -123,7 +123,6 @@ const registerAssetCatalogCommand = (
|
|||||||
.default(1_000)
|
.default(1_000)
|
||||||
.describe("Maximum records to return"),
|
.describe("Maximum records to return"),
|
||||||
}),
|
}),
|
||||||
alias: {},
|
|
||||||
run: ({ args, options }) =>
|
run: ({ args, options }) =>
|
||||||
logCliCommand(logger, CLI_COMMANDS.inspectAssetCatalog, () =>
|
logCliCommand(logger, CLI_COMMANDS.inspectAssetCatalog, () =>
|
||||||
runProviderAnalysis(
|
runProviderAnalysis(
|
||||||
@@ -170,11 +169,6 @@ const registerInterfaceBuilderCommand = (
|
|||||||
.default(40_000)
|
.default(40_000)
|
||||||
.describe("Maximum decoded connections"),
|
.describe("Maximum decoded connections"),
|
||||||
}),
|
}),
|
||||||
alias: {
|
|
||||||
maxDocuments: "max-documents",
|
|
||||||
maxObjects: "max-objects",
|
|
||||||
maxConnections: "max-connections",
|
|
||||||
},
|
|
||||||
run: ({ args, options }) =>
|
run: ({ args, options }) =>
|
||||||
logCliCommand(logger, "decode-interface-builder", () =>
|
logCliCommand(logger, "decode-interface-builder", () =>
|
||||||
runProviderAnalysis(
|
runProviderAnalysis(
|
||||||
@@ -233,9 +227,6 @@ const registerInspectionCommand = (
|
|||||||
.default("fail")
|
.default("fail")
|
||||||
.describe("Behavior when declared artifact integrity does not match"),
|
.describe("Behavior when declared artifact integrity does not match"),
|
||||||
}),
|
}),
|
||||||
alias: {
|
|
||||||
integrityPolicy: "integrity-policy",
|
|
||||||
},
|
|
||||||
run: ({ args, options }) =>
|
run: ({ args, options }) =>
|
||||||
logCliCommand(logger, "inspect-artifact", () =>
|
logCliCommand(logger, "inspect-artifact", () =>
|
||||||
runProviderAnalysis(
|
runProviderAnalysis(
|
||||||
|
|||||||
@@ -15,13 +15,14 @@ import {
|
|||||||
type AnalysisInputIssue,
|
type AnalysisInputIssue,
|
||||||
} from "../domain/analysisErrorCore.js";
|
} from "../domain/analysisErrorCore.js";
|
||||||
import { browserCaptureComparisonInputSchema } from "../domain/browserCaptureComparison.js";
|
import { browserCaptureComparisonInputSchema } from "../domain/browserCaptureComparison.js";
|
||||||
|
import { projectInputIssues } from "../domain/inputIssueProjection.js";
|
||||||
import { discoverWebMcpToolsInputSchema } from "../domain/webMcpDiscovery.js";
|
import { discoverWebMcpToolsInputSchema } from "../domain/webMcpDiscovery.js";
|
||||||
import {
|
import {
|
||||||
captureWebScreenshotInputSchema,
|
captureWebScreenshotInputSchema,
|
||||||
compareWebScreenshotsInputSchema,
|
compareWebScreenshotsInputSchema,
|
||||||
} from "../domain/webScreenshot.js";
|
} from "../domain/webScreenshot.js";
|
||||||
import type { JsonValue } from "../domain/jsonValue.js";
|
import type { JsonValue } from "../domain/jsonValue.js";
|
||||||
import { safeParseJson } from "../domain/safeJson.js";
|
import { parseCliJsonInput } from "../cliJsonInput.js";
|
||||||
import type { Logger } from "pino";
|
import type { Logger } from "pino";
|
||||||
import { CLI_COMMANDS } from "../cliCommandNames.js";
|
import { CLI_COMMANDS } from "../cliCommandNames.js";
|
||||||
|
|
||||||
@@ -79,46 +80,39 @@ const registerCaptureDiff = (
|
|||||||
description:
|
description:
|
||||||
"Compare two normalized passive captures or recorded browser scenarios",
|
"Compare two normalized passive captures or recorded browser scenarios",
|
||||||
args: z.object({
|
args: z.object({
|
||||||
beforeJson: z.string().describe("Earlier normalized web capture JSON"),
|
beforeJson: z
|
||||||
afterJson: z.string().describe("Later normalized web capture JSON"),
|
.string()
|
||||||
|
.describe("Earlier normalized web capture JSON or JSON file path"),
|
||||||
|
afterJson: z
|
||||||
|
.string()
|
||||||
|
.describe("Later normalized web capture JSON or JSON file path"),
|
||||||
}),
|
}),
|
||||||
options: z.object({
|
options: z.object({
|
||||||
normalizationJson: z
|
normalizationJson: z
|
||||||
.string()
|
.string()
|
||||||
.default('{"rules":[]}')
|
.default('{"rules":[]}')
|
||||||
.describe(
|
.describe(
|
||||||
"Recorded literal normalization policy for browser scenarios",
|
"Recorded literal normalization policy JSON or JSON file path for browser scenarios",
|
||||||
),
|
),
|
||||||
}),
|
}),
|
||||||
run: ({ args, options }) =>
|
run: ({ args, options }) =>
|
||||||
logCliCommand(logger, "compare-web-captures", async () => {
|
logCliCommand(logger, "compare-web-captures", async () => {
|
||||||
const before = safeParseJson(args.beforeJson);
|
const inputs = await parseJsonArguments("compare_web_captures", {
|
||||||
const after = safeParseJson(args.afterJson);
|
before: args.beforeJson,
|
||||||
const normalization = safeParseJson(options.normalizationJson);
|
after: args.afterJson,
|
||||||
if (!before.ok)
|
normalization: options.normalizationJson,
|
||||||
return inputError("compare_web_captures", [
|
});
|
||||||
invalidJsonIssue("before"),
|
if (!inputs.ok) return inputs.error;
|
||||||
]);
|
const { before, after, normalization } = inputs.value;
|
||||||
if (!after.ok)
|
|
||||||
return inputError("compare_web_captures", [
|
|
||||||
invalidJsonIssue("after"),
|
|
||||||
]);
|
|
||||||
if (!normalization.ok)
|
|
||||||
return inputError("compare_web_captures", [
|
|
||||||
invalidJsonIssue("normalization"),
|
|
||||||
]);
|
|
||||||
const scenarioComparison =
|
const scenarioComparison =
|
||||||
browserCaptureComparisonInputSchema.safeParse({
|
browserCaptureComparisonInputSchema.safeParse({
|
||||||
before_scenario: before.value,
|
before_scenario: before,
|
||||||
after_scenario: after.value,
|
after_scenario: after,
|
||||||
normalization: normalization.value,
|
normalization,
|
||||||
});
|
});
|
||||||
const parsed = scenarioComparison.success
|
const parsed = scenarioComparison.success
|
||||||
? scenarioComparison
|
? scenarioComparison
|
||||||
: browserCaptureComparisonInputSchema.safeParse({
|
: browserCaptureComparisonInputSchema.safeParse({ before, after });
|
||||||
before: before.value,
|
|
||||||
after: after.value,
|
|
||||||
});
|
|
||||||
if (!parsed.success) return inputError("compare_web_captures");
|
if (!parsed.success) return inputError("compare_web_captures");
|
||||||
const result = await compareWebCaptureEvidence(
|
const result = await compareWebCaptureEvidence(
|
||||||
new CdpBrowserProvider(),
|
new CdpBrowserProvider(),
|
||||||
@@ -168,8 +162,12 @@ const registerScreenshotDiff = (
|
|||||||
cli.command(CLI_COMMANDS.compareWebScreenshots, {
|
cli.command(CLI_COMMANDS.compareWebScreenshots, {
|
||||||
description: "Compare two self-verifying PNG artifact JSON values",
|
description: "Compare two self-verifying PNG artifact JSON values",
|
||||||
args: z.object({
|
args: z.object({
|
||||||
beforeJson: z.string().describe("Earlier screenshot artifact JSON"),
|
beforeJson: z
|
||||||
afterJson: z.string().describe("Later screenshot artifact JSON"),
|
.string()
|
||||||
|
.describe("Earlier screenshot artifact JSON or JSON file path"),
|
||||||
|
afterJson: z
|
||||||
|
.string()
|
||||||
|
.describe("Later screenshot artifact JSON or JSON file path"),
|
||||||
}),
|
}),
|
||||||
options: z.object({
|
options: z.object({
|
||||||
channelThreshold: z
|
channelThreshold: z
|
||||||
@@ -182,12 +180,27 @@ const registerScreenshotDiff = (
|
|||||||
}),
|
}),
|
||||||
run: ({ args, options }) =>
|
run: ({ args, options }) =>
|
||||||
logCliCommand(logger, "compare-web-screenshots", async () => {
|
logCliCommand(logger, "compare-web-screenshots", async () => {
|
||||||
const parsed = compareWebScreenshotsInputSchema.safeParse({
|
const operation = "compare_web_screenshots";
|
||||||
before: parseJson(args.beforeJson),
|
const inputs = await parseJsonArguments(operation, {
|
||||||
after: parseJson(args.afterJson),
|
before: args.beforeJson,
|
||||||
channel_threshold: options.channelThreshold,
|
after: args.afterJson,
|
||||||
});
|
});
|
||||||
if (!parsed.success) return inputError("compare_web_screenshots");
|
if (!inputs.ok) return inputs.error;
|
||||||
|
const { before, after } = inputs.value;
|
||||||
|
const input = {
|
||||||
|
before,
|
||||||
|
after,
|
||||||
|
channel_threshold: options.channelThreshold,
|
||||||
|
};
|
||||||
|
const parsed = compareWebScreenshotsInputSchema.safeParse(input);
|
||||||
|
if (!parsed.success)
|
||||||
|
return browserCliError(
|
||||||
|
new AnalysisInputError(
|
||||||
|
operation,
|
||||||
|
{ cause: parsed.error },
|
||||||
|
projectInputIssues(parsed.error.issues, input),
|
||||||
|
),
|
||||||
|
);
|
||||||
const result = await compareWebScreenshotEvidence(
|
const result = await compareWebScreenshotEvidence(
|
||||||
new CdpBrowserProvider(),
|
new CdpBrowserProvider(),
|
||||||
parsed.data,
|
parsed.data,
|
||||||
@@ -197,11 +210,40 @@ const registerScreenshotDiff = (
|
|||||||
});
|
});
|
||||||
};
|
};
|
||||||
|
|
||||||
const parseJson = (value: string): unknown => {
|
/**
|
||||||
const parsed = safeParseJson(value);
|
* Parse each named inline JSON argument or local JSON file, like other
|
||||||
return parsed.ok ? parsed.value : undefined;
|
* JSON-input commands. Screenshot artifacts and scenario captures embed PNG
|
||||||
|
* bytes that can exceed the host's command-line length limit.
|
||||||
|
*/
|
||||||
|
const parseJsonArguments = async (
|
||||||
|
operation: string,
|
||||||
|
fields: Readonly<Record<string, string>>,
|
||||||
|
): Promise<
|
||||||
|
| { readonly ok: true; readonly value: Readonly<Record<string, unknown>> }
|
||||||
|
| { readonly ok: false; readonly error: JsonValue }
|
||||||
|
> => {
|
||||||
|
const values: Record<string, unknown> = {};
|
||||||
|
for (const [field, value] of Object.entries(fields)) {
|
||||||
|
const parsed = await parseCliJsonInput(value, operation);
|
||||||
|
if (!parsed.ok)
|
||||||
|
return isFileInputFailure(parsed.error)
|
||||||
|
? parsed
|
||||||
|
: {
|
||||||
|
ok: false,
|
||||||
|
error: inputError(operation, [invalidJsonIssue(field)]),
|
||||||
|
};
|
||||||
|
values[field] = parsed.value;
|
||||||
|
}
|
||||||
|
return { ok: true, value: values };
|
||||||
};
|
};
|
||||||
|
|
||||||
|
/** File read and decode failures name the selected path; inline JSON does not. */
|
||||||
|
const isFileInputFailure = (error: JsonValue): boolean =>
|
||||||
|
typeof error === "object" &&
|
||||||
|
error !== null &&
|
||||||
|
!Array.isArray(error) &&
|
||||||
|
"input_path" in error;
|
||||||
|
|
||||||
const invalidJsonIssue = (field: string): AnalysisInputIssue => ({
|
const invalidJsonIssue = (field: string): AnalysisInputIssue => ({
|
||||||
path: [field],
|
path: [field],
|
||||||
reason: "invalid_format",
|
reason: "invalid_format",
|
||||||
|
|||||||
@@ -54,7 +54,6 @@ const registerOverviewCommands = (
|
|||||||
options: overviewOptions.extend({
|
options: overviewOptions.extend({
|
||||||
integrityPolicy: javascriptApplicationOptions.shape.integrityPolicy,
|
integrityPolicy: javascriptApplicationOptions.shape.integrityPolicy,
|
||||||
}),
|
}),
|
||||||
alias: { integrityPolicy: "integrity-policy" },
|
|
||||||
run: async ({ args, options }) => {
|
run: async ({ args, options }) => {
|
||||||
// Route JavaScript targets exactly like analyze-javascript-application,
|
// Route JavaScript targets exactly like analyze-javascript-application,
|
||||||
// including typed cancellation; cancellation wraps logging so its exit
|
// including typed cancellation; cancellation wraps logging so its exit
|
||||||
@@ -274,9 +273,6 @@ const registerTraceCommand = (
|
|||||||
"target-format": formatSelectionOption,
|
"target-format": formatSelectionOption,
|
||||||
provider: providerSelectionOption,
|
provider: providerSelectionOption,
|
||||||
}),
|
}),
|
||||||
alias: {
|
|
||||||
caseSensitive: "case-sensitive",
|
|
||||||
},
|
|
||||||
run: ({ args, options }) =>
|
run: ({ args, options }) =>
|
||||||
logCliCommand(logger, "trace", () =>
|
logCliCommand(logger, "trace", () =>
|
||||||
runDirectAnalysis(
|
runDirectAnalysis(
|
||||||
@@ -455,7 +451,6 @@ const registerSearchCommand = (
|
|||||||
"target-format": formatSelectionOption,
|
"target-format": formatSelectionOption,
|
||||||
provider: providerSelectionOption,
|
provider: providerSelectionOption,
|
||||||
}),
|
}),
|
||||||
alias: { caseSensitive: "case-sensitive" },
|
|
||||||
run: ({ args, options }) =>
|
run: ({ args, options }) =>
|
||||||
logCliCommand(logger, "search", () =>
|
logCliCommand(logger, "search", () =>
|
||||||
runDirectAnalysis(
|
runDirectAnalysis(
|
||||||
|
|||||||
@@ -157,13 +157,6 @@ export const registerCoreNativeCommands = (
|
|||||||
"target-format": formatSelectionOption,
|
"target-format": formatSelectionOption,
|
||||||
provider: providerSelectionOption,
|
provider: providerSelectionOption,
|
||||||
}),
|
}),
|
||||||
alias: {
|
|
||||||
maxDepth: "max-depth",
|
|
||||||
maxFunctions: "max-functions",
|
|
||||||
maxCallSites: "max-call-sites",
|
|
||||||
maxNodes: "max-nodes",
|
|
||||||
maxEdges: "max-edges",
|
|
||||||
},
|
|
||||||
run: ({ args, options }) =>
|
run: ({ args, options }) =>
|
||||||
logCliCommand(logger, CLI_COMMANDS.traceNativeValues, () =>
|
logCliCommand(logger, CLI_COMMANDS.traceNativeValues, () =>
|
||||||
runDirectAnalysis(
|
runDirectAnalysis(
|
||||||
@@ -283,7 +276,6 @@ const registerNativeDispatchMetadataCommand = (
|
|||||||
"target-format": formatSelectionOption,
|
"target-format": formatSelectionOption,
|
||||||
provider: providerSelectionOption,
|
provider: providerSelectionOption,
|
||||||
}),
|
}),
|
||||||
alias: { maxRecords: "max-records" },
|
|
||||||
run: async ({ args, options }) =>
|
run: async ({ args, options }) =>
|
||||||
logCliCommand(logger, "inspect-native-dispatch-metadata", () =>
|
logCliCommand(logger, "inspect-native-dispatch-metadata", () =>
|
||||||
runDirectAnalysis(
|
runDirectAnalysis(
|
||||||
@@ -343,11 +335,6 @@ const registerNativeUiActionCommand = (
|
|||||||
"target-format": formatSelectionOption,
|
"target-format": formatSelectionOption,
|
||||||
provider: providerSelectionOption,
|
provider: providerSelectionOption,
|
||||||
}),
|
}),
|
||||||
alias: {
|
|
||||||
maxDepth: "max-depth",
|
|
||||||
maxNodes: "max-nodes",
|
|
||||||
maxEdges: "max-edges",
|
|
||||||
},
|
|
||||||
run: async ({ args, options }) =>
|
run: async ({ args, options }) =>
|
||||||
logCliCommand(logger, "trace-native-ui-action", () =>
|
logCliCommand(logger, "trace-native-ui-action", () =>
|
||||||
runDirectAnalysis(
|
runDirectAnalysis(
|
||||||
|
|||||||
@@ -190,7 +190,6 @@ const registerJavaScriptApplicationCommand = (
|
|||||||
path: z.string().describe("ASAR or extracted application path"),
|
path: z.string().describe("ASAR or extracted application path"),
|
||||||
}),
|
}),
|
||||||
options: javascriptApplicationOptions,
|
options: javascriptApplicationOptions,
|
||||||
alias: { integrityPolicy: "integrity-policy" },
|
|
||||||
run: ({ args, options }) =>
|
run: ({ args, options }) =>
|
||||||
withCommandCancellation((signal) =>
|
withCommandCancellation((signal) =>
|
||||||
logCliCommand(logger, CLI_COMMANDS.analyzeJavaScriptApplication, () =>
|
logCliCommand(logger, CLI_COMMANDS.analyzeJavaScriptApplication, () =>
|
||||||
|
|||||||
+15
-3
@@ -4,16 +4,27 @@ import { bufferedJsonParts, jsonParts } from "../domain/jsonSerialization.js";
|
|||||||
|
|
||||||
type JsonOutputFormat = "json" | "jsonl";
|
type JsonOutputFormat = "json" | "jsonl";
|
||||||
|
|
||||||
/** Write ordinary JSON without constructing a document-sized string. */
|
/**
|
||||||
|
* Write ordinary JSON without constructing a document-sized string.
|
||||||
|
*
|
||||||
|
* `json` is indented for a terminal reader. Piped or redirected output, which
|
||||||
|
* agents and scripts consume, is compact: indentation adds no information and
|
||||||
|
* measured 16-45% more model tokens on real results. The value is identical.
|
||||||
|
*/
|
||||||
export const writeJsonOutput = async (
|
export const writeJsonOutput = async (
|
||||||
value: unknown,
|
value: unknown,
|
||||||
destination: Writable,
|
destination: Writable,
|
||||||
format: JsonOutputFormat = "json",
|
format: JsonOutputFormat = "json",
|
||||||
): Promise<void> => {
|
): Promise<void> => {
|
||||||
for (const part of bufferedJsonParts(jsonOutputParts(value, format)))
|
for (const part of bufferedJsonParts(
|
||||||
|
jsonOutputParts(value, format, isTerminal(destination)),
|
||||||
|
))
|
||||||
await writePart(destination, part);
|
await writePart(destination, part);
|
||||||
};
|
};
|
||||||
|
|
||||||
|
const isTerminal = (destination: Writable): boolean =>
|
||||||
|
"isTTY" in destination && destination.isTTY === true;
|
||||||
|
|
||||||
const writePart = (destination: Writable, part: string): Promise<void> =>
|
const writePart = (destination: Writable, part: string): Promise<void> =>
|
||||||
new Promise((resolve, reject) => {
|
new Promise((resolve, reject) => {
|
||||||
const onError = (cause: unknown): void => reject(cause);
|
const onError = (cause: unknown): void => reject(cause);
|
||||||
@@ -39,6 +50,7 @@ const writePart = (destination: Writable, part: string): Promise<void> =>
|
|||||||
export function* jsonOutputParts(
|
export function* jsonOutputParts(
|
||||||
value: unknown,
|
value: unknown,
|
||||||
format: JsonOutputFormat,
|
format: JsonOutputFormat,
|
||||||
|
pretty = true,
|
||||||
): Generator<string> {
|
): Generator<string> {
|
||||||
if (format === "jsonl" && Array.isArray(value)) {
|
if (format === "jsonl" && Array.isArray(value)) {
|
||||||
for (const item of value) {
|
for (const item of value) {
|
||||||
@@ -47,6 +59,6 @@ export function* jsonOutputParts(
|
|||||||
}
|
}
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
yield* jsonParts(value, format === "json");
|
yield* jsonParts(value, pretty && format === "json");
|
||||||
yield "\n";
|
yield "\n";
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -148,7 +148,6 @@ const registerMaintenanceCommands = (
|
|||||||
.default(false)
|
.default(false)
|
||||||
.describe("Also remove REA caches and state"),
|
.describe("Also remove REA caches and state"),
|
||||||
}),
|
}),
|
||||||
alias: { purgeData: "purge-data" },
|
|
||||||
run: ({ options }) =>
|
run: ({ options }) =>
|
||||||
logCliCommand(
|
logCliCommand(
|
||||||
logger,
|
logger,
|
||||||
|
|||||||
@@ -65,10 +65,6 @@ export const registerUtilityCommands = (
|
|||||||
.default(500)
|
.default(500)
|
||||||
.describe("Maximum accessibility nodes per capture"),
|
.describe("Maximum accessibility nodes per capture"),
|
||||||
}),
|
}),
|
||||||
alias: {
|
|
||||||
windowId: "window-id",
|
|
||||||
maxNodes: "max-nodes",
|
|
||||||
},
|
|
||||||
run: ({ args, options }) =>
|
run: ({ args, options }) =>
|
||||||
logCliCommand(logger, command, async () => {
|
logCliCommand(logger, command, async () => {
|
||||||
const parameters = {
|
const parameters = {
|
||||||
@@ -172,7 +168,6 @@ const registerNativeCommands = (
|
|||||||
"Plist path relative to the app root (default: Contents/Info.plist)",
|
"Plist path relative to the app root (default: Contents/Info.plist)",
|
||||||
),
|
),
|
||||||
}),
|
}),
|
||||||
alias: { relativePath: "relative-path" },
|
|
||||||
run: ({ args, options }) =>
|
run: ({ args, options }) =>
|
||||||
logCliCommand(logger, "inspect-plist", () =>
|
logCliCommand(logger, "inspect-plist", () =>
|
||||||
runProviderAnalysis(
|
runProviderAnalysis(
|
||||||
|
|||||||
@@ -2,7 +2,6 @@ import { Cli } from "incur";
|
|||||||
import { z } from "zod";
|
import { z } from "zod";
|
||||||
|
|
||||||
import { captureBrowserScenario } from "./application/BrowserScenarioCaptureService.js";
|
import { captureBrowserScenario } from "./application/BrowserScenarioCaptureService.js";
|
||||||
import { createBrowserScenarioProvider } from "./composition/browserScenario.js";
|
|
||||||
import { CLI_COMMANDS } from "./cliCommandNames.js";
|
import { CLI_COMMANDS } from "./cliCommandNames.js";
|
||||||
import { parseCliJsonInput, resolveCliJsonPaths } from "./cliJsonInput.js";
|
import { parseCliJsonInput, resolveCliJsonPaths } from "./cliJsonInput.js";
|
||||||
import { logCliCommand } from "./cliLogging.js";
|
import { logCliCommand } from "./cliLogging.js";
|
||||||
@@ -44,6 +43,9 @@ export const registerBrowserScenarioCommands = (
|
|||||||
projectInputIssues(scenario.error.issues, input.value),
|
projectInputIssues(scenario.error.issues, input.value),
|
||||||
),
|
),
|
||||||
);
|
);
|
||||||
|
// Load Playwright only for this command, not during CLI startup.
|
||||||
|
const { createBrowserScenarioProvider } =
|
||||||
|
await import("./composition/browserScenario.js");
|
||||||
const result = await captureBrowserScenario(
|
const result = await captureBrowserScenario(
|
||||||
createBrowserScenarioProvider(environment),
|
createBrowserScenarioProvider(environment),
|
||||||
scenario.data,
|
scenario.data,
|
||||||
|
|||||||
@@ -8,14 +8,14 @@ import { evidenceResultOf } from "./toolOutputSchemaPrimitives.js";
|
|||||||
|
|
||||||
const RETAINED_EXAMPLE_ID = `ev_${"a".repeat(64)}`;
|
const RETAINED_EXAMPLE_ID = `ev_${"a".repeat(64)}`;
|
||||||
|
|
||||||
/** Selected views of already completed layout and JavaScript analysis Evidence. */
|
/** Selected views of already completed layout, JavaScript and native function Evidence. */
|
||||||
export const ANALYSIS_VIEW_TOOL_CONTRACTS = [
|
export const ANALYSIS_VIEW_TOOL_CONTRACTS = [
|
||||||
{
|
{
|
||||||
name: "inspect_analysis_view",
|
name: "inspect_analysis_view",
|
||||||
...toolContractMetadata("inspect_analysis_view"),
|
...toolContractMetadata("inspect_analysis_view"),
|
||||||
kind: "application",
|
kind: "application",
|
||||||
description:
|
description:
|
||||||
"Inspect a selected view of completed binary-layout or JavaScript application Evidence without repeating analysis. Use an exact same-session retained evidence_id or portable inline Evidence. Select a summary, layout mitigations/linkage facet, one section/symbol/module, or a stable page with a positive limit. Module pages contain JavaScript assets, bundled modules and source modules; use node_id when paths are ambiguous or unavailable. Returns selected facts inline with artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the retained Evidence when needed.",
|
"Inspect selected views of completed binary-layout, JavaScript application or native analyze_function Evidence without repeating analysis. Use an exact same-session retained evidence_id or portable inline Evidence. Select a summary, layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Native views select procedure, pseudocode, assembly, callers/callees, references and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.",
|
||||||
inputSchema: inspectAnalysisViewInputSchema,
|
inputSchema: inspectAnalysisViewInputSchema,
|
||||||
outputSchema: evidenceResultOf(analysisViewResultSchema),
|
outputSchema: evidenceResultOf(analysisViewResultSchema),
|
||||||
examples: [
|
examples: [
|
||||||
@@ -43,6 +43,21 @@ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [
|
|||||||
},
|
},
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
title: "Page native high-pcode operations from retained Evidence",
|
||||||
|
input: {
|
||||||
|
source: {
|
||||||
|
kind: "retained-evidence",
|
||||||
|
evidence_id: RETAINED_EXAMPLE_ID,
|
||||||
|
},
|
||||||
|
view: {
|
||||||
|
kind: "native",
|
||||||
|
facet: "value_flow_operations",
|
||||||
|
offset: 0,
|
||||||
|
limit: 32,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
{
|
{
|
||||||
title: "Page module identities from retained application Evidence",
|
title: "Page module identities from retained application Evidence",
|
||||||
input: {
|
input: {
|
||||||
|
|||||||
@@ -63,7 +63,7 @@ const artifact = <
|
|||||||
export const ARTIFACT_TOOL_CONTRACTS = [
|
export const ARTIFACT_TOOL_CONTRACTS = [
|
||||||
artifact(
|
artifact(
|
||||||
"inspect_artifact",
|
"inspect_artifact",
|
||||||
"Inspect an active archive or application package in one call. Returns the complete content-addressed artifact graph, source Evidence, observations, relationships, integrity contradictions, hypotheses, unexplored branches, limitations, and next probes inline. Read-only DMG mounting is used automatically on supported macOS hosts. Integrity mismatches fail by default; record-and-continue keeps mismatches explicitly untrusted. It does not extract files; use extract_artifact to materialize its regular contents.",
|
"Inspect an active archive or application package in one call. Returns the complete content-addressed artifact graph once, as the nested inventory Evidence, with root and integrity observations, integrity contradictions, hypotheses, unexplored branches, limitations, and next probes inline. Read-only DMG mounting is used automatically on supported macOS hosts. Integrity mismatches fail by default; record-and-continue keeps mismatches explicitly untrusted. It does not extract files; use extract_artifact to materialize its regular contents.",
|
||||||
artifactInventoryInputSchema,
|
artifactInventoryInputSchema,
|
||||||
),
|
),
|
||||||
artifact(
|
artifact(
|
||||||
|
|||||||
@@ -59,7 +59,7 @@ const referencedDescription = (
|
|||||||
const describeSchema = (
|
const describeSchema = (
|
||||||
schema: unknown,
|
schema: unknown,
|
||||||
root: Readonly<Record<string, unknown>>,
|
root: Readonly<Record<string, unknown>>,
|
||||||
describeProperty: (property: string) => string,
|
describeProperty: (property: string) => string | undefined,
|
||||||
): unknown => {
|
): unknown => {
|
||||||
if (!isObject(schema)) return schema;
|
if (!isObject(schema)) return schema;
|
||||||
return Object.fromEntries(
|
return Object.fromEntries(
|
||||||
@@ -70,16 +70,16 @@ const describeSchema = (
|
|||||||
Object.fromEntries(
|
Object.fromEntries(
|
||||||
Object.entries(value).map(([name, child]) => {
|
Object.entries(value).map(([name, child]) => {
|
||||||
const described = describeSchema(child, root, describeProperty);
|
const described = describeSchema(child, root, describeProperty);
|
||||||
|
if (key !== "properties" || !isObject(described))
|
||||||
|
return [name, described];
|
||||||
|
const description =
|
||||||
|
referencedDescription(root, described) ??
|
||||||
|
describeProperty(name);
|
||||||
return [
|
return [
|
||||||
name,
|
name,
|
||||||
key === "properties" && isObject(described)
|
description === undefined
|
||||||
? {
|
? described
|
||||||
...described,
|
: { ...described, description },
|
||||||
description:
|
|
||||||
referencedDescription(root, described) ??
|
|
||||||
describeProperty(name),
|
|
||||||
}
|
|
||||||
: described,
|
|
||||||
];
|
];
|
||||||
}),
|
}),
|
||||||
),
|
),
|
||||||
@@ -223,7 +223,7 @@ const flattenRootObjectUnion = (
|
|||||||
/** Describe canonical input fields and present the root as one object. */
|
/** Describe canonical input fields and present the root as one object. */
|
||||||
export const presentInputJsonSchema = (
|
export const presentInputJsonSchema = (
|
||||||
root: Record<string, unknown>,
|
root: Record<string, unknown>,
|
||||||
describeProperty: (property: string) => string,
|
describeProperty: (property: string) => string | undefined,
|
||||||
): Record<string, unknown> => {
|
): Record<string, unknown> => {
|
||||||
const flattened = flattenRootObjectUnion(root);
|
const flattened = flattenRootObjectUnion(root);
|
||||||
const described = describeSchema(flattened, flattened, describeProperty);
|
const described = describeSchema(flattened, flattened, describeProperty);
|
||||||
|
|||||||
@@ -97,9 +97,14 @@ describe("advertised input references", () => {
|
|||||||
before: { description: "Exact byte range in the selected artifact." },
|
before: { description: "Exact byte range in the selected artifact." },
|
||||||
after: { description: "Exact byte range in the selected artifact." },
|
after: { description: "Exact byte range in the selected artifact." },
|
||||||
metadata: { default: literal, examples: [literal] },
|
metadata: { default: literal, examples: [literal] },
|
||||||
constructor: { description: "Value for constructor." },
|
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
// Inherited object keys never supply a description.
|
||||||
|
expect(inline).not.toHaveProperty([
|
||||||
|
"properties",
|
||||||
|
"constructor",
|
||||||
|
"description",
|
||||||
|
]);
|
||||||
const ajv =
|
const ajv =
|
||||||
target === "draft-2020-12"
|
target === "draft-2020-12"
|
||||||
? new Ajv2020({ strict: false })
|
? new Ajv2020({ strict: false })
|
||||||
|
|||||||
@@ -1,11 +1,69 @@
|
|||||||
import { Ajv } from "ajv";
|
import { Ajv } from "ajv";
|
||||||
import { Ajv2020 } from "ajv/dist/2020.js";
|
import { Ajv2020 } from "ajv/dist/2020.js";
|
||||||
|
import { AjvJsonSchemaValidator } from "@modelcontextprotocol/client/validators/ajv";
|
||||||
import { expect, it } from "vitest";
|
import { expect, it } from "vitest";
|
||||||
import { z } from "zod";
|
import { z } from "zod";
|
||||||
|
|
||||||
import { toolContract } from "./toolContracts.js";
|
import { toolContract } from "./toolContracts.js";
|
||||||
import { toolOutputSchemaWithMetadata } from "./toolSchemaMetadata.js";
|
import { toolOutputSchemaWithMetadata } from "./toolSchemaMetadata.js";
|
||||||
|
|
||||||
|
it("keeps dialect-specific validators distinct after JSON schema serialization", () => {
|
||||||
|
const contract = {
|
||||||
|
...toolContract("binary_session"),
|
||||||
|
outputSchema: z.strictObject({ value: z.number().int().min(1).max(8) }),
|
||||||
|
};
|
||||||
|
const output = toolOutputSchemaWithMetadata(contract);
|
||||||
|
const provider = new AjvJsonSchemaValidator();
|
||||||
|
for (const target of ["draft-2020-12", "draft-07"] as const) {
|
||||||
|
const projected = output["~standard"].jsonSchema.output({ target });
|
||||||
|
const parsed = z
|
||||||
|
.record(z.string(), z.unknown())
|
||||||
|
.parse(JSON.parse(JSON.stringify(projected)));
|
||||||
|
const validate = provider.getValidator(parsed);
|
||||||
|
expect(validate({ value: 8 }).valid).toBe(true);
|
||||||
|
expect(validate({ value: 9 }).valid).toBe(false);
|
||||||
|
expect(validate({ value: 1.5 }).valid).toBe(false);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("preserves explicit schema identity and relative reference resolution", () => {
|
||||||
|
const value = z.string();
|
||||||
|
const canonical = z.strictObject({ value });
|
||||||
|
const contract = {
|
||||||
|
...toolContract("binary_session"),
|
||||||
|
outputSchema: canonical,
|
||||||
|
};
|
||||||
|
const projected = toolOutputSchemaWithMetadata(contract)[
|
||||||
|
"~standard"
|
||||||
|
].jsonSchema.output({
|
||||||
|
target: "draft-2020-12",
|
||||||
|
libraryOptions: {
|
||||||
|
override: (context: {
|
||||||
|
readonly zodSchema: unknown;
|
||||||
|
readonly jsonSchema: Record<string, unknown>;
|
||||||
|
}) => {
|
||||||
|
if (context.zodSchema !== value) return;
|
||||||
|
delete context.jsonSchema.type;
|
||||||
|
context.jsonSchema.$ref = "child.json";
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
|
const ajv = new Ajv2020({ strict: false });
|
||||||
|
ajv.addSchema({ $id: "child.json", type: "string", minLength: 3 });
|
||||||
|
const validate = ajv.compile(projected);
|
||||||
|
expect(validate({ value: "abc" })).toBe(true);
|
||||||
|
expect(validate({ value: "ab" })).toBe(false);
|
||||||
|
|
||||||
|
const identified = toolOutputSchemaWithMetadata({
|
||||||
|
...contract,
|
||||||
|
outputSchema: canonical.meta({ $id: "https://example.test/result" }),
|
||||||
|
})["~standard"].jsonSchema.output({ target: "draft-2020-12" });
|
||||||
|
expect(identified.$id).toBe("https://example.test/result");
|
||||||
|
const explicit = ajv.compile(identified);
|
||||||
|
expect(explicit({ value: "abc" })).toBe(true);
|
||||||
|
expect(explicit({ value: 1 })).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
it.each(["draft-2020-12", "draft-07"] as const)(
|
it.each(["draft-2020-12", "draft-07"] as const)(
|
||||||
"preserves refined numeric bounds and strict repeated objects through %s references",
|
"preserves refined numeric bounds and strict repeated objects through %s references",
|
||||||
(target) => {
|
(target) => {
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ import { z } from "zod";
|
|||||||
|
|
||||||
import type { ToolContract } from "./toolContractTypes.js";
|
import type { ToolContract } from "./toolContractTypes.js";
|
||||||
import { presentInputJsonSchema } from "./inputSchemaPresentation.js";
|
import { presentInputJsonSchema } from "./inputSchemaPresentation.js";
|
||||||
|
import { digestCanonicalValue } from "../domain/canonicalDigest.js";
|
||||||
|
|
||||||
const PROPERTY_DESCRIPTIONS: Readonly<Record<string, string>> = {
|
const PROPERTY_DESCRIPTIONS: Readonly<Record<string, string>> = {
|
||||||
addresses: "Ordered provider-normalized procedure addresses to analyze.",
|
addresses: "Ordered provider-normalized procedure addresses to analyze.",
|
||||||
@@ -69,20 +70,55 @@ export const toolInputSchemaWithMetadata = <Contract extends ToolContract>(
|
|||||||
},
|
},
|
||||||
);
|
);
|
||||||
|
|
||||||
/** Share repeated output definitions without changing canonical validation. */
|
/** Share output definitions and content-bound validator identities on the wire. */
|
||||||
export const toolOutputSchemaWithMetadata = <Contract extends ToolContract>(
|
export const toolOutputSchemaWithMetadata = <Contract extends ToolContract>(
|
||||||
contract: Contract,
|
contract: Contract,
|
||||||
): Contract["outputSchema"] =>
|
): Contract["outputSchema"] =>
|
||||||
withAdvertisedJsonSchema(
|
withAdvertisedJsonSchema(
|
||||||
contract.outputSchema,
|
contract.outputSchema,
|
||||||
"output",
|
"output",
|
||||||
(project) => (options) =>
|
(project) => (options) => {
|
||||||
project({
|
const shared = project({
|
||||||
...options,
|
...options,
|
||||||
libraryOptions: { reused: "ref", ...options.libraryOptions },
|
libraryOptions: { reused: "ref", ...options.libraryOptions },
|
||||||
}),
|
});
|
||||||
|
// An explicit ID owns the reference base. Relative external references
|
||||||
|
// also depend on that base, so leave those projections unchanged.
|
||||||
|
if ("$id" in shared || hasRelativeSchemaReference(shared)) return shared;
|
||||||
|
return {
|
||||||
|
...shared,
|
||||||
|
$id: `urn:rea:tool-output:sha256:${digestCanonicalValue(
|
||||||
|
{ target: options.target, schema: shared },
|
||||||
|
"Tool output schema",
|
||||||
|
)}`,
|
||||||
|
};
|
||||||
|
},
|
||||||
);
|
);
|
||||||
|
|
||||||
|
const hasRelativeSchemaReference = (
|
||||||
|
schema: Record<string, unknown>,
|
||||||
|
): boolean => {
|
||||||
|
const pending: unknown[] = [schema];
|
||||||
|
const visited = new WeakSet<object>();
|
||||||
|
while (pending.length > 0) {
|
||||||
|
const current = pending.pop();
|
||||||
|
if (current === null || typeof current !== "object" || visited.has(current))
|
||||||
|
continue;
|
||||||
|
visited.add(current);
|
||||||
|
for (const [key, value] of Object.entries(current)) {
|
||||||
|
if (
|
||||||
|
["$id", "$ref", "$dynamicRef", "$recursiveRef"].includes(key) &&
|
||||||
|
typeof value === "string" &&
|
||||||
|
!value.startsWith("#") &&
|
||||||
|
!URL.canParse(value)
|
||||||
|
)
|
||||||
|
return true;
|
||||||
|
if (value !== null && typeof value === "object") pending.push(value);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
};
|
||||||
|
|
||||||
const withAdvertisedJsonSchema = <Schema extends z.ZodType>(
|
const withAdvertisedJsonSchema = <Schema extends z.ZodType>(
|
||||||
canonical: Schema,
|
canonical: Schema,
|
||||||
io: "input" | "output",
|
io: "input" | "output",
|
||||||
@@ -128,7 +164,7 @@ const memoizeByTarget = (
|
|||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
const fallbackPropertyDescription = (property: string): string => {
|
const fallbackPropertyDescription = (property: string): string | undefined => {
|
||||||
const explicit = Object.hasOwn(PROPERTY_DESCRIPTIONS, property)
|
const explicit = Object.hasOwn(PROPERTY_DESCRIPTIONS, property)
|
||||||
? PROPERTY_DESCRIPTIONS[property]
|
? PROPERTY_DESCRIPTIONS[property]
|
||||||
: undefined;
|
: undefined;
|
||||||
@@ -156,5 +192,6 @@ const fallbackPropertyDescription = (property: string): string => {
|
|||||||
return `Local filesystem ${words} selected for this operation.`;
|
return `Local filesystem ${words} selected for this operation.`;
|
||||||
if (property.startsWith("is_") || property.startsWith("has_"))
|
if (property.startsWith("is_") || property.startsWith("has_"))
|
||||||
return `Whether ${words}.`;
|
return `Whether ${words}.`;
|
||||||
return `Value for ${words}.`;
|
// A generic sentence would only restate the property name in every schema.
|
||||||
|
return undefined;
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -191,7 +191,7 @@ export const analysisErrorUserMessage = (error: AnalysisError): string => {
|
|||||||
const standardMessage = standardErrorMessage(error._tag);
|
const standardMessage = standardErrorMessage(error._tag);
|
||||||
if (standardMessage !== undefined) return standardMessage;
|
if (standardMessage !== undefined) return standardMessage;
|
||||||
if (error instanceof ArtifactOperationError)
|
if (error instanceof ArtifactOperationError)
|
||||||
return artifactMessage(error.operation, error.reason);
|
return artifactMessage(error.operation, error.reason, error.detail);
|
||||||
if (error instanceof EvidenceReferenceError)
|
if (error instanceof EvidenceReferenceError)
|
||||||
return error.reason === "missing"
|
return error.reason === "missing"
|
||||||
? `Evidence ${error.evidenceId} is not retained in this session. Supply complete inline Evidence, re-run its producer, or import its bundle before using this reference.`
|
? `Evidence ${error.evidenceId} is not retained in this session. Supply complete inline Evidence, re-run its producer, or import its bundle before using this reference.`
|
||||||
@@ -329,13 +329,13 @@ const START_FAILURE_TAGS: ReadonlySet<AnalysisErrorTag> = new Set([
|
|||||||
const artifactMessage = (
|
const artifactMessage = (
|
||||||
operation: ArtifactOperationError["operation"],
|
operation: ArtifactOperationError["operation"],
|
||||||
reason: ArtifactOperationError["reason"],
|
reason: ArtifactOperationError["reason"],
|
||||||
|
detail?: string,
|
||||||
): string => {
|
): string => {
|
||||||
if (reason === "cancelled")
|
if (reason === "cancelled")
|
||||||
return "Artifact operation was cancelled. Start it again when ready.";
|
return "Artifact operation was cancelled. Start it again when ready.";
|
||||||
if (reason === "limit")
|
if (reason === "limit")
|
||||||
return "Artifact is too large to process safely. Narrow the requested path or use a smaller artifact.";
|
return "Artifact is too large to process safely. Narrow the requested path or use a smaller artifact.";
|
||||||
if (reason === "path")
|
if (reason === "path") return artifactPathMessage(detail);
|
||||||
return "Artifact contains an unsafe or conflicting internal path. Inspect the reported path and correct the artifact before retrying.";
|
|
||||||
if (reason === "unavailable")
|
if (reason === "unavailable")
|
||||||
return "Artifact processing is unavailable for the current target or host. Check artifact support and required tools.";
|
return "Artifact processing is unavailable for the current target or host. Check artifact support and required tools.";
|
||||||
if (reason === "integrity" && operation === "analyze_javascript_application")
|
if (reason === "integrity" && operation === "analyze_javascript_application")
|
||||||
@@ -345,6 +345,23 @@ const artifactMessage = (
|
|||||||
return "Artifact could not be read or written. Check file access and try again.";
|
return "Artifact could not be read or written. Check file access and try again.";
|
||||||
};
|
};
|
||||||
|
|
||||||
|
const artifactPathMessage = (detail: string | undefined): string => {
|
||||||
|
// Kept in step with DESTINATION_CASE_COLLISION_PREFIX. Domain does not import
|
||||||
|
// the artifact adapter that produces the detail.
|
||||||
|
if (
|
||||||
|
detail !== undefined &&
|
||||||
|
detail.startsWith("Destination filesystem cannot store both ")
|
||||||
|
)
|
||||||
|
return detail;
|
||||||
|
if (
|
||||||
|
detail !== undefined &&
|
||||||
|
(detail.startsWith("Artifact path is absolute or unsafe:") ||
|
||||||
|
detail.startsWith("Artifact path is not normalized:"))
|
||||||
|
)
|
||||||
|
return "Artifact contains an unsafe internal path. Inspect the reported path and correct the artifact before retrying.";
|
||||||
|
return "Artifact contains a conflicting internal path. Inspect the reported path before retrying.";
|
||||||
|
};
|
||||||
|
|
||||||
const evidenceFileMessage = ({
|
const evidenceFileMessage = ({
|
||||||
operation,
|
operation,
|
||||||
reason,
|
reason,
|
||||||
|
|||||||
@@ -1,6 +1,8 @@
|
|||||||
import { expect, it } from "vitest";
|
import { expect, it } from "vitest";
|
||||||
|
|
||||||
import { resolveJavaScriptSourceMapReference } from "../javascript/javascriptSourceMapPaths.js";
|
import { resolveJavaScriptSourceMapReference } from "../javascript/javascriptSourceMapPaths.js";
|
||||||
|
import { functionDossierSchema } from "../hopperValues.js";
|
||||||
|
import { ghidraFunctionDossier } from "../ghidraValues.fixture.js";
|
||||||
import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "../../contracts/javascript/javascriptRuntimeReconciliationExample.js";
|
import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "../../contracts/javascript/javascriptRuntimeReconciliationExample.js";
|
||||||
import {
|
import {
|
||||||
analysisViewBindJavaScriptGraphs,
|
analysisViewBindJavaScriptGraphs,
|
||||||
@@ -48,21 +50,190 @@ const javascriptParent = (
|
|||||||
limitations: JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE.limitations,
|
limitations: JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE.limitations,
|
||||||
});
|
});
|
||||||
|
|
||||||
it("accepts caller-selected page sizes and rejects malformed bounds", () => {
|
const nativeParent = (
|
||||||
const input = {
|
result: unknown = ghidraFunctionDossier(),
|
||||||
source: { kind: "retained-evidence", evidence_id: `ev_${"a".repeat(64)}` },
|
): AnalysisViewParent => ({
|
||||||
view: { kind: "page", collection: "sections", offset: 0, limit: 500 },
|
evidenceId: "ev_" + "b".repeat(64),
|
||||||
};
|
operation: "analyze_function",
|
||||||
expect(inspectAnalysisViewInputSchema.safeParse(input).success).toBe(true);
|
normalizedResult: result,
|
||||||
for (const limit of [0, -1, 0.5, Infinity, Number.MAX_SAFE_INTEGER + 1])
|
limitations: ["Retained provider limitation"],
|
||||||
expect(
|
artifact: { path: "/fixtures/example.exe", sha256: "a".repeat(64) },
|
||||||
inspectAnalysisViewInputSchema.safeParse({
|
|
||||||
...input,
|
|
||||||
view: { ...input.view, limit },
|
|
||||||
}).success,
|
|
||||||
).toBe(false);
|
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("projects native procedure and bounded assembly and high-pcode with exact parent identity", () => {
|
||||||
|
const parent = nativeParent();
|
||||||
|
const procedure = projectAnalysisView(parent, {
|
||||||
|
kind: "native",
|
||||||
|
facet: "procedure",
|
||||||
|
offset: 0,
|
||||||
|
limit: 64,
|
||||||
|
});
|
||||||
|
if (!procedure.ok) throw procedure.error;
|
||||||
|
expect(procedure.value).toMatchObject({
|
||||||
|
kind: "native",
|
||||||
|
parent_operation: "analyze_function",
|
||||||
|
artifact: parent.artifact,
|
||||||
|
item: { address: "0x401000" },
|
||||||
|
});
|
||||||
|
expect(procedure.value.view_digest).not.toBe(parent.evidenceId.slice(3));
|
||||||
|
const assembly = projectAnalysisView(parent, {
|
||||||
|
kind: "native",
|
||||||
|
facet: "assembly",
|
||||||
|
offset: 0,
|
||||||
|
limit: 1,
|
||||||
|
});
|
||||||
|
if (!assembly.ok) throw assembly.error;
|
||||||
|
expect(assembly.value).toMatchObject({
|
||||||
|
coverage: { examined: 1, total: 2, next_offset: 1 },
|
||||||
|
item: ["0x401000: CALL 0x401020"],
|
||||||
|
});
|
||||||
|
const flow = projectAnalysisView(parent, {
|
||||||
|
kind: "native",
|
||||||
|
facet: "value_flow_operations",
|
||||||
|
offset: 0,
|
||||||
|
limit: 1,
|
||||||
|
});
|
||||||
|
if (!flow.ok) throw flow.error;
|
||||||
|
expect(flow.value).toMatchObject({ item: [{ opcode: "COPY" }] });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects invalid native parents, unsupported source operations and singleton offsets", () => {
|
||||||
|
const view = {
|
||||||
|
kind: "native",
|
||||||
|
facet: "assembly",
|
||||||
|
offset: 0,
|
||||||
|
limit: 4,
|
||||||
|
} as const;
|
||||||
|
expect(projectAnalysisView(nativeParent({}), view)).toMatchObject({
|
||||||
|
ok: false,
|
||||||
|
error: { _tag: "AnalysisInputError" },
|
||||||
|
});
|
||||||
|
expect(
|
||||||
|
projectAnalysisView({ ...nativeParent(), artifact: null }, view),
|
||||||
|
).toMatchObject({
|
||||||
|
ok: false,
|
||||||
|
error: { _tag: "AnalysisInputError" },
|
||||||
|
});
|
||||||
|
expect(projectAnalysisView(layoutParent(), view)).toMatchObject({
|
||||||
|
ok: false,
|
||||||
|
error: { _tag: "AnalysisInputError" },
|
||||||
|
});
|
||||||
|
expect(projectAnalysisView(javascriptParent(), view)).toMatchObject({
|
||||||
|
ok: false,
|
||||||
|
error: { _tag: "AnalysisInputError" },
|
||||||
|
});
|
||||||
|
expect(
|
||||||
|
projectAnalysisView(nativeParent(), {
|
||||||
|
kind: "native",
|
||||||
|
facet: "procedure",
|
||||||
|
offset: 1,
|
||||||
|
limit: 2,
|
||||||
|
}),
|
||||||
|
).toMatchObject({ ok: false, error: { _tag: "AnalysisInputError" } });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("preserves native API limitations and unavailable residual unknowns", () => {
|
||||||
|
const original = functionDossierSchema.parse(ghidraFunctionDossier());
|
||||||
|
const view = {
|
||||||
|
kind: "native",
|
||||||
|
facet: "procedure",
|
||||||
|
offset: 0,
|
||||||
|
limit: 64,
|
||||||
|
} as const;
|
||||||
|
const unavailable = projectAnalysisView(
|
||||||
|
nativeParent({
|
||||||
|
...original,
|
||||||
|
native_api: {
|
||||||
|
available: false,
|
||||||
|
reason: "No decompiler",
|
||||||
|
residual_unknowns: ["Calling convention unresolved"],
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
view,
|
||||||
|
);
|
||||||
|
if (!unavailable.ok) throw unavailable.error;
|
||||||
|
expect(unavailable.value.unknowns).toContain("Calling convention unresolved");
|
||||||
|
const available = projectAnalysisView(nativeParent(), view);
|
||||||
|
if (!available.ok) throw available.error;
|
||||||
|
if (!original.native_api?.available) throw new Error("expected API fixture");
|
||||||
|
expect(available.value.limitations).toEqual(
|
||||||
|
expect.arrayContaining(original.native_api.limitations),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reports the malformed native dossier field", () => {
|
||||||
|
const result = projectAnalysisView(
|
||||||
|
nativeParent({
|
||||||
|
...functionDossierSchema.parse(ghidraFunctionDossier()),
|
||||||
|
pseudocode: 12,
|
||||||
|
}),
|
||||||
|
{
|
||||||
|
kind: "native",
|
||||||
|
facet: "pseudocode",
|
||||||
|
offset: 0,
|
||||||
|
limit: 64,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
expect(result).toMatchObject({
|
||||||
|
ok: false,
|
||||||
|
error: {
|
||||||
|
issues: expect.arrayContaining([
|
||||||
|
expect.objectContaining({
|
||||||
|
path: ["source", "normalized_result", "pseudocode"],
|
||||||
|
}),
|
||||||
|
]),
|
||||||
|
},
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("native pseudocode pages do not split surrogate pairs", () => {
|
||||||
|
const original = functionDossierSchema.parse(ghidraFunctionDossier());
|
||||||
|
const parent = nativeParent({ ...original, pseudocode: "ab😀cd" });
|
||||||
|
expect(
|
||||||
|
projectAnalysisView(parent, {
|
||||||
|
kind: "native",
|
||||||
|
facet: "pseudocode",
|
||||||
|
offset: 3,
|
||||||
|
limit: 2,
|
||||||
|
}),
|
||||||
|
).toMatchObject({ ok: false, error: { _tag: "AnalysisInputError" } });
|
||||||
|
const safe = projectAnalysisView(parent, {
|
||||||
|
kind: "native",
|
||||||
|
facet: "pseudocode",
|
||||||
|
offset: 2,
|
||||||
|
limit: 3,
|
||||||
|
});
|
||||||
|
if (!safe.ok) throw safe.error;
|
||||||
|
expect(safe.value).toMatchObject({
|
||||||
|
item: { text: "😀c", unit: "utf16-code-units" },
|
||||||
|
coverage: { examined: 3, next_offset: 5 },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each([
|
||||||
|
{ kind: "page", collection: "sections", offset: 0, limit: 500 },
|
||||||
|
{ kind: "native", facet: "assembly", offset: 0, limit: 500 },
|
||||||
|
])(
|
||||||
|
"accepts caller-selected page sizes and rejects malformed bounds: $kind",
|
||||||
|
(view) => {
|
||||||
|
const input = {
|
||||||
|
source: {
|
||||||
|
kind: "retained-evidence",
|
||||||
|
evidence_id: `ev_${"a".repeat(64)}`,
|
||||||
|
},
|
||||||
|
view,
|
||||||
|
};
|
||||||
|
expect(inspectAnalysisViewInputSchema.safeParse(input).success).toBe(true);
|
||||||
|
for (const limit of [0, -1, 0.5, Infinity, Number.MAX_SAFE_INTEGER + 1])
|
||||||
|
expect(
|
||||||
|
inspectAnalysisViewInputSchema.safeParse({
|
||||||
|
...input,
|
||||||
|
view: { ...input.view, limit },
|
||||||
|
}).success,
|
||||||
|
).toBe(false);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
it("projects layout summary, facet, item, and stable pages", () => {
|
it("projects layout summary, facet, item, and stable pages", () => {
|
||||||
const parent = layoutParent();
|
const parent = layoutParent();
|
||||||
const layout = analysisViewLayoutFixture();
|
const layout = analysisViewLayoutFixture();
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ import { jsonObjectSchema, jsonValueSchema } from "../jsonValue.js";
|
|||||||
import { err, ok, type Result } from "../result.js";
|
import { err, ok, type Result } from "../result.js";
|
||||||
import type { AnalysisError } from "../analysisErrorBase.js";
|
import type { AnalysisError } from "../analysisErrorBase.js";
|
||||||
import { binaryLayoutSchema } from "../native/binaryLayout.js";
|
import { binaryLayoutSchema } from "../native/binaryLayout.js";
|
||||||
|
import { functionDossierSchema } from "../hopperValues.js";
|
||||||
import {
|
import {
|
||||||
parseOwnedJavaScriptApplicationAnalysisSteps,
|
parseOwnedJavaScriptApplicationAnalysisSteps,
|
||||||
type JavaScriptApplicationAnalysisResult,
|
type JavaScriptApplicationAnalysisResult,
|
||||||
@@ -19,6 +20,7 @@ import {
|
|||||||
import { analysisInputErrorFromIssues } from "../inputIssueProjection.js";
|
import { analysisInputErrorFromIssues } from "../inputIssueProjection.js";
|
||||||
import { projectBinaryLayoutView } from "./binaryLayoutView.js";
|
import { projectBinaryLayoutView } from "./binaryLayoutView.js";
|
||||||
import { projectJavaScriptApplicationView } from "./javascriptApplicationView.js";
|
import { projectJavaScriptApplicationView } from "./javascriptApplicationView.js";
|
||||||
|
import { projectNativeFunctionView } from "./nativeFunctionView.js";
|
||||||
|
|
||||||
/** Public name of the selected-view workflow. */
|
/** Public name of the selected-view workflow. */
|
||||||
export const INSPECT_ANALYSIS_VIEW_OPERATION = "inspect_analysis_view" as const;
|
export const INSPECT_ANALYSIS_VIEW_OPERATION = "inspect_analysis_view" as const;
|
||||||
@@ -73,12 +75,35 @@ const pageViewSchema = z.strictObject({
|
|||||||
limit: z.number().int().positive(),
|
limit: z.number().int().positive(),
|
||||||
});
|
});
|
||||||
|
|
||||||
|
/** Bounded native function facets of already authenticated analyze_function Evidence. */
|
||||||
|
const nativeViewSchema = z.strictObject({
|
||||||
|
kind: z.literal("native"),
|
||||||
|
facet: z.enum([
|
||||||
|
"procedure",
|
||||||
|
"pseudocode",
|
||||||
|
"assembly",
|
||||||
|
"callers",
|
||||||
|
"callees",
|
||||||
|
"incoming_references",
|
||||||
|
"outgoing_references",
|
||||||
|
"value_flow_summary",
|
||||||
|
"value_flow_operations",
|
||||||
|
"value_flow_def_use",
|
||||||
|
"value_flow_effects",
|
||||||
|
"value_flow_parameters",
|
||||||
|
"value_flow_parameter_uses",
|
||||||
|
]),
|
||||||
|
offset: z.number().int().nonnegative().default(0),
|
||||||
|
limit: z.number().int().positive().default(64),
|
||||||
|
});
|
||||||
|
|
||||||
/** Caller-selected projection of already completed analysis Evidence. */
|
/** Caller-selected projection of already completed analysis Evidence. */
|
||||||
export const analysisViewRequestSchema = z.discriminatedUnion("kind", [
|
export const analysisViewRequestSchema = z.discriminatedUnion("kind", [
|
||||||
summaryViewSchema,
|
summaryViewSchema,
|
||||||
facetViewSchema,
|
facetViewSchema,
|
||||||
itemViewSchema,
|
itemViewSchema,
|
||||||
pageViewSchema,
|
pageViewSchema,
|
||||||
|
nativeViewSchema,
|
||||||
]);
|
]);
|
||||||
|
|
||||||
/** Inline Evidence or an exact same-session retained reference. */
|
/** Inline Evidence or an exact same-session retained reference. */
|
||||||
@@ -108,7 +133,9 @@ const coverageSchema = z.strictObject({
|
|||||||
});
|
});
|
||||||
|
|
||||||
const artifactSchema = z.strictObject({
|
const artifactSchema = z.strictObject({
|
||||||
path: z.string().min(1),
|
path: z
|
||||||
|
.string()
|
||||||
|
.describe("Exact recorded artifact path, including an empty string."),
|
||||||
sha256: digestSchema,
|
sha256: digestSchema,
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -150,6 +177,7 @@ const viewResultBase = {
|
|||||||
parent_operation: z.enum([
|
parent_operation: z.enum([
|
||||||
"inspect_binary_layout",
|
"inspect_binary_layout",
|
||||||
"analyze_javascript_application",
|
"analyze_javascript_application",
|
||||||
|
"analyze_function",
|
||||||
]),
|
]),
|
||||||
parent_digest: digestSchema,
|
parent_digest: digestSchema,
|
||||||
view_digest: digestSchema,
|
view_digest: digestSchema,
|
||||||
@@ -185,6 +213,13 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [
|
|||||||
items: z.array(jsonValueSchema),
|
items: z.array(jsonValueSchema),
|
||||||
...viewResultBase,
|
...viewResultBase,
|
||||||
}),
|
}),
|
||||||
|
z.strictObject({
|
||||||
|
kind: z.literal("native"),
|
||||||
|
view: nativeViewSchema,
|
||||||
|
procedure_address: z.string().nullable(),
|
||||||
|
item: jsonValueSchema,
|
||||||
|
...viewResultBase,
|
||||||
|
}),
|
||||||
]);
|
]);
|
||||||
|
|
||||||
export type AnalysisViewRequest = z.output<typeof analysisViewRequestSchema>;
|
export type AnalysisViewRequest = z.output<typeof analysisViewRequestSchema>;
|
||||||
@@ -200,6 +235,7 @@ export interface AnalysisViewParent {
|
|||||||
readonly operation: string;
|
readonly operation: string;
|
||||||
readonly normalizedResult: unknown;
|
readonly normalizedResult: unknown;
|
||||||
readonly limitations: readonly string[];
|
readonly limitations: readonly string[];
|
||||||
|
readonly artifact?: { readonly path: string; readonly sha256: string } | null;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Construct a typed input failure for one selected-view constraint. */
|
/** Construct a typed input failure for one selected-view constraint. */
|
||||||
@@ -216,7 +252,11 @@ export type UnsignedAnalysisView =
|
|||||||
>
|
>
|
||||||
| Omit<Extract<AnalysisViewResult, { readonly kind: "facet" }>, "view_digest">
|
| Omit<Extract<AnalysisViewResult, { readonly kind: "facet" }>, "view_digest">
|
||||||
| Omit<Extract<AnalysisViewResult, { readonly kind: "item" }>, "view_digest">
|
| Omit<Extract<AnalysisViewResult, { readonly kind: "item" }>, "view_digest">
|
||||||
| Omit<Extract<AnalysisViewResult, { readonly kind: "page" }>, "view_digest">;
|
| Omit<Extract<AnalysisViewResult, { readonly kind: "page" }>, "view_digest">
|
||||||
|
| Omit<
|
||||||
|
Extract<AnalysisViewResult, { readonly kind: "native" }>,
|
||||||
|
"view_digest"
|
||||||
|
>;
|
||||||
|
|
||||||
/** Attach the content digest that must change when projected bytes change. */
|
/** Attach the content digest that must change when projected bytes change. */
|
||||||
export const sealAnalysisView = (
|
export const sealAnalysisView = (
|
||||||
@@ -261,7 +301,7 @@ const unsupportedParent = (
|
|||||||
new AnalysisUnsupportedTargetError(
|
new AnalysisUnsupportedTargetError(
|
||||||
INSPECT_ANALYSIS_VIEW_OPERATION,
|
INSPECT_ANALYSIS_VIEW_OPERATION,
|
||||||
parentArtifactPath(parent),
|
parentArtifactPath(parent),
|
||||||
`Parent Evidence operation ${JSON.stringify(parent.operation)} is not inspect_binary_layout or analyze_javascript_application.`,
|
`Parent Evidence operation ${JSON.stringify(parent.operation)} is not inspect_binary_layout, analyze_javascript_application or analyze_function.`,
|
||||||
);
|
);
|
||||||
|
|
||||||
const parentArtifactPath = (parent: AnalysisViewParent): string => {
|
const parentArtifactPath = (parent: AnalysisViewParent): string => {
|
||||||
@@ -327,5 +367,22 @@ export const projectAnalysisView = (
|
|||||||
);
|
);
|
||||||
return projected.ok ? ok(sealAnalysisView(projected.value)) : projected;
|
return projected.ok ? ok(sealAnalysisView(projected.value)) : projected;
|
||||||
}
|
}
|
||||||
|
if (parent.operation === "analyze_function") {
|
||||||
|
const native = functionDossierSchema.safeParse(parent.normalizedResult);
|
||||||
|
if (!native.success)
|
||||||
|
return err(
|
||||||
|
analysisInputErrorFromIssues(
|
||||||
|
INSPECT_ANALYSIS_VIEW_OPERATION,
|
||||||
|
native.error.issues.map((issue) => ({
|
||||||
|
...issue,
|
||||||
|
path: ["source", "normalized_result", ...issue.path],
|
||||||
|
})),
|
||||||
|
{ source: { normalized_result: parent.normalizedResult } },
|
||||||
|
{ cause: native.error },
|
||||||
|
),
|
||||||
|
);
|
||||||
|
const projected = projectNativeFunctionView(parent, native.data, view);
|
||||||
|
return projected.ok ? ok(sealAnalysisView(projected.value)) : projected;
|
||||||
|
}
|
||||||
return err(unsupportedParent(parent));
|
return err(unsupportedParent(parent));
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -152,6 +152,10 @@ export const projectBinaryLayoutView = (
|
|||||||
view: AnalysisViewRequest,
|
view: AnalysisViewRequest,
|
||||||
): Result<UnsignedAnalysisView, AnalysisError> => {
|
): Result<UnsignedAnalysisView, AnalysisError> => {
|
||||||
const shared = parentFields(parent, layout);
|
const shared = parentFields(parent, layout);
|
||||||
|
if (view.kind === "native")
|
||||||
|
return err(
|
||||||
|
layoutIncompatible("Native views require analyze_function Evidence."),
|
||||||
|
);
|
||||||
if (view.kind === "summary")
|
if (view.kind === "summary")
|
||||||
return ok({
|
return ok({
|
||||||
kind: "summary",
|
kind: "summary",
|
||||||
|
|||||||
@@ -243,6 +243,10 @@ export const projectJavaScriptApplicationView = (
|
|||||||
): Result<UnsignedAnalysisView, AnalysisError> => {
|
): Result<UnsignedAnalysisView, AnalysisError> => {
|
||||||
const shared = parentFields(parent, analysis);
|
const shared = parentFields(parent, analysis);
|
||||||
const modules = moduleEntries(analysis.graph.nodes);
|
const modules = moduleEntries(analysis.graph.nodes);
|
||||||
|
if (view.kind === "native")
|
||||||
|
return err(
|
||||||
|
jsIncompatible("Native views require analyze_function Evidence."),
|
||||||
|
);
|
||||||
if (view.kind === "summary")
|
if (view.kind === "summary")
|
||||||
return ok({
|
return ok({
|
||||||
kind: "summary",
|
kind: "summary",
|
||||||
|
|||||||
@@ -0,0 +1,202 @@
|
|||||||
|
import type { AnalysisError } from "../analysisErrorBase.js";
|
||||||
|
import { uniqueSorted } from "../canonicalOrdering.js";
|
||||||
|
import type { FunctionDossier } from "../hopperValues.js";
|
||||||
|
import { jsonValueSchema } from "../jsonValue.js";
|
||||||
|
import { err, ok, type Result } from "../result.js";
|
||||||
|
import {
|
||||||
|
analysisViewInputError,
|
||||||
|
type AnalysisViewCoverage,
|
||||||
|
type AnalysisViewParent,
|
||||||
|
type AnalysisViewRequest,
|
||||||
|
type UnsignedAnalysisView,
|
||||||
|
} from "./analysisView.js";
|
||||||
|
import {
|
||||||
|
completeWithinViewCoverage,
|
||||||
|
pageViewCoverage,
|
||||||
|
} from "./analysisViewCoverage.js";
|
||||||
|
|
||||||
|
type Artifact = { readonly path: string; readonly sha256: string };
|
||||||
|
|
||||||
|
const viewError = (message: string): AnalysisError =>
|
||||||
|
analysisViewInputError([
|
||||||
|
{ path: ["view"], reason: "invalid_value", message },
|
||||||
|
]);
|
||||||
|
|
||||||
|
const parentFields = (
|
||||||
|
parent: AnalysisViewParent,
|
||||||
|
dossier: FunctionDossier,
|
||||||
|
artifact: Artifact,
|
||||||
|
) => {
|
||||||
|
const flow = dossier.native_value_flow;
|
||||||
|
const api = dossier.native_api;
|
||||||
|
const unknowns: string[] = [];
|
||||||
|
if (flow === null) unknowns.push("native_value_flow was not recorded");
|
||||||
|
else if (!flow.available)
|
||||||
|
unknowns.push("native_value_flow unavailable: " + flow.reason);
|
||||||
|
else if (flow.truncated)
|
||||||
|
unknowns.push(
|
||||||
|
"native_value_flow is truncated; omitted observations remain unknown",
|
||||||
|
);
|
||||||
|
if (api === null) unknowns.push("native_api was not recorded");
|
||||||
|
else if (!api.available)
|
||||||
|
unknowns.push(
|
||||||
|
"native_api unavailable: " + api.reason,
|
||||||
|
...api.residual_unknowns,
|
||||||
|
);
|
||||||
|
return {
|
||||||
|
parent_evidence_id: parent.evidenceId,
|
||||||
|
parent_operation: "analyze_function" as const,
|
||||||
|
parent_digest: parent.evidenceId.slice(3),
|
||||||
|
procedure_address: dossier.procedure.address,
|
||||||
|
artifact,
|
||||||
|
limitations: uniqueSorted([
|
||||||
|
...parent.limitations,
|
||||||
|
...dossier.limitations,
|
||||||
|
...(flow?.limitations ?? []),
|
||||||
|
...(api?.available ? api.limitations : []),
|
||||||
|
]),
|
||||||
|
unknowns: uniqueSorted(unknowns),
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
const pseudocodePage = (
|
||||||
|
body: string,
|
||||||
|
offset: number,
|
||||||
|
limit: number,
|
||||||
|
): Result<
|
||||||
|
{
|
||||||
|
readonly item: { readonly text: string; readonly unit: "utf16-code-units" };
|
||||||
|
readonly coverage: AnalysisViewCoverage;
|
||||||
|
},
|
||||||
|
AnalysisError
|
||||||
|
> => {
|
||||||
|
const start = Math.min(offset, body.length);
|
||||||
|
const high = (n: number) => n >= 0xd800 && n <= 0xdbff;
|
||||||
|
const low = (n: number) => n >= 0xdc00 && n <= 0xdfff;
|
||||||
|
if (
|
||||||
|
start > 0 &&
|
||||||
|
start < body.length &&
|
||||||
|
high(body.charCodeAt(start - 1)) &&
|
||||||
|
low(body.charCodeAt(start))
|
||||||
|
)
|
||||||
|
return err(viewError("Pseudocode offset splits a UTF-16 surrogate pair."));
|
||||||
|
let end = Math.min(start + limit, body.length);
|
||||||
|
if (
|
||||||
|
end > start &&
|
||||||
|
end < body.length &&
|
||||||
|
high(body.charCodeAt(end - 1)) &&
|
||||||
|
low(body.charCodeAt(end))
|
||||||
|
)
|
||||||
|
end--;
|
||||||
|
if (end === start && start < body.length)
|
||||||
|
return err(
|
||||||
|
viewError(
|
||||||
|
"Increase pseudocode limit to include the next complete Unicode character.",
|
||||||
|
),
|
||||||
|
);
|
||||||
|
return ok({
|
||||||
|
item: { text: body.slice(start, end), unit: "utf16-code-units" },
|
||||||
|
coverage: pageViewCoverage(start, end - start, body.length),
|
||||||
|
});
|
||||||
|
};
|
||||||
|
|
||||||
|
const nativeRows = (
|
||||||
|
dossier: FunctionDossier,
|
||||||
|
facet: Exclude<
|
||||||
|
Extract<AnalysisViewRequest, { readonly kind: "native" }>["facet"],
|
||||||
|
"procedure" | "pseudocode" | "value_flow_summary"
|
||||||
|
>,
|
||||||
|
) => {
|
||||||
|
const sourceFlow = dossier.native_value_flow;
|
||||||
|
const flow = sourceFlow?.available ? sourceFlow : undefined;
|
||||||
|
return {
|
||||||
|
assembly: dossier.assembly,
|
||||||
|
callers: dossier.callers,
|
||||||
|
callees: dossier.callees,
|
||||||
|
incoming_references: dossier.incoming_references,
|
||||||
|
outgoing_references: dossier.outgoing_references,
|
||||||
|
value_flow_operations: flow?.operations,
|
||||||
|
value_flow_def_use: flow?.def_use,
|
||||||
|
value_flow_effects: flow?.effects,
|
||||||
|
value_flow_parameters: flow?.parameters,
|
||||||
|
value_flow_parameter_uses: flow?.parameter_uses,
|
||||||
|
}[facet];
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Project bounded native function facets from complete retained Evidence without reanalysis. */
|
||||||
|
export const projectNativeFunctionView = (
|
||||||
|
parent: AnalysisViewParent,
|
||||||
|
dossier: FunctionDossier,
|
||||||
|
view: AnalysisViewRequest,
|
||||||
|
): Result<UnsignedAnalysisView, AnalysisError> => {
|
||||||
|
if (view.kind !== "native")
|
||||||
|
return err(viewError("analyze_function Evidence requires a native view."));
|
||||||
|
const artifact = parent.artifact;
|
||||||
|
if (!artifact)
|
||||||
|
return err(
|
||||||
|
analysisViewInputError([
|
||||||
|
{
|
||||||
|
path: ["source", "subject"],
|
||||||
|
reason: "invalid_value",
|
||||||
|
message:
|
||||||
|
"Native function Evidence requires an artifact-bound subject.",
|
||||||
|
},
|
||||||
|
]),
|
||||||
|
);
|
||||||
|
const common = parentFields(parent, dossier, artifact);
|
||||||
|
const singleton = (item: unknown): UnsignedAnalysisView => ({
|
||||||
|
kind: "native",
|
||||||
|
view,
|
||||||
|
item: jsonValueSchema.parse(item),
|
||||||
|
coverage: completeWithinViewCoverage(1, 1),
|
||||||
|
...common,
|
||||||
|
});
|
||||||
|
const flow = dossier.native_value_flow;
|
||||||
|
if (view.facet === "procedure" || view.facet === "value_flow_summary") {
|
||||||
|
if (view.offset !== 0)
|
||||||
|
return err(viewError("Singleton native facets require offset 0."));
|
||||||
|
if (view.facet === "procedure") return ok(singleton(dossier.procedure));
|
||||||
|
const status =
|
||||||
|
flow === null
|
||||||
|
? { available: false, reason: "not recorded" }
|
||||||
|
: !flow.available
|
||||||
|
? { available: false, reason: flow.reason }
|
||||||
|
: {
|
||||||
|
available: true,
|
||||||
|
provenance: flow.provenance,
|
||||||
|
truncated: flow.truncated,
|
||||||
|
omitted_operations_lower_bound:
|
||||||
|
flow.omitted_operations_lower_bound,
|
||||||
|
known_omitted_inputs: flow.known_omitted_inputs,
|
||||||
|
known_omitted_edges: flow.known_omitted_edges,
|
||||||
|
operations: flow.operations.length,
|
||||||
|
def_use: flow.def_use.length,
|
||||||
|
effects: flow.effects.length,
|
||||||
|
parameters: flow.parameters.length,
|
||||||
|
parameter_uses: flow.parameter_uses.length,
|
||||||
|
};
|
||||||
|
return ok(singleton(status));
|
||||||
|
}
|
||||||
|
if (view.facet === "pseudocode") {
|
||||||
|
const page = pseudocodePage(dossier.pseudocode, view.offset, view.limit);
|
||||||
|
return page.ok
|
||||||
|
? ok({ kind: "native", view, ...page.value, ...common })
|
||||||
|
: page;
|
||||||
|
}
|
||||||
|
const rows = nativeRows(dossier, view.facet);
|
||||||
|
if (rows === undefined)
|
||||||
|
return ok(
|
||||||
|
singleton({
|
||||||
|
available: false,
|
||||||
|
reason: flow?.available === false ? flow.reason : "not recorded",
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
const items = rows.slice(view.offset, view.offset + view.limit);
|
||||||
|
return ok({
|
||||||
|
kind: "native",
|
||||||
|
view,
|
||||||
|
item: jsonValueSchema.parse(items),
|
||||||
|
coverage: pageViewCoverage(view.offset, items.length, rows.length),
|
||||||
|
...common,
|
||||||
|
});
|
||||||
|
};
|
||||||
@@ -125,6 +125,8 @@ export const projectAndroidApplication = (
|
|||||||
"Manifest, resource, signing, and bytecode semantics require a dedicated Android provider; this projection reports exact inventory paths and hashes only.",
|
"Manifest, resource, signing, and bytecode semantics require a dedicated Android provider; this projection reports exact inventory paths and hashes only.",
|
||||||
"Runtime families are inferred from inventory formats and paths; filename suffixes do not establish valid DEX or JVM class bytes.",
|
"Runtime families are inferred from inventory formats and paths; filename suffixes do not establish valid DEX or JVM class bytes.",
|
||||||
"Bridge candidates are path-based hypotheses, not decoded JNI declarations or observed runtime calls.",
|
"Bridge candidates are path-based hypotheses, not decoded JNI declarations or observed runtime calls.",
|
||||||
|
"A bridge basis is inferred from the native path and is repeated for every managed component.",
|
||||||
|
"ZIP inventory cannot see the APK Signing Block, so an empty signing array does not mean the APK is unsigned.",
|
||||||
...(bridgeProjection.coverage.status === "partial"
|
...(bridgeProjection.coverage.status === "partial"
|
||||||
? [
|
? [
|
||||||
`Bridge candidate pairs exceeded the projection safety budget; ${bridgeProjection.coverage.omitted_candidates} hypotheses are omitted. Component arrays still include every component from the supplied inventory pages.`,
|
`Bridge candidate pairs exceeded the projection safety budget; ${bridgeProjection.coverage.omitted_candidates} hypotheses are omitted. Component arrays still include every component from the supplied inventory pages.`,
|
||||||
@@ -157,23 +159,12 @@ export const projectAndroidApplication = (
|
|||||||
};
|
};
|
||||||
|
|
||||||
const classify = (all: readonly Component[]) => ({
|
const classify = (all: readonly Component[]) => ({
|
||||||
manifests: all.filter(
|
manifests: all.filter(({ path }) =>
|
||||||
({ path, format }) =>
|
/(?:^|\/)AndroidManifest\.xml$/u.test(path),
|
||||||
format === "android-manifest" ||
|
|
||||||
/(?:^|\/)AndroidManifest\.xml$/u.test(path),
|
|
||||||
),
|
|
||||||
resources: all.filter(
|
|
||||||
({ path, format }) =>
|
|
||||||
format === "android-resources" || /(?:^|\/)resources\.arsc$/u.test(path),
|
|
||||||
),
|
|
||||||
dex: all.filter(
|
|
||||||
({ path, format }) =>
|
|
||||||
format === "dex" || /(?:^|\/)[^/]+\.dex$/iu.test(path),
|
|
||||||
),
|
|
||||||
jvm_classes: all.filter(
|
|
||||||
({ path, format }) =>
|
|
||||||
format === "jvm-class" || /(?:^|\/)[^/]+\.class$/iu.test(path),
|
|
||||||
),
|
),
|
||||||
|
resources: all.filter(({ path }) => /(?:^|\/)resources\.arsc$/u.test(path)),
|
||||||
|
dex: all.filter(({ path }) => /(?:^|\/)[^/]+\.dex$/iu.test(path)),
|
||||||
|
jvm_classes: all.filter(({ path }) => /(?:^|\/)[^/]+\.class$/iu.test(path)),
|
||||||
native_libraries: all.filter(
|
native_libraries: all.filter(
|
||||||
({ path, format }) =>
|
({ path, format }) =>
|
||||||
format === "elf" || /(?:^|\/)lib\/[^/]+\/[^/]+\.so$/iu.test(path),
|
format === "elf" || /(?:^|\/)lib\/[^/]+\/[^/]+\.so$/iu.test(path),
|
||||||
@@ -189,19 +180,9 @@ const runtimeFamilies = (all: readonly Component[]) => {
|
|||||||
const families = new Set<
|
const families = new Set<
|
||||||
AndroidApplicationProjectionResult["runtime_families"][number]
|
AndroidApplicationProjectionResult["runtime_families"][number]
|
||||||
>();
|
>();
|
||||||
if (
|
if (all.some(({ path }) => path.toLowerCase().endsWith(".dex")))
|
||||||
all.some(
|
|
||||||
({ path, format }) =>
|
|
||||||
format === "dex" || path.toLowerCase().endsWith(".dex"),
|
|
||||||
)
|
|
||||||
)
|
|
||||||
families.add("dalvik-art");
|
families.add("dalvik-art");
|
||||||
if (
|
if (all.some(({ path }) => path.toLowerCase().endsWith(".class")))
|
||||||
all.some(
|
|
||||||
({ path, format }) =>
|
|
||||||
format === "jvm-class" || path.toLowerCase().endsWith(".class"),
|
|
||||||
)
|
|
||||||
)
|
|
||||||
families.add("java-kotlin");
|
families.add("java-kotlin");
|
||||||
if (all.some(({ format }) => format === "elf")) families.add("native");
|
if (all.some(({ format }) => format === "elf")) families.add("native");
|
||||||
if (all.some(({ format }) => format === "javascript-bundle"))
|
if (all.some(({ format }) => format === "javascript-bundle"))
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user