Compare commits
63
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
627c2682f1 | ||
|
|
69e097a8ca | ||
|
|
3bdc8730bd | ||
|
|
3bcc68c037 | ||
|
|
83a4445f44 | ||
|
|
783fcb06a4 | ||
|
|
e55ef72135 | ||
|
|
48c352b6bf | ||
|
|
6fb3a345fe | ||
|
|
1c64fe3eff | ||
|
|
538f8cc722 | ||
|
|
0d78c99c11 | ||
|
|
21eff3bae1 | ||
|
|
b7f563e15f | ||
|
|
4a4d36f881 | ||
|
|
d0249282c6 | ||
|
|
a1b5d51a8d | ||
|
|
bd74c7af2b | ||
|
|
9f67e44e5d | ||
|
|
ee50ed121e | ||
|
|
c0904b4c1a | ||
|
|
d9d2a17381 | ||
|
|
04089d5bec | ||
|
|
880a603fe8 | ||
|
|
64b71b180a | ||
|
|
2ff850de68 | ||
|
|
42b0ed126d | ||
|
|
b8fab74297 | ||
|
|
edf46248f2 | ||
|
|
5dfb9d050c | ||
|
|
b38429cb6a | ||
|
|
87b9793c53 | ||
|
|
d93fbda390 | ||
|
|
6611bcf6e8 | ||
|
|
e19879328a | ||
|
|
b5ededd0f9 | ||
|
|
aada0ca812 | ||
|
|
f889a69fb9 | ||
|
|
adde8a8092 | ||
|
|
091cb81fc3 | ||
|
|
ed6af7eb8c | ||
|
|
976c2c083e | ||
|
|
eaf88aca2a | ||
|
|
1febc22907 | ||
|
|
4b653fe6ad | ||
|
|
0d11f71e0d | ||
|
|
43f9c39dfb | ||
|
|
a149d183ef | ||
|
|
f28f338283 | ||
|
|
e9fe6437a7 | ||
|
|
6395e8d300 | ||
|
|
1f17ac8228 | ||
|
|
89f5e489ad | ||
|
|
7285e15839 | ||
|
|
8bd5ba2d23 | ||
|
|
0677468877 | ||
|
|
55b61485e9 | ||
|
|
264615059c | ||
|
|
74d7da6ced | ||
|
|
dcfc5d403a | ||
|
|
9c8511409c | ||
|
|
f1924cf9c1 | ||
|
|
b777e5d4d8 |
@@ -77,6 +77,12 @@ explicit path or endpoint and do not need it.
|
||||
argument strings and mutability as inferred candidates; this performs no chain
|
||||
lookup or target execution. See the
|
||||
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/evm-bytecode.md).
|
||||
- Embedded Go compiler, module or build-setting metadata: `inspect_go_binary`
|
||||
with the explicit local file. This built-in target-free inspection requires no
|
||||
Go installation or disassembler. Preserve raw module bytes and source ranges;
|
||||
absent metadata does not prove a binary is not Go, and unparsed text records
|
||||
remain explicit. See the
|
||||
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/go-binaries.md).
|
||||
- Explicit Linux ELF file for offline layout, symbols, relocations or static
|
||||
mitigation evidence: `inspect_binary_layout`. This target-free operation uses
|
||||
caller-supplied pwntools without opening a disassembler database. Preserve its
|
||||
@@ -115,8 +121,14 @@ ask when matches are ambiguous. Never choose an example app on the user's behalf
|
||||
In a target-free session, use `open_binary` to bind any archive/package or
|
||||
native target whose analysis tool operates on the active target. Do not call an
|
||||
unadvertised tool; inspect `binary_session` with `{}` and its
|
||||
`result.tool_availability` for availability reasons and remediation. The tool
|
||||
list stays complete; availability depends on the operation, target, and host.
|
||||
`result.tool_availability` for availability reasons and remediation. Expand each
|
||||
group's `names` into entries, inheriting the inventory-level client requirements
|
||||
unless the group overrides them. In its closed effect and annotation vocabularies,
|
||||
listed flags are true and omitted known flags are false. The grouped name union
|
||||
is the complete tool inventory; use `tools/list` for canonical tool schemas.
|
||||
Provider candidate facts describe possible providers, while top-level session
|
||||
capabilities describe the current route and `analysis_provider_binding` names the
|
||||
selected provider. Do not treat those scopes as interchangeable.
|
||||
|
||||
## Work summary-first
|
||||
|
||||
|
||||
@@ -11,14 +11,14 @@ main and an older release expose the same arguments.
|
||||
|
||||
## Choose the remedy from the observed failure
|
||||
|
||||
| Observation | Meaning and next step |
|
||||
| --------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| No REA tools in the active session | Inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
|
||||
| Missing, malformed or stale registration | Prepare a scoped setup plan; apply only authorized changes with backups. |
|
||||
| Aligned registration, but no connected tools | Restart/reconnect the client. Doctor checks registration; it cannot establish the current agent's connection. Inspect the client's launch error if reconnection fails. |
|
||||
| `MCP tool call requires approval, but approval policy is never` | Codex rejected the call before REA executed it. Do not install an engine or repeat setup to solve this client-policy decision. See the approval section below. |
|
||||
| REA returns `provider_unavailable` or an unavailable tool entry | Read that operation's provider diagnostics or `binary_session.result.tool_availability`. Resolve only the prerequisite needed for the task; a complete tool catalog does not imply every operation is available. |
|
||||
| `rea` is missing in a child shell, or Node is incompatible | Inspect the actual shell PATH and launcher/runtime. A prefix installation may be absent from a login shell's PATH; this does not establish a broken MCP registration. |
|
||||
| Observation | Meaning and next step |
|
||||
| --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| No REA tools in the active session | Inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
|
||||
| Missing, malformed or stale registration | Prepare a scoped setup plan; apply only authorized changes with backups. |
|
||||
| Aligned registration, but no connected tools | Restart/reconnect the client. Doctor checks registration; it cannot establish the current agent's connection. Inspect the client's launch error if reconnection fails. |
|
||||
| `MCP tool call requires approval, but approval policy is never` | Codex rejected the call before REA executed it. Do not install an engine or repeat setup to solve this client-policy decision. See the approval section below. |
|
||||
| REA returns `provider_unavailable` or an unavailable tool entry | Read that operation's provider diagnostics or `binary_session.result.tool_availability`. Expand each group's tool names and inherit the shared client requirements unless overridden. Use the operation's availability, reason, and remediation; a complete tool catalog does not imply every operation is available. |
|
||||
| `rea` is missing in a child shell, or Node is incompatible | Inspect the actual shell PATH and launcher/runtime. A prefix installation may be absent from a login shell's PATH; this does not establish a broken MCP registration. |
|
||||
|
||||
If the client cannot launch its shell/sandbox helper or read the installed skill,
|
||||
inspect its launch diagnostic. For example, Codex can refuse helper aliases when
|
||||
@@ -46,7 +46,7 @@ rea setup --client codex --dry-run --json
|
||||
Use the current supported client's ID: `claude_code`, `claude_desktop`, `codex`,
|
||||
`cursor`, `gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`,
|
||||
`copilot_cli`, `commandcode`, `qwen_code`, `vscode`, `grok_build`, `omp`, `pi`,
|
||||
`hermes`, or `grok_bot`. If unknown, inspect `doctor --json` before choosing a
|
||||
`hermes`, `dsh`, or `grok_bot`. If unknown, inspect `doctor --json` before choosing a
|
||||
scope. `grok_bot` uses a chat registration step rather than local `mcp.json`.
|
||||
For unsupported clients, use version-pinned manual stdio registration or CLI.
|
||||
|
||||
|
||||
@@ -46,3 +46,9 @@ observed exit signal stays in `exit.signal`, `exit.code` is null for every
|
||||
deadline exit, and the code of a finalizer that exited on its own is in
|
||||
`exit.finalization.exit_code` (null on a signal exit or when no exit was
|
||||
observed).
|
||||
|
||||
Use `inspect_analysis_view` with `view: {kind: "api-surface"}` on retained
|
||||
`inspect_web_network_capture` HAR Evidence to group observed request lines by
|
||||
method, origin and path template. Format matches are inferences, not handler
|
||||
identity or schemas. Group pages cite parent record ordinals; consult record
|
||||
coverage and exclusions before claiming completeness. No requests are replayed.
|
||||
|
||||
@@ -1,5 +1,16 @@
|
||||
# Native, managed, and packaged artifacts
|
||||
|
||||
## Go build metadata
|
||||
|
||||
Use `inspect_go_binary` for an explicit local ELF, PE or thin Mach-O when the
|
||||
question concerns the embedded compiler version, module/dependency versions,
|
||||
local replacements or build settings. No `open_binary`, Go installation or
|
||||
target execution is required. The CLI equivalent is `inspect-go-binary PATH`.
|
||||
Preserve artifact SHA-256, original file ranges, framed module bytes and every
|
||||
unparsed module-text line. A missing record is an unknown, not proof of a
|
||||
non-Go binary; observed metadata does not authenticate its claims. Function,
|
||||
source and type recovery remain a separate native-analysis question.
|
||||
|
||||
## Recorded Linux crashes
|
||||
|
||||
Use `inspect_recorded_crash` with an explicit Linux x86-64 ELF core path to
|
||||
@@ -50,10 +61,19 @@ runs and may change files or use the network. Hardened-runtime targets need the
|
||||
`get-task-allow` entitlement.
|
||||
|
||||
Use `binary_session` with no arguments to check the open target, selected
|
||||
provider, and alignment. Its default `result.tool_availability` includes the
|
||||
complete tool inventory with availability reasons and remediation. When choosing
|
||||
a tool, use its entry in that result; `tools/list` retains the complete catalog
|
||||
when the target or provider state changes.
|
||||
provider, and alignment. Its default `result.tool_availability` carries a
|
||||
complete inventory grouped by shared facts. Expand each group's `names` and
|
||||
inherit the inventory-level client requirements unless a group overrides them;
|
||||
availability, reasons, remediation, modes, and flags remain inline. The
|
||||
`effects` and `annotations` arrays name true flags from fixed vocabularies, so an
|
||||
omitted known flag is false. `tools/list` remains the canonical source for full
|
||||
tool schemas when target or provider state changes.
|
||||
|
||||
Provider capability groups carry provider identity and ordered limitations.
|
||||
Prepend each set's `shared_limitations` to a group's `limitations` to recover the
|
||||
operation's full ordered list. Candidate capabilities describe possible
|
||||
providers. Top-level session capabilities describe the current route, and
|
||||
`analysis_provider_binding` identifies the selected provider.
|
||||
|
||||
## DOS MZ with Ghidra
|
||||
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
name: Real Go binary metadata
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
pull_request:
|
||||
paths:
|
||||
- ".github/workflows/real-go-binary.yml"
|
||||
- "src/go/**"
|
||||
- "src/domain/go/**"
|
||||
- "src/domain/dosMz.ts"
|
||||
- "src/application/go/**"
|
||||
- "src/contracts/go/**"
|
||||
- "src/composition/go.ts"
|
||||
- "src/cli/goCommands.ts"
|
||||
- "src/server/registerGoTools.ts"
|
||||
- "src/server/createServer.ts"
|
||||
- "src/cli.ts"
|
||||
- "src/cliCommandNames.ts"
|
||||
- "src/contracts/toolContracts.ts"
|
||||
- "src/contracts/toolEffects.ts"
|
||||
- "src/artifacts/readStableArtifact*.ts"
|
||||
- "src/artifacts/ArtifactReader.ts"
|
||||
- "src/process/PrivateRuntimeRoot.ts"
|
||||
- "scripts/verify-go-binary.mjs"
|
||||
- "scripts/lib/mcp-verifier-results.mjs"
|
||||
- "scripts/rea.mjs"
|
||||
- "package.json"
|
||||
- "package-lock.json"
|
||||
- ".nvmrc"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: real-go-binary-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
NODE_OPTIONS: --max-old-space-size=768
|
||||
HUSKY: "0"
|
||||
|
||||
jobs:
|
||||
real-go-binary:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version: "1.26.1"
|
||||
cache: false
|
||||
- run: npm ci
|
||||
- name: Verify compiler-produced ELF, PE and Mach-O through public CLI and MCP
|
||||
run: npm run verify:go:binary
|
||||
@@ -14,7 +14,9 @@ Preserve caller-selected inputs and local evidence, including output, URLs, path
|
||||
|
||||
Dependencies flow inward: domain semantics and contracts, provider adapters, shared application workflows, then CLI/MCP adapters. Keep engine protocols and provider-specific code out of domain/application layers. See [docs/architecture.mermaid](docs/architecture.mermaid) when changing composition.
|
||||
|
||||
Give each interpretation and resource a clear owner. Preserve identity, omission, uncertainty, and source evidence across boundaries. Retain ownership of resources whose cleanup failed. Reuse `src/process/` supervision and identity primitives; a PID and executable path alone do not establish ownership after exit or reuse.
|
||||
Give each interpretation and resource a clear owner. Consumers must use that owner's parsed facts rather than reinterpret raw source. Preserve identity, omission, uncertainty, and source evidence across boundaries. Retain ownership of resources whose cleanup failed. Reuse `src/process/` supervision and identity primitives; a PID and executable path alone do not establish ownership after exit or reuse.
|
||||
|
||||
Keep proof of ownership separate from validity, editability, and readiness. Malformed owned data may still be safely removable. Unresolved identity or applicability must remain unknown rather than be treated as absent, foreign, or aligned.
|
||||
|
||||
Assess brittleness in touched code during implementation, debugging, review, and maintenance. Fix the producing representation or ownership rule before adding downstream repairs. Normalize once at the owning boundary using format-aware parsers. Remove superseded representations, aliases, wrappers, and redundant tests after verifying affected consumers. Introduce abstractions only when they remove observed complexity.
|
||||
|
||||
|
||||
@@ -154,6 +154,7 @@ Additional tools and host support depend on the target:
|
||||
| ---------------------- | ------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra or IDA; [native analysis](https://rea.tools/guides/native/) |
|
||||
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/binary-diagnostics.md) |
|
||||
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
|
||||
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/evm-bytecode.md) |
|
||||
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/recorded-crashes.md) |
|
||||
@@ -316,7 +317,7 @@ Report vulnerabilities through [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Star history
|
||||
|
||||
🎉 **90,000 GitHub stars — thank you!**
|
||||
🎉 **110,000 GitHub stars — thank you!**
|
||||
|
||||
Thanks to everyone using REA, reporting bugs, requesting features, testing builds, and contributing fixes.
|
||||
|
||||
|
||||
+1
-1
@@ -260,7 +260,7 @@ npx rea-agents@latest setup
|
||||
|
||||
## سجل النجوم
|
||||
|
||||
🎉 **90,000 نجمة على GitHub — شكرًا لكم!**
|
||||
🎉 **110,000 نجمة على GitHub — شكرًا لكم!**
|
||||
|
||||
شكرًا لكل من يستخدم REA، ويبلّغ عن الأخطاء، ويقترح ميزات، ويختبر إصدارات البناء، ويساهم بالإصلاحات.
|
||||
|
||||
|
||||
+1
-1
@@ -258,7 +258,7 @@ Melde Sicherheitslücken gemäß [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Sternverlauf
|
||||
|
||||
🎉 **90.000 GitHub-Sterne – vielen Dank!**
|
||||
🎉 **110.000 GitHub-Sterne – vielen Dank!**
|
||||
|
||||
Danke an alle, die REA nutzen, Fehler melden, Funktionswünsche äußern, Builds testen und Fehlerbehebungen beitragen.
|
||||
|
||||
|
||||
+1
-1
@@ -258,7 +258,7 @@ Informa de vulnerabilidades siguiendo [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Historial de estrellas
|
||||
|
||||
🎉 **90.000 estrellas en GitHub: ¡gracias!**
|
||||
🎉 **110.000 estrellas en GitHub: ¡gracias!**
|
||||
|
||||
Gracias a quienes usan REA, informan de errores, solicitan funciones, prueban compilaciones y contribuyen con correcciones.
|
||||
|
||||
|
||||
+1
-1
@@ -243,7 +243,7 @@ npx rea-agents@latest setup
|
||||
|
||||
## تاریخچهٔ ستارهها
|
||||
|
||||
🎉 **۹۰٬۰۰۰ ستارهٔ GitHub — سپاسگزاریم!**
|
||||
🎉 **۱۱۰٬۰۰۰ ستارهٔ GitHub — سپاسگزاریم!**
|
||||
|
||||
از همهٔ کسانی که از REA استفاده میکنند، خطاها را گزارش میدهند، قابلیتهای جدید پیشنهاد میکنند، buildها را آزمایش میکنند و در بهبود پروژه مشارکت دارند سپاسگزاریم.
|
||||
|
||||
|
||||
+1
-1
@@ -262,7 +262,7 @@ Signalez les vulnérabilités via [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Historique des étoiles
|
||||
|
||||
🎉 **90 000 étoiles GitHub — merci !**
|
||||
🎉 **110 000 étoiles GitHub — merci !**
|
||||
|
||||
Merci à toutes les personnes qui utilisent REA, signalent des bugs, proposent des fonctionnalités, testent les builds et contribuent des correctifs.
|
||||
|
||||
|
||||
+1
-1
@@ -283,7 +283,7 @@ Laporkan kerentanan melalui [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Riwayat Bintang
|
||||
|
||||
🎉 **90.000 bintang GitHub, terima kasih!**
|
||||
🎉 **110.000 bintang GitHub, terima kasih!**
|
||||
|
||||
Terima kasih kepada semua orang yang menggunakan REA, melaporkan bug, meminta fitur, menguji build, dan berkontribusi dalam perbaikan.
|
||||
|
||||
|
||||
+1
-1
@@ -262,7 +262,7 @@ Segnala vulnerabilità tramite [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Cronologia delle stelle
|
||||
|
||||
🎉 **90.000 stelle su GitHub — grazie!**
|
||||
🎉 **110.000 stelle su GitHub — grazie!**
|
||||
|
||||
Grazie a tutti coloro che usano REA, segnalano bug, propongono funzionalità, testano le build e contribuiscono con correzioni.
|
||||
|
||||
|
||||
+1
-1
@@ -260,7 +260,7 @@ npx rea-agents@latest setup
|
||||
|
||||
## スター履歴
|
||||
|
||||
🎉 **GitHub スター 90,000 件、ありがとうございます!**
|
||||
🎉 **GitHub スター 110,000 件、ありがとうございます!**
|
||||
|
||||
REA の利用、バグ報告、機能の要望、ビルドのテスト、修正への貢献に感謝します。
|
||||
|
||||
|
||||
+1
-1
@@ -260,7 +260,7 @@ npx rea-agents@latest setup
|
||||
|
||||
## 스타 기록
|
||||
|
||||
🎉 **GitHub 스타 90,000개, 감사합니다!**
|
||||
🎉 **GitHub 스타 110,000개, 감사합니다!**
|
||||
|
||||
REA를 사용하고, 버그를 보고하고, 기능을 요청하고, 빌드를 테스트하고, 수정에 기여해 주신 모든 분께 감사드립니다.
|
||||
|
||||
|
||||
+1
-1
@@ -258,7 +258,7 @@ Zgłaszaj podatności zgodnie z [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Historia gwiazdek
|
||||
|
||||
🎉 **90 000 gwiazdek na GitHubie — dziękujemy!**
|
||||
🎉 **110 000 gwiazdek na GitHubie — dziękujemy!**
|
||||
|
||||
Dziękujemy wszystkim, którzy używają REA, zgłaszają błędy, proponują nowe funkcje, testują kompilacje i przesyłają poprawki.
|
||||
|
||||
|
||||
+1
-1
@@ -258,7 +258,7 @@ Relate vulnerabilidades conforme [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Histórico de estrelas
|
||||
|
||||
🎉 **90.000 estrelas no GitHub — muito obrigado!**
|
||||
🎉 **110.000 estrelas no GitHub — muito obrigado!**
|
||||
|
||||
Obrigado a todos que usam o REA, relatam bugs, sugerem funcionalidades, testam builds e contribuem com correções.
|
||||
|
||||
|
||||
+1
-1
@@ -258,7 +258,7 @@ npx rea-agents@latest setup
|
||||
|
||||
## История звёзд
|
||||
|
||||
🎉 **90 000 звёзд на GitHub — спасибо!**
|
||||
🎉 **110 000 звёзд на GitHub — спасибо!**
|
||||
|
||||
Спасибо всем, кто использует REA, сообщает об ошибках, предлагает новые функции, тестирует сборки и помогает с исправлениями.
|
||||
|
||||
|
||||
+1
-1
@@ -258,7 +258,7 @@ npx rea-agents@latest setup
|
||||
|
||||
## ประวัติดาว
|
||||
|
||||
🎉 **90,000 ดาวบน GitHub — ขอบคุณทุกคน!**
|
||||
🎉 **110,000 ดาวบน GitHub — ขอบคุณทุกคน!**
|
||||
|
||||
ขอบคุณทุกคนที่ใช้ REA รายงานบั๊ก เสนอฟีเจอร์ ทดสอบบิลด์ และช่วยแก้ไข
|
||||
|
||||
|
||||
+1
-1
@@ -258,7 +258,7 @@ Güvenlik açıklarını [SECURITY.md](SECURITY.md) üzerinden bildirin.
|
||||
|
||||
## Yıldız geçmişi
|
||||
|
||||
🎉 **GitHub'da 90.000 yıldız — teşekkürler!**
|
||||
🎉 **GitHub'da 110.000 yıldız — teşekkürler!**
|
||||
|
||||
REA'yı kullanan, hata bildiren, özellik talep eden, derlemeleri test eden ve düzeltmelere katkıda bulunan herkese teşekkürler.
|
||||
|
||||
|
||||
+1
-1
@@ -258,7 +258,7 @@ npx rea-agents@latest setup
|
||||
|
||||
## Історія зірок
|
||||
|
||||
🎉 **90 000 зірок на GitHub — дякуємо!**
|
||||
🎉 **110 000 зірок на GitHub — дякуємо!**
|
||||
|
||||
Дякуємо всім, хто використовує REA, повідомляє про помилки, пропонує нові функції, тестує збірки й допомагає з виправленнями.
|
||||
|
||||
|
||||
+1
-1
@@ -258,7 +258,7 @@ Báo cáo lỗ hổng theo [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Lịch sử sao
|
||||
|
||||
🎉 **90.000 sao trên GitHub — xin cảm ơn!**
|
||||
🎉 **110.000 sao trên GitHub — xin cảm ơn!**
|
||||
|
||||
Cảm ơn mọi người đã sử dụng REA, báo lỗi, đề xuất tính năng, kiểm thử bản dựng và đóng góp bản sửa lỗi.
|
||||
|
||||
|
||||
+1
-1
@@ -260,7 +260,7 @@ npx rea-agents@latest setup
|
||||
|
||||
## Star 歷史
|
||||
|
||||
🎉 **GitHub Star 達到 90,000 個,感謝大家!**
|
||||
🎉 **GitHub Star 達到 110,000 個,感謝大家!**
|
||||
|
||||
感謝每一位使用 REA、報告錯誤、提出功能需求、測試建置和貢獻修復的朋友。
|
||||
|
||||
|
||||
+1
-1
@@ -260,7 +260,7 @@ npx rea-agents@latest setup
|
||||
|
||||
## Star 历史
|
||||
|
||||
🎉 **GitHub Star 达到 90,000 个,感谢大家!**
|
||||
🎉 **GitHub Star 达到 110,000 个,感谢大家!**
|
||||
|
||||
感谢每一位使用 REA、报告错误、提出功能需求、测试构建和贡献修复的朋友。
|
||||
|
||||
|
||||
@@ -908,6 +908,19 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
result.add("procedure", function == null ? JsonNull.INSTANCE : GSON.toJsonTree(canonicalAddress(function.getEntryPoint())));
|
||||
result.addProperty("architecture", currentProgram.getLanguageID().getIdAsString());
|
||||
result.addProperty("mode", currentProgram.getLanguage().getLanguageDescription().getVariant());
|
||||
if (currentProgram.getLanguageID().getIdAsString().startsWith("MIPS:")) {
|
||||
JsonArray decoderContext = new JsonArray();
|
||||
var modeRegister = currentProgram.getRegister("ISA_MODE");
|
||||
if (modeRegister != null) {
|
||||
var modeValue = currentProgram.getProgramContext().getValue(modeRegister, address, false);
|
||||
JsonObject context = new JsonObject();
|
||||
context.addProperty("register", modeRegister.getName());
|
||||
context.add("value", modeValue == null ? JsonNull.INSTANCE : GSON.toJsonTree(modeValue.toString()));
|
||||
context.addProperty("source", "ghidra-program-context");
|
||||
decoderContext.add(context);
|
||||
}
|
||||
result.add("decoder_context", decoderContext);
|
||||
}
|
||||
JsonArray operands = new JsonArray();
|
||||
JsonArray references = new JsonArray();
|
||||
JsonArray destinations = new JsonArray();
|
||||
|
||||
@@ -112,6 +112,11 @@ enum RunTokenReadError: Error {
|
||||
case ambiguousEnvironmentBoundary
|
||||
}
|
||||
|
||||
struct ParsedRunToken {
|
||||
let runId: String?
|
||||
let environmentEntryCount: Int
|
||||
}
|
||||
|
||||
@_silgen_name("csops")
|
||||
private func csops(
|
||||
_ pid: Int32,
|
||||
@@ -122,19 +127,67 @@ private func csops(
|
||||
|
||||
private let csOpsStatus: UInt32 = 0
|
||||
private let csPlatformBinary: UInt32 = 0x0400_0000
|
||||
private let csRestrict: UInt32 = 0x0000_0800
|
||||
|
||||
/**
|
||||
* Whether the target image is an Apple platform binary. Current macOS omits
|
||||
* the environment of platform binaries from KERN_PROCARGS2 for every caller,
|
||||
* so this distinguishes a policy-withheld environment from a process whose
|
||||
* environment is genuinely absent or transiently unreadable.
|
||||
*/
|
||||
func isPlatformBinary(pid: Int32) -> Bool {
|
||||
/** Read the target's current code-signing flags when csops permits inspection. */
|
||||
private func codeSigningFlags(pid: Int32) -> UInt32? {
|
||||
var flags: UInt32 = 0
|
||||
let result = withUnsafeMutablePointer(to: &flags) { pointer in
|
||||
csops(pid, csOpsStatus, UnsafeMutableRawPointer(pointer), MemoryLayout<UInt32>.size)
|
||||
}
|
||||
return result == 0 && (flags & csPlatformBinary) != 0
|
||||
return result == 0 ? flags : nil
|
||||
}
|
||||
|
||||
/** Derive one process-token observation from parser facts and native flags. */
|
||||
func processRunTokenObservation(
|
||||
pid: Int32,
|
||||
parse: Result<ParsedRunToken, RunTokenReadError>,
|
||||
codeSigningFlags: () -> UInt32?
|
||||
) -> RunTokenObservation {
|
||||
switch parse {
|
||||
case .success(let parsed):
|
||||
if let runId = parsed.runId {
|
||||
return RunTokenObservation(pid: pid, state: "readable", run_id: runId, reason: nil)
|
||||
}
|
||||
// A non-empty environment without REA's token proves this candidate is
|
||||
// unrelated. CS_RESTRICT cannot turn that readable fact into uncertainty.
|
||||
if parsed.environmentEntryCount > 0 {
|
||||
return RunTokenObservation(pid: pid, state: "readable", run_id: nil, reason: nil)
|
||||
}
|
||||
let flags = codeSigningFlags()
|
||||
if let flags, (flags & csPlatformBinary) != 0 {
|
||||
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
|
||||
}
|
||||
if let flags, (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return RunTokenObservation(pid: pid, state: "readable", run_id: nil, reason: nil)
|
||||
case .failure(let failure):
|
||||
switch failure {
|
||||
case .environmentUnavailable:
|
||||
let flags = codeSigningFlags()
|
||||
if let flags, (flags & csPlatformBinary) != 0 {
|
||||
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
|
||||
}
|
||||
if let flags, (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return unavailable(pid: pid, reason: "environment_unavailable")
|
||||
case .ambiguousEnvironmentBoundary:
|
||||
if let flags = codeSigningFlags(), (flags & csRestrict) != 0 {
|
||||
return unavailable(pid: pid, reason: "restricted_process_environment_withheld")
|
||||
}
|
||||
return unavailable(pid: pid, reason: "ambiguous_environment_boundary")
|
||||
case .duplicateToken:
|
||||
return unavailable(pid: pid, reason: "duplicate_run_token")
|
||||
case .invalidTokenEncoding:
|
||||
return unavailable(pid: pid, reason: "invalid_run_token_encoding")
|
||||
case .appleVectorUnavailable:
|
||||
return unavailable(pid: pid, reason: "apple_vector_unavailable")
|
||||
case .invalidBuffer:
|
||||
return unavailable(pid: pid, reason: "malformed_procargs")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func readRunToken(pid: Int32, pointerSize: Int) -> RunTokenObservation {
|
||||
@@ -151,33 +204,19 @@ private func readRunToken(pid: Int32, pointerSize: Int) -> RunTokenObservation {
|
||||
return unavailable(pid: pid, reason: "sysctl_failed_\(errno)")
|
||||
}
|
||||
bytes = Array(bytes.prefix(size))
|
||||
let parse: Result<ParsedRunToken, RunTokenReadError>
|
||||
do {
|
||||
let runId = try token(in: bytes, pointerSize: pointerSize)
|
||||
// A complete empty environment is readable for an ordinary image. Apple
|
||||
// platform images can expose the same layout after the kernel withholds
|
||||
// their environment, so they remain explicitly unverified without a token.
|
||||
if runId == nil && isPlatformBinary(pid: pid) {
|
||||
return unavailable(pid: pid, reason: "platform_binary_environment_withheld")
|
||||
}
|
||||
return RunTokenObservation(pid: pid, state: "readable", run_id: runId, reason: nil)
|
||||
} catch RunTokenReadError.duplicateToken {
|
||||
return unavailable(pid: pid, reason: "duplicate_run_token")
|
||||
} catch RunTokenReadError.invalidTokenEncoding {
|
||||
return unavailable(pid: pid, reason: "invalid_run_token_encoding")
|
||||
} catch RunTokenReadError.environmentUnavailable {
|
||||
return unavailable(
|
||||
pid: pid,
|
||||
reason: isPlatformBinary(pid: pid)
|
||||
? "platform_binary_environment_withheld"
|
||||
: "environment_unavailable"
|
||||
)
|
||||
} catch RunTokenReadError.appleVectorUnavailable {
|
||||
return unavailable(pid: pid, reason: "apple_vector_unavailable")
|
||||
} catch RunTokenReadError.ambiguousEnvironmentBoundary {
|
||||
return unavailable(pid: pid, reason: "ambiguous_environment_boundary")
|
||||
parse = .success(try parseProcessEnvironment(in: bytes, pointerSize: pointerSize))
|
||||
} catch let failure as RunTokenReadError {
|
||||
parse = .failure(failure)
|
||||
} catch {
|
||||
return unavailable(pid: pid, reason: "malformed_procargs")
|
||||
parse = .failure(.invalidBuffer)
|
||||
}
|
||||
return processRunTokenObservation(
|
||||
pid: pid,
|
||||
parse: parse,
|
||||
codeSigningFlags: { codeSigningFlags(pid: pid) }
|
||||
)
|
||||
}
|
||||
|
||||
func readRunTokens(pids: [Int32]) -> [RunTokenObservation] {
|
||||
@@ -201,7 +240,10 @@ func readRunTokens(pids: [Int32]) -> [RunTokenObservation] {
|
||||
}
|
||||
}
|
||||
|
||||
func token(in bytes: [UInt8], pointerSize: Int = MemoryLayout<UnsafeRawPointer>.size) throws -> String? {
|
||||
func parseProcessEnvironment(
|
||||
in bytes: [UInt8],
|
||||
pointerSize: Int = MemoryLayout<UnsafeRawPointer>.size
|
||||
) throws -> ParsedRunToken {
|
||||
guard bytes.count >= MemoryLayout<Int32>.size else { throw RunTokenReadError.invalidBuffer }
|
||||
let argc = bytes.withUnsafeBytes { $0.loadUnaligned(as: Int32.self).littleEndian }
|
||||
guard argc >= 0, Int(argc) <= bytes.count else { throw RunTokenReadError.invalidBuffer }
|
||||
@@ -277,7 +319,7 @@ func token(in bytes: [UInt8], pointerSize: Int = MemoryLayout<UnsafeRawPointer>.
|
||||
found = decoded
|
||||
}
|
||||
}
|
||||
return found
|
||||
return ParsedRunToken(runId: found, environmentEntryCount: environmentEntries.count)
|
||||
}
|
||||
|
||||
private func readString(
|
||||
|
||||
@@ -206,6 +206,11 @@ def inspect_elf(path, cache):
|
||||
"visibility": raw.st_other.visibility, "section_index": raw.st_shndx,
|
||||
})
|
||||
if h.sh_type in ("SHT_REL", "SHT_RELA"):
|
||||
if h.sh_size % h.sh_entsize != 0:
|
||||
raise LayoutFailure(
|
||||
"format",
|
||||
f"ELF relocation section {index} size is not a multiple of its {h.sh_entsize}-byte entries.",
|
||||
)
|
||||
if image.header.e_type == "ET_REL" and h.sh_info >= len(all_sections):
|
||||
raise LayoutFailure("format", f"ELF relocation section {index} references missing target section {h.sh_info}.")
|
||||
symbol_table = None
|
||||
|
||||
@@ -26,6 +26,7 @@ export default defineConfig({
|
||||
text: "Investigation guides",
|
||||
items: [
|
||||
{ text: "Native binaries", link: "/native-investigation" },
|
||||
{ text: "Go build metadata", link: "/go-binaries" },
|
||||
{
|
||||
text: "JavaScript and Electron",
|
||||
link: "/javascript-artifact-reconstruction",
|
||||
|
||||
@@ -88,7 +88,7 @@ existing analysis families directly, for example
|
||||
| `list_adb_packages` | read | `pm list packages -f` with scope; base APK device paths |
|
||||
| `resolve_adb_packages` | read | Case-insensitive package-name substring search with exact-match flag |
|
||||
| `inspect_adb_package` | read | `dumpsys package` projection: versions, install times, installer, flags, permission count |
|
||||
| `list_adb_directory` | read | `ls -la`: kinds, permissions, owner, group, sizes, symlink targets |
|
||||
| `list_adb_directory` | read | NUL-framed immediate children, including hidden entries: kinds, permissions, owner, group, sizes, symlink targets |
|
||||
| `pull_adb_package` | read | Complete split APK set from `pm path`, each file digested |
|
||||
| `pull_adb_file` | read | One absolute device path pulled and digested |
|
||||
| `capture_adb_screen` | read | One `screencap -p` frame (exec-out, no device-side file) with dimensions |
|
||||
@@ -102,6 +102,9 @@ existing analysis families directly, for example
|
||||
|
||||
## Interpret the result
|
||||
|
||||
- **Directory names retain whitespace and newlines.** Symlink targets do too,
|
||||
including trailing newlines. Records containing invalid UTF-8 remain as
|
||||
labelled hex in `unparsed_lines` and make directory coverage partial.
|
||||
- **Inferences carry their basis.** Device `kind` reports
|
||||
`kind_basis` (serial prefix or USB transport); APK `role` reports
|
||||
`role_basis: file_name`; `emulator_observed` is the build's own
|
||||
@@ -109,12 +112,12 @@ existing analysis families directly, for example
|
||||
- **Device paths are not host paths.** `base_apk_device_path` values live on
|
||||
the device filesystem. Only pulls and captures produce host-local paths,
|
||||
each paired with byte size and SHA-256 digest. A pulled file's host name is
|
||||
the device path's last segment, accepted only when every host can store it:
|
||||
a `pull_adb_file` whose device path ends in a separator, `.`/`..`, a
|
||||
control or reserved character, a Windows reserved device name, a trailing
|
||||
dot or space, or a `:` alternate-data-stream marker is refused as
|
||||
`invalid_request` before any `adb pull` runs, and REA never renames a
|
||||
device-reported file.
|
||||
the device path's final POSIX segment. REA preserves the original device
|
||||
path and filename; a missing final segment or `.`/`..` is refused before
|
||||
output creation. Windows hosts follow [Windows filename rules](https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file),
|
||||
while POSIX hosts retain valid POSIX filenames such as names with spaces or
|
||||
colons. A package pull checks reported names against the destination
|
||||
filesystem before pulling, and refuses names that would share a local path.
|
||||
- **Splits are part of the answer.** `pm path` enumerates the base APK and
|
||||
every installed split; pulls cover the whole set. System packages whose
|
||||
APKs keep other names report the `unknown` role rather than guessing.
|
||||
|
||||
@@ -107,7 +107,11 @@ the same source byte limit and explicit truncation reporting.
|
||||
upstream engine and should be checked against the retained XML when needed.
|
||||
Signature verification is explicitly `not_performed`.
|
||||
- Class search consumes all pages and checks their counts. An empty `query`
|
||||
inventories all class names; class names are decompiler representations.
|
||||
inventories all class-name rows; class names are decompiler representations.
|
||||
Repeated names remain repeated in `matches` so the inventory row count and
|
||||
producer evidence stay visible. `duplicate_names` identifies matching names
|
||||
with multiple rows and their occurrence counts; a name alone does not
|
||||
distinguish which provider class node a later lookup selects.
|
||||
- Class inventories read parsed metadata without generating source and return
|
||||
full reported display types, fields and inner class names. Synthetic members
|
||||
that code generation hides can appear in the inventory; it stays consistent
|
||||
|
||||
+40
@@ -37,8 +37,44 @@ module page. See
|
||||
[JavaScript application workflows](javascript-application-workflows.md) for
|
||||
views, feature traces and comparisons that take the same saved Evidence.
|
||||
|
||||
## Managed assemblies (.NET)
|
||||
|
||||
For a .NET PE/CLI assembly, inspect its identity and deployment form before
|
||||
inspecting members. These commands read metadata and Common Intermediate
|
||||
Language (CIL) without loading or executing the assembly and require no native
|
||||
analysis engine:
|
||||
|
||||
```bash
|
||||
rea inspect-managed-artifact /absolute/path/Assembly.dll --json
|
||||
rea inspect-managed-members /absolute/path/Assembly.dll --json
|
||||
rea inspect-managed-members /absolute/path/Assembly.dll --type Example.Parser --json
|
||||
```
|
||||
|
||||
Replace `Example.Parser` with an exact `full_name` from the member inspection,
|
||||
or a declaring type name you already know. Omit `--type` to inspect all members,
|
||||
or repeat it to select several types.
|
||||
Selection limits decoding to the selected types; member references remain an
|
||||
assembly-wide inventory, so the output can still be large.
|
||||
|
||||
Choose the workflow from the artifact's contents rather than its `.exe` or
|
||||
`.dll` extension. The managed parser requires PE/CLI metadata; it does not
|
||||
unpack .NET single-file hosts or decode IL2CPP metadata. NativeAOT method bodies
|
||||
require native analysis. See the [managed-code guide](managed-code-analysis.md)
|
||||
for supported deployment forms, selection semantics, and coverage limits.
|
||||
|
||||
## Native analysis
|
||||
|
||||
Inspect embedded Go build metadata without installing Go or configuring a
|
||||
disassembler:
|
||||
|
||||
```bash
|
||||
rea inspect-go-binary /absolute/path/to/program --json
|
||||
```
|
||||
|
||||
The shared CLI/MCP workflow reports compiler version, modules, replacements,
|
||||
build settings, original bytes and SHA-256. See [Go binaries](go-binaries.md)
|
||||
for supported containers and absent or malformed metadata.
|
||||
|
||||
Configure [Hopper or Ghidra](installation.md#hopper), or the
|
||||
[IDA adapter](ida-provider.md), before analyzing a native target. Substitute
|
||||
your target, search text and function name or address in these examples:
|
||||
@@ -114,6 +150,10 @@ an exact result when the target bytes, operation, parameters, provider and
|
||||
settings match. Mutations and cursor-dependent calls are excluded from the
|
||||
cache. Snapshot files are local and use owner-only permissions.
|
||||
|
||||
Dispatching a metadata write invalidates the active session's snapshot cache,
|
||||
even if its reply fails: the provider may already have applied the write.
|
||||
Close and reopen the target before importing or saving another snapshot.
|
||||
|
||||
Snapshots retain eligible target-scoped question histories in full, including
|
||||
mutation Evidence recorded while another target was active. Observations and
|
||||
related questions must still belong to the saved target. If those dependencies
|
||||
|
||||
+65
-5
@@ -16,9 +16,10 @@ PE, managed and Mach-O architecture sets are unchanged.
|
||||
The Ghidra lane admits ELF32 `ET_EXEC` and `ET_REL`, little or big endian, with
|
||||
explicit o32 and `EF_MIPS_ARCH_32R2` flags and no machine-specific or ASE
|
||||
encoding flags. These are header declarations, not proof that every instruction
|
||||
conforms to that ISA. ELF64, n32/n64, unspecified ABI, MIPS16/microMIPS,
|
||||
machine-specific variants, shared objects and PSP PRX are not covered by this
|
||||
lane. Their family may be identified, but provider selection returns the
|
||||
conforms to that ISA. ELF64, n32/n64, unspecified ABI, microMIPS,
|
||||
machine-specific variants and PSP PRX are not covered by this standard-instruction
|
||||
lane. MIPS16 shared images have the separate bounded profile below; other shared
|
||||
or compressed images remain unsupported. Their family may be identified, but provider selection returns the
|
||||
specific unsupported constraint rather than guessing a compatible profile.
|
||||
|
||||
Explicit `EF_MIPS_FP64` and `EF_MIPS_NAN2008` declarations are refused. Their
|
||||
@@ -49,6 +50,63 @@ values are retained for diagnosis but do not gain provider support. These checks
|
||||
bind the declared interpretation; the fixtures do not establish full
|
||||
floating-point behavioral conformance.
|
||||
|
||||
### MIPS16 shared images
|
||||
|
||||
A separate profile admits ELF32 `ET_DYN`, either byte order, declaring explicit
|
||||
o32, MIPS32r2 and exactly the MIPS16 ASE. Machine variants, n32, FP64 and NAN2008
|
||||
remain refused. Source inspection must examine the complete dynamic relocation
|
||||
declarations through uniquely mapped file-backed `PT_LOAD` ranges. This profile
|
||||
verifies `R_MIPS_NONE` and symbol-index-zero `R_MIPS_REL32`; section declarations
|
||||
must describe that same dynamic table. Unavailable section headers and additional
|
||||
section relocation tables (including `--emit-relocs` output) remain unsupported.
|
||||
Symbol-bearing REL32,
|
||||
other relocation types, RELA and PLT declarations remain unsupported. A real
|
||||
negative control shows that Ghidra can normalize a symbol-bearing MIPS16 function
|
||||
pointer to an even address, losing the source mode tag. REA refuses that shape
|
||||
rather than rewriting the source or silently repairing the analysis result.
|
||||
|
||||
The profile pins the endian-matched `MIPS:<LE|BE>:32:default` processor and default
|
||||
compiler specification, and checks the bridge-reported language after import.
|
||||
The generic processor preserves mixed MIPS32/MIPS16 code through per-address
|
||||
`ISA_MODE` context; a global `16e` processor would impose one mode across the
|
||||
image. A caller-selected incompatible processor or compiler specification is
|
||||
explicitly refused without replacing the requested choice.
|
||||
|
||||
An observed ABI flags record must agree with the header and MIPS16 ASE, and
|
||||
must declare the same verified floating-point combinations as the standard
|
||||
lane. With no observed record, every ABI declaration table must have been
|
||||
completely examined. The header still declares o32, but floating-point ABI
|
||||
semantics remain unknown and are stated in Evidence; selecting the default
|
||||
compiler specification does not establish floating-point storage. Partial
|
||||
section tables do not prove record absence. The existing soft-float prototype
|
||||
policy applies when the source explicitly declares soft-float.
|
||||
|
||||
Returned addresses are analysis coordinates, not runtime load addresses.
|
||||
`inspect_native_load_image` preserves original relocation bytes, current loaded
|
||||
memory bytes and provider status. External GOT bindings remain unverified
|
||||
without their defining images. The provider's `modified_sha256` describes its
|
||||
ephemeral relocated image and can differ from `original_sha256`; it does not
|
||||
mean that the caller's artifact was written. `inspect_native_instruction` retains the program
|
||||
language variant in `mode` and reports actual per-address `ISA_MODE` separately
|
||||
in `decoder_context`, including null when unavailable. Artifact bytes and digests
|
||||
remain unchanged; this workflow does not execute the shared library.
|
||||
|
||||
The optional real-provider check builds eight source-owned GNU controls, covering
|
||||
both byte orders, mixed MIPS32/MIPS16 code, declared and absent ABI records, and
|
||||
the rejected symbol-bearing and additional section relocations. It compares source instruction bytes,
|
||||
function symbols and the local function-pointer tag with actual CLI/MCP results,
|
||||
checks context through a snapshot round trip, and verifies process cleanup.
|
||||
It requires caller-supplied GNU MIPS cross-binutils and Ghidra; no fixture target
|
||||
is executed or patched after linking. Floating-point behavior of compressed
|
||||
functions is not established by these integer controls.
|
||||
|
||||
```bash
|
||||
REA_MIPS_AS=/absolute/path/to/mips-linux-gnu-as \
|
||||
REA_MIPS_LD=/absolute/path/to/mips-linux-gnu-ld \
|
||||
GHIDRA_INSTALL_DIR=/absolute/path/to/ghidra \
|
||||
npm run verify:ghidra:mips-shared
|
||||
```
|
||||
|
||||
MIPS header and ABI record facts are committed in the analysis profile so
|
||||
interpretation changes invalidate profile-bound snapshots. The support-lane
|
||||
identity is versioned independently of file identity. Evidence, lifecycle
|
||||
@@ -186,8 +244,10 @@ from the original file bytes. HI16/LO16 are checked as the effective address
|
||||
each LO16 forms with each HI16 it resolves, including several HI16s before one
|
||||
LO16, so a missing signed-low carry fails and a HI16 without a following LO16
|
||||
fails. The lane also requires matching Ghidra relocation entries and statuses,
|
||||
section file offsets and sizes, coverage of a shared HI16, a LO16 carry,
|
||||
external code and data and `R_MIPS_32` words, direct calls from the entry
|
||||
section file offsets and sizes, coverage of a shared HI16, a LO16 resolving
|
||||
several pending HI16s (from an assembly block, since Clang does not emit that
|
||||
from C), a LO16 carry, external code and data and `R_MIPS_32` words, direct
|
||||
calls from the entry
|
||||
function, the relocatable support lane in the snapshot profile, the
|
||||
relocatable limitation in Evidence and an unchanged source object. The fixture
|
||||
also calls `powf`, an external whose float prototype comes from Ghidra's C
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
# Go binary build metadata
|
||||
|
||||
`inspect_go_binary` / `inspect-go-binary` answers which Go compiler version,
|
||||
modules, replacements and build settings are embedded in one selected local
|
||||
executable or library. It reads the file without running it and requires no
|
||||
Go installation, active binary target, Hopper, Ghidra or IDA.
|
||||
|
||||
```bash
|
||||
rea inspect-go-binary /absolute/path/to/application --json
|
||||
```
|
||||
|
||||
The CLI resolves relative paths against the caller's working directory. MCP
|
||||
requires an absolute filesystem path and preserves that exact selection:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_go_binary",
|
||||
"arguments": { "path": "/artifacts/application" }
|
||||
}
|
||||
```
|
||||
|
||||
## Observations
|
||||
|
||||
The inline Evidence result contains the artifact path, SHA-256, byte length,
|
||||
container format, architecture, word size and byte order. A recognized
|
||||
`build_info` record contains:
|
||||
|
||||
- `go_version`, preserved as embedded UTF-8 text rather than forced into a
|
||||
semantic version, or `null` when the embedded bytes are not UTF-8;
|
||||
- `go_version_bytes_base64`, preserving the exact compiler-string bytes;
|
||||
- exact file offsets and byte lengths for the header, compiler string and module
|
||||
string;
|
||||
- `module_text`, the decoded module-information body, or `null` when the complete
|
||||
body cannot be represented as UTF-8;
|
||||
- `module_bytes_base64`, the original module-string bytes including Go framing;
|
||||
- `module.path`, the embedded package path;
|
||||
- `module.main`, the main module's path, version, checksum and optional
|
||||
replacement;
|
||||
- `module.dependencies`, with the same fields and their original order;
|
||||
- `module.settings`, retaining ordered key/value entries and duplicate keys;
|
||||
- `module.unparsed_lines`, `module.unparsed_line_bytes_base64` and
|
||||
`module.complete`, so unfamiliar, malformed, conflicting or undecodable records
|
||||
do not disappear behind a complete result.
|
||||
|
||||
Missing checksums are `null`; empty fields and development/local replacement
|
||||
versions remain unchanged. Build settings such as `GOOS`, `GOARCH` or `vcs.revision`
|
||||
are reported when present. Their absence does not establish a build choice.
|
||||
Go strings may contain arbitrary bytes. Non-UTF-8 records remain available as
|
||||
base64 while other decodable module records are still reported; the reader does
|
||||
not insert replacement characters or reject the entire artifact for those
|
||||
strings.
|
||||
|
||||
## Supported boundary
|
||||
|
||||
The built-in TypeScript reader supports ELF, PE and thin Mach-O images with Go's
|
||||
inline or historical pointer build-info encoding. Source ranges refer to the
|
||||
original stable file snapshot, including pointer-backed strings. It does not
|
||||
scan arbitrary byte strings as compiler metadata or interpret a fat Mach-O
|
||||
container as one selected architecture.
|
||||
|
||||
Historical moduleless Go binaries can store their empty module-string header in
|
||||
declared zero-initialized virtual memory rather than file-backed bytes. The reader
|
||||
recognizes this only when the complete optional module header lies in that memory
|
||||
without a conflicting file mapping. Its module contents are empty and
|
||||
`module_location` has zero bytes, anchored at the build-info field containing the
|
||||
module-header pointer. Compiler strings and other unmapped pointers remain strict
|
||||
file-backed reads.
|
||||
|
||||
`build_info: null` means no recognized record was found in the inspected image.
|
||||
It does not prove that the binary is not Go. Stripping, old builds, deliberate
|
||||
removal and obfuscation can limit metadata. A malformed supported container or
|
||||
recognized build-info record returns an input-format failure. Unsupported
|
||||
containers receive a separate unsupported-target failure.
|
||||
|
||||
Symlinks and nonregular files are rejected. The shared stable reader detects
|
||||
replacement and observed in-place changes during acquisition. Missing files,
|
||||
host permission denial, changed artifacts and cancellation retain distinct
|
||||
errors. The selected file is never modified; this inspection creates no
|
||||
temporary files, starts no subprocesses and accesses no network endpoint.
|
||||
|
||||
Resource guards bound the selected file to 256 MiB, aggregate embedded strings
|
||||
to 1 MiB, structural table decoding to 16 MiB and the complete normalized metadata
|
||||
report to 8 MiB. Oversized results fail without partial success. MCP also applies
|
||||
its [response transport budget](mcp-contracts.md), which counts both text and
|
||||
structured content; a report within the metadata limit can still require export
|
||||
from retained Evidence. These limits are not operating-system memory or CPU
|
||||
isolation.
|
||||
|
||||
Embedded metadata is evidence of file content. It does not authenticate a
|
||||
compiler, dependency, commit or supply chain, and it does not establish runtime
|
||||
behavior. Function names, source mappings, types and decompiled code are outside
|
||||
this tool's coverage. Use the [native workflow](native-investigation.md) when
|
||||
those questions require a disassembler.
|
||||
|
||||
## Real producer verification
|
||||
|
||||
`npm run verify:go:binary` exercises compiler-built fixtures through the public
|
||||
CLI and MCP, comparing their metadata with Go's standard `debug/buildinfo`
|
||||
reader. The verification lane requires Go; normal inspection does not.
|
||||
+90
-17
@@ -89,8 +89,11 @@ running server or its registration.
|
||||
|
||||
The npm release checked on **2026-10-11** was **6.4.0**, published from
|
||||
[`a56f7bb6`](https://github.com/morluto/rea/commit/a56f7bb668ab23fec8d33f7af353ac7ed2fe1895).
|
||||
The actual registry package accepted dry-run setup plans for all 17 client IDs
|
||||
listed below, including Qwen Code, Pi and Hermes. OpenCode's plan selected
|
||||
The actual registry package accepted dry-run setup plans for the 17 client IDs
|
||||
in that release, including Qwen Code, Pi and Hermes. DeepSeek Harness (`dsh`)
|
||||
is on main and is not in npm 6.4.0; `rea setup --client dsh` against `@latest`
|
||||
reports an unsupported client until a release containing it is published.
|
||||
OpenCode's plan selected
|
||||
`OPENCODE_CONFIG_DIR` even when that directory did not yet exist. These checks
|
||||
used an isolated home on macOS with Node 24; they verify planning and argument
|
||||
admission, without writing client configuration or establishing live integration
|
||||
@@ -162,6 +165,10 @@ listed after the table because its connector is not one of these files:
|
||||
| OMP | `omp` |
|
||||
| Pi | `pi` |
|
||||
| Hermes | `hermes` |
|
||||
| DeepSeek Harness | `dsh` |
|
||||
|
||||
`dsh` is main-only until a release after 6.4.0. The registry check above does
|
||||
not cover it.
|
||||
|
||||
For GitHub Copilot CLI 1.0.95, `gpt-4.1` model metadata can block ordinary
|
||||
chat with REA's full input-schema profile before sending a model request
|
||||
@@ -182,6 +189,35 @@ schema constraints; REA still validates complete canonical inputs. Full-profile
|
||||
`gpt-5.4` metadata is also verified locally. Live model API acceptance and
|
||||
Windows remain unverified; see [native client verification](testing.md).
|
||||
|
||||
Claude Code normally defers MCP tool definitions through tool search. It
|
||||
[turns tool search off](https://code.claude.com/docs/en/mcp#configure-tool-search)
|
||||
when `ANTHROPIC_BASE_URL` points to a non-first-party host, such as DeepSeek's
|
||||
Anthropic-compatible endpoint. It then sends every REA tool's name, description
|
||||
and input schema with every request
|
||||
([#2019](https://github.com/morluto/rea/issues/2019)). On main at 04089d5be,
|
||||
with 172 tools, that payload measured about 496 KB with the default profile and
|
||||
about 290 KB with `compact`. By the o200k and cl100k tokenizers, that is roughly
|
||||
120k and 68k tokens. Compaction cannot remove tool definitions. If they nearly
|
||||
fill the window Claude Code assumes, the context refills after every compaction
|
||||
and Claude Code reports that autocompact is thrashing. To avoid this:
|
||||
|
||||
- Make Claude Code assume the model's actual context window. DeepSeek's Claude
|
||||
Code guide selects `deepseek-flash[1m]`; the `[1m]` suffix makes Claude Code
|
||||
assume a one-million-token window. For other unrecognized model IDs, set
|
||||
`CLAUDE_CODE_MAX_CONTEXT_TOKENS` to the window the provider documents.
|
||||
- If the endpoint forwards `tool_reference` blocks, set `ENABLE_TOOL_SEARCH=true`
|
||||
to restore deferred loading. DeepSeek's Anthropic API guide does not list
|
||||
`tool_reference` among its supported content blocks, so this setting may cause
|
||||
its requests to fail.
|
||||
- To shrink the upfront catalog, add
|
||||
`"REA_MCP_INPUT_SCHEMA_PROFILE": "compact"` to the `env` object of the
|
||||
`mcpServers.rea` entry in `~/.claude.json` (`$CLAUDE_CONFIG_DIR/.claude.json`
|
||||
when that variable is set), then restart Claude Code. Setup keeps this
|
||||
override.
|
||||
|
||||
The sizes above come from REA's own `tools/list`. A live Claude Code session
|
||||
against DeepSeek's API is unverified.
|
||||
|
||||
For [Pi](https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/mcp.md),
|
||||
setup writes a `type: "stdio"` entry under `mcpServers.rea` in the user-level
|
||||
`~/.pi/agent/mcp.json` and installs the bundled skill under `~/.agents/skills`.
|
||||
@@ -259,11 +295,15 @@ configuration and skill destinations; doctor and uninstall use those same paths.
|
||||
|
||||
### DeepSeek Harness
|
||||
|
||||
DeepSeek Harness uses a Cordis YAML patch sequence rather than a `mcpServers`
|
||||
object. REA's `setup --client` does not edit this format. For Harness
|
||||
`0.2.0-rc.2`, append the following entry to `$DSH_HOME/cordis.patch.yml`
|
||||
(`~/.dsh/cordis.patch.yml` when `DSH_HOME` is unset). Preserve existing patches
|
||||
and replace the executable paths with absolute paths from your installation:
|
||||
DeepSeek Harness registers MCP clients through a Cordis YAML patch sequence
|
||||
rather than a `mcpServers` object. `rea setup --client dsh` appends (or
|
||||
replaces) the REA entry in the home patch `$DSH_HOME/cordis.patch.yml`
|
||||
(`~/.dsh/cordis.patch.yml` when `DSH_HOME` is unset), preserving unrelated
|
||||
patches and comments. `rea doctor --client dsh` checks that same entry id.
|
||||
`rea uninstall` removes it by scanning supported clients; uninstall takes no
|
||||
`--client` flag. The home patch applies to every Harness profile, so the
|
||||
registration is written once and no per-profile edit is needed. The written
|
||||
entry is:
|
||||
|
||||
```yaml
|
||||
- insert:
|
||||
@@ -274,17 +314,20 @@ and replace the executable paths with absolute paths from your installation:
|
||||
transport: stdio
|
||||
command: /absolute/path/to/node
|
||||
args: ["/absolute/path/to/rea/scripts/rea.mjs", mcp]
|
||||
cwd: !!js process.cwd()
|
||||
```
|
||||
|
||||
Setup preserves caller-owned Cordis expressions and replaces launcher fields
|
||||
with literal values that doctor can verify. It refuses to edit an expression
|
||||
that produces the entire REA config, or a group-targeted REA insertion. Use a
|
||||
literal config and a root-level insertion as shown above.
|
||||
|
||||
For a global npm installation, the launcher is under
|
||||
`$(npm root -g)/rea-agents/scripts/rea.mjs`. Check the Node executable with
|
||||
`command -v node`. Register `serverName: rea` only once in each active profile,
|
||||
then restart Harness. It exposes tools as `mcp__rea__<tool_name>`.
|
||||
Run `rea setup --skill --dry-run`, review the destination, then
|
||||
`rea setup --skill --yes` to install the workflow in `~/.agents/skills`.
|
||||
Harness discovers that directory, or `$DSH_AGENTS_HOME/skills` when configured;
|
||||
its native `skill` tool can load `reverse-engineer-anything`.
|
||||
`$(npm root -g)/rea-agents/scripts/rea.mjs`. Restart Harness after setup; it
|
||||
exposes tools as `mcp__rea__<tool_name>`. Run `rea setup --skill --dry-run`,
|
||||
review the destination, then `rea setup --skill --yes` to install the workflow
|
||||
in `~/.agents/skills` (`$DSH_AGENTS_HOME/skills` when configured); Harness
|
||||
discovers that directory, and its native `skill` tool can load
|
||||
`reverse-engineer-anything`.
|
||||
|
||||
REA's optional [native client verification](testing.md#end-to-end-integration-and-golden-evidence)
|
||||
checks headless chat, skill loading, all tool input schemas and actual analysis
|
||||
@@ -779,6 +822,32 @@ is no fixed per-operation or response-size ceiling. Unresolved computed calls
|
||||
remain unknown, reference-kind provenance is preserved, and provider-specific
|
||||
pseudocode is never treated as original source or Hopper-equivalent text.
|
||||
|
||||
### Missing macOS native decompiler
|
||||
|
||||
If `rea doctor --json` reports `native_decompiler` as missing on macOS x64,
|
||||
the extracted Ghidra archive may include the decompiler source without the
|
||||
macOS executable. For Ghidra 12.1.4, you can build it manually with an existing
|
||||
Xcode Command Line Tools installation, `make`, and `bison`:
|
||||
|
||||
```bash
|
||||
cd "$GHIDRA_INSTALL_DIR/Ghidra/Features/Decompiler/src/decompile/cpp"
|
||||
make ghidra_opt
|
||||
decompiler_dir="$GHIDRA_INSTALL_DIR/Ghidra/Features/Decompiler/os/mac_x86_64"
|
||||
mkdir -p "$decompiler_dir"
|
||||
test ! -e "$decompiler_dir/decompile" && test ! -L "$decompiler_dir/decompile" &&
|
||||
install -m 755 ghidra_opt "$decompiler_dir/decompile"
|
||||
rea doctor --json
|
||||
```
|
||||
|
||||
The copy step requires the destination to be absent and preserves an existing
|
||||
decompiler. This recipe is specific to macOS x64: the
|
||||
[Ghidra 12.1.4 Makefile](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/Ghidra/Features/Decompiler/src/decompile/cpp/Makefile)
|
||||
explicitly selects `-arch x86_64` on Darwin. For other architectures or a Gradle
|
||||
build, follow the matching release's
|
||||
[native component instructions](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/GhidraDocs/GettingStarted.md#building-native-components).
|
||||
REA accepts the matching executable under either `os/<platform>/` or
|
||||
`build/os/<platform>/`; it does not build the component during setup or diagnosis.
|
||||
|
||||
### Ghidra import language and analysis seeds
|
||||
|
||||
Ghidra detects the language from the executable header. To set it (for example, Borland C++ PE images):
|
||||
@@ -882,8 +951,12 @@ verification workflows.
|
||||
|
||||
Unexpected request failures retain their original internal cause. CLI and MCP
|
||||
errors expose Error names, messages, and available codes under
|
||||
`details.diagnostics.failure_cause`; other rejection values retain their
|
||||
primitive value or an explicit type. Shutdown warnings include the failure
|
||||
`details.diagnostics.failure_cause`, including bounded nested causes and explicit
|
||||
truncation markers. Darwin ownership-helper compilation failures also retain
|
||||
the compiler outcome, exit code or signal, and bounded stderr; a timed-out
|
||||
compiler remains distinguishable from a compiler that exited with an error.
|
||||
Other rejection values retain their primitive value or an explicit type.
|
||||
Shutdown warnings include the failure
|
||||
kind, message, and diagnostics, while process and temporary-project cleanup
|
||||
continues. These messages redact known bridge authentication tokens and
|
||||
preserve local paths and other analysis context.
|
||||
|
||||
@@ -174,6 +174,24 @@ default). Reaching it fails the current source with the same memory resource
|
||||
failure and reports `worker_process_memory_limit_bytes`, also a coverage limit;
|
||||
the next source starts a new analyzer. Bun elsewhere enforces no analyzer limit.
|
||||
|
||||
The default semantic capacities are 100000 nodes across the application and
|
||||
20000 per source. Large bundles may explicitly select `max_semantic_nodes` and
|
||||
`max_semantic_nodes_per_file` (CLI: `--max-semantic-nodes` and
|
||||
`--max-semantic-nodes-per-file`), each from 1 to 1000000. The per-file offer is
|
||||
also bounded by the remaining application capacity. Raising these controls does
|
||||
not bypass worker heap, execution deadline or parent result admission. Truncated
|
||||
coverage reports both effective node ceilings; omitted relationships remain
|
||||
unknown.
|
||||
|
||||
While sources remain, the sequential worker reserves one next compact static
|
||||
checkpoint at its 8 MiB bound with the same 6x retention expansion used at
|
||||
admission. Earlier checkpoints are already charged to retained bytes; the
|
||||
remaining inventory does not multiply the reserve. Streamed file transfers keep the default 512 MiB
|
||||
allowance, and can grow with the parent's offered retention budget (8x byte
|
||||
expansion), up to 2 GiB. Individual records remain bounded to 8 MiB. The parent
|
||||
still independently checks actual bytes, record overhead, digests and source
|
||||
identities before adopting results.
|
||||
|
||||
A source that exhausts the analyzer heap does not terminate the CLI/MCP
|
||||
process. Completed static facts are retained, other files can still be analyzed,
|
||||
and unexamined semantic relationships remain explicit unknowns. The parent
|
||||
@@ -366,9 +384,30 @@ Source-map contents are parsed as local input. The graph stores original source
|
||||
|
||||
Only explicitly present BrowserWindow values are observations. REA does not
|
||||
substitute version-dependent Electron defaults for omitted `webPreferences`.
|
||||
Dynamic option objects, bridge keys, API objects, and IPC channel expressions
|
||||
Unresolved option objects, bridge keys, API objects, and IPC channel expressions
|
||||
remain unknown and make coverage partial.
|
||||
|
||||
Recognized literal module-table bootstraps also support bounded lexical
|
||||
substitution through module exports, import helper bodies, local wrapper calls,
|
||||
returned closures, object spreads, and callback parameters. Recovered Electron
|
||||
paths retain source ranges, module identities, literal channels and branch
|
||||
conditions as static candidates. Unknown spreads, reassigned bindings, accessors,
|
||||
dynamic channels and unsupported syntax do not become exact implementations.
|
||||
This pass neither executes the bundle nor proves that runtime guards pass.
|
||||
An ambiguous or unresolved handler retains its channel registration but does
|
||||
not acquire a path to one arbitrarily selected callback. Unknown factory
|
||||
conditions retain both return branches, including fallthrough. Short-circuit
|
||||
selection distinguishes truthiness from nullishness; uncertain `??` paths
|
||||
retain a nullish guard rather than a falsy guard.
|
||||
|
||||
These paths appear as contextual `javascript-expression` nodes. An exact `api`
|
||||
seed such as `desktop.copy` can trace an exposed member through a unique IPC
|
||||
pairing to an external API call. `addListener` is normalized to `on`. Semantic
|
||||
projection retains these boundary paths before bulk function definitions, within
|
||||
the existing per-file node budget; omitted general semantics remain explicitly
|
||||
partial. Wrapper expansion is bounded by one million visits/substitutions and
|
||||
128 nested substitutions, with exhaustion reported as unknown coverage.
|
||||
|
||||
Electron API identity follows proven lexical module aliases. Namespace interop
|
||||
also preserves identity when the complete Rolldown helper operations, lexical
|
||||
roles, live getters, and captured intrinsic origins match, or when a stable
|
||||
@@ -415,7 +454,7 @@ do not establish that an application has no Electron boundary usage.
|
||||
|
||||
IPC pairing is an inference, not an observation. A renderer `invoke` pairs only
|
||||
with one unique `ipcMain.handle`/`handleOnce` candidate on the same exact literal
|
||||
channel; a renderer send pairs only with one unique `ipcMain.on`/`once`
|
||||
channel; a renderer send pairs only with one unique `ipcMain.on`/`once`/`addListener`
|
||||
candidate. Dynamic channels are never paired, and multiple compatible handlers
|
||||
are reported as ambiguous without adding a caller-to-handler edge. A matching
|
||||
channel still does not prove registration order, reachability, or runtime use.
|
||||
|
||||
@@ -36,7 +36,9 @@ application starts, and REA never resumes it or sends
|
||||
`Runtime.runIfWaitingForDebugger`. Resume a paused target with a separate
|
||||
caller-owned debugger before requesting application runtime observations.
|
||||
|
||||
The caller supplies the literal-loopback HTTP endpoint directly. Selecting an endpoint exposes all targets it serves and their local script locations or renderer origins. A target must match the exact requested target ID and expose a same-port WebSocket reported by that endpoint. Targets already marked attached are rejected rather than displacing another debugger. Local paths are canonicalized after symlink resolution.
|
||||
The caller supplies the literal-loopback HTTP endpoint directly. Selecting an endpoint exposes all targets it serves and their local script locations or renderer origins. A target must match the exact requested target ID and expose a same-port WebSocket reported by that endpoint. Targets explicitly marked attached are rejected rather than displacing another debugger. Node's `/json/list` omits attachment state, so REA preserves it as `null` (unknown), reports that limitation, and permits observation unless the endpoint explicitly reports `attached: true`. Local paths are canonicalized after symlink resolution.
|
||||
|
||||
The `attached` field in JavaScript runtime and browser target results is nullable: `true` and `false` are reported only when discovery supplied a boolean; `null` means the endpoint omitted or could not report the state. CDP `TargetInfo` worker records require an `attached` boolean; REA excludes a worker record with a missing or malformed value as an invalid protocol value.
|
||||
|
||||
`list_javascript_runtime_targets` returns every available target in one inline
|
||||
array.
|
||||
|
||||
@@ -58,6 +58,34 @@ reports `resource-limit` with unavailable or partial coverage; omitted facts do
|
||||
not become empty observations. Ambiguous PE section mappings are rejected
|
||||
rather than interpreted according to section order.
|
||||
|
||||
### Selected member inspection
|
||||
|
||||
When whole-assembly member decoding reaches the representation budget, select
|
||||
known declaring types before their signatures and CIL bodies are decoded:
|
||||
|
||||
```bash
|
||||
rea inspect-managed-members /absolute/path/Assembly.dll --type Example.Parser
|
||||
```
|
||||
|
||||
MCP uses `inspect_managed_members` with
|
||||
`{"path":"/absolute/path/Assembly.dll","type_names":["Example.Parser"]}`.
|
||||
Repeat `--type` or supply multiple unique `type_names` for several types.
|
||||
Names match the reported `full_name` exactly. Top-level names are namespace
|
||||
qualified; nested types currently report bare names, which can select several
|
||||
types with that name.
|
||||
|
||||
The result preserves the requested names in `selection.type_names` and reports
|
||||
unmatched names separately. If metadata names are unreadable, limitations retain
|
||||
that uncertainty. With no selection, `selection` is null and whole-assembly
|
||||
inspection keeps its existing resource policy. Selection restricts types,
|
||||
fields, methods, call edges and field accesses; member references remain an
|
||||
assembly-wide inventory, so output size is not guaranteed to be small.
|
||||
|
||||
Coverage describes the selected members. Comparisons leave members outside the
|
||||
opposite observation's selection unknown, and application graphs report their
|
||||
selected scope as partial. Neither makes an absence claim about the whole
|
||||
assembly. Older member Evidence without `selection` remains readable.
|
||||
|
||||
## Analysis objective
|
||||
|
||||
REA's managed-code track is intended to answer five different questions without
|
||||
|
||||
+57
-11
@@ -113,17 +113,48 @@ Schema does not express those cross-field invariants.
|
||||
|
||||
Call `binary_session` with `{}` and read `result.tool_availability` to choose a
|
||||
callable operation for the current target, provider, host, and negotiated client
|
||||
capabilities. The default result includes the complete inventory with each
|
||||
tool's availability, reason, and remediation. Each entry also reports required
|
||||
and optional negotiated client features plus the currently missing features.
|
||||
The optional inputs `expected_package_version`, `expected_catalog_digest`, and
|
||||
capabilities. The default result contains the complete inventory in groups that
|
||||
share status facts. `tool_availability.client_requirements` is the baseline for
|
||||
each group; a group's optional `client_requirements` replaces that baseline for
|
||||
the group's tools. Expand each group's `names` into one tool entry, carrying its
|
||||
surface, availability, reason, remediation, default-mode status, modes, and
|
||||
client requirements. Compare the union of group names with `tools/list` when
|
||||
checking that a client received the full catalog. An operation absent from all
|
||||
groups is missing from the complete inventory; it is not an implicit
|
||||
unavailable entry.
|
||||
|
||||
The `effects` and `annotations` arrays list the true members of their fixed
|
||||
vocabularies. A known member omitted from either array is false. This applies
|
||||
only to these closed flag sets; it does not make an omitted tool, provider, mode,
|
||||
or limitation false. Keep availability reasons, remediation, client-feature
|
||||
requirements, and mode facts inline with the grouped tool names. The optional
|
||||
inputs `expected_package_version`, `expected_catalog_digest`, and
|
||||
`expected_server_path` compare the live session with the caller's expectations.
|
||||
|
||||
`server_identity.catalog` contains catalog counts and schema digests. Grouped
|
||||
tool availability carries the name set and current per-tool facts; `tools/list`
|
||||
is the canonical source for complete tool definitions, including input and
|
||||
output schemas. Clients that previously read `server_identity.catalog.tools`
|
||||
should expand `tool_availability.groups` and use `tools/list` for schemas. The
|
||||
same catalog identity summary is returned by CLI `capabilities` and `providers`;
|
||||
MCP tool definitions remain available through `tools/list`.
|
||||
|
||||
Provider capabilities are grouped by provider identity and shared facts. In each
|
||||
set, `shared_limitations` is the ordered common prefix; append a group's ordered
|
||||
`limitations` suffix to recover each operation's exact limitations. Group
|
||||
`effects` arrays use the same true-only closed-vocabulary rule as tool
|
||||
availability. Expand every group operation while retaining its provider
|
||||
identity, availability, reason, rejection code, effects, and ordered
|
||||
limitations.
|
||||
|
||||
`binary_session.analysis_provider_candidates` is authoritative for deep-engine
|
||||
discovery. Target-free discovery is sorted by provider ID, reports host
|
||||
availability and `unknown` target support, and does not create an analysis
|
||||
client. `open_binary.provider_id` accepts a concrete provider ID or `auto`; it
|
||||
uses the same parser and selection policy as CLI `--provider` and
|
||||
discovery. Each candidate's capability sets describe that possible provider;
|
||||
they do not select it. The top-level `capabilities` describe the current route,
|
||||
while `analysis_provider_binding` identifies the concrete selected route. These
|
||||
fields are not interchangeable. Target-free discovery is sorted by provider ID,
|
||||
reports host availability and `unknown` target support, and does not create an
|
||||
analysis client. `open_binary.provider_id` accepts a concrete provider ID or
|
||||
`auto`; it uses the same parser and selection policy as CLI `--provider` and
|
||||
`REA_ANALYSIS_PROVIDER`. A successful deep open exposes one immutable provider,
|
||||
concrete version, selection source, and complete analysis profile through
|
||||
`analysis_provider_binding`. Ambiguity and unknown, unavailable, or unsupported
|
||||
@@ -165,6 +196,11 @@ Provider calls receive the request cancellation signal. Artifact traversal,
|
||||
hashing, version comparisons, Hopper requests, and process capture
|
||||
check the same signal. Cancellation is distinct from timeout. A cleanup failure
|
||||
uses `cleanup_incomplete` and lists only the owned resource kinds that remain.
|
||||
An atomic Evidence bundle or snapshot write can publish its complete output
|
||||
before staging cleanup fails. In that case, `details.published: true`,
|
||||
`details.published_path`, and `details.bytes` retain the publication receipt.
|
||||
Use that receipt to distinguish completed publication from a write that never
|
||||
committed. Staging resources remain owned until cleanup is verified.
|
||||
Native call tracing and process capture retain available observations in
|
||||
`details.partial_observation` on failure, including when cleanup succeeds.
|
||||
The observation reports its partial coverage; cleanup details describe host
|
||||
@@ -310,9 +346,12 @@ directly to `compare_functions`, and `inspect_artifact` Evidence to
|
||||
`compare_artifacts`. Use `get_evidence_bundle` when the task needs broader
|
||||
retained session history or an explicit bundle for transfer.
|
||||
|
||||
`get_call_graph` keeps provider-limited relationship reads on their node as
|
||||
`limitations`, alongside any observed `calls`. An empty call list with such a
|
||||
limitation is incomplete coverage, not proof that the function has no calls.
|
||||
`get_call_graph` returns a `graph` keyed by traversal depth and a top-level
|
||||
`limitations` array with deduplicated execution caveats. Each node keeps its
|
||||
relationship coverage and read-specific `limitations`, alongside observed
|
||||
`calls`. When a provider does not classify its caveats, they remain on that
|
||||
node and coverage is `unknown`. An empty call list with partial or unknown
|
||||
coverage is not proof that the function has no calls.
|
||||
`trace_call_path` reports each limited expansion in `residual_unknowns` and
|
||||
sets `truncated: true`; any path already supported by observed edges remains
|
||||
visible. Tool errors remain in their existing error or failure fields.
|
||||
@@ -380,6 +419,13 @@ analysis, so later views and application workflows read it without repeating
|
||||
analysis or moving the complete record through the response. A server without
|
||||
session retention refuses summary detail with `capability_unavailable`.
|
||||
|
||||
`inspect_analysis_view` also accepts `api-surface` for retained HAR capture Evidence,
|
||||
grouping request lines by method, origin and format-rule path template. Templated
|
||||
segments are inferences. Because paging follows the full grouping, every page is
|
||||
inferred Evidence whenever any group is templated, and derived Evidence only when
|
||||
no group was templated.
|
||||
See [derived API surfaces](web-network-captures.md#derived-api-surface).
|
||||
|
||||
`inspect_analysis_view` projects a caller-selected view of already completed
|
||||
`inspect_binary_layout`, `analyze_javascript_application`, or `analyze_function`
|
||||
Evidence. Native views select procedure, pseudocode, assembly, basic blocks,
|
||||
|
||||
@@ -6,11 +6,20 @@ coverage and unknowns. The CLI and MCP use the same application workflows.
|
||||
|
||||
## Static inspection
|
||||
|
||||
`inspect_macho` retains otool segment `file_offset` values relative to the
|
||||
selected Mach-O slice. Its evidence file-offset ranges address the original
|
||||
input file and include the observed lipo slice offset for universal binaries.
|
||||
If that offset is unavailable, segment evidence locations are omitted with an
|
||||
explicit limitation; architecture inventory locations remain available.
|
||||
`inspect_macho.selected_architecture` identifies the exact selected lipo slice
|
||||
and its `container_file_offset` in the input file. A thin Mach-O has a known
|
||||
offset of zero; an unknown universal-slice offset stays null. The Evidence
|
||||
subject format describes the input container: `mach-o-universal` for FAT files,
|
||||
including those with one slice. The normalized result describes the selected
|
||||
Mach-O image. Architecture
|
||||
`file_offset` values are container-relative. Entrypoint, segment and section
|
||||
`file_offset` values retain their source meaning and are relative to the
|
||||
selected Mach-O slice. Raw load-command fields retain their original values.
|
||||
Evidence locations map the LC_MAIN entrypoint and nonempty file-backed segment
|
||||
ranges to container offsets by adding the observed slice offset. Zero-fill
|
||||
segments do not produce file ranges. When the selected slice's container offset
|
||||
is unknown, those mapped locations are omitted with an explicit limitation;
|
||||
architecture inventory locations remain available.
|
||||
|
||||
Hopper's `address_to_file_offset` combines the native image-relative mapping with
|
||||
original-file coordinates only after checking the source against the session's
|
||||
@@ -53,6 +62,10 @@ complete observation.
|
||||
classes are preserved. Select `root`, `offset` and `limit` explicitly.
|
||||
UID 0 cannot distinguish conditional nil from ordinary nil. Limits are
|
||||
64 MiB of input, 200,000 objects/references and 128 nesting levels.
|
||||
XML dictionary entries decoded as `__proto__` are omitted with an explicit
|
||||
count because REA results cannot represent that key. XML tag whitespace and
|
||||
comments do not change that handling. Other values and the original archive
|
||||
digest are preserved; key interpretation follows the pinned plist decoder.
|
||||
- `decode_interface_builder` / `rea decode-interface-builder <app>` reads
|
||||
compiled storyboard and nib resources into objects, outlets, actions,
|
||||
connections and controller/class names. `verify:interface-builder` compiles
|
||||
@@ -192,8 +205,14 @@ rea inspect-native-data-type <binary> --address <typed-data-address> --provider
|
||||
Instruction facts include bytes, length, decoder text, ordered register/scalar/
|
||||
address tokens, typed references and flow destinations. Mid-instruction, data,
|
||||
outside-memory and undecodable addresses have separate outcomes. Effective
|
||||
memory base/index/displacement roles and per-instruction context mode remain
|
||||
unavailable; `mode` is the program language variant.
|
||||
memory base/index/displacement roles remain unavailable. Ghidra reports its program
|
||||
language variant in `mode`; other providers retain their own mode label. Ghidra
|
||||
MIPS results additionally expose `decoder_context`
|
||||
with the observed `ISA_MODE` register at the queried address and its provider
|
||||
source. A null value is unavailable, not mode zero; an empty array reports no
|
||||
available context register, and omission preserves older captures or providers
|
||||
that do not expose this facet. The context is an analysis observation, not proof
|
||||
of a runtime processor state.
|
||||
|
||||
`inspect_native_instruction` also returns `incoming_references` to the exact
|
||||
queried address, including data, interior-instruction and undecodable addresses.
|
||||
|
||||
+23
-7
@@ -112,8 +112,12 @@ Set `REA_VERIFY_HERMES_STICKY_PROFILE=1` to exercise native Hermes selection of
|
||||
a named sticky profile with `HERMES_HOME` still pointing to its root.
|
||||
|
||||
`verify:gemini-client` requires an installed Gemini CLI (verified with
|
||||
`@google/gemini-cli@0.63.0`); select it with `REA_VERIFY_GEMINI_COMMAND`.
|
||||
The optional POSIX lane uses native `GEMINI_CLI_HOME` discovery in an isolated
|
||||
`@google/gemini-cli@0.63.0` on POSIX and native Windows); select it with
|
||||
`REA_VERIFY_GEMINI_COMMAND`. A selected `.js`, `.mjs` or `.cjs` entry point runs
|
||||
under the verifier's Node. On Windows, select the npm install's
|
||||
`node_modules/@google/gemini-cli/bundle/gemini.js`, because only a `gemini.cmd`
|
||||
shim reaches `PATH`.
|
||||
The optional lane uses native `GEMINI_CLI_HOME` discovery in an isolated
|
||||
Git project, checks setup plans, backups and idempotence, activates the installed
|
||||
personal skill, validates all forwarded input JSON Schemas against their declared
|
||||
dialect, forwards the complete REA catalog and checks full JavaScript
|
||||
@@ -121,7 +125,7 @@ Evidence for a Unicode path. Use `-- chat` for ordinary chat with REA enabled,
|
||||
or `REA_VERIFY_RUNTIME_ROOT` for a production-only installed REA package.
|
||||
The local fixture exercises the real Gemini API adapter, including its
|
||||
`untrusted_context` tool-result envelope. Token counts are synthetic; live
|
||||
Google API and native Windows compatibility remain unverified. The lane disables
|
||||
Google API compatibility remains unverified. The lane disables
|
||||
the client's memory-based relaunch to preserve the caller's Node heap budget.
|
||||
|
||||
`verify:opencode-client` requires an installed OpenCode (verified with
|
||||
@@ -268,9 +272,13 @@ unverified; skill discovery uses an explicit isolated custom directory.
|
||||
`verify:deepseek-client` requires an installed DeepSeek Harness (`dsh`;
|
||||
verified with `@deepseek-ai/dsh@0.2.0-rc.2`) and Git. Set
|
||||
`REA_VERIFY_DEEPSEEK_COMMAND` to select its executable. This optional POSIX lane
|
||||
installs the skill through `rea setup --skill`, writes the native Cordis MCP
|
||||
patch into an isolated `DSH_HOME`, and binds `HOME` and `USERPROFILE` to the
|
||||
disposable account. The native default `~/.agents/skills` search path discovers
|
||||
seeds caller model settings and a stale Cordis registration in an isolated
|
||||
`DSH_HOME`, then runs public `setup --client dsh` dry-run, setup, repeated setup,
|
||||
and scoped doctor. It checks the backup and unrelated patches before booting
|
||||
Harness with the generated registration. After native execution it uninstalls
|
||||
through the public CLI and checks that only caller patches remain. `HOME` and
|
||||
`USERPROFILE` point to the disposable account. The native default
|
||||
`~/.agents/skills` search path discovers
|
||||
the installed skill without a `DSH_AGENTS_HOME` override. A separate Git root prevents inherited project skills from
|
||||
masking a missing installation. The default `call` mode checks skill loading,
|
||||
complete catalog discovery, the actual forwarded input schemas and regexes,
|
||||
@@ -281,6 +289,12 @@ context capacity declared by its default DeepSeek Flash model; it does not
|
||||
establish live DeepSeek API or native Windows coverage. Receipts include
|
||||
request artifacts, result digests and owned-process lineage.
|
||||
|
||||
The DSH acceptance journey also exercises public setup, approval, doctor,
|
||||
generated-launcher MCP startup and uninstall without an installed Harness.
|
||||
That journey proves REA workflow behavior, not native Cordis expression
|
||||
evaluation. Filesystem regressions retain tagged values and unrelated rows
|
||||
across alias dependencies and check refusal of unsafe YAML without mutation.
|
||||
|
||||
Full E2E tests invoke the production command dispatcher and real providers,
|
||||
without fake launchers, runners or responses. `verify:keyed-archive` writes
|
||||
Foundation binary, XML, and integer archives using one run-owned Swift module
|
||||
@@ -1387,7 +1401,9 @@ reports a named skip; `--require-tools` makes missing configuration fail.
|
||||
|
||||
The verifier creates and validates real modules with custom sections, multiple
|
||||
import/export kinds, escaped names, an empty module, and a bulk-memory
|
||||
DataCount section. It compares exact WAT,
|
||||
DataCount section. Debug-name fixtures cover compiler-generated module names
|
||||
and preserved name-section payloads with empty, multiline, Unicode, long and
|
||||
NUL-containing names. It compares exact WAT,
|
||||
section output and selected/tool digests through the compiled CLI and real MCP
|
||||
SDK, validates advertised schemas, checks identical CLI/MCP Evidence IDs and
|
||||
session bundle readback, retains invalid/truncated module diagnostics, preserves
|
||||
|
||||
@@ -55,6 +55,11 @@ satisfy. A plan without execution does not establish runtime behavior.
|
||||
- Keep observed, derived, and inferred relationships distinct. Preserve
|
||||
unresolved edges and partial provider facts instead of requiring every
|
||||
provider to supply the same metadata.
|
||||
- For caller/callee reads, `relationship_coverage` describes only the
|
||||
provider-observed direct relationship set. `complete` means that set was
|
||||
returned without a known read limit; it does not claim every possible runtime
|
||||
edge. Provider and execution caveats stay in `limitations` and do not imply
|
||||
incomplete relationship coverage. Missing coverage remains `unknown`.
|
||||
- Declare process, filesystem, network, UI, and mutation effects truthfully;
|
||||
enforce the requested target and lifecycle.
|
||||
- Add limits only for real format, protocol, authority, or measured resource
|
||||
|
||||
@@ -86,8 +86,22 @@ harm_rank,location,trigger,current_copy,user_risk,proposed_replacement,implement
|
||||
77,install.sh:node missing,node is absent,"Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+ is required; install it from nodejs.org and retry.",Low risk; exact versions and recovery,Keep current copy,unchanged,verified by shell branch inspection
|
||||
78,install.sh:npm missing,npm is absent,npm is required; install it with Node.js and retry.,Low risk; exact recovery,Keep current copy,unchanged,verified by shell branch inspection
|
||||
79,install.sh:node unreadable,node exists but does not report a parseable version,could not determine the active Node.js version.,No cause-local recovery,"The active Node.js version could not be read. Check that node works and is on PATH, then retry.",implemented,verified by exact unreadable-Node-version installer test
|
||||
80,install.sh:Node 22 floor,Node 22 minor version is below 19,"Node.js VERSION is unsupported; use Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+.",Low risk; exact detected version and supported alternatives,Keep current copy,unchanged,verified by exact installer test using Node 20 and branch inspection
|
||||
81,install.sh:Node 24 floor,Node 24 minor version is below 11,"Node.js VERSION is unsupported; use Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+.",Low risk; exact detected version and floor,Keep current copy,unchanged,verified by shell branch inspection
|
||||
80,install.sh:Node 22 floor,Node 22 minor version is below 19,"Node.js VERSION is unsupported; use Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+.",No install guidance; operator must guess between nvm/fnm/system packages/nodejs.org,"Node.js VERSION is unsupported.
|
||||
|
||||
REA requires Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+. Install one via:
|
||||
nvm: nvm install 24 && nvm use 24 (https://github.com/nvm-sh/nvm)
|
||||
fnm: fnm install 24 && fnm use 24 (https://github.com/Schniz/fnm)
|
||||
Direct download for OS/ARCH:
|
||||
https://nodejs.org/dist/latest-v24.x/ (grab the OS-ARCH tarball)
|
||||
Then rerun this installer.",implemented,verified by exact installer test using Node 20 against the `linux/x64` fake host and by shell branch inspection
|
||||
81,install.sh:Node 24 floor,Node 24 minor version is below 11,"Node.js VERSION is unsupported; use Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+.",No install guidance; operator must guess between nvm/fnm/system packages/nodejs.org,"Node.js VERSION is unsupported.
|
||||
|
||||
REA requires Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+. Install one via:
|
||||
nvm: nvm install 24 && nvm use 24 (https://github.com/nvm-sh/nvm)
|
||||
fnm: fnm install 24 && fnm use 24 (https://github.com/Schniz/fnm)
|
||||
Direct download for OS/ARCH:
|
||||
https://nodejs.org/dist/latest-v24.x/ (grab the OS-ARCH tarball)
|
||||
Then rerun this installer.",implemented,verified by shell branch inspection
|
||||
82,install.sh:version syntax,Explicit version is not exact semantic version,version must be an exact semantic version.,Low risk; exact correction,Keep current copy,unchanged,verified by shell branch inspection
|
||||
83,install.sh:release lookup,Latest GitHub release request fails,could not resolve the latest GitHub release.,No fallback despite supporting pinned versions,"The latest REA release could not be resolved. Check network access or pass --version VERSION, then retry.",implemented,verified by exact release-lookup installer test
|
||||
84,install.sh:release response,Latest release response is not valid expected JSON,Node JSON parser stack followed by the safe release-response message,Leaks a parser stack and buries recovery guidance,The release response was invalid. Retry later or pass --version VERSION.,implemented,verified by exact malformed-release test with parser stderr suppressed
|
||||
|
||||
|
+11
-1
@@ -4,6 +4,15 @@
|
||||
local WASM bytes using caller-supplied WABT 1.0.42. No active binary target is
|
||||
required. REA does not install tools or fetch artifacts and does not execute WASM.
|
||||
|
||||
`open_binary` is not the WASM inspection route. A recognized WASM v1 header
|
||||
returns `target_unavailable` with category `unsupported_target` and constraint
|
||||
`wasm_requires_static_inspection`, including the selected path in a valid
|
||||
`inspect_wasm_artifact` call. Use that target-free tool or
|
||||
`rea inspect-wasm-artifact <path> --format json` with the toolchain below.
|
||||
Check `binary_session` for configured capability availability. A recognized
|
||||
header is not validation of the complete module; missing WABT is reported
|
||||
separately as an unavailable capability, not as missing native engine support.
|
||||
|
||||
Set `REA_WABT_BIN_DIRECTORY` to an absolute directory containing executable
|
||||
`wasm-validate`, `wasm-objdump` and `wasm2wat` (with `.exe` on Windows). Missing
|
||||
executables, unsupported banners and inaccessible tools produce actionable
|
||||
@@ -37,7 +46,8 @@ into a private workspace as read-only `module.wasm`; its digest is rechecked
|
||||
before and after each tool. WABT validates with `--enable-all`,
|
||||
reports sections through `wasm-objdump -h -x`, and decodes through
|
||||
`wasm2wat --enable-all`. The result retains selected byte SHA-256 and size,
|
||||
section payload ranges, full header/detail output, exact import/export WAT
|
||||
section payload ranges, full header/detail output (including module-name
|
||||
preambles), exact import/export WAT
|
||||
forms, decoded WAT and its UTF-8 SHA-256. Producer format drift and invalid or
|
||||
truncated modules are failures, with validation diagnostics and partial Evidence
|
||||
retaining selected byte identity and tool provenance.
|
||||
|
||||
@@ -100,6 +100,11 @@ resources follow the same explicit declarations without changing error types;
|
||||
sensitive issue pointers identify a real ancestor instead of a fabricated
|
||||
coordinate. Header/cookie authentication values and known
|
||||
transport URL userinfo are excluded structurally, including native backups.
|
||||
HAR redaction and API-surface projection share the same recorded-URL parser.
|
||||
Canonical userinfo spans are removed without rewriting the remaining source
|
||||
text. Credential-bearing spellings without a safe raw span, such as slashless
|
||||
or extra-slash HTTP URLs, are excluded as a whole transport field. Ordinary
|
||||
extension text and noncredential path/query values remain unchanged.
|
||||
|
||||
Generated free-text limitations follow the same literal declarations, including
|
||||
global, per-record and Evidence explanations added during projection. Contract
|
||||
@@ -112,6 +117,75 @@ wire fragments. Native dictionary duplicate keys have the unchanged upstream
|
||||
decoder's last-value semantics; this adapter does not claim to recover discarded
|
||||
duplicates or the original numeric spelling of parsed native floats.
|
||||
|
||||
## Derived API surface
|
||||
|
||||
Pass retained HAR Evidence to `inspect_analysis_view` with
|
||||
`view: {"kind":"api-surface","offset":0,"limit":50}`. Omitted offset and limit
|
||||
are 0 and 50. The view groups request lines by method, WHATWG URL origin and
|
||||
path template, sorted by origin, template and method. Query strings, fragments,
|
||||
credentials, headers and bodies are not copied into groups.
|
||||
|
||||
| Rule | Exact segment format | Placeholder |
|
||||
| ---------- | ---------------------------------------------------- | ------------ |
|
||||
| `numeric` | ASCII digits only | `{numeric}` |
|
||||
| `uuid` | RFC 9562 textual UUID, any version, case-insensitive | `{uuid}` |
|
||||
| `hex` | At least 32 hexadecimal digits | `{hex}` |
|
||||
| `iso-date` | Valid calendar date `YYYY-MM-DD` | `{iso-date}` |
|
||||
|
||||
Rules apply in table order to the recorded, still-encoded request path, read
|
||||
before WHATWG dot-segment removal, so `/a/%2e%2e/admin` and `/admin` stay distinct. Segment indexes start
|
||||
at zero after the leading slash. Each matched segment lists its observed literals
|
||||
and `basis: "inferred"`. `templated_groups` counts templated groups across every
|
||||
page. Because paging follows the full grouping, every page is inferred Evidence
|
||||
when that count is nonzero, and derived Evidence only when no group was templated. Version prefixes, short opaque IDs
|
||||
and digit-bearing route names stay literal. Samples merge only when method,
|
||||
origin, and exact segment rules match. The view does not infer further
|
||||
cross-sample patterns.
|
||||
|
||||
Groups cite parent record ordinals and distinct statuses and MIME types (lowercase
|
||||
type/subtype without parameters). An invalid type/subtype essence is unknown;
|
||||
parameters do not affect this projection. Flags report single samples, mixed
|
||||
statuses or mixed content types.
|
||||
`coverage` pages groups; `records` separately reports examined, total and excluded
|
||||
parent records. Ordinal subsets carry a limitation. Null URLs are excluded as
|
||||
`url-redacted`; unparseable URLs, URLs without an origin, spellings without
|
||||
`scheme://authority` (such as `https:host/path`), and authorities containing a
|
||||
tab or newline as `url-unparseable`.
|
||||
Unavailable request objects or methods are explicitly excluded. Unavailable
|
||||
response fields, including HAR's `status: 0` and an empty `mimeType`, remain
|
||||
unknown, not invented statuses or MIME types.
|
||||
|
||||
The view makes no handler-identity, schema or causality claim. It performs no
|
||||
network activity. Native mitmproxy captures return a typed unsupported error.
|
||||
Parents inspected with `sensitive_values`, or whose Evidence does not record
|
||||
`sensitive_value_count`, return a typed unsupported error whose reason is the
|
||||
canonical token `sensitive_values_unretained`. The view would write path
|
||||
templates and limitation text that it cannot check against declarations the
|
||||
parent does not retain, so the refusal does not repeat declaration text.
|
||||
|
||||
One accumulator owns group identity, observed-value/status/MIME sets, ordinals,
|
||||
exclusions and deduplicated limitations. It scans path spans without splitting
|
||||
literal paths into arrays. Before schema parsing, it reserves the capture clone
|
||||
and validation scratch by walking the parent without cloning it. Inline requests
|
||||
borrow the Evidence value through CLI/MCP envelope validation; the application
|
||||
admits the entire portable envelope before authenticated parsing or hashing.
|
||||
The envelope clone, hash scratch and capture clone stay charged during grouping.
|
||||
Temporary URL/path/MIME work and retained state must
|
||||
fit together within a 96 MiB projection representation budget. A whole group or
|
||||
merge is admitted before allocating its retained sets, map entries or array
|
||||
values. Fixed group storage, distinct text, exclusions and limitations count as
|
||||
well as matched segments. Per-record reservations are released after each
|
||||
record; repeated values do not accumulate another retained charge. Only the
|
||||
requested page is materialized as output groups.
|
||||
|
||||
The representation budget uses conservative storage and worst-case JSON-escaping
|
||||
allowances; it is not an enforced V8 heap or process-memory limit. It is separate
|
||||
from the decoder's actual reply-byte limit and MCP transport admission. Exceeding
|
||||
it returns `resource_constraint` without a partial view. Limitations on the
|
||||
parent envelope, capture and examined records are deduplicated onto the view.
|
||||
Portable captures must retain unique ordinals within their declared total;
|
||||
caller-selected subsets can preserve any requested ordinal order.
|
||||
|
||||
## Resource and lifecycle boundaries
|
||||
|
||||
Complete evidence has a 32 MiB input, 96 MiB decoder reply and 64-level nesting
|
||||
|
||||
+33
-1
@@ -25,6 +25,38 @@ fail() {
|
||||
exit 1
|
||||
}
|
||||
|
||||
# Resolve the current host's Node.js tarball label pair; a label we cannot map
|
||||
# is kept verbatim so the operator can still read it. Falls back on the raw
|
||||
# `uname` outputs rather than inventing a label the Node.js release index does
|
||||
# not publish, so a mismatch is obvious instead of silently wrong.
|
||||
node_host_labels() {
|
||||
case "$(uname -m)" in
|
||||
arm64|aarch64) node_arch_label="arm64" ;;
|
||||
x86_64|amd64) node_arch_label="x64" ;;
|
||||
*) node_arch_label="$(uname -m)" ;;
|
||||
esac
|
||||
case "$(uname -s)" in
|
||||
Darwin) node_os_label="darwin" ;;
|
||||
Linux) node_os_label="linux" ;;
|
||||
*) node_os_label="$(uname -s | tr '[:upper:]' '[:lower:]')" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# Exit with actionable remediation when the active Node.js is unsupported.
|
||||
# The engine floor stays in sync with package.json#engines.node — change both.
|
||||
fail_node_version() {
|
||||
local observed="$1"
|
||||
node_host_labels
|
||||
printf 'REA installation failed: Node.js %s is unsupported.\n\n' "$observed" >&2
|
||||
printf 'REA requires Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+. Install one via:\n' >&2
|
||||
printf ' nvm: nvm install 24 && nvm use 24 (https://github.com/nvm-sh/nvm)\n' >&2
|
||||
printf ' fnm: fnm install 24 && fnm use 24 (https://github.com/Schniz/fnm)\n' >&2
|
||||
printf ' Direct download for %s/%s:\n' "$node_os_label" "$node_arch_label" >&2
|
||||
printf ' https://nodejs.org/dist/latest-v24.x/ (grab the %s-%s tarball)\n' "$node_os_label" "$node_arch_label" >&2
|
||||
printf 'Then rerun this installer.\n' >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage: install.sh [--version VERSION] [--dry-run] [--no-setup] [--no-prompt] [--verbose]
|
||||
@@ -78,7 +110,7 @@ main() {
|
||||
[[ "$node_major" =~ ^[0-9]+$ && "$node_minor" =~ ^[0-9]+$ ]] || fail "the active Node.js version could not be read. Check that node works and is on PATH, then retry."
|
||||
if ! [[ "$_node_patch" =~ ^[0-9]+(\+[0-9A-Za-z.-]+)?$ ]] ||
|
||||
! ((node_major == 22 && node_minor >= 19 || node_major == 24 && node_minor >= 11 || node_major >= 26)); then
|
||||
fail "Node.js $node_version is unsupported; use Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+."
|
||||
fail_node_version "$node_version"
|
||||
fi
|
||||
|
||||
if [[ -n "$version" ]]; then
|
||||
|
||||
Generated
+1
-1
@@ -6707,7 +6707,7 @@
|
||||
},
|
||||
"node_modules/rolldown": {
|
||||
"version": "1.2.12",
|
||||
"resolved": "https://mirrors.cloud.tencent.com/npm/rolldown/-/rolldown-1.2.12.tgz",
|
||||
"resolved": "https://registry.npmjs.org/rolldown/-/rolldown-1.2.12.tgz",
|
||||
"integrity": "sha512-8wafseiaG80xmXSfqidUNqZcylTlhmPZZt+za2m+js2sFZ8dTNlhIOV2WcbIPx2hgwPBJpEUGFAMZ9bgBBLTSQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
|
||||
+4
-2
@@ -165,6 +165,7 @@
|
||||
"verify:ghidra": "npm run build:cached && node scripts/verify-real-ghidra.mjs",
|
||||
"verify:ghidra:seeds": "npm run build:cached && node scripts/verify/ghidra/seeds.mjs",
|
||||
"verify:ghidra:mips": "npm run build:cached && node scripts/verify-real-ghidra-mips.mjs",
|
||||
"verify:ghidra:mips-shared": "npm run build:cached && node scripts/verify-real-ghidra-mips-shared.mjs",
|
||||
"verify:ghidra:mips-relocatable": "npm run build:cached && node scripts/verify-real-ghidra-mips-relocatable.mjs",
|
||||
"verify:ghidra:swift": "npm run build:cached && node scripts/verify-real-ghidra-swift.mjs",
|
||||
"verify:ida": "npm run build:cached && node scripts/verify-real-ida.mjs",
|
||||
@@ -250,13 +251,14 @@
|
||||
"verify:browser:runtime": "npm run build:cached && node scripts/verify-browser-runtime.mjs",
|
||||
"verify:web:network-captures": "npm run build:cached && node scripts/verify-web-network-captures.mjs",
|
||||
"verify:binary:layout": "npm run build:cached && node scripts/verify-binary-layout.mjs",
|
||||
"verify:go:binary": "npm run build:cached && node scripts/verify-go-binary.mjs",
|
||||
"verify:wasm:package": "npm run build:cached && node scripts/verify-packaged-wasm-artifact.mjs",
|
||||
"verify:wasm:artifact": "npm run build:cached && node scripts/verify-wasm-artifact.mjs",
|
||||
"verify:evm:interface": "npm run build:cached && node scripts/verify-evm-interface.mjs",
|
||||
"verify:recorded:crash": "npm run build:cached && node scripts/verify-recorded-crash.mjs",
|
||||
"mcp-catalog:check:built": "node scripts/generate-mcp-tool-catalog.mjs --check",
|
||||
"compile": "node scripts/clean-build-output.mjs && tsc -p tsconfig.build.json && node scripts/bundle-mcp-runtime.mjs",
|
||||
"compile:termux": "node scripts/clean-build-output.mjs && node --max-old-space-size=3072 node_modules/typescript-termux/lib/tsc.js -p tsconfig.build.json && node scripts/bundle-mcp-runtime.mjs",
|
||||
"compile": "node scripts/clean-build-output.mjs && tsc -p tsconfig.build.json && node scripts/bundle-runtime.mjs",
|
||||
"compile:termux": "node scripts/clean-build-output.mjs && node --max-old-space-size=3072 node_modules/typescript-termux/lib/tsc.js -p tsconfig.build.json && node scripts/bundle-runtime.mjs",
|
||||
"artifacts:mcp-catalog": "node scripts/generate-mcp-tool-catalog.mjs",
|
||||
"artifacts:skills": "node scripts/generate-skill-metadata.mjs",
|
||||
"artifacts:product-catalog": "node scripts/generate-product-catalog.mjs",
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
export const bundleMcpRuntime: (root: string) => Promise<void>;
|
||||
@@ -1,104 +0,0 @@
|
||||
import { parse } from "@babel/parser";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
|
||||
import { fileURLToPath, pathToFileURL } from "node:url";
|
||||
|
||||
/**
|
||||
* Reduce filesystem resolution during MCP startup. Keep dependencies external,
|
||||
* dynamic imports lazy, and the original compiled files for CLI/worker entrypoints.
|
||||
*/
|
||||
export const bundleMcpRuntime = async (root) => {
|
||||
const compiledRoot = join(root, "dist");
|
||||
const { rolldown, RUNTIME_MODULE_ID } = await import("rolldown");
|
||||
const { imports = {} } = JSON.parse(
|
||||
await readFile(join(root, "package.json"), "utf8"),
|
||||
);
|
||||
const bundle = await rolldown({
|
||||
input: { main: join(compiledRoot, "main.js") },
|
||||
platform: "node",
|
||||
treeshake: false,
|
||||
external: (source) => !source.startsWith(".") && !isAbsolute(source),
|
||||
plugins: [
|
||||
{
|
||||
name: "preserve-compiled-module-locations",
|
||||
transform(code, id) {
|
||||
if (id === RUNTIME_MODULE_ID) return null;
|
||||
if (!code.includes("import.meta")) return null;
|
||||
const sourcePath = relative(compiledRoot, id);
|
||||
if (
|
||||
isAbsolute(sourcePath) ||
|
||||
sourcePath === ".." ||
|
||||
sourcePath.startsWith(`..${sep}`)
|
||||
)
|
||||
throw new Error(`MCP bundle input is outside dist: ${id}`);
|
||||
// All chunks are directly under dist/mcp. Preserve the original
|
||||
// module URL, including URL escaping, for bridges and createRequire.
|
||||
const moduleUrl = `new URL(${JSON.stringify(`../${sourcePath.split(sep).map(encodeURIComponent).join("/")}`)}, import.meta.url).href`;
|
||||
const edits = [];
|
||||
const visit = (node) => {
|
||||
if (node === null || typeof node !== "object") return;
|
||||
const isMeta = (value) =>
|
||||
value?.type === "MetaProperty" &&
|
||||
value.meta.name === "import" &&
|
||||
value.property.name === "meta";
|
||||
if (
|
||||
node.type === "CallExpression" &&
|
||||
node.callee.type === "MemberExpression" &&
|
||||
!node.callee.computed &&
|
||||
isMeta(node.callee.object) &&
|
||||
node.callee.property.name === "resolve" &&
|
||||
node.arguments.length === 1 &&
|
||||
node.arguments[0].type === "StringLiteral" &&
|
||||
Object.hasOwn(imports, node.arguments[0].value)
|
||||
)
|
||||
// Package imports resolve in the same package scope after bundling.
|
||||
return;
|
||||
if (
|
||||
node.type === "MemberExpression" &&
|
||||
!node.computed &&
|
||||
isMeta(node.object) &&
|
||||
node.property.name === "url"
|
||||
) {
|
||||
edits.push({ start: node.start, end: node.end });
|
||||
return;
|
||||
}
|
||||
if (isMeta(node))
|
||||
throw new Error(
|
||||
`Unsupported import.meta use in MCP bundle: ${id}`,
|
||||
);
|
||||
for (const value of Object.values(node)) {
|
||||
if (Array.isArray(value)) value.forEach(visit);
|
||||
else visit(value);
|
||||
}
|
||||
};
|
||||
visit(parse(code, { sourceType: "module" }).program);
|
||||
for (const { start, end } of edits.sort((a, b) => b.start - a.start))
|
||||
code = code.slice(0, start) + moduleUrl + code.slice(end);
|
||||
return { code, map: null };
|
||||
},
|
||||
},
|
||||
],
|
||||
});
|
||||
try {
|
||||
await bundle.write({
|
||||
dir: join(compiledRoot, "mcp"),
|
||||
format: "esm",
|
||||
entryFileNames: "[name].js",
|
||||
chunkFileNames: "[name]-[hash].js",
|
||||
codeSplitting: true,
|
||||
strictExecutionOrder: true,
|
||||
minify: false,
|
||||
sourcemap: false,
|
||||
});
|
||||
} finally {
|
||||
await bundle.close();
|
||||
}
|
||||
};
|
||||
|
||||
if (
|
||||
process.argv[1] !== undefined &&
|
||||
pathToFileURL(resolve(process.argv[1])).href === import.meta.url
|
||||
)
|
||||
await bundleMcpRuntime(
|
||||
resolve(dirname(fileURLToPath(import.meta.url)), ".."),
|
||||
);
|
||||
@@ -0,0 +1 @@
|
||||
export const bundleRuntime: (root: string) => Promise<void>;
|
||||
@@ -0,0 +1,140 @@
|
||||
import { parse } from "@babel/parser";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
|
||||
import { fileURLToPath, pathToFileURL } from "node:url";
|
||||
|
||||
/**
|
||||
* Reduce filesystem resolution during CLI and MCP startup. Keep dependencies
|
||||
* external, dynamic imports lazy, and original compiled worker entrypoints.
|
||||
*/
|
||||
export const bundleRuntime = async (root) => {
|
||||
const compiledRoot = resolve(root, "dist");
|
||||
// CLI, MCP, workers and updater fixtures share one release-identity owner.
|
||||
const metadataModule = join(compiledRoot, "generatedPackageMetadata.js");
|
||||
const isMetadataModule = (id) =>
|
||||
isAbsolute(id) && resolve(id) === metadataModule;
|
||||
const { rolldown, RUNTIME_MODULE_ID } = await import("rolldown");
|
||||
const { imports = {} } = JSON.parse(
|
||||
await readFile(join(root, "package.json"), "utf8"),
|
||||
);
|
||||
for (const [directory, entries] of [
|
||||
["mcp", { main: "main.js" }],
|
||||
[
|
||||
"cli-runtime",
|
||||
{
|
||||
cli: "cli.js",
|
||||
cliOutput: "cliOutput.js",
|
||||
streamedJsonOutput: "cli/streamedJsonOutput.js",
|
||||
mcpDoctor: "mcpDoctor.js",
|
||||
},
|
||||
],
|
||||
]) {
|
||||
const bundle = await rolldown({
|
||||
input: Object.fromEntries(
|
||||
Object.entries(entries).map(([name, path]) => [
|
||||
name,
|
||||
join(compiledRoot, path),
|
||||
]),
|
||||
),
|
||||
platform: "node",
|
||||
treeshake: false,
|
||||
external: (source) =>
|
||||
isMetadataModule(source) ||
|
||||
(!source.startsWith(".") && !isAbsolute(source)),
|
||||
plugins: [
|
||||
{
|
||||
name: "preserve-compiled-module-locations",
|
||||
transform(code, id) {
|
||||
if (id === RUNTIME_MODULE_ID) return null;
|
||||
if (!code.includes("import.meta")) return null;
|
||||
const sourcePath = relative(compiledRoot, id);
|
||||
if (
|
||||
isAbsolute(sourcePath) ||
|
||||
sourcePath === ".." ||
|
||||
sourcePath.startsWith(`..${sep}`)
|
||||
)
|
||||
throw new Error(`Runtime bundle input is outside dist: ${id}`);
|
||||
// Each output directory is directly under dist. Preserve the original
|
||||
// module URL, including URL escaping, for bridges and createRequire.
|
||||
const moduleUrl = `new URL(${JSON.stringify(`../${sourcePath.split(sep).map(encodeURIComponent).join("/")}`)}, import.meta.url).href`;
|
||||
const edits = [];
|
||||
const visit = (node) => {
|
||||
if (node === null || typeof node !== "object") return;
|
||||
const isMeta = (value) =>
|
||||
value?.type === "MetaProperty" &&
|
||||
value.meta.name === "import" &&
|
||||
value.property.name === "meta";
|
||||
if (
|
||||
node.type === "CallExpression" &&
|
||||
node.callee.type === "MemberExpression" &&
|
||||
!node.callee.computed &&
|
||||
isMeta(node.callee.object) &&
|
||||
node.callee.property.name === "resolve" &&
|
||||
node.arguments.length === 1 &&
|
||||
node.arguments[0].type === "StringLiteral" &&
|
||||
Object.hasOwn(imports, node.arguments[0].value)
|
||||
)
|
||||
// Package imports resolve in the same package scope after bundling.
|
||||
return;
|
||||
if (
|
||||
node.type === "MemberExpression" &&
|
||||
!node.computed &&
|
||||
isMeta(node.object) &&
|
||||
node.property.name === "url"
|
||||
) {
|
||||
edits.push({ start: node.start, end: node.end });
|
||||
return;
|
||||
}
|
||||
if (isMeta(node))
|
||||
throw new Error(
|
||||
`Unsupported import.meta use in runtime bundle: ${id}`,
|
||||
);
|
||||
for (const value of Object.values(node)) {
|
||||
if (Array.isArray(value)) value.forEach(visit);
|
||||
else visit(value);
|
||||
}
|
||||
};
|
||||
visit(parse(code, { sourceType: "module" }).program);
|
||||
for (const { start, end } of edits.sort(
|
||||
(a, b) => b.start - a.start,
|
||||
))
|
||||
code = code.slice(0, start) + moduleUrl + code.slice(end);
|
||||
return { code, map: null };
|
||||
},
|
||||
},
|
||||
],
|
||||
});
|
||||
try {
|
||||
await bundle.write({
|
||||
dir: join(compiledRoot, directory),
|
||||
format: "esm",
|
||||
entryFileNames: "[name].js",
|
||||
chunkFileNames:
|
||||
directory === "mcp"
|
||||
? "[name]-[hash].js"
|
||||
: (chunk) => {
|
||||
// Keep lazy facades distinguishable when source basenames repeat.
|
||||
if (typeof chunk.facadeModuleId !== "string")
|
||||
return "[name]-[hash].js";
|
||||
const path = relative(compiledRoot, chunk.facadeModuleId);
|
||||
return `${path.replace(/\.js$/u, "").split(sep).join("__")}-[hash].js`;
|
||||
},
|
||||
paths: (id) =>
|
||||
isMetadataModule(id) ? "../generatedPackageMetadata.js" : id,
|
||||
minifyInternalExports: directory === "mcp",
|
||||
codeSplitting: true,
|
||||
strictExecutionOrder: true,
|
||||
minify: false,
|
||||
sourcemap: false,
|
||||
});
|
||||
} finally {
|
||||
await bundle.close();
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
if (
|
||||
process.argv[1] !== undefined &&
|
||||
pathToFileURL(resolve(process.argv[1])).href === import.meta.url
|
||||
)
|
||||
await bundleRuntime(resolve(dirname(fileURLToPath(import.meta.url)), ".."));
|
||||
@@ -150,7 +150,16 @@ async function verifyCase(executable, entrypoint, scenario) {
|
||||
phase = "public-map-assertions";
|
||||
observed = item?.mappings;
|
||||
assert.equal(item.status, "included");
|
||||
assert.deepEqual(item.mappings, expected);
|
||||
assert.equal(item.mappings.length, expected.length);
|
||||
for (const [index, coordinates] of expected.entries()) {
|
||||
const mapping = item.mappings[index];
|
||||
for (const [key, value] of Object.entries(coordinates))
|
||||
assert.deepEqual(mapping[key], value);
|
||||
// This compiler fixture has one source declaration in one wrapped leaf.
|
||||
assert.equal(mapping.source_index, 0);
|
||||
assert.equal(mapping.declaration_index, 0);
|
||||
}
|
||||
assert.equal(item.original_sources.length, 1);
|
||||
assert.equal(item.artifact.sha256, sha(site.map));
|
||||
assert.equal(item.artifact.bytes, Buffer.byteLength(site.map));
|
||||
assert.equal(item.original_sources[0].artifact.sha256, sha(site.original));
|
||||
|
||||
@@ -27,6 +27,7 @@ export const toolFamilyCatalog = (sources) => {
|
||||
contracts: [
|
||||
...sources.artifactContracts.ARTIFACT_TOOL_CONTRACTS,
|
||||
...sources.evmContracts.EVM_TOOL_CONTRACTS,
|
||||
...sources.goContracts.GO_TOOL_CONTRACTS,
|
||||
...sources.wasmContracts.WASM_TOOL_CONTRACTS,
|
||||
],
|
||||
},
|
||||
@@ -188,6 +189,10 @@ export const providerCatalog = (sources) => {
|
||||
identity: sources.evmProvider.EVMOLE_PROVIDER_IDENTITY,
|
||||
contracts: sources.evmContracts.EVM_TOOL_CONTRACTS,
|
||||
},
|
||||
{
|
||||
identity: sources.goProvider.GO_BINARY_PROVIDER_IDENTITY,
|
||||
contracts: sources.goContracts.GO_TOOL_CONTRACTS,
|
||||
},
|
||||
{
|
||||
identity: sources.artifactProviders.ARTIFACT_GRAPH_PROVIDER,
|
||||
contracts: sources.artifactContracts.ARTIFACT_TOOL_CONTRACTS,
|
||||
|
||||
@@ -49,6 +49,8 @@ const SOURCE_PATHS = {
|
||||
wasmProvider: "dist/wasm/WabtRelease.js",
|
||||
evmContracts: "dist/contracts/evm/evmToolContracts.js",
|
||||
evmProvider: "dist/evm/EvmoleRelease.js",
|
||||
goContracts: "dist/contracts/go/goToolContracts.js",
|
||||
goProvider: "dist/go/GoBinaryProvider.js",
|
||||
artifactContracts: "dist/contracts/artifactToolContracts.js",
|
||||
managedContracts: "dist/contracts/managed/managedToolContracts.js",
|
||||
firmwareContracts: "dist/contracts/firmware/firmwareToolContracts.js",
|
||||
|
||||
@@ -1,5 +1,11 @@
|
||||
import { analysisErrorProjectionSchema } from "../../dist/contracts/errorSchemas.js";
|
||||
import { toolContract } from "../../dist/contracts/toolContracts.js";
|
||||
import {
|
||||
expandProviderCapabilities,
|
||||
expandToolAvailability,
|
||||
providerCapabilitySet,
|
||||
} from "../../dist/contracts/toolOutputSchemaPrimitives.js";
|
||||
import { compareUnicodeCodePoints } from "../../dist/domain/unicodeCodePointOrder.js";
|
||||
|
||||
/** Read the textual JSON projection from one MCP tool call. */
|
||||
export const mcpTextValue = (result) => {
|
||||
@@ -10,6 +16,10 @@ export const mcpTextValue = (result) => {
|
||||
|
||||
const jsonValue = (result) => JSON.parse(mcpTextValue(result));
|
||||
|
||||
/** Sort canonical contract names using REA's code-point ordering. */
|
||||
export const sortVerifierNames = (names) =>
|
||||
[...names].sort(compareUnicodeCodePoints);
|
||||
|
||||
/** Read the complete canonical diagnostic from an MCP tool failure. */
|
||||
export const requireMcpToolError = (result) => {
|
||||
if (result.isError !== true) throw new Error("Expected an MCP tool failure");
|
||||
@@ -35,6 +45,44 @@ const operationResultKey = (contract) => {
|
||||
return evidence ? "normalized_result" : "result";
|
||||
};
|
||||
|
||||
/** Expand compact session metadata once at the verifier boundary. */
|
||||
export const expandMcpSessionStatus = (status) => ({
|
||||
...status,
|
||||
capabilities: expandProviderCapabilities(status.capabilities),
|
||||
analysis_provider_candidates: expandProviderCandidates(
|
||||
status.analysis_provider_candidates,
|
||||
),
|
||||
tool_availability: expandToolAvailability(status.tool_availability),
|
||||
});
|
||||
|
||||
/** Expand provider capability groups while retaining candidate identity/state. */
|
||||
export const expandProviderCandidates = (candidates) =>
|
||||
candidates.map((candidate) => ({
|
||||
...candidate,
|
||||
capabilities: expandProviderCapabilities(
|
||||
providerCapabilitySet.array().parse(candidate.capabilities),
|
||||
),
|
||||
}));
|
||||
|
||||
/** Expand CLI provider capability groups at the same owning boundary. */
|
||||
export const expandProviderOutput = (providers) => ({
|
||||
...providers,
|
||||
capabilities: expandProviderCapabilities(
|
||||
providerCapabilitySet.array().parse(providers.capabilities),
|
||||
),
|
||||
analysis_provider_candidates: expandProviderCandidates(
|
||||
providers.analysis_provider_candidates,
|
||||
),
|
||||
});
|
||||
|
||||
/** Expand the CLI capabilities command's compact provider descriptor groups. */
|
||||
export const expandCapabilitiesOutput = (capabilities) => ({
|
||||
...capabilities,
|
||||
capabilities: expandProviderCapabilities(
|
||||
providerCapabilitySet.array().parse(capabilities.capabilities),
|
||||
),
|
||||
});
|
||||
|
||||
/** Validate an Evidence-producing tool output and return its normalized result. */
|
||||
export const requireMcpEvidenceResult = (result, operation) => {
|
||||
const contract = toolContract(operation);
|
||||
@@ -48,7 +96,10 @@ export const requireMcpLifecycleResult = (result, operation) => {
|
||||
const contract = toolContract(operation);
|
||||
if (operationResultKey(contract) !== "result")
|
||||
throw new Error(`${operation} requires the Evidence result extractor`);
|
||||
return requireOutput(result, operation, contract).result;
|
||||
const projected = requireOutput(result, operation, contract).result;
|
||||
return operation === "binary_session"
|
||||
? expandMcpSessionStatus(projected)
|
||||
: projected;
|
||||
};
|
||||
|
||||
/** Project a mixed verifier call using its exact named canonical output schema. */
|
||||
@@ -58,7 +109,10 @@ export const requireMcpOperationResult = (
|
||||
contract = toolContract(operation),
|
||||
) => {
|
||||
const key = operationResultKey(contract);
|
||||
return requireOutput(result, operation, contract)[key];
|
||||
const projected = requireOutput(result, operation, contract)[key];
|
||||
return operation === "binary_session"
|
||||
? expandMcpSessionStatus(projected)
|
||||
: projected;
|
||||
};
|
||||
|
||||
/** Verify provider identity directly from the tool result. */
|
||||
|
||||
@@ -6,6 +6,7 @@ import { fileURLToPath, pathToFileURL } from "node:url";
|
||||
const PURE_LAYERS = new Set(["domain", "contracts"]);
|
||||
const MIGRATED_PROVIDER_ROOTS = new Set([
|
||||
"evm",
|
||||
"go",
|
||||
"wasm",
|
||||
"android",
|
||||
"firmware",
|
||||
|
||||
@@ -11,6 +11,7 @@ import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { parseEvidence } from "../../dist/domain/evidence.js";
|
||||
import { analysisProfileSchema } from "../../dist/domain/analysisProfile.js";
|
||||
import { nativeValueTraceSchema } from "../../dist/domain/native/nativeValueTrace.js";
|
||||
import { requireMcpLifecycleResult } from "./mcp-verifier-results.mjs";
|
||||
import { verifyGhidraBoundaries } from "./real-ghidra-boundaries.mjs";
|
||||
|
||||
const defaultEntrypoint = fileURLToPath(new URL("../rea.mjs", import.meta.url));
|
||||
@@ -172,13 +173,16 @@ async function verifyNativeValueAdapters(
|
||||
{ timeout: 180000 },
|
||||
);
|
||||
assert.notEqual(opened.isError, true, JSON.stringify(opened));
|
||||
const inventory = await client.callTool({
|
||||
const inventoryReply = await client.callTool({
|
||||
name: "binary_session",
|
||||
arguments: {},
|
||||
});
|
||||
assert.notEqual(inventory.isError, true, JSON.stringify(inventory));
|
||||
const inventory = requireMcpLifecycleResult(
|
||||
inventoryReply,
|
||||
"binary_session",
|
||||
);
|
||||
assert.ok(
|
||||
inventory.structuredContent?.result?.tool_availability.some(
|
||||
inventory.tool_availability.some(
|
||||
(tool) => tool.name === "trace_native_values" && tool.available,
|
||||
),
|
||||
"Native value tracing is unavailable in real provider discovery",
|
||||
@@ -267,11 +271,14 @@ async function verifyNativeValueAdapters(
|
||||
arguments: {},
|
||||
});
|
||||
assert.notEqual(closed.isError, true, JSON.stringify(closed));
|
||||
const status = await client.callTool({
|
||||
const statusReply = await client.callTool({
|
||||
name: "binary_session",
|
||||
arguments: {},
|
||||
});
|
||||
assert.equal(status.structuredContent?.result?.open, false);
|
||||
assert.equal(
|
||||
requireMcpLifecycleResult(statusReply, "binary_session").open,
|
||||
false,
|
||||
);
|
||||
assert.deepEqual(
|
||||
(await readdir(env.TMPDIR)).filter((name) =>
|
||||
name.startsWith("rea-ghidra-"),
|
||||
|
||||
@@ -13,8 +13,6 @@ import { promisify } from "node:util";
|
||||
import Ajv from "ajv/dist/2020.js";
|
||||
import addFormats from "ajv-formats";
|
||||
|
||||
import { analysisErrorProjectionSchema } from "../../dist/contracts/errorSchemas.js";
|
||||
|
||||
import { verifyLegacyGhidraReferenceSnapshot } from "./ghidra-reference-snapshot-e2e.mjs";
|
||||
import { verifyGhidraSnapshotLifecycle } from "./real-ghidra-snapshot-lifecycle.mjs";
|
||||
import { verifyGhidraTargetAdmission } from "./real-ghidra-target-admission.mjs";
|
||||
|
||||
@@ -1,5 +1,17 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { requireFunctionDossierOracle } from "../../dist/application/RealHopperAssertions.js";
|
||||
|
||||
/**
|
||||
* `requireFunctionDossierOracle` is only used inside `verifyRealHopperFixture`,
|
||||
* which runs against the real Hopper bridge after `build:cached`. Loading it
|
||||
* lazily keeps this module importable from `test:fast` (which docs promise to
|
||||
* run "without build" — see docs/testing.md § test scripts) so that mock-only
|
||||
* callers such as `requireCurrentDocument` and `resolveFixtureProcedure` do
|
||||
* not require `dist/`.
|
||||
*/
|
||||
const loadFunctionDossierOracle = () =>
|
||||
import("../../dist/application/RealHopperAssertions.js").then(
|
||||
(module) => module.requireFunctionDossierOracle,
|
||||
);
|
||||
|
||||
/** Require truthful tri-state permissions from real Hopper memory regions. */
|
||||
export const requireTruthfulMemoryRegions = (segments) => {
|
||||
@@ -87,6 +99,7 @@ export async function verifyRealHopperFixture({
|
||||
oracle,
|
||||
normalizedResult,
|
||||
}) {
|
||||
const requireFunctionDossierOracle = await loadFunctionDossierOracle();
|
||||
const main = await resolveFixtureProcedure(
|
||||
client,
|
||||
options,
|
||||
|
||||
@@ -37,7 +37,7 @@ export async function verifyHopperWorkflows(call, invalid, procedure) {
|
||||
);
|
||||
assert.equal(cycle.length, 2, "fixture omitted its source-owned call cycle");
|
||||
const cycleGraph = await call("get_call_graph", { address: cycle[0].value });
|
||||
const cycleNodes = Object.values(cycleGraph).flat();
|
||||
const cycleNodes = Object.values(cycleGraph.graph).flat();
|
||||
assert.equal(cycleNodes.length, 2);
|
||||
assert.deepEqual(
|
||||
cycleNodes.map((node) => node.address).sort(),
|
||||
|
||||
@@ -3,6 +3,17 @@ import { lstat } from "node:fs/promises";
|
||||
import { promisify } from "node:util";
|
||||
import { Ajv2020 } from "ajv/dist/2020.js";
|
||||
|
||||
/**
|
||||
* `mcp-verifier-results.mjs` reaches into `dist/` at module load, so eager
|
||||
* importing it propagates that build dependency to every consumer in this
|
||||
* file. `verifyCompleteToolCatalog` is the only consumer, so we lazy-load
|
||||
* the module there and keep the other helpers (`exec`, `run`, `json`,
|
||||
* `pathExists`) usable from `test:fast` callers that only need `dist/`-free
|
||||
* utilities — see docs/testing.md § test scripts, which promises `test:fast`
|
||||
* runs "without build".
|
||||
*/
|
||||
const loadMcpVerifierResults = () => import("./mcp-verifier-results.mjs");
|
||||
|
||||
/** Promisified child_process execution helper. */
|
||||
export const exec = promisify(execFile);
|
||||
|
||||
@@ -31,6 +42,8 @@ export const json = (text) => JSON.parse(text);
|
||||
|
||||
/** Verify schema validity and that the MCP catalog matches session metadata. */
|
||||
export const verifyCompleteToolCatalog = async (client, options) => {
|
||||
const { requireMcpLifecycleResult, sortVerifierNames } =
|
||||
await loadMcpVerifierResults();
|
||||
const listed = await client.listTools(undefined, options);
|
||||
const ajv = new Ajv2020({ strict: false, validateFormats: false });
|
||||
for (const tool of listed.tools) {
|
||||
@@ -46,11 +59,10 @@ export const verifyCompleteToolCatalog = async (client, options) => {
|
||||
{ name: "binary_session", arguments: {} },
|
||||
options,
|
||||
);
|
||||
const availability = status.structuredContent?.result?.tool_availability;
|
||||
if (!Array.isArray(availability))
|
||||
throw new Error("packaged MCP omitted tool availability");
|
||||
const expected = availability.map(({ name }) => name).sort();
|
||||
const observed = listed.tools.map(({ name }) => name).sort();
|
||||
const expanded = requireMcpLifecycleResult(status, "binary_session");
|
||||
const availability = expanded.tool_availability;
|
||||
const expected = sortVerifierNames(availability.map(({ name }) => name));
|
||||
const observed = sortVerifierNames(listed.tools.map(({ name }) => name));
|
||||
if (JSON.stringify(observed) !== JSON.stringify(expected))
|
||||
throw new Error(
|
||||
"packaged MCP tool inventory diverged from session metadata",
|
||||
|
||||
@@ -47,6 +47,10 @@ if (await pathExists(join(root, "src"))) {
|
||||
for (const path of [
|
||||
"dist/main.js",
|
||||
"dist/mcp/main.js",
|
||||
"dist/cli-runtime/cli.js",
|
||||
"dist/cli-runtime/cliOutput.js",
|
||||
"dist/cli-runtime/streamedJsonOutput.js",
|
||||
"dist/cli-runtime/mcpDoctor.js",
|
||||
"dist/mcpDoctor.js",
|
||||
"dist/cli.js",
|
||||
"dist/cliOutput.js",
|
||||
|
||||
+11
-6
@@ -33,8 +33,12 @@ if (
|
||||
const runtimeFiles = isMcpMode
|
||||
? ["dist/mcp/main.js"]
|
||||
: isMcpDoctorMode
|
||||
? ["dist/mcp/main.js", "dist/mcpDoctor.js"]
|
||||
: ["dist/cli.js", "dist/cliOutput.js", "dist/cli/streamedJsonOutput.js"];
|
||||
? ["dist/mcp/main.js", "dist/cli-runtime/mcpDoctor.js"]
|
||||
: [
|
||||
"dist/cli-runtime/cli.js",
|
||||
"dist/cli-runtime/cliOutput.js",
|
||||
"dist/cli-runtime/streamedJsonOutput.js",
|
||||
];
|
||||
|
||||
if (!(await compiledRuntimeExists(runtimeFiles))) {
|
||||
process.stderr.write(
|
||||
@@ -45,7 +49,8 @@ if (!(await compiledRuntimeExists(runtimeFiles))) {
|
||||
const { runEntrypoint } = await import("../dist/mcp/main.js");
|
||||
await runEntrypoint();
|
||||
} else if (isMcpDoctorMode) {
|
||||
const { runProductionMcpDoctorCli } = await import("../dist/mcpDoctor.js");
|
||||
const { runProductionMcpDoctorCli } =
|
||||
await import("../dist/cli-runtime/mcpDoctor.js");
|
||||
const result = await runProductionMcpDoctorCli(args.slice(2), {
|
||||
dispatcherPath: fileURLToPath(import.meta.url),
|
||||
packageRoot,
|
||||
@@ -53,14 +58,14 @@ if (!(await compiledRuntimeExists(runtimeFiles))) {
|
||||
process.stdout.write(result.output);
|
||||
process.exitCode = result.exitCode;
|
||||
} else {
|
||||
const { createCli } = await import("../dist/cli.js");
|
||||
const { createCli } = await import("../dist/cli-runtime/cli.js");
|
||||
const { createStreamedCliJsonOutput } =
|
||||
await import("../dist/cli/streamedJsonOutput.js");
|
||||
await import("../dist/cli-runtime/streamedJsonOutput.js");
|
||||
const {
|
||||
renderCliOutputArgumentError,
|
||||
renderEmptyFilteredCliOutput,
|
||||
validateCliOutputArguments,
|
||||
} = await import("../dist/cliOutput.js");
|
||||
} = await import("../dist/cli-runtime/cliOutput.js");
|
||||
const outputArguments = validateCliOutputArguments(args);
|
||||
if (!outputArguments.ok) {
|
||||
process.stdout.write(renderCliOutputArgumentError(outputArguments));
|
||||
|
||||
+80
-67
@@ -1,7 +1,5 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdtemp, readFile, rm, stat } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { readFile, stat } from "node:fs/promises";
|
||||
|
||||
import { ApktoolResourceAnalysisService } from "../dist/application/apktool/ApktoolResourceAnalysisService.js";
|
||||
import { createApktoolResourceAnalysisProvider } from "../dist/composition/apktool.js";
|
||||
@@ -11,8 +9,8 @@ if (options.help) {
|
||||
process.stdout.write(
|
||||
"Usage: npm run verify:apktool [-- --apk PATH]\n" +
|
||||
"Requires a caller-selected apktool launcher (REA_APKTOOL_COMMAND or\n" +
|
||||
"PATH) and a Java runtime. Without --apk, a small signed probe APK is\n" +
|
||||
"expected at the path given by --apk; nothing is downloaded or built.\n",
|
||||
"PATH) and a Java runtime. Without --apk, only client availability is\n" +
|
||||
"checked. Supply a local APK to verify decoding; nothing is downloaded or built.\n",
|
||||
);
|
||||
process.exit(0);
|
||||
}
|
||||
@@ -38,70 +36,80 @@ const run = async (operation, input) => {
|
||||
return result.value.normalized_result;
|
||||
};
|
||||
|
||||
const client = await run("inspect_apktool_client", {});
|
||||
if (client !== null) {
|
||||
check(
|
||||
"apktool launcher reports a version",
|
||||
typeof client.client.apktool_version === "string" &&
|
||||
client.client.apktool_version !== "",
|
||||
`command=${client.client.command} version=${client.client.apktool_version}`,
|
||||
);
|
||||
}
|
||||
try {
|
||||
const client = await run("inspect_apktool_client", {});
|
||||
if (client !== null) {
|
||||
check(
|
||||
"apktool launcher reports a version",
|
||||
typeof client.client.apktool_version === "string" &&
|
||||
client.client.apktool_version !== "",
|
||||
`command=${client.client.command} version=${client.client.apktool_version}`,
|
||||
);
|
||||
}
|
||||
|
||||
if (options.apk !== null) {
|
||||
const decoded = await run("decode_android_resources", {
|
||||
path: options.apk,
|
||||
include_strings: true,
|
||||
});
|
||||
if (decoded !== null) {
|
||||
const bytes = await stat(options.apk).then((info) => info.size);
|
||||
const digest = createHash("sha256")
|
||||
.update(await readFile(options.apk))
|
||||
.digest("hex");
|
||||
check(
|
||||
"decoded target identity matches the file on disk",
|
||||
decoded.target.bytes === bytes && decoded.target.sha256 === digest,
|
||||
`${bytes} bytes`,
|
||||
);
|
||||
check(
|
||||
"metadata carries version and SDK facts",
|
||||
typeof decoded.metadata.version_name === "string" &&
|
||||
decoded.metadata.min_sdk_version !== null &&
|
||||
decoded.metadata.target_sdk_version !== null,
|
||||
`version=${String(decoded.metadata.version_name)} sdk=${String(decoded.metadata.min_sdk_version)}..${String(decoded.metadata.target_sdk_version)}`,
|
||||
);
|
||||
check(
|
||||
"decoded manifest names the package",
|
||||
typeof decoded.metadata.package_name === "string" &&
|
||||
decoded.manifest !== null &&
|
||||
decoded.manifest.includes(decoded.metadata.package_name),
|
||||
String(decoded.metadata.package_name),
|
||||
);
|
||||
check(
|
||||
"string resources projected",
|
||||
decoded.strings.length > 0,
|
||||
`${decoded.strings.length} string(s)`,
|
||||
);
|
||||
if (decoded.locales.length > 0) {
|
||||
const locale = await run("decode_android_resources", {
|
||||
path: options.apk,
|
||||
include_strings: true,
|
||||
locale: decoded.locales[0],
|
||||
});
|
||||
if (locale !== null)
|
||||
check(
|
||||
`locale table projected for ${String(decoded.locales[0])}`,
|
||||
Array.isArray(locale.strings),
|
||||
`${locale.strings.length} string(s)`,
|
||||
);
|
||||
if (options.apk !== null) {
|
||||
const decoded = await run("decode_android_resources", {
|
||||
path: options.apk,
|
||||
include_strings: true,
|
||||
});
|
||||
if (decoded !== null) {
|
||||
const bytes = await stat(options.apk).then((info) => info.size);
|
||||
const digest = createHash("sha256")
|
||||
.update(await readFile(options.apk))
|
||||
.digest("hex");
|
||||
check(
|
||||
"decoded target identity matches the file on disk",
|
||||
decoded.target.bytes === bytes && decoded.target.sha256 === digest,
|
||||
`${bytes} bytes`,
|
||||
);
|
||||
check(
|
||||
"metadata carries version and SDK facts",
|
||||
typeof decoded.metadata.version_name === "string" &&
|
||||
decoded.metadata.min_sdk_version !== null &&
|
||||
decoded.metadata.target_sdk_version !== null,
|
||||
`version=${String(decoded.metadata.version_name)} sdk=${String(decoded.metadata.min_sdk_version)}..${String(decoded.metadata.target_sdk_version)}`,
|
||||
);
|
||||
check(
|
||||
"decoded manifest names the package",
|
||||
typeof decoded.metadata.package_name === "string" &&
|
||||
decoded.manifest !== null &&
|
||||
decoded.manifest.includes(decoded.metadata.package_name),
|
||||
String(decoded.metadata.package_name),
|
||||
);
|
||||
check(
|
||||
"string resources projected",
|
||||
decoded.strings.length > 0,
|
||||
`${decoded.strings.length} string(s)`,
|
||||
);
|
||||
if (decoded.locales.length > 0) {
|
||||
const locale = await run("decode_android_resources", {
|
||||
path: options.apk,
|
||||
include_strings: true,
|
||||
locale: decoded.locales[0],
|
||||
});
|
||||
if (locale !== null)
|
||||
check(
|
||||
`locale table projected for ${String(decoded.locales[0])}`,
|
||||
Array.isArray(locale.strings),
|
||||
`${locale.strings.length} string(s)`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
await provider.close();
|
||||
} catch {
|
||||
// The provider owns per-call workspaces only; closing is best-effort.
|
||||
} catch (cause) {
|
||||
failures += 1;
|
||||
process.stdout.write(
|
||||
`[FAIL] verifier execution — ${cause instanceof Error ? cause.message : String(cause)}\n`,
|
||||
);
|
||||
} finally {
|
||||
try {
|
||||
await provider.close();
|
||||
} catch (cause) {
|
||||
failures += 1;
|
||||
process.stdout.write(
|
||||
`[FAIL] provider cleanup — ${cause instanceof Error ? cause.message : String(cause)}\n`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if (failures > 0) {
|
||||
@@ -116,7 +124,12 @@ function parseArguments(argv) {
|
||||
const argument = argv[index];
|
||||
if (argument === "--help" || argument === "-h") parsed.help = true;
|
||||
else if (argument === "--apk") {
|
||||
parsed.apk = argv[index + 1] ?? null;
|
||||
const path = argv[index + 1];
|
||||
if (path === undefined) {
|
||||
process.stderr.write("--apk requires a local APK path\n");
|
||||
process.exit(2);
|
||||
}
|
||||
parsed.apk = path;
|
||||
index += 1;
|
||||
} else {
|
||||
process.stderr.write(`unknown argument: ${argument}\n`);
|
||||
|
||||
@@ -662,6 +662,16 @@ try {
|
||||
bytes.writeBigUInt64LE(8n, header + 56);
|
||||
return [`undersized-entry-${type}`, bytes, "invalid_input"];
|
||||
});
|
||||
const nonintegralSymbolEntries = ["SHT_SYMTAB", "SHT_DYNSYM"].map((type) => {
|
||||
const section = protectedReport.sections.find((item) => item.type === type);
|
||||
assert.notEqual(section, undefined);
|
||||
const bytes = Buffer.from(sectionBearingBytes);
|
||||
const header = Number(BigInt(section.header_location.offset));
|
||||
const size = bytes.readBigUInt64LE(header + 32);
|
||||
assert.ok(size + 1n <= BigInt(bytes.length) - BigInt(section.offset));
|
||||
bytes.writeBigUInt64LE(size + 1n, header + 32);
|
||||
return [`nonintegral-entry-count-${type}`, bytes, "invalid_input"];
|
||||
});
|
||||
const relocationSection = relocatable.sections.find(
|
||||
(item) => item.type === "SHT_RELA",
|
||||
);
|
||||
@@ -677,6 +687,17 @@ try {
|
||||
bytes.writeBigUInt64LE(8n, header + 56);
|
||||
return [`undersized-entry-${type}`, bytes, "invalid_input"];
|
||||
});
|
||||
const nonintegralRelocationEntries = [
|
||||
["RELA", 4, 24],
|
||||
["REL", 9, 16],
|
||||
].map(([type, sectionType, entrySize]) => {
|
||||
const bytes = Buffer.from(object);
|
||||
const header = Number(BigInt(relocationSection.header_location.offset));
|
||||
bytes.writeUInt32LE(sectionType, header + 4);
|
||||
bytes.writeBigUInt64LE(BigInt(entrySize + 1), header + 32);
|
||||
bytes.writeBigUInt64LE(BigInt(entrySize), header + 56);
|
||||
return [`nonintegral-entry-count-${type}`, bytes, "invalid_input"];
|
||||
});
|
||||
const invalidRelocationReferences = [
|
||||
["missing-symbol-table", relocatable.sections.length + 100],
|
||||
["wrong-symbol-table-kind", 1],
|
||||
@@ -1066,7 +1087,9 @@ try {
|
||||
],
|
||||
...zeroSymbolEntries,
|
||||
...undersizedSymbolEntries,
|
||||
...nonintegralSymbolEntries,
|
||||
...undersizedRelocationEntries,
|
||||
...nonintegralRelocationEntries,
|
||||
...invalidRelocationReferences,
|
||||
["unterminated-dynamic-segment", shortDynamic, "invalid_input"],
|
||||
["unterminated-interpreter", unterminatedInterpreter, "invalid_input"],
|
||||
|
||||
@@ -5,11 +5,16 @@ import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import Ajv2020 from "ajv/dist/2020.js";
|
||||
import { CATALOG_IDENTITY } from "../dist/catalogIdentity.js";
|
||||
import {
|
||||
TOOL_CONTRACTS,
|
||||
toolContract,
|
||||
} from "../dist/contracts/toolContracts.js";
|
||||
import { exec } from "./lib/verify-package-core.mjs";
|
||||
import {
|
||||
expandMcpSessionStatus,
|
||||
sortVerifierNames,
|
||||
} from "./lib/mcp-verifier-results.mjs";
|
||||
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
|
||||
import { createAnthropicModelFixture } from "./verify/clients/anthropic-model-fixture.mjs";
|
||||
|
||||
@@ -58,6 +63,8 @@ await writeFile(
|
||||
'export function result() { return { clientCompatibility: "REA_CLIENT_FIXTURE", count: 7 }; }\n',
|
||||
);
|
||||
let skillRequested = false;
|
||||
let sessionRequested = false;
|
||||
let sessionStatusSeen = false;
|
||||
let skillLoaded = false;
|
||||
let expectedSkillBody;
|
||||
let evidenceSeen = false;
|
||||
@@ -73,7 +80,7 @@ const containsText = (value, text) => {
|
||||
return Object.values(value).some((nested) => containsText(nested, text));
|
||||
return false;
|
||||
};
|
||||
const extractEvidence = (value) => {
|
||||
const extractInlineResult = (value, matches) => {
|
||||
if (typeof value === "string") {
|
||||
try {
|
||||
// Native Claude appends this resource hint to tool results. Original bodies remain intact in request artifacts.
|
||||
@@ -81,20 +88,16 @@ const extractEvidence = (value) => {
|
||||
/\n\n<system-reminder>\n<total_tokens>\d+ tokens left<\/total_tokens>\n<\/system-reminder>$/u,
|
||||
"",
|
||||
);
|
||||
return extractEvidence(JSON.parse(json));
|
||||
return extractInlineResult(JSON.parse(json), matches);
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
if (value && typeof value === "object") {
|
||||
if (
|
||||
typeof value.evidence_id === "string" &&
|
||||
value.normalized_result?.graph?.nodes?.length > 0
|
||||
)
|
||||
return value;
|
||||
if (matches(value)) return value;
|
||||
for (const nested of Object.values(value)) {
|
||||
const evidence = extractEvidence(nested);
|
||||
if (evidence) return evidence;
|
||||
const result = extractInlineResult(nested, matches);
|
||||
if (result) return result;
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
@@ -125,10 +128,12 @@ const fixture = await createAnthropicModelFixture({
|
||||
0,
|
||||
-"analyze_javascript_application".length,
|
||||
);
|
||||
const expected = TOOL_CONTRACTS.map((t) => prefix + t.name).sort();
|
||||
const expected = sortVerifierNames(
|
||||
TOOL_CONTRACTS.map((t) => prefix + t.name),
|
||||
);
|
||||
const reaTools = tools.filter((t) => t.name.startsWith(prefix));
|
||||
assert.deepEqual(
|
||||
reaTools.map((t) => t.name).sort(),
|
||||
sortVerifierNames(reaTools.map((t) => t.name)),
|
||||
expected,
|
||||
"Claude must forward the complete REA catalog after connection",
|
||||
);
|
||||
@@ -146,11 +151,61 @@ const fixture = await createAnthropicModelFixture({
|
||||
schemasChecked++;
|
||||
}
|
||||
}
|
||||
if (mode === "call" && !sessionStatusSeen) {
|
||||
const observedStatus = extractInlineResult(
|
||||
body.messages,
|
||||
(value) =>
|
||||
Boolean(value.result?.server_identity) &&
|
||||
Array.isArray(value.result?.tool_availability?.groups),
|
||||
);
|
||||
if (observedStatus) {
|
||||
const parsedStatus =
|
||||
toolContract("binary_session").outputSchema.parse(observedStatus);
|
||||
const status = {
|
||||
...parsedStatus,
|
||||
result: expandMcpSessionStatus(parsedStatus.result),
|
||||
};
|
||||
assert.deepEqual(
|
||||
status.result.server_identity.catalog,
|
||||
CATALOG_IDENTITY,
|
||||
"binary_session must return the exact compact catalog counts and digests",
|
||||
);
|
||||
assert.equal(
|
||||
Object.hasOwn(status.result.server_identity.catalog, "tools"),
|
||||
false,
|
||||
"binary_session must not inline the duplicated canonical tool catalog",
|
||||
);
|
||||
assert.deepEqual(
|
||||
sortVerifierNames(
|
||||
status.result.tool_availability.map((entry) => entry.name),
|
||||
),
|
||||
sortVerifierNames(TOOL_CONTRACTS.map((contract) => contract.name)),
|
||||
"binary_session must retain complete canonical tool availability",
|
||||
);
|
||||
sessionStatusSeen = true;
|
||||
} else {
|
||||
assert(
|
||||
!sessionRequested,
|
||||
"Claude must return the inline binary_session result before continuing",
|
||||
);
|
||||
const session = tools.find(
|
||||
(tool) => tool.name === `${prefix}binary_session`,
|
||||
);
|
||||
assert(session, "Claude must expose the REA binary_session tool");
|
||||
sessionRequested = true;
|
||||
return call(session.name, {});
|
||||
}
|
||||
}
|
||||
skillLoaded ||=
|
||||
expectedSkillBody !== undefined &&
|
||||
containsText(body.messages, expectedSkillBody) &&
|
||||
containsText(body.messages, skillDirectory);
|
||||
const observed = extractEvidence(body.messages);
|
||||
const observed = extractInlineResult(
|
||||
body.messages,
|
||||
(value) =>
|
||||
typeof value.evidence_id === "string" &&
|
||||
value.normalized_result?.graph?.nodes?.length > 0,
|
||||
);
|
||||
if (observed) {
|
||||
const evidence = toolContract(
|
||||
"analyze_javascript_application",
|
||||
@@ -184,7 +239,7 @@ const fixture = await createAnthropicModelFixture({
|
||||
await writeFile(join(lab, "claude-evidence.json"), bytes);
|
||||
evidenceSeen = true;
|
||||
}
|
||||
if (mode === "chat" || evidenceSeen) {
|
||||
if (mode === "chat" || (sessionStatusSeen && evidenceSeen)) {
|
||||
finalSeen = true;
|
||||
return [{ type: "text", text: "REA_CLIENT_COMPATIBILITY_OK" }];
|
||||
}
|
||||
@@ -294,6 +349,7 @@ try {
|
||||
"Skill",
|
||||
"WaitForMcpServers",
|
||||
"mcp__rea__analyze_javascript_application",
|
||||
...(mode === "call" ? ["mcp__rea__binary_session"] : []),
|
||||
],
|
||||
},
|
||||
}),
|
||||
@@ -324,7 +380,7 @@ try {
|
||||
assert.equal(parsed.result, "REA_CLIENT_COMPATIBILITY_OK");
|
||||
assert.deepEqual(parsed.permission_denials, []);
|
||||
assert(finalSeen);
|
||||
if (mode === "call") assert(evidenceSeen && skillLoaded);
|
||||
if (mode === "call") assert(evidenceSeen && skillLoaded && sessionStatusSeen);
|
||||
} catch (cause) {
|
||||
failure = cause;
|
||||
if (typeof cause?.stdout === "string")
|
||||
@@ -355,6 +411,7 @@ try {
|
||||
schemasChecked,
|
||||
pendingServerWaits,
|
||||
evidenceSeen,
|
||||
sessionStatusSeen,
|
||||
skillLoaded,
|
||||
artifactDigest,
|
||||
finalSeen,
|
||||
|
||||
@@ -5,11 +5,12 @@ import { tmpdir } from "node:os";
|
||||
import { dirname, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { Ajv2020 } from "ajv/dist/2020.js";
|
||||
import { parse as parseYaml } from "yaml";
|
||||
import {
|
||||
TOOL_CONTRACTS,
|
||||
toolContract,
|
||||
} from "../dist/contracts/toolContracts.js";
|
||||
import { exec } from "./lib/verify-package-core.mjs";
|
||||
import { exec, pathExists } from "./lib/verify-package-core.mjs";
|
||||
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
|
||||
import { createOpenAiModelFixture } from "./verify/clients/model-fixture.mjs";
|
||||
|
||||
@@ -32,6 +33,7 @@ const skillPath = join(
|
||||
"SKILL.md",
|
||||
);
|
||||
const cli = join(runtimeRoot, "scripts", "rea.mjs");
|
||||
const configPath = join(profile, "cordis.patch.yml");
|
||||
const verifier = createVerifierRun();
|
||||
await Promise.all([
|
||||
mkdir(account),
|
||||
@@ -66,32 +68,32 @@ const clientVersion = (
|
||||
timeout: 30_000,
|
||||
})
|
||||
).stdout.trim();
|
||||
for (const args of [
|
||||
["setup", "--skill", "--dry-run"],
|
||||
["setup", "--skill", "--yes"],
|
||||
]) {
|
||||
const lifecycle = [];
|
||||
const runRea = async (args, artifact) => {
|
||||
const result = await exec(process.execPath, [cli, ...args, "--json"], {
|
||||
env: environment,
|
||||
cwd: workspace,
|
||||
timeout: 60_000,
|
||||
});
|
||||
const parsed = JSON.parse(result.stdout);
|
||||
assert.equal(parsed.status, args.includes("--dry-run") ? "planned" : "ready");
|
||||
if (args.includes("--dry-run")) {
|
||||
assert.equal(parsed.plannedActions.length, 1);
|
||||
assert.equal(parsed.plannedActions[0].kind, "install_skill");
|
||||
assert.equal(parsed.plannedActions[0].target, dirname(skillPath));
|
||||
}
|
||||
await writeFile(
|
||||
join(lab, args.includes("--dry-run") ? "setup-plan.json" : "setup.json"),
|
||||
result.stdout,
|
||||
);
|
||||
}
|
||||
assert(
|
||||
(await readFile(skillPath, "utf8")).includes(
|
||||
"name: reverse-engineer-anything",
|
||||
),
|
||||
);
|
||||
await writeFile(join(lab, `${artifact}.json`), result.stdout);
|
||||
return JSON.parse(result.stdout);
|
||||
};
|
||||
const patches = (text) =>
|
||||
parseYaml(text, {
|
||||
customTags: [
|
||||
{
|
||||
tag: "tag:yaml.org,2002:js",
|
||||
resolve: (source) => ({ __jsExpr: source }),
|
||||
},
|
||||
],
|
||||
});
|
||||
const withoutRea = (text) =>
|
||||
patches(text).flatMap((op) => {
|
||||
if (op.id === "rea-mcp") return [];
|
||||
if (!Array.isArray(op.insert)) return [op];
|
||||
const insert = op.insert.filter((entry) => entry.id !== "rea-mcp");
|
||||
return insert.length === 0 ? [] : [{ ...op, insert }];
|
||||
});
|
||||
|
||||
let catalog;
|
||||
let patternCount = 0;
|
||||
@@ -247,10 +249,9 @@ const fixture = await createOpenAiModelFixture({
|
||||
});
|
||||
let failure;
|
||||
try {
|
||||
// Use the client's native Cordis patch DSL; generic `rea setup --client` does not edit it.
|
||||
await writeFile(
|
||||
join(profile, "cordis.patch.yml"),
|
||||
`
|
||||
// Seed model settings and a stale registration; the public setup workflow
|
||||
// must produce the registration this native client actually boots.
|
||||
const original = `
|
||||
- id: llm-pi-ai
|
||||
config:
|
||||
providers:
|
||||
@@ -275,13 +276,59 @@ try {
|
||||
config:
|
||||
serverName: rea
|
||||
transport: stdio
|
||||
command: ${JSON.stringify(process.execPath)}
|
||||
args: ${JSON.stringify([cli, "mcp"])}
|
||||
command: stale-launcher
|
||||
args: [mcp]
|
||||
cwd: !!js process.cwd()
|
||||
env:
|
||||
REA_PROCESS_RUN_ID: ${JSON.stringify(verifier.run_id)}
|
||||
`,
|
||||
`;
|
||||
await writeFile(configPath, original);
|
||||
const planned = await runRea(
|
||||
["setup", "--client", "dsh", "--dry-run"],
|
||||
"setup-plan",
|
||||
);
|
||||
assert.equal(planned.status, "planned");
|
||||
assert(planned.plannedActions.some((action) => action.target === configPath));
|
||||
assert(
|
||||
planned.plannedActions.some(
|
||||
(action) => action.target === dirname(skillPath),
|
||||
),
|
||||
);
|
||||
assert(
|
||||
planned.plannedActions.every(
|
||||
(action) =>
|
||||
action.target === configPath || action.target === dirname(skillPath),
|
||||
),
|
||||
);
|
||||
assert.equal(await readFile(configPath, "utf8"), original);
|
||||
assert.equal(await pathExists(`${configPath}.rea.backup`), false);
|
||||
lifecycle.push("dry-run");
|
||||
const setup = await runRea(["setup", "--client", "dsh", "--yes"], "setup");
|
||||
assert.equal(setup.status, "ready");
|
||||
assert.equal(await readFile(`${configPath}.rea.backup`, "utf8"), original);
|
||||
const configured = await readFile(configPath, "utf8");
|
||||
assert.deepEqual(withoutRea(configured), withoutRea(original));
|
||||
assert(configured.includes("!!js process.cwd()"));
|
||||
assert(
|
||||
(await readFile(skillPath, "utf8")).includes(
|
||||
"name: reverse-engineer-anything",
|
||||
),
|
||||
);
|
||||
lifecycle.push("setup");
|
||||
const repeated = await runRea(
|
||||
["setup", "--client", "dsh", "--yes"],
|
||||
"setup-repeat",
|
||||
);
|
||||
assert.deepEqual(repeated.appliedActions, []);
|
||||
assert.equal(await readFile(configPath, "utf8"), configured);
|
||||
assert.equal(await readFile(`${configPath}.rea.backup`, "utf8"), original);
|
||||
lifecycle.push("idempotent-setup");
|
||||
const doctor = await runRea(
|
||||
["doctor", "--client", "dsh", "--skill"],
|
||||
"doctor",
|
||||
);
|
||||
assert.equal(doctor.healthy, true);
|
||||
lifecycle.push("doctor");
|
||||
const execution = exec(
|
||||
command,
|
||||
[
|
||||
@@ -304,6 +351,20 @@ try {
|
||||
assert.equal(fixture.failure, undefined);
|
||||
assert(finalSeen && result.stdout.includes("REA_CLIENT_COMPATIBILITY_OK"));
|
||||
if (mode === "call") assert(evidenceSeen && skillLoaded);
|
||||
lifecycle.push(mode === "call" ? "native-tool-invocation" : "native-chat");
|
||||
const uninstall = await runRea(["uninstall"], "uninstall");
|
||||
assert.equal(uninstall.status, "complete");
|
||||
assert(
|
||||
uninstall.items.some(
|
||||
(item) => item.name === "dsh" && item.status === "removed",
|
||||
),
|
||||
);
|
||||
assert.deepEqual(
|
||||
patches(await readFile(configPath, "utf8")),
|
||||
withoutRea(original),
|
||||
);
|
||||
assert.equal(await pathExists(skillPath), false);
|
||||
lifecycle.push("uninstall");
|
||||
} catch (cause) {
|
||||
failure = cause;
|
||||
if (typeof cause?.stdout === "string")
|
||||
@@ -316,6 +377,7 @@ try {
|
||||
client: "deepseek-harness",
|
||||
clientVersion,
|
||||
mode,
|
||||
lifecycle,
|
||||
passed: failure === undefined,
|
||||
artifacts: lab,
|
||||
runtimeRoot,
|
||||
|
||||
@@ -2,7 +2,7 @@ import assert from "node:assert/strict";
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { join, resolve, sep } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import Ajv2020 from "ajv/dist/2020.js";
|
||||
import {
|
||||
@@ -13,14 +13,18 @@ import { exec } from "./lib/verify-package-core.mjs";
|
||||
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
|
||||
import { createGeminiModelFixture } from "./verify/clients/gemini-model-fixture.mjs";
|
||||
|
||||
assert(
|
||||
process.platform !== "win32",
|
||||
"Native Gemini verifier requires POSIX; Windows is unverified",
|
||||
);
|
||||
const repo = fileURLToPath(new URL("..", import.meta.url));
|
||||
const runtimeRoot = process.env.REA_VERIFY_RUNTIME_ROOT ?? repo;
|
||||
const command = process.env.REA_VERIFY_GEMINI_COMMAND ?? "gemini";
|
||||
const mode = process.argv[2] ?? "call";
|
||||
// npm installs of Gemini CLI provide a JavaScript entry point; on Windows only a gemini.cmd shim reaches
|
||||
// PATH, which Node does not spawn without a shell, so a selected .js entry runs under this Node. It is
|
||||
// resolved once, because the preflight runs from the caller's directory and the client from the workspace.
|
||||
const entryPoint = /\.(c|m)?js$/iu.test(command) ? resolve(command) : undefined;
|
||||
const launch = (args) =>
|
||||
entryPoint === undefined
|
||||
? [command, args]
|
||||
: [process.execPath, [entryPoint, ...args]];
|
||||
assert(
|
||||
["chat", "call"].includes(mode),
|
||||
"Usage: verify-gemini-client.mjs [chat|call]",
|
||||
@@ -60,11 +64,15 @@ const environment = {
|
||||
let clientVersion;
|
||||
try {
|
||||
clientVersion = (
|
||||
await exec(command, ["--version"], { env: environment, timeout: 30_000 })
|
||||
await exec(...launch(["--version"]), { env: environment, timeout: 30_000 })
|
||||
).stdout.trim();
|
||||
} catch (cause) {
|
||||
const windowsHint =
|
||||
process.platform === "win32"
|
||||
? " On Windows, select the JavaScript entry point; npm installs provide it as node_modules/@google/gemini-cli/bundle/gemini.js."
|
||||
: "";
|
||||
throw new Error(
|
||||
`Gemini preflight failed for ${command}; install Gemini CLI or set REA_VERIFY_GEMINI_COMMAND. ${String(cause)}`,
|
||||
`Gemini preflight failed for ${command}; install Gemini CLI or set REA_VERIFY_GEMINI_COMMAND.${windowsHint} ${String(cause)}`,
|
||||
);
|
||||
}
|
||||
const original = JSON.stringify(
|
||||
@@ -103,11 +111,11 @@ for (const args of [
|
||||
assert(parsed.plannedActions.some((a) => a.target === configPath));
|
||||
assert(
|
||||
parsed.plannedActions.some((a) =>
|
||||
a.target.startsWith(join(profile, ".agents", "skills") + "/"),
|
||||
a.target.startsWith(join(profile, ".agents", "skills") + sep),
|
||||
),
|
||||
);
|
||||
assert(
|
||||
parsed.plannedActions.every((a) => a.target.startsWith(profile + "/")),
|
||||
parsed.plannedActions.every((a) => a.target.startsWith(profile + sep)),
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -300,8 +308,7 @@ const fixture = await createGeminiModelFixture({
|
||||
let failure;
|
||||
try {
|
||||
const response = await exec(
|
||||
command,
|
||||
[
|
||||
...launch([
|
||||
"--model",
|
||||
"gemini-2.5-flash",
|
||||
"--approval-mode",
|
||||
@@ -314,7 +321,7 @@ try {
|
||||
mode === "chat"
|
||||
? "Say hello briefly."
|
||||
: `Use REA to analyze the JavaScript directory ${target}. Report the result.`,
|
||||
],
|
||||
]),
|
||||
{
|
||||
env: {
|
||||
...environment,
|
||||
|
||||
@@ -0,0 +1,518 @@
|
||||
#!/usr/bin/env node
|
||||
import assert from "node:assert/strict";
|
||||
import { isUtf8 } from "node:buffer";
|
||||
import { execFile } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import { isAbsolute, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { promisify } from "node:util";
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { parseEvidence } from "../dist/domain/evidence.js";
|
||||
import { goBinarySchema } from "../dist/domain/go/goBinary.js";
|
||||
import { PrivateRuntimeRoot } from "../dist/process/PrivateRuntimeRoot.js";
|
||||
import { mcpTextValue } from "./lib/mcp-verifier-results.mjs";
|
||||
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
|
||||
|
||||
const execute = promisify(execFile);
|
||||
const compiler = process.env.REA_VERIFY_GO_COMMAND ?? "go";
|
||||
if (process.env.REA_VERIFY_GO_COMMAND !== undefined && !isAbsolute(compiler))
|
||||
throw new Error(
|
||||
"REA_VERIFY_GO_COMMAND must be an absolute Go compiler path.",
|
||||
);
|
||||
let compilerVersion = null;
|
||||
const run = createVerifierRun();
|
||||
const root = await PrivateRuntimeRoot.create({ prefix: "rea-go-verifier-" });
|
||||
const entrypoint = fileURLToPath(new URL("./rea.mjs", import.meta.url));
|
||||
const environment = Object.fromEntries(
|
||||
Object.entries(process.env).filter(([, value]) => typeof value === "string"),
|
||||
);
|
||||
delete environment.HOPPER_TARGET_PATH;
|
||||
const goEnvironment = {
|
||||
...environment,
|
||||
CGO_ENABLED: "0",
|
||||
GOCACHEPROG: "",
|
||||
GO111MODULE: "on",
|
||||
GOENV: "off",
|
||||
GOFLAGS: "",
|
||||
GOMAXPROCS: "2",
|
||||
GOTOOLCHAIN: "local",
|
||||
GOWORK: "off",
|
||||
GOPROXY: "off",
|
||||
GOSUMDB: "off",
|
||||
GOPATH: join(root.path, "gopath"),
|
||||
GOCACHE: join(root.path, "cache"),
|
||||
GOMODCACHE: join(root.path, "modules"),
|
||||
GOTMPDIR: join(root.path, "tmp"),
|
||||
// Go's own verifier seam isolates telemetry; no global configuration is changed.
|
||||
TEST_TELEMETRY_DIR: join(root.path, "telemetry"),
|
||||
};
|
||||
const client = new Client({ name: "go-binary-verifier", version: "1" });
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [entrypoint, "mcp"],
|
||||
env: environment,
|
||||
stderr: "pipe",
|
||||
});
|
||||
const targets = [
|
||||
["linux", "amd64", "elf", 64, "little"],
|
||||
["linux", "386", "elf", 32, "little"],
|
||||
["linux", "arm64", "elf", 64, "little"],
|
||||
["linux", "ppc64", "elf", 64, "big"],
|
||||
["linux", "mips", "elf", 32, "big"],
|
||||
["windows", "amd64", "pe", 64, "little"],
|
||||
["windows", "386", "pe", 32, "little"],
|
||||
["darwin", "amd64", "macho", 64, "little"],
|
||||
["darwin", "arm64", "macho", 64, "little"],
|
||||
];
|
||||
const verified = [];
|
||||
const omittedCoverage = [];
|
||||
const failures = [];
|
||||
try {
|
||||
await prepareFixture();
|
||||
try {
|
||||
compilerVersion = (
|
||||
await execute(compiler, ["version"], {
|
||||
cwd: root.path,
|
||||
env: goEnvironment,
|
||||
timeout: 10_000,
|
||||
})
|
||||
).stdout.trim();
|
||||
} catch (cause) {
|
||||
throw new Error(
|
||||
"verify:go:binary requires Go on PATH or absolute REA_VERIFY_GO_COMMAND; Go is only a fixture producer, not a REA runtime dependency.",
|
||||
{ cause },
|
||||
);
|
||||
}
|
||||
const host = (await go(["env", "GOHOSTOS", "GOHOSTARCH"])).stdout
|
||||
.trim()
|
||||
.split("\n");
|
||||
const oracle = join(
|
||||
root.path,
|
||||
host[0] === "windows" ? "buildinfo-oracle.exe" : "buildinfo-oracle",
|
||||
);
|
||||
await go(
|
||||
["build", "-trimpath", "-buildvcs=false", "-o", oracle, "./oracle"],
|
||||
{ GOOS: host[0], GOARCH: host[1] },
|
||||
);
|
||||
await client.connect(transport);
|
||||
for (const target of targets) {
|
||||
const [os, arch] = target;
|
||||
const path = join(
|
||||
root.path,
|
||||
`${os}-${arch}${os === "windows" ? ".exe" : ""}`,
|
||||
);
|
||||
await go(
|
||||
[
|
||||
"build",
|
||||
"-trimpath",
|
||||
"-buildvcs=false",
|
||||
"-ldflags=-s -w",
|
||||
"-o",
|
||||
path,
|
||||
".",
|
||||
],
|
||||
{ GOOS: os, GOARCH: arch },
|
||||
);
|
||||
await verify(path, target, oracle);
|
||||
if (os === "linux" && arch === "amd64") {
|
||||
const sectionless = join(root.path, "linux-amd64-sectionless");
|
||||
const bytes = Buffer.from(await readFile(path));
|
||||
bytes.fill(0, 40, 48); // ELF64 e_shoff.
|
||||
bytes.fill(0, 58, 64); // e_shentsize, e_shnum and e_shstrndx.
|
||||
await writeFile(sectionless, bytes);
|
||||
await verify(sectionless, target, oracle);
|
||||
}
|
||||
}
|
||||
if (process.platform === "win32")
|
||||
omittedCoverage.push(
|
||||
"Compiler-produced non-UTF-8 argv strings were not verified on Windows; Windows argv cannot preserve arbitrary byte arguments.",
|
||||
);
|
||||
else {
|
||||
const path = join(root.path, "linux-amd64-byte-strings");
|
||||
const expected = JSON.parse(
|
||||
(
|
||||
await execute(oracle, ["--build-byte-fixture", compiler, path], {
|
||||
cwd: root.path,
|
||||
env: { ...goEnvironment, GOOS: "linux", GOARCH: "amd64" },
|
||||
timeout: 120_000,
|
||||
maxBuffer: 1024 * 1024,
|
||||
})
|
||||
).stdout,
|
||||
);
|
||||
await verify(path, targets[0], oracle, expected);
|
||||
}
|
||||
} catch (cause) {
|
||||
failures.push(cause);
|
||||
} finally {
|
||||
for (const close of [
|
||||
() => client.close(),
|
||||
() => transport.close(),
|
||||
() => root.close(),
|
||||
]) {
|
||||
try {
|
||||
await close();
|
||||
} catch (cause) {
|
||||
failures.push(cause);
|
||||
}
|
||||
}
|
||||
}
|
||||
const verifier = await completeVerifierRun(run);
|
||||
try {
|
||||
await assert.rejects(readFile(join(root.path, "go.mod")), { code: "ENOENT" });
|
||||
if (verifier.process_lineage.status === "verified")
|
||||
assert.deepEqual(verifier.process_lineage.descendants, []);
|
||||
} catch (cause) {
|
||||
failures.push(cause);
|
||||
}
|
||||
if (failures.length > 0) {
|
||||
console.error(
|
||||
JSON.stringify(
|
||||
{
|
||||
status: "failed",
|
||||
compiler: compilerVersion,
|
||||
verified,
|
||||
omitted_coverage: omittedCoverage,
|
||||
verifier,
|
||||
},
|
||||
null,
|
||||
2,
|
||||
),
|
||||
);
|
||||
throw new AggregateError(
|
||||
failures,
|
||||
"Real Go metadata verification failed; cleanup failures are retained.",
|
||||
);
|
||||
}
|
||||
console.log(
|
||||
JSON.stringify(
|
||||
{
|
||||
status: "passed",
|
||||
compiler: compilerVersion,
|
||||
binaries: verified.length,
|
||||
public_cases: verified.length * 2,
|
||||
verified,
|
||||
omitted_coverage: omittedCoverage,
|
||||
fixture_execution:
|
||||
"not-requested; only the Go fixture producer and test-owned buildinfo oracle/helper were executed",
|
||||
verifier,
|
||||
},
|
||||
null,
|
||||
2,
|
||||
),
|
||||
);
|
||||
|
||||
async function go(args, overrides = {}) {
|
||||
return execute(compiler, args, {
|
||||
cwd: root.path,
|
||||
env: { ...goEnvironment, ...overrides },
|
||||
timeout: 120_000,
|
||||
maxBuffer: 1024 * 1024,
|
||||
});
|
||||
}
|
||||
|
||||
async function prepareFixture() {
|
||||
await mkdir(join(root.path, "tmp"));
|
||||
await mkdir(join(root.path, "telemetry"));
|
||||
await writeFile(join(root.path, "telemetry", "mode"), "off\n");
|
||||
await mkdir(join(root.path, "dependency"));
|
||||
await mkdir(join(root.path, "oracle"));
|
||||
await writeFile(
|
||||
join(root.path, "go.mod"),
|
||||
"module example.test/rea-go-fixture\n\ngo 1.20\n\nrequire example.test/dependency v1.2.3\nreplace example.test/dependency => ./dependency\n",
|
||||
);
|
||||
await writeFile(
|
||||
join(root.path, "dependency", "go.mod"),
|
||||
"module example.test/dependency\n\ngo 1.20\n",
|
||||
);
|
||||
await writeFile(
|
||||
join(root.path, "dependency", "dependency.go"),
|
||||
'package dependency\nfunc Value() string { return "REA fixture" }\n',
|
||||
);
|
||||
await writeFile(
|
||||
join(root.path, "main.go"),
|
||||
'package main\nimport ("fmt"; "example.test/dependency")\nfunc main() { fmt.Println(dependency.Value()) }\n',
|
||||
);
|
||||
await writeFile(
|
||||
join(root.path, "oracle", "main.go"),
|
||||
String.raw`package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"debug/buildinfo"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"os/exec"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
)
|
||||
|
||||
func main() {
|
||||
byteFixture := len(os.Args) == 4 && os.Args[1] == "--build-byte-fixture"
|
||||
path := os.Args[1]
|
||||
if byteFixture {
|
||||
path = os.Args[3]
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, os.Args[2], "build", "-buildvcs=false",
|
||||
"-tags=" + string([]byte{0xff}),
|
||||
"-ldflags=-X runtime.buildVersion=custom" + string([]byte{0xff}),
|
||||
"-o", path, ".")
|
||||
cmd.Stdout = os.Stdout
|
||||
cmd.Stderr = os.Stderr
|
||||
if err := cmd.Run(); err != nil { panic(err) }
|
||||
}
|
||||
info, err := buildinfo.ReadFile(path)
|
||||
if err != nil { panic(err) }
|
||||
var result interface{} = info
|
||||
if byteFixture {
|
||||
settings := make([]map[string]string, 0)
|
||||
byteSettings := make([]map[string]string, 0)
|
||||
for _, setting := range info.Settings {
|
||||
if utf8.ValidString(setting.Key) && utf8.ValidString(setting.Value) {
|
||||
settings = append(settings, map[string]string{"Key": setting.Key, "Value": setting.Value})
|
||||
} else {
|
||||
byteSettings = append(byteSettings, map[string]string{
|
||||
"KeyBytesBase64": base64.StdEncoding.EncodeToString([]byte(setting.Key)),
|
||||
"ValueBytesBase64": base64.StdEncoding.EncodeToString([]byte(setting.Value)),
|
||||
})
|
||||
}
|
||||
}
|
||||
result = map[string]interface{}{
|
||||
"GoVersion": nil,
|
||||
"GoVersionBytesBase64": base64.StdEncoding.EncodeToString([]byte(info.GoVersion)),
|
||||
"Path": info.Path, "Main": info.Main, "Deps": info.Deps,
|
||||
"Settings": settings, "ByteSettings": byteSettings,
|
||||
}
|
||||
}
|
||||
if err := json.NewEncoder(os.Stdout).Encode(result); err != nil { panic(err) }
|
||||
}
|
||||
`,
|
||||
);
|
||||
}
|
||||
|
||||
async function inspect(mode, path) {
|
||||
let envelope;
|
||||
if (mode === "cli") {
|
||||
const { stdout } = await execute(
|
||||
process.execPath,
|
||||
[entrypoint, "inspect-go-binary", path, "--json"],
|
||||
{ env: environment, timeout: 30_000, maxBuffer: 8 * 1024 * 1024 },
|
||||
);
|
||||
envelope = JSON.parse(stdout);
|
||||
} else {
|
||||
const response = await client.callTool(
|
||||
{ name: "inspect_go_binary", arguments: { path } },
|
||||
{ timeout: 30_000 },
|
||||
);
|
||||
assert.notEqual(response.isError, true, mcpTextValue(response));
|
||||
envelope = JSON.parse(mcpTextValue(response));
|
||||
}
|
||||
const evidence = parseEvidence(envelope);
|
||||
assert.equal(evidence.subject.local_path, path);
|
||||
const report = goBinarySchema.parse(evidence.normalized_result);
|
||||
assert.equal(evidence.subject.digest.sha256, report.artifact.sha256);
|
||||
return { evidence, report };
|
||||
}
|
||||
|
||||
async function verify(path, target, oracle, byteExpected = null) {
|
||||
const bytes = await readFile(path);
|
||||
const expected =
|
||||
byteExpected ??
|
||||
JSON.parse(
|
||||
(
|
||||
await execute(oracle, [path], {
|
||||
timeout: 10_000,
|
||||
maxBuffer: 1024 * 1024,
|
||||
})
|
||||
).stdout,
|
||||
);
|
||||
const reports = [];
|
||||
for (const mode of ["cli", "mcp"]) {
|
||||
const { evidence, report } = await inspect(mode, path);
|
||||
assert.equal(report.format, target[2]);
|
||||
assert.equal(report.architecture, target[1]);
|
||||
assert.equal(
|
||||
evidence.subject.architecture,
|
||||
{ amd64: "x86_64", 386: "x86", arm64: "arm64", mips: "mips" }[
|
||||
target[1]
|
||||
] ?? null,
|
||||
);
|
||||
assert.equal(report.bits, target[3]);
|
||||
assert.equal(report.byte_order, target[4]);
|
||||
assert.equal(report.artifact.path, path);
|
||||
assert.equal(report.artifact.bytes, bytes.length);
|
||||
assert.equal(
|
||||
report.artifact.sha256,
|
||||
createHash("sha256").update(bytes).digest("hex"),
|
||||
);
|
||||
const info = report.build_info;
|
||||
assert.notEqual(info, null);
|
||||
assert.equal(info.go_version, expected.GoVersion);
|
||||
assert.equal(info.module.path, expected.Path);
|
||||
assert.deepEqual(
|
||||
normalizeObservedModule(info.module.main),
|
||||
normalizeModule(expected.Main),
|
||||
);
|
||||
assert.deepEqual(
|
||||
info.module.dependencies.map(normalizeObservedModule),
|
||||
(expected.Deps ?? []).map(normalizeModule),
|
||||
);
|
||||
assert.deepEqual(
|
||||
info.module.settings,
|
||||
(expected.Settings ?? []).map(({ Key, Value }) => ({
|
||||
key: Key,
|
||||
value: Value,
|
||||
})),
|
||||
);
|
||||
assert.equal(info.module.complete, byteExpected === null);
|
||||
if (byteExpected === null) {
|
||||
assert.deepEqual(info.module.unparsed_lines, []);
|
||||
assert.deepEqual(info.module.unparsed_line_bytes_base64, []);
|
||||
} else verifyByteStrings(info, expected);
|
||||
verifySource(info, bytes);
|
||||
reports.push(report);
|
||||
assert.deepEqual(
|
||||
await readFile(path),
|
||||
bytes,
|
||||
`${mode} changed source bytes`,
|
||||
);
|
||||
}
|
||||
assert.deepEqual(
|
||||
reports[0],
|
||||
reports[1],
|
||||
"Public CLI and MCP projections disagree",
|
||||
);
|
||||
verified.push({
|
||||
target: `${target[0]}/${target[1]}`,
|
||||
file:
|
||||
byteExpected === null
|
||||
? path.endsWith("sectionless")
|
||||
? "sectionless"
|
||||
: "stripped"
|
||||
: "byte-valued-strings",
|
||||
go_version: expected.GoVersion,
|
||||
...(byteExpected === null
|
||||
? {}
|
||||
: { go_version_bytes_base64: expected.GoVersionBytesBase64 }),
|
||||
});
|
||||
}
|
||||
|
||||
function verifyByteStrings(info, expected) {
|
||||
const rawByte = Buffer.from([0xff]);
|
||||
const compilerBytes = Buffer.concat([Buffer.from("custom"), rawByte]);
|
||||
const ldflagsBytes = Buffer.concat([
|
||||
Buffer.from("-X runtime.buildVersion=custom"),
|
||||
rawByte,
|
||||
]);
|
||||
assert.equal(info.go_version, null);
|
||||
assert.equal(info.module_text, null);
|
||||
assert.equal(info.go_version_bytes_base64, compilerBytes.toString("base64"));
|
||||
assert.equal(info.go_version_bytes_base64, expected.GoVersionBytesBase64);
|
||||
assert.deepEqual(expected.ByteSettings, [
|
||||
{
|
||||
KeyBytesBase64: Buffer.from("-ldflags").toString("base64"),
|
||||
ValueBytesBase64: ldflagsBytes.toString("base64"),
|
||||
},
|
||||
{
|
||||
KeyBytesBase64: Buffer.from("-tags").toString("base64"),
|
||||
ValueBytesBase64: rawByte.toString("base64"),
|
||||
},
|
||||
]);
|
||||
assert.deepEqual(info.module.unparsed_lines, [
|
||||
'build\t-ldflags="-X runtime.buildVersion=custom\\xff"',
|
||||
]);
|
||||
assert.deepEqual(info.module.unparsed_line_bytes_base64, [
|
||||
Buffer.concat([Buffer.from("build\t-tags="), rawByte]).toString("base64"),
|
||||
]);
|
||||
}
|
||||
|
||||
function normalizeModule(module) {
|
||||
const identity = ({ Path, Version, Sum }) => ({
|
||||
path: Path,
|
||||
version: Version,
|
||||
sum: Sum === "" || Sum === undefined ? null : Sum,
|
||||
});
|
||||
return {
|
||||
...identity(module),
|
||||
replacement: module.Replace == null ? null : identity(module.Replace),
|
||||
};
|
||||
}
|
||||
|
||||
function normalizeObservedModule(module) {
|
||||
// debug/buildinfo JSON represents both absent and explicitly empty sums as "".
|
||||
const identity = (value) => ({
|
||||
...value,
|
||||
sum: value.sum === "" ? null : value.sum,
|
||||
});
|
||||
return {
|
||||
...identity(module),
|
||||
replacement:
|
||||
module.replacement === null ? null : identity(module.replacement),
|
||||
};
|
||||
}
|
||||
|
||||
function verifySource(info, bytes) {
|
||||
assert.deepEqual(
|
||||
bytes.subarray(info.header_offset, info.header_offset + 14),
|
||||
Buffer.from("ff20476f206275696c64696e663a", "hex"),
|
||||
);
|
||||
assert.equal(
|
||||
info.encoding,
|
||||
"inline",
|
||||
"Current compiler fixture must use inline string encoding",
|
||||
);
|
||||
const version = inlineString(bytes, info.header_offset + 32);
|
||||
const module = inlineString(bytes, version.offset + version.bytes.length);
|
||||
assert.deepEqual(info.version_location, {
|
||||
offset: version.offset,
|
||||
bytes: version.bytes.length,
|
||||
});
|
||||
assert.deepEqual(info.module_location, {
|
||||
offset: module.offset,
|
||||
bytes: module.bytes.length,
|
||||
});
|
||||
assert.equal(
|
||||
isUtf8(version.bytes) ? version.bytes.toString("utf8") : null,
|
||||
info.go_version,
|
||||
);
|
||||
assert.equal(version.bytes.toString("base64"), info.go_version_bytes_base64);
|
||||
assert.equal(module.bytes.toString("base64"), info.module_bytes_base64);
|
||||
assert.deepEqual(
|
||||
module.bytes.subarray(0, 16),
|
||||
Buffer.from("3077af0c9274080241e1c107e6d618e6", "hex"),
|
||||
);
|
||||
assert.deepEqual(
|
||||
module.bytes.subarray(-16),
|
||||
Buffer.from("f932433186182072008242104116d8f2", "hex"),
|
||||
);
|
||||
const moduleBody = module.bytes.subarray(16, -16);
|
||||
assert.equal(
|
||||
isUtf8(moduleBody) ? moduleBody.toString("utf8") : null,
|
||||
info.module_text,
|
||||
);
|
||||
assert.equal(moduleBody.at(-1), 10);
|
||||
}
|
||||
|
||||
function inlineString(bytes, start) {
|
||||
let length = 0;
|
||||
let offset = start;
|
||||
let shift = 0;
|
||||
for (;;) {
|
||||
assert.ok(
|
||||
offset < bytes.length && shift < 49,
|
||||
"Fixture string length is invalid",
|
||||
);
|
||||
const byte = bytes[offset++];
|
||||
length += (byte & 127) * 2 ** shift;
|
||||
if ((byte & 128) === 0) break;
|
||||
shift += 7;
|
||||
}
|
||||
assert.ok(
|
||||
length <= bytes.length - offset,
|
||||
"Fixture string contents are truncated",
|
||||
);
|
||||
return { offset, bytes: bytes.subarray(offset, offset + length) };
|
||||
}
|
||||
@@ -1,13 +1,19 @@
|
||||
import { json, run } from "./lib/verify-package-core.mjs";
|
||||
import {
|
||||
expandCapabilitiesOutput,
|
||||
expandProviderOutput,
|
||||
} from "./lib/mcp-verifier-results.mjs";
|
||||
|
||||
/** Verify packaged capabilities, providers, and non-Linux search results. */
|
||||
export async function verifyPackageCapabilitiesAndSearch({ cli, environment }) {
|
||||
const capabilities = json(
|
||||
await run(cli, ["capabilities", "--json"], environment),
|
||||
const capabilities = expandCapabilitiesOutput(
|
||||
json(await run(cli, ["capabilities", "--json"], environment)),
|
||||
);
|
||||
if (!Array.isArray(capabilities.capabilities))
|
||||
throw new Error("packaged capabilities CLI failed");
|
||||
const providers = json(await run(cli, ["providers", "--json"], environment));
|
||||
const providers = expandProviderOutput(
|
||||
json(await run(cli, ["providers", "--json"], environment)),
|
||||
);
|
||||
const candidates = providers.analysis_provider_candidates;
|
||||
if (
|
||||
!Array.isArray(candidates) ||
|
||||
|
||||
@@ -1,4 +1,7 @@
|
||||
import { requireMcpToolError } from "./lib/mcp-verifier-results.mjs";
|
||||
import {
|
||||
requireMcpLifecycleResult,
|
||||
requireMcpToolError,
|
||||
} from "./lib/mcp-verifier-results.mjs";
|
||||
import assert from "node:assert/strict";
|
||||
import { join } from "node:path";
|
||||
import { execFile } from "node:child_process";
|
||||
@@ -21,14 +24,14 @@ const verifyMcpToolsAndPrompts = async (client, mcpOptions) => {
|
||||
};
|
||||
|
||||
const verifyMcpTargetFree = async (client, mcpOptions) => {
|
||||
const status = await client.callTool(
|
||||
const result = await client.callTool(
|
||||
{ name: "binary_session", arguments: {} },
|
||||
mcpOptions,
|
||||
);
|
||||
const currentDocument =
|
||||
status.structuredContent?.result?.tool_availability?.find(
|
||||
({ name }) => name === "current_document",
|
||||
);
|
||||
const status = requireMcpLifecycleResult(result, "binary_session");
|
||||
const currentDocument = status.tool_availability.find(
|
||||
({ name }) => name === "current_document",
|
||||
);
|
||||
if (
|
||||
currentDocument?.available !== false ||
|
||||
currentDocument.reason !== "target_required"
|
||||
@@ -67,15 +70,13 @@ const verifyMcpOpenAndBind = async (client, mcpOptions) => {
|
||||
);
|
||||
if (opened.isError === true)
|
||||
throw new Error("packaged MCP could not open a binary");
|
||||
const providerStatusEnvelope = json(
|
||||
prompts.mcpText(
|
||||
await client.callTool(
|
||||
{ name: "binary_session", arguments: {} },
|
||||
mcpOptions,
|
||||
),
|
||||
const providerStatus = requireMcpLifecycleResult(
|
||||
await client.callTool(
|
||||
{ name: "binary_session", arguments: {} },
|
||||
mcpOptions,
|
||||
),
|
||||
"binary_session",
|
||||
);
|
||||
const providerStatus = providerStatusEnvelope.result;
|
||||
if (
|
||||
providerStatus.analysis_provider_binding?.provider?.id !== "hopper" ||
|
||||
providerStatus.analysis_provider_binding?.selection_source !== "request"
|
||||
|
||||
@@ -1,7 +1,10 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import assert from "node:assert/strict";
|
||||
import { requireMcpOperationResult } from "./lib/mcp-verifier-results.mjs";
|
||||
import {
|
||||
expandProviderOutput,
|
||||
requireMcpOperationResult,
|
||||
} from "./lib/mcp-verifier-results.mjs";
|
||||
import { verifyGhidraCompactReferences } from "./lib/real-ghidra-compact-references.mjs";
|
||||
import { execFile } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
@@ -230,14 +233,16 @@ const assertCleanup = async () => {
|
||||
assert.equal(entries.length, 0, "Provider runtime cleanup left files.");
|
||||
};
|
||||
try {
|
||||
const providers = JSON.parse(
|
||||
(
|
||||
await exec(process.execPath, [entry, "providers", "--json"], {
|
||||
env: environment,
|
||||
cwd: callerDirectory,
|
||||
timeout: 30_000,
|
||||
})
|
||||
).stdout,
|
||||
const providers = expandProviderOutput(
|
||||
JSON.parse(
|
||||
(
|
||||
await exec(process.execPath, [entry, "providers", "--json"], {
|
||||
env: environment,
|
||||
cwd: callerDirectory,
|
||||
timeout: 30_000,
|
||||
})
|
||||
).stdout,
|
||||
),
|
||||
);
|
||||
assert.equal(providers.analysis_provider_binding, null);
|
||||
assert.ok(Array.isArray(providers.analysis_provider_candidates));
|
||||
|
||||
@@ -10,6 +10,11 @@ import { pathToFileURL } from "node:url";
|
||||
import { lt, rcompare } from "semver";
|
||||
|
||||
import { CATALOG_IDENTITY } from "../dist/catalogIdentity.js";
|
||||
import { TOOL_CONTRACTS } from "../dist/contracts/toolContracts.js";
|
||||
import {
|
||||
requireMcpLifecycleResult,
|
||||
sortVerifierNames,
|
||||
} from "./lib/mcp-verifier-results.mjs";
|
||||
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
|
||||
|
||||
const verifierRun = createVerifierRun();
|
||||
@@ -54,27 +59,39 @@ try {
|
||||
await client.connect(transport);
|
||||
const tools = await client.listTools();
|
||||
const toolNames = tools.tools.map(({ name }) => name);
|
||||
const canonicalToolNames = new Set(
|
||||
CATALOG_IDENTITY.tools.map(({ name }) => name),
|
||||
);
|
||||
const canonicalToolNames = new Set(TOOL_CONTRACTS.map(({ name }) => name));
|
||||
const unknownToolNames = toolNames.filter(
|
||||
(name) => !canonicalToolNames.has(name),
|
||||
);
|
||||
const publishedToolNames = new Set(toolNames);
|
||||
const missingToolNames = [...canonicalToolNames].filter(
|
||||
(name) => !publishedToolNames.has(name),
|
||||
);
|
||||
publishedToolCount = toolNames.length;
|
||||
if (
|
||||
publishedToolCount === 0 ||
|
||||
new Set(toolNames).size !== publishedToolCount ||
|
||||
unknownToolNames.length > 0
|
||||
publishedToolNames.size !== publishedToolCount ||
|
||||
unknownToolNames.length > 0 ||
|
||||
missingToolNames.length > 0
|
||||
)
|
||||
throw new Error(
|
||||
`Published MCP exposed an invalid capability-scoped tool projection (${String(publishedToolCount)} tools, unknown: ${unknownToolNames.join(", ") || "none"})`,
|
||||
`Published MCP exposed an invalid canonical tool catalog (${String(publishedToolCount)} tools, unknown: ${unknownToolNames.join(", ") || "none"}, missing: ${missingToolNames.join(", ") || "none"})`,
|
||||
);
|
||||
const status = await client.callTool({
|
||||
const statusReply = await client.callTool({
|
||||
name: "binary_session",
|
||||
arguments: {},
|
||||
});
|
||||
if (status.isError === true)
|
||||
throw new Error("Published target-free MCP session check failed");
|
||||
const status = requireMcpLifecycleResult(statusReply, "binary_session");
|
||||
const availabilityNames = sortVerifierNames(
|
||||
status.tool_availability.map(({ name }) => name),
|
||||
);
|
||||
if (
|
||||
JSON.stringify(availabilityNames) !==
|
||||
JSON.stringify(sortVerifierNames(canonicalToolNames))
|
||||
)
|
||||
throw new Error(
|
||||
"Published binary_session omitted canonical tool availability",
|
||||
);
|
||||
update = await verifyPublishedUpdate({ version, canaryRoot });
|
||||
} catch (cause) {
|
||||
throw new Error(`Published package verification failed: ${stderr}`, {
|
||||
|
||||
@@ -158,7 +158,7 @@ try {
|
||||
observations: {
|
||||
...x.observations,
|
||||
source_files: x.observations.source_files.map(
|
||||
({ name, ...file }) => file,
|
||||
({ name: _name, ...file }) => file,
|
||||
),
|
||||
},
|
||||
});
|
||||
|
||||
@@ -163,6 +163,7 @@ async function verifyFixture(byteOrder, kernelModule) {
|
||||
});
|
||||
assert.equal(coverage.unplaced, 0, "Fixture references an empty section");
|
||||
assert.ok(coverage.shared_hi16, "No HI16 shared by several LO16s");
|
||||
assert.ok(coverage.multiple_hi16, "No LO16 resolved several pending HI16s");
|
||||
assert.ok(coverage.lo16_carry, "No LO16 >= 0x8000 forced a HI16 carry");
|
||||
assert.ok(coverage.external_code, "No external code relocation");
|
||||
assert.ok(coverage.external_data, "No external data relocation");
|
||||
|
||||
@@ -0,0 +1,338 @@
|
||||
#!/usr/bin/env node
|
||||
import assert from "node:assert/strict";
|
||||
import { execFile } from "node:child_process";
|
||||
import { mkdtemp, mkdir, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { parseBinaryTarget } from "../dist/application/BinaryTargetResolver.js";
|
||||
import { parseEvidence } from "../dist/domain/evidence.js";
|
||||
import { parseAnalysisSnapshot } from "../dist/domain/analysisSnapshot.js";
|
||||
import { buildMips16SharedFixture } from "../tests/conformance/ghidra/mips16-shared-fixture.mjs";
|
||||
import {
|
||||
requireMcpOperationResult,
|
||||
requireMcpToolError,
|
||||
} from "./lib/mcp-verifier-results.mjs";
|
||||
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
|
||||
|
||||
if (
|
||||
process.env.GHIDRA_INSTALL_DIR === undefined ||
|
||||
!["linux", "darwin"].includes(process.platform)
|
||||
)
|
||||
throw new Error(
|
||||
"This optional lane requires BYO Ghidra on Linux/macOS and GNU MIPS cross-binutils.",
|
||||
);
|
||||
const execute = promisify(execFile);
|
||||
const run = createVerifierRun();
|
||||
const workspace = await mkdtemp(join(tmpdir(), "rea-mips-shared-proof-"));
|
||||
const runtime = join(workspace, "runtime");
|
||||
await mkdir(runtime);
|
||||
const env = {
|
||||
PATH: process.env.PATH ?? "/usr/bin:/bin",
|
||||
TMPDIR: runtime,
|
||||
REA_LOG_LEVEL: "silent",
|
||||
REA_ANALYSIS_PROVIDER: "ghidra",
|
||||
REA_PROCESS_RUN_ID: run.run_id,
|
||||
GHIDRA_INSTALL_DIR: process.env.GHIDRA_INSTALL_DIR,
|
||||
...(process.env.JAVA_HOME === undefined
|
||||
? {}
|
||||
: { JAVA_HOME: process.env.JAVA_HOME }),
|
||||
...(process.env.GHIDRA_HEADLESS_MAXMEM === undefined
|
||||
? {}
|
||||
: { GHIDRA_HEADLESS_MAXMEM: process.env.GHIDRA_HEADLESS_MAXMEM }),
|
||||
};
|
||||
const reports = [];
|
||||
let failure;
|
||||
try {
|
||||
for (const byteOrder of ["big", "little"]) {
|
||||
for (const variant of [
|
||||
"declared",
|
||||
"recordless",
|
||||
"symbol-relative",
|
||||
"section-relocations",
|
||||
]) {
|
||||
const fixture = await buildMips16SharedFixture(
|
||||
workspace,
|
||||
byteOrder,
|
||||
variant,
|
||||
);
|
||||
const resolved = await parseBinaryTarget(fixture.path);
|
||||
if (!resolved.ok) throw resolved.error;
|
||||
assert.equal(resolved.value.sha256, fixture.sha256);
|
||||
assert.equal(resolved.value.architecture, "mips");
|
||||
if (variant === "section-relocations") {
|
||||
assert.equal(resolved.value.mips.sharedLoad.status, "unavailable");
|
||||
assert.match(
|
||||
resolved.value.mips.sharedLoad.reason,
|
||||
/section relocation declarations/u,
|
||||
);
|
||||
} else {
|
||||
assert.equal(resolved.value.mips.sharedLoad.status, "complete");
|
||||
assert.deepEqual(
|
||||
resolved.value.mips.sharedLoad.relocationTypes,
|
||||
[0, 3],
|
||||
);
|
||||
assert.equal(
|
||||
resolved.value.mips.sharedLoad.symbolRelativeCount,
|
||||
variant === "symbol-relative" ? 1 : 0,
|
||||
);
|
||||
}
|
||||
if (variant === "recordless")
|
||||
assert.equal(resolved.value.mips.abiFlags, null);
|
||||
else {
|
||||
assert.equal(resolved.value.mips.abiFlags.fpAbi, 5);
|
||||
assert.equal(resolved.value.mips.abiFlags.ases, 0x400);
|
||||
}
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: ["scripts/rea.mjs", "mcp"],
|
||||
cwd: process.cwd(),
|
||||
env,
|
||||
stderr: "pipe",
|
||||
});
|
||||
const client = new Client({
|
||||
name: "rea-mips16-shared-conformance",
|
||||
version: "1",
|
||||
});
|
||||
let stderr = "";
|
||||
transport.stderr?.on("data", (chunk) => {
|
||||
stderr = (stderr + chunk.toString()).slice(-262144);
|
||||
});
|
||||
let calls = 0;
|
||||
const reply = async (name, args = {}) => {
|
||||
const response = await client.callTool(
|
||||
{ name, arguments: args },
|
||||
{ timeout: 180000 },
|
||||
);
|
||||
await writeFile(
|
||||
join(workspace, `${byteOrder}-${variant}-${++calls}-${name}.json`),
|
||||
JSON.stringify(response, null, 2),
|
||||
);
|
||||
return response;
|
||||
};
|
||||
const call = async (name, args) =>
|
||||
requireMcpOperationResult(await reply(name, args), name);
|
||||
const original = await readFile(fixture.path);
|
||||
try {
|
||||
await client.connect(transport);
|
||||
await client.listTools();
|
||||
if (
|
||||
variant === "symbol-relative" ||
|
||||
variant === "section-relocations"
|
||||
) {
|
||||
const refusal = requireMcpToolError(
|
||||
await reply("open_binary", {
|
||||
path: fixture.path,
|
||||
provider_id: "ghidra",
|
||||
}),
|
||||
);
|
||||
assert.equal(refusal.category, "unsupported_provider");
|
||||
assert.match(
|
||||
JSON.stringify(refusal),
|
||||
variant === "symbol-relative"
|
||||
? /Symbol-bearing R_MIPS_REL32/u
|
||||
: /section relocation declarations/u,
|
||||
);
|
||||
assert.deepEqual(await readFile(fixture.path), original);
|
||||
await client.ping();
|
||||
reports.push({ ...fixture, status: "refused", ping: true });
|
||||
continue;
|
||||
}
|
||||
await call("open_binary", {
|
||||
path: fixture.path,
|
||||
provider_id: "ghidra",
|
||||
});
|
||||
const loadedReply = await reply("inspect_native_load_image");
|
||||
const loaded = requireMcpOperationResult(
|
||||
loadedReply,
|
||||
"inspect_native_load_image",
|
||||
);
|
||||
const evidence = parseEvidence(
|
||||
JSON.parse(
|
||||
loadedReply.content.find((item) => item.type === "text").text,
|
||||
),
|
||||
);
|
||||
const language = `MIPS:${byteOrder === "little" ? "LE" : "BE"}:32:default`;
|
||||
assert.equal(loaded.observations.language_id, language);
|
||||
assert.equal(loaded.observations.compiler_spec_id, "default");
|
||||
assert(
|
||||
loaded.observations.source_files.some(
|
||||
(file) => file.original_sha256 === fixture.sha256,
|
||||
),
|
||||
);
|
||||
assert(
|
||||
evidence.limitations.some((item) =>
|
||||
item.includes("External GOT bindings"),
|
||||
),
|
||||
);
|
||||
if (variant === "recordless")
|
||||
assert(
|
||||
evidence.limitations.some((item) =>
|
||||
item.includes("does not establish the floating-point ABI"),
|
||||
),
|
||||
);
|
||||
const base = BigInt(loaded.observations.image_base);
|
||||
const address = (source, clearTag = false) =>
|
||||
`0x${(base + (clearTag ? BigInt(source) & ~1n : BigInt(source))).toString(16)}`;
|
||||
const compressed = address(fixture.symbols.rea_mips16_probe, true);
|
||||
const standard = address(fixture.symbols.rea_mips32_probe);
|
||||
const procedures = await call("list_procedures");
|
||||
assert(
|
||||
procedures.some(
|
||||
(item) =>
|
||||
item.address === compressed && item.value === "rea_mips16_probe",
|
||||
),
|
||||
);
|
||||
assert(
|
||||
procedures.some(
|
||||
(item) =>
|
||||
item.address === standard && item.value === "rea_mips32_probe",
|
||||
),
|
||||
);
|
||||
let firstInstruction;
|
||||
const expected =
|
||||
byteOrder === "little"
|
||||
? ["5a6a", "8eea", "074a", "a0e8"]
|
||||
: ["6a5a", "ea8e", "4a07", "e8a0"];
|
||||
for (let i = 0; i < 4; i += 1) {
|
||||
const instruction = await call("inspect_native_instruction", {
|
||||
address: `0x${(BigInt(compressed) + BigInt(2 * i)).toString(16)}`,
|
||||
});
|
||||
assert.equal(instruction.status, "decoded");
|
||||
assert.equal(instruction.mode, "default");
|
||||
assert.equal(instruction.length, 2);
|
||||
assert.equal(instruction.bytes, expected[i]);
|
||||
assert.deepEqual(instruction.decoder_context, [
|
||||
{
|
||||
register: "ISA_MODE",
|
||||
value: "1",
|
||||
source: "ghidra-program-context",
|
||||
},
|
||||
]);
|
||||
if (i === 0) firstInstruction = instruction;
|
||||
}
|
||||
const standardInstruction = await call("inspect_native_instruction", {
|
||||
address: standard,
|
||||
});
|
||||
assert.equal(standardInstruction.length, 4);
|
||||
assert.equal(
|
||||
standardInstruction.bytes,
|
||||
byteOrder === "little" ? "03008224" : "24820003",
|
||||
);
|
||||
assert.equal(
|
||||
standardInstruction.decoder_context[0].register,
|
||||
"ISA_MODE",
|
||||
);
|
||||
assert(
|
||||
[null, "0"].includes(standardInstruction.decoder_context[0].value),
|
||||
);
|
||||
const pointer = await call("read_bytes", {
|
||||
address: address(fixture.symbols.rea_mips16_pointer),
|
||||
length: 4,
|
||||
});
|
||||
const pointerBytes = Buffer.from(pointer.bytes_hex, "hex");
|
||||
const pointerValue =
|
||||
byteOrder === "little"
|
||||
? pointerBytes.readUInt32LE(0)
|
||||
: pointerBytes.readUInt32BE(0);
|
||||
assert.equal(
|
||||
BigInt(pointerValue),
|
||||
base + BigInt(fixture.symbols.rea_mips16_probe),
|
||||
);
|
||||
const dossier = await call("analyze_function", {
|
||||
procedure: compressed,
|
||||
});
|
||||
assert.match(dossier.pseudocode, /\^\s*0x5a/u);
|
||||
assert.match(dossier.pseudocode, /\+\s*(?:7|0x7)\b/u);
|
||||
const standardDossier = await call("analyze_function", {
|
||||
procedure: standard,
|
||||
});
|
||||
assert.match(standardDossier.pseudocode, /\+\s*(?:3|0x3)\b/u);
|
||||
const snapshotPath = join(
|
||||
workspace,
|
||||
`${byteOrder}-${variant}.snapshot.json`,
|
||||
);
|
||||
await call("close_binary", { snapshot_path: snapshotPath });
|
||||
const snapshot = parseAnalysisSnapshot(
|
||||
JSON.parse(await readFile(snapshotPath, "utf8")),
|
||||
);
|
||||
assert.equal(
|
||||
snapshot.binding.analysis_profile.parameters.language_selection,
|
||||
"provider-pinned-v1",
|
||||
);
|
||||
assert.equal(
|
||||
snapshot.binding.analysis_profile.parameters.language_id,
|
||||
language,
|
||||
);
|
||||
await call("open_binary", {
|
||||
path: fixture.path,
|
||||
provider_id: "ghidra",
|
||||
snapshot_path: snapshotPath,
|
||||
});
|
||||
assert.deepEqual(
|
||||
await call("inspect_native_instruction", { address: compressed }),
|
||||
firstInstruction,
|
||||
);
|
||||
await call("close_binary");
|
||||
await client.ping();
|
||||
const cli = await execute(
|
||||
process.execPath,
|
||||
[
|
||||
"scripts/rea.mjs",
|
||||
"inspect-native-instruction",
|
||||
fixture.path,
|
||||
compressed,
|
||||
"--provider",
|
||||
"ghidra",
|
||||
"--format",
|
||||
"json",
|
||||
],
|
||||
{ env, timeout: 180000, maxBuffer: 8 * 1024 * 1024 },
|
||||
);
|
||||
assert.deepEqual(
|
||||
parseEvidence(JSON.parse(cli.stdout)).normalized_result,
|
||||
firstInstruction,
|
||||
);
|
||||
assert.deepEqual(await readFile(fixture.path), original);
|
||||
reports.push({
|
||||
...fixture,
|
||||
status: "passed",
|
||||
language_id: language,
|
||||
mixed_mode: true,
|
||||
standard_decoder_context: standardInstruction.decoder_context,
|
||||
pointer_tag_preserved: true,
|
||||
snapshot_context_preserved: true,
|
||||
cli_mcp_parity: true,
|
||||
source_unchanged: true,
|
||||
ping: true,
|
||||
});
|
||||
} finally {
|
||||
await client.close();
|
||||
await transport.close();
|
||||
await writeFile(
|
||||
join(workspace, `${byteOrder}-${variant}.stderr`),
|
||||
stderr,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (cause) {
|
||||
failure = cause;
|
||||
}
|
||||
const completed = await completeVerifierRun(run);
|
||||
if (
|
||||
completed.process_lineage.status !== "verified" ||
|
||||
completed.process_lineage.descendants.length !== 0
|
||||
)
|
||||
throw new Error(`Cleanup unverified; retained workspace: ${workspace}`, {
|
||||
cause: failure,
|
||||
});
|
||||
if (failure !== undefined)
|
||||
throw new Error(
|
||||
`MIPS16 shared verification failed; retained workspace: ${workspace}`,
|
||||
{ cause: failure },
|
||||
);
|
||||
await rm(workspace, { recursive: true, force: true });
|
||||
console.log(JSON.stringify({ ...completed, fixtures: reports }, null, 2));
|
||||
@@ -30,6 +30,7 @@ import {
|
||||
requireMcpOperationResult,
|
||||
requireMcpLifecycleResult,
|
||||
requireMcpEvidenceResult,
|
||||
sortVerifierNames,
|
||||
requireWorkflowEvidenceProvider,
|
||||
} from "./lib/mcp-verifier-results.mjs";
|
||||
import {
|
||||
@@ -283,8 +284,10 @@ try {
|
||||
const availableTools = initialStatus.tool_availability;
|
||||
if (!Array.isArray(availableTools))
|
||||
throw new Error("The real session omitted tool availability");
|
||||
const expectedNames = availableTools.map(({ name }) => name).sort();
|
||||
const actualNames = listed.tools.map(({ name }) => name).sort();
|
||||
const expectedNames = sortVerifierNames(
|
||||
availableTools.map(({ name }) => name),
|
||||
);
|
||||
const actualNames = sortVerifierNames(listed.tools.map(({ name }) => name));
|
||||
if (JSON.stringify(actualNames) !== JSON.stringify(expectedNames)) {
|
||||
const expected = new Set(expectedNames);
|
||||
const actual = new Set(actualNames);
|
||||
|
||||
@@ -131,15 +131,35 @@ try {
|
||||
i32.const 1
|
||||
memory.init 0
|
||||
data.drop 0))\n`,
|
||||
];
|
||||
for (const [index, source] of sources.entries()) {
|
||||
].map((source) => ({ source, debugNames: false }));
|
||||
sources.push(
|
||||
{
|
||||
source:
|
||||
'(module $named (func (export "answer") (result i32) i32.const 42))\n',
|
||||
debugNames: true,
|
||||
},
|
||||
{
|
||||
source: await readFile(
|
||||
new URL("../tests/fixtures/wasm/module-names.wat", import.meta.url),
|
||||
"utf8",
|
||||
),
|
||||
debugNames: false,
|
||||
},
|
||||
);
|
||||
for (const [index, { source, debugNames }] of sources.entries()) {
|
||||
const fixtureRoot = join(root.path, `fixture-${index}`);
|
||||
await mkdir(fixtureRoot);
|
||||
const path = join(fixtureRoot, "module.wasm");
|
||||
await writeFile(join(fixtureRoot, "source.wat"), source);
|
||||
await tool(
|
||||
"wat2wasm",
|
||||
["--enable-all", "source.wat", "-o", "module.wasm"],
|
||||
[
|
||||
"--enable-all",
|
||||
...(debugNames ? ["--debug-names"] : []),
|
||||
"source.wat",
|
||||
"-o",
|
||||
"module.wasm",
|
||||
],
|
||||
fixtureRoot,
|
||||
);
|
||||
await tool("wasm-validate", ["--enable-all", "module.wasm"], fixtureRoot);
|
||||
@@ -227,8 +247,13 @@ try {
|
||||
'"test\nline"',
|
||||
'"fake"\nSection Details:\n Type start=0x0000000a end=0x0000000b (size=0x00000001) count: 0\n"end"',
|
||||
'"unicode-é\\\t"',
|
||||
'"visible"',
|
||||
],
|
||||
);
|
||||
if (debugNames) {
|
||||
assert.ok(report.headers.includes("\nmodule name: <named>\n"));
|
||||
assert.deepEqual(report.exports, ['(export "answer" (func 0))']);
|
||||
}
|
||||
if (index === 4) {
|
||||
assert.match(
|
||||
headers,
|
||||
|
||||
@@ -913,6 +913,30 @@ async function inspect(mode, input, errorCategory) {
|
||||
}
|
||||
assert.notEqual(result.isError, true, mcpTextValue(result));
|
||||
assertSensitiveLimitations(value, input.sensitive_values ?? []);
|
||||
if (input.format === "har") {
|
||||
const surface = await client.callTool({
|
||||
name: "inspect_analysis_view",
|
||||
arguments: {
|
||||
source: { kind: "retained-evidence", evidence_id: value.evidence_id },
|
||||
view: { kind: "api-surface" },
|
||||
},
|
||||
});
|
||||
if ((input.sensitive_values ?? []).length > 0) {
|
||||
// Derived view text cannot be checked against undisclosed declarations.
|
||||
assert.equal(surface.isError, true);
|
||||
const refused = JSON.parse(mcpTextValue(surface));
|
||||
for (const literal of input.sensitive_values)
|
||||
assert.ok(!JSON.stringify(refused).includes(literal));
|
||||
} else {
|
||||
assert.notEqual(surface.isError, true, mcpTextValue(surface));
|
||||
const projected = JSON.parse(mcpTextValue(surface));
|
||||
assert.equal(projected.normalized_result.kind, "api-surface");
|
||||
assert.equal(
|
||||
projected.normalized_result.records.examined,
|
||||
value.normalized_result.records.length,
|
||||
);
|
||||
}
|
||||
}
|
||||
return value.normalized_result;
|
||||
}
|
||||
const args = [
|
||||
|
||||
@@ -1,4 +1,8 @@
|
||||
import { requireMcpToolError } from "../../lib/mcp-verifier-results.mjs";
|
||||
import {
|
||||
requireMcpLifecycleResult,
|
||||
requireMcpToolError,
|
||||
sortVerifierNames,
|
||||
} from "../../lib/mcp-verifier-results.mjs";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
access,
|
||||
@@ -273,22 +277,23 @@ const verifyAndroidReadiness = async ({
|
||||
try {
|
||||
await client.connect(transport);
|
||||
const catalog = await client.listTools();
|
||||
const catalogNames = catalog.tools.map(({ name }) => name).sort();
|
||||
const status = await client.callTool({
|
||||
const catalogNames = sortVerifierNames(
|
||||
catalog.tools.map(({ name }) => name),
|
||||
);
|
||||
const statusReply = await client.callTool({
|
||||
name: "binary_session",
|
||||
arguments: {},
|
||||
});
|
||||
assert.notEqual(status.isError, true, JSON.stringify(status));
|
||||
const availability = status.structuredContent?.result?.tool_availability;
|
||||
assert.ok(Array.isArray(availability));
|
||||
const status = requireMcpLifecycleResult(statusReply, "binary_session");
|
||||
const availability = status.tool_availability;
|
||||
assert.deepEqual(
|
||||
availability.map(({ name }) => name).sort(),
|
||||
TOOL_CONTRACTS.map(({ name }) => name).sort(),
|
||||
sortVerifierNames(availability.map(({ name }) => name)),
|
||||
sortVerifierNames(TOOL_CONTRACTS.map(({ name }) => name)),
|
||||
"Readiness must retain one availability record for the complete canonical catalog",
|
||||
);
|
||||
assert.deepEqual(
|
||||
catalogNames,
|
||||
TOOL_CONTRACTS.map(({ name }) => name).sort(),
|
||||
sortVerifierNames(TOOL_CONTRACTS.map(({ name }) => name)),
|
||||
"Readiness must not truncate the SDK tool catalog",
|
||||
);
|
||||
const byName = new Map(availability.map((item) => [item.name, item]));
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { execFile } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
@@ -48,6 +48,19 @@ export async function verifyNativeProvenance(app, executable) {
|
||||
assert.equal(mcp.raw_result, null);
|
||||
assert.ok(cli.normalized_result.provenance.length > 0);
|
||||
assert.deepEqual(mcp.normalized_result, cli.normalized_result);
|
||||
if (operation === "inspect_macho") {
|
||||
const delayed = cli.normalized_result.load_commands.items.find(
|
||||
({ fields }) => fields.name === "@rpath/libdelay.dylib (offset 28)",
|
||||
);
|
||||
assert.equal(delayed?.kind, "LC_LOAD_DYLIB");
|
||||
assert.equal(delayed.fields.options, "delay-init");
|
||||
assert.equal(delayed.fields["time stamp"], undefined);
|
||||
assert.ok(
|
||||
cli.normalized_result.dependencies.items.some(
|
||||
({ path }) => path === "@rpath/libdelay.dylib",
|
||||
),
|
||||
);
|
||||
}
|
||||
assert.deepEqual(
|
||||
JSON.parse(response.content.find((item) => item.type === "text").text),
|
||||
response.structuredContent,
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { readFile, readdir } from "node:fs/promises";
|
||||
import { join, relative, sep } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
@@ -100,7 +100,9 @@ const verifyRegions = (evidence) => {
|
||||
);
|
||||
assert.equal(gzip.reported_size, oracle.length);
|
||||
assert.equal(gzip.size_basis, "provider_reported_validation_unknown");
|
||||
return result.regions.map(({ provider_id, ...region }) => region);
|
||||
return result.regions.map(
|
||||
({ provider_id: _provider_id, ...region }) => region,
|
||||
);
|
||||
};
|
||||
const verifyFiles = async (evidence) => {
|
||||
const result = firmwareResultSchemas.extract_firmware.parse(
|
||||
|
||||
@@ -77,6 +77,8 @@ async function verifyTarget(child, targetPath, name) {
|
||||
assert.equal(discovery.length, 1);
|
||||
const raw = discovery[0];
|
||||
assert.equal(typeof raw.id, "string");
|
||||
const expectedAttached =
|
||||
typeof raw.attached === "boolean" ? raw.attached : null;
|
||||
// Independent oracle: the fixture owns the pathname, and Node's discovery
|
||||
// serialization loses quotes/backslashes. Never decode underscores into it.
|
||||
const unresolved = process.platform === "win32" || /[_"\\]/u.test(targetPath);
|
||||
@@ -96,6 +98,7 @@ async function verifyTarget(child, targetPath, name) {
|
||||
const inventory = listed.normalized_result;
|
||||
assert.equal(inventory.targets.length, 1);
|
||||
assert.equal(inventory.targets[0].target_id, raw.id);
|
||||
assert.equal(inventory.targets[0].attached, expectedAttached);
|
||||
assert.deepEqual(inventory.targets[0].location, expectedLocation);
|
||||
assert.equal(inventory.excluded.unsupported_location, 0);
|
||||
const mcpList = await runMcp("list_javascript_runtime_targets", {
|
||||
@@ -122,7 +125,13 @@ async function verifyTarget(child, targetPath, name) {
|
||||
observation_ms: 100,
|
||||
});
|
||||
for (const result of [observed.normalized_result, mcpObservation])
|
||||
verifyObservation(result, raw.id, expectedLocation, targetPath);
|
||||
verifyObservation(
|
||||
result,
|
||||
raw.id,
|
||||
expectedLocation,
|
||||
targetPath,
|
||||
expectedAttached,
|
||||
);
|
||||
assert.deepEqual(mcpObservation.target, observed.normalized_result.target);
|
||||
process.stdout.write(
|
||||
`${JSON.stringify({
|
||||
@@ -133,6 +142,7 @@ async function verifyTarget(child, targetPath, name) {
|
||||
target_id: raw.id,
|
||||
discovery_url: raw.url,
|
||||
target_location: expectedLocation,
|
||||
attached: inventory.targets[0].attached,
|
||||
cli_scripts: observed.normalized_result.scripts.items.length,
|
||||
mcp_scripts: mcpObservation.scripts.items.length,
|
||||
listed_evidence_id: listed.evidence_id,
|
||||
@@ -141,9 +151,16 @@ async function verifyTarget(child, targetPath, name) {
|
||||
);
|
||||
}
|
||||
|
||||
function verifyObservation(result, targetId, expectedLocation, targetPath) {
|
||||
function verifyObservation(
|
||||
result,
|
||||
targetId,
|
||||
expectedLocation,
|
||||
targetPath,
|
||||
expectedAttached,
|
||||
) {
|
||||
assert.equal(result.target.target_id, targetId);
|
||||
assert.equal(result.target.runtime_kind, "node");
|
||||
assert.equal(result.target.attached, expectedAttached);
|
||||
assert.deepEqual(result.target.location, expectedLocation);
|
||||
assert.equal(result.capture.truncated, false);
|
||||
assert.ok(Array.isArray(result.directly_observed));
|
||||
|
||||
@@ -239,4 +239,35 @@ describe("parsePmPathOutput", () => {
|
||||
role: "unknown",
|
||||
});
|
||||
});
|
||||
|
||||
it("preserves whitespace and the exact POSIX device path", () => {
|
||||
const devicePath = "/data/app/My App/résumé:final\\draft*.apk";
|
||||
const { paths, rejectedPaths, unparsedLines } = parsePmPathOutput(
|
||||
`package:${devicePath}\r\n`,
|
||||
);
|
||||
expect(paths).toEqual([
|
||||
{
|
||||
device_path: devicePath,
|
||||
file_name: "résumé:final\\draft*.apk",
|
||||
role: "unknown",
|
||||
},
|
||||
]);
|
||||
expect(rejectedPaths).toEqual([]);
|
||||
expect(unparsedLines).toEqual([]);
|
||||
});
|
||||
|
||||
it("rejects an empty final component without normalizing the source", () => {
|
||||
const devicePath = "/data/app/pkg/base.apk/";
|
||||
const parsed = parsePmPathOutput(`package:${devicePath}\n`);
|
||||
expect(parsed.paths).toEqual([]);
|
||||
expect(parsed.rejectedPaths).toEqual([
|
||||
{ device_path: devicePath, reason: "empty_file_name" },
|
||||
]);
|
||||
});
|
||||
|
||||
it("retains malformed lines including surrounding whitespace", () => {
|
||||
expect(
|
||||
parsePmPathOutput(" package:/data/app/pkg/base.apk \n").unparsedLines,
|
||||
).toEqual([" package:/data/app/pkg/base.apk "]);
|
||||
});
|
||||
});
|
||||
|
||||
+26
-11
@@ -6,6 +6,11 @@
|
||||
* their evidence basis.
|
||||
*/
|
||||
|
||||
import {
|
||||
adbHostFileName,
|
||||
type AdbHostFileNameReason,
|
||||
} from "./AdbHostFileName.js";
|
||||
|
||||
/** One device exactly as `adb devices -l` reports it. */
|
||||
export interface AdbDeviceEntry {
|
||||
readonly serial: string;
|
||||
@@ -205,11 +210,6 @@ export interface AdbPackagePathEntry {
|
||||
readonly role: "base" | "split" | "unknown";
|
||||
}
|
||||
|
||||
const fileNameOf = (devicePath: string): string => {
|
||||
const parts = devicePath.split(/\//u);
|
||||
return parts[parts.length - 1] ?? devicePath;
|
||||
};
|
||||
|
||||
/**
|
||||
* Parse `pm path PACKAGE` lines. The base/split role is derived from the file
|
||||
* name (`base.apk`, `split_*.apk`), which is the only signal the output
|
||||
@@ -219,20 +219,35 @@ export const parsePmPathOutput = (
|
||||
stdout: string,
|
||||
): {
|
||||
paths: readonly AdbPackagePathEntry[];
|
||||
rejectedPaths: readonly {
|
||||
device_path: string;
|
||||
reason: AdbHostFileNameReason;
|
||||
}[];
|
||||
unparsedLines: readonly string[];
|
||||
} => {
|
||||
const paths: AdbPackagePathEntry[] = [];
|
||||
const rejectedPaths: {
|
||||
device_path: string;
|
||||
reason: AdbHostFileNameReason;
|
||||
}[] = [];
|
||||
const unparsedLines: string[] = [];
|
||||
for (const rawLine of stdout.split(/\r?\n/u)) {
|
||||
const line = rawLine.trim();
|
||||
if (line === "") continue;
|
||||
const match = /^package:(\S+)$/u.exec(line);
|
||||
if (rawLine === "") continue;
|
||||
const match = /^package:(.*)$/u.exec(rawLine);
|
||||
if (match === null) {
|
||||
unparsedLines.push(line);
|
||||
unparsedLines.push(rawLine);
|
||||
continue;
|
||||
}
|
||||
const devicePath = match[1]!;
|
||||
const fileName = fileNameOf(devicePath);
|
||||
const validated = adbHostFileName(devicePath);
|
||||
if (!validated.ok) {
|
||||
rejectedPaths.push({
|
||||
device_path: devicePath,
|
||||
reason: validated.reason,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
const fileName = validated.fileName;
|
||||
paths.push({
|
||||
device_path: devicePath,
|
||||
file_name: fileName,
|
||||
@@ -244,5 +259,5 @@ export const parsePmPathOutput = (
|
||||
: "unknown",
|
||||
});
|
||||
}
|
||||
return { paths, unparsedLines };
|
||||
return { paths, rejectedPaths, unparsedLines };
|
||||
};
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
import { isUtf8 } from "node:buffer";
|
||||
|
||||
interface AdbDirectoryEntry {
|
||||
readonly name: string;
|
||||
readonly kind: "directory" | "file" | "symlink" | "other";
|
||||
readonly permissions: string | null;
|
||||
readonly owner: string | null;
|
||||
readonly group: string | null;
|
||||
readonly bytes: number | null;
|
||||
readonly date: string | null;
|
||||
readonly link_target: string | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* List direct children with Android 8 Toybox's batched `find -exec` and
|
||||
* `stat -c` support. Names and link targets are NUL-framed; neither is ever
|
||||
* inserted into a line-oriented metadata record or command substitution.
|
||||
*/
|
||||
export const ADB_DIRECTORY_BATCH_SCRIPT = String.raw`metadata=$(stat -c '%A|%U|%G|%s|%y' "$@") || exit
|
||||
printf '%s\n' "$metadata" | {
|
||||
while IFS= read -r metadata; do
|
||||
[ "$#" -gt 0 ] || exit 1
|
||||
path=$1
|
||||
shift
|
||||
printf '%s\000%s\000' "$metadata" "${"${path##*/}"}"
|
||||
case "$metadata" in
|
||||
l*) readlink -n "$path" || exit 1 ;;
|
||||
esac
|
||||
printf '\000'
|
||||
done
|
||||
[ "$#" -eq 0 ]
|
||||
}`;
|
||||
|
||||
const METADATA =
|
||||
/^([bcdlps-][rwxstST-]{9})\|([^|\n]+)\|([^|\n]+)\|(\d+)\|([^|\n]+)$/u;
|
||||
|
||||
const parseEntry = (
|
||||
metadata: string,
|
||||
name: string,
|
||||
linkTarget: string,
|
||||
): AdbDirectoryEntry | null => {
|
||||
const match = METADATA.exec(metadata);
|
||||
if (match === null || name === "" || name.includes("/")) return null;
|
||||
const permissions = match[1]!;
|
||||
const bytes = Number(match[4]!);
|
||||
if (!Number.isSafeInteger(bytes)) return null;
|
||||
|
||||
const isSymlink = permissions.startsWith("l");
|
||||
if (isSymlink ? linkTarget === "" : linkTarget !== "") return null;
|
||||
|
||||
return {
|
||||
name,
|
||||
kind: permissions.startsWith("d")
|
||||
? "directory"
|
||||
: isSymlink
|
||||
? "symlink"
|
||||
: permissions.startsWith("-")
|
||||
? "file"
|
||||
: "other",
|
||||
permissions,
|
||||
owner: match[2]!,
|
||||
group: match[3]!,
|
||||
bytes,
|
||||
date: match[5]!,
|
||||
link_target: isSymlink ? linkTarget : null,
|
||||
};
|
||||
};
|
||||
|
||||
/** Parse metadata/name/target NUL triples from `ADB_DIRECTORY_BATCH_SCRIPT`. */
|
||||
export const parseAdbDirectoryListing = (
|
||||
stdout: Buffer,
|
||||
): {
|
||||
entries: readonly AdbDirectoryEntry[];
|
||||
unparsedLines: readonly string[];
|
||||
} => {
|
||||
const entries: AdbDirectoryEntry[] = [];
|
||||
const unparsedLines: string[] = [];
|
||||
let offset = 0;
|
||||
while (offset < stdout.length) {
|
||||
const metadataEnd = stdout.indexOf(0, offset);
|
||||
const nameEnd = metadataEnd < 0 ? -1 : stdout.indexOf(0, metadataEnd + 1);
|
||||
const targetEnd = nameEnd < 0 ? -1 : stdout.indexOf(0, nameEnd + 1);
|
||||
if (targetEnd < 0) {
|
||||
unparsedLines.push(describeUnparsedRecord(stdout.subarray(offset)));
|
||||
break;
|
||||
}
|
||||
const rawRecord = stdout.subarray(offset, targetEnd + 1);
|
||||
if (!isUtf8(rawRecord)) {
|
||||
unparsedLines.push(describeUnparsedRecord(rawRecord));
|
||||
} else {
|
||||
const entry = parseEntry(
|
||||
stdout.toString("utf8", offset, metadataEnd),
|
||||
stdout.toString("utf8", metadataEnd + 1, nameEnd),
|
||||
stdout.toString("utf8", nameEnd + 1, targetEnd),
|
||||
);
|
||||
if (entry === null) unparsedLines.push(rawRecord.toString("utf8"));
|
||||
else entries.push(entry);
|
||||
}
|
||||
offset = targetEnd + 1;
|
||||
}
|
||||
|
||||
return { entries, unparsedLines };
|
||||
};
|
||||
|
||||
const describeUnparsedRecord = (record: Buffer): string =>
|
||||
isUtf8(record)
|
||||
? record.toString("utf8")
|
||||
: `Invalid UTF-8 directory record (hex): ${record.toString("hex")}`;
|
||||
@@ -14,28 +14,50 @@ describe("adbHostFileName", () => {
|
||||
],
|
||||
["/data/local/tmp/résumé.xml", "résumé.xml"],
|
||||
["/data/local/tmp/a-b_c.d", "a-b_c.d"],
|
||||
["/data/local/tmp/", "tmp"],
|
||||
])("accepts the portable device path %s", (devicePath, fileName) => {
|
||||
["/data/local/tmp/a\\b", "a\\b"],
|
||||
["/data/local/tmp/report:2024", "report:2024"],
|
||||
["/data/local/tmp/a*b.txt", "a*b.txt"],
|
||||
["/data/local/tmp/re\u007fport", "re\u007fport"],
|
||||
])("accepts the POSIX device path %s", (devicePath, fileName) => {
|
||||
expect(adbHostFileName(devicePath)).toEqual({ ok: true, fileName });
|
||||
});
|
||||
|
||||
it.each([
|
||||
["/", "empty_file_name"],
|
||||
["data/local/tmp/file", "not_absolute_path"],
|
||||
["/data/local/tmp/", "empty_file_name"],
|
||||
["/data/local/tmp/..", "dot_segment"],
|
||||
["/data/local/tmp/.", "dot_segment"],
|
||||
["/data/local/tmp/a\\b", "path_separator"],
|
||||
["/data/local/tmp/re\u007fport", "control_character"],
|
||||
["/data/local/tmp/ta\tb", "control_character"],
|
||||
["/data\0/local/tmp/file", "unsupported_character"],
|
||||
["/data/local/tmp/a\0b", "unsupported_character"],
|
||||
] as const)("refuses %s as %s", (devicePath, reason) => {
|
||||
expect(adbHostFileName(devicePath)).toEqual({ ok: false, reason });
|
||||
});
|
||||
|
||||
it.each([
|
||||
["/data/local/tmp/a\\b", "unsupported_character"],
|
||||
["/data/local/tmp/ta\tb", "unsupported_character"],
|
||||
["/data/local/tmp/CON", "reserved_name"],
|
||||
["/data/local/tmp/nul.txt", "reserved_name"],
|
||||
["/data/local/tmp/com1", "reserved_name"],
|
||||
["/data/local/tmp/LPT9", "reserved_name"],
|
||||
["/data/local/tmp/report:2024", "alternate_data_stream"],
|
||||
["/data/local/tmp/COM¹.txt", "reserved_name"],
|
||||
["/data/local/tmp/lpt³", "reserved_name"],
|
||||
["/data/local/tmp/report:2024", "unsupported_character"],
|
||||
["/data/local/tmp/report.", "trailing_dot_or_space"],
|
||||
["/data/local/tmp/report ", "trailing_dot_or_space"],
|
||||
["/data/local/tmp/a*b.txt", "unsupported_character"],
|
||||
["/data/local/tmp/a?b.txt", "unsupported_character"],
|
||||
] as const)("refuses %s as %s", (devicePath, reason) => {
|
||||
expect(adbHostFileName(devicePath)).toEqual({ ok: false, reason });
|
||||
] as const)("rejects %s on Windows as %s", (devicePath, reason) => {
|
||||
expect(adbHostFileName(devicePath, "win32")).toEqual({ ok: false, reason });
|
||||
});
|
||||
|
||||
it("does not impose non-Windows control restrictions on Windows filenames", () => {
|
||||
expect(adbHostFileName("/data/local/tmp/name\u007f.txt", "win32").ok).toBe(
|
||||
true,
|
||||
);
|
||||
expect(adbHostFileName("/data/local/tmp/name\u0085.txt", "win32").ok).toBe(
|
||||
true,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
+34
-42
@@ -1,61 +1,53 @@
|
||||
import { posix } from "node:path";
|
||||
|
||||
/** Why a device-reported path cannot be used as a host file name. */
|
||||
export type AdbHostFileNameReason =
|
||||
| "empty_file_name"
|
||||
| "not_absolute_path"
|
||||
| "dot_segment"
|
||||
| "path_separator"
|
||||
| "control_character"
|
||||
| "unsupported_character"
|
||||
| "reserved_name"
|
||||
| "trailing_dot_or_space"
|
||||
| "alternate_data_stream";
|
||||
| "trailing_dot_or_space";
|
||||
|
||||
/** A portable host file name, or the exact reason the device name was refused. */
|
||||
export type AdbHostFileNameResult =
|
||||
| { readonly ok: true; readonly fileName: string }
|
||||
| { readonly ok: false; readonly reason: AdbHostFileNameReason };
|
||||
|
||||
/**
|
||||
* Win32 resolves these basenames to a device regardless of case and of any
|
||||
* extension (`NUL`, `nul.txt`), so a portable host file name must avoid them on
|
||||
* every host. `(?:\.|$)` keeps ordinary names such as `console.txt` admissible.
|
||||
*/
|
||||
const WINDOWS_RESERVED_BASENAME =
|
||||
/^(?:con|prn|aux|nul|com[1-9]|lpt[1-9])(?:\.|$)/iu;
|
||||
|
||||
/** Characters that are not portable in a host file name on Windows or POSIX. */
|
||||
const UNSUPPORTED_CHARACTER = /[*?"<>|]/u;
|
||||
|
||||
/** Control characters (`Cc`), including C0 controls and DEL. */
|
||||
const CONTROL_CHARACTER = /\p{Cc}/u;
|
||||
/^(?:con|prn|aux|nul|com[1-9¹²³]|lpt[1-9¹²³])(?:\.|$)/iu;
|
||||
|
||||
/**
|
||||
* Derive a portable host file name from a device-reported path.
|
||||
*
|
||||
* The device path is a POSIX path on the device filesystem, so its last
|
||||
* `/`-separated segment is taken with POSIX semantics rather than the host
|
||||
* `basename` (which would vary between POSIX and Windows hosts). A name is
|
||||
* accepted only when every host can store it and address it back unambiguously:
|
||||
* no separator, control, or reserved-device characters, no trailing dot or
|
||||
* space, no `:` alternate-data-stream marker, and no `.`/`..` segment.
|
||||
* Derive the final component of a device POSIX path without normalizing the
|
||||
* source. A trailing slash has no filename and must not silently become the
|
||||
* preceding component as `basename` would make it.
|
||||
*/
|
||||
export const adbHostFileName = (devicePath: string): AdbHostFileNameResult => {
|
||||
const fileName = posix.basename(devicePath);
|
||||
if (fileName === "") return { ok: false, reason: "empty_file_name" };
|
||||
export const adbHostFileName = (
|
||||
devicePath: string,
|
||||
platform: NodeJS.Platform = process.platform,
|
||||
): AdbHostFileNameResult => {
|
||||
if (devicePath === "" || devicePath.endsWith("/"))
|
||||
return { ok: false, reason: "empty_file_name" };
|
||||
if (devicePath.includes("\0"))
|
||||
return { ok: false, reason: "unsupported_character" };
|
||||
if (!devicePath.startsWith("/"))
|
||||
return { ok: false, reason: "not_absolute_path" };
|
||||
const fileName = devicePath.slice(devicePath.lastIndexOf("/") + 1);
|
||||
if (fileName === "." || fileName === "..")
|
||||
return { ok: false, reason: "dot_segment" };
|
||||
if (fileName.includes("\\") || fileName.includes("/"))
|
||||
return { ok: false, reason: "path_separator" };
|
||||
if (CONTROL_CHARACTER.test(fileName))
|
||||
return { ok: false, reason: "control_character" };
|
||||
if (fileName.includes(":"))
|
||||
return { ok: false, reason: "alternate_data_stream" };
|
||||
if (UNSUPPORTED_CHARACTER.test(fileName))
|
||||
return { ok: false, reason: "unsupported_character" };
|
||||
if (WINDOWS_RESERVED_BASENAME.test(fileName))
|
||||
return { ok: false, reason: "reserved_name" };
|
||||
if (/[. ]$/u.test(fileName))
|
||||
return { ok: false, reason: "trailing_dot_or_space" };
|
||||
if (platform === "win32") {
|
||||
if (/[\\:*?"<>|]/u.test(fileName) || hasWindowsControl(fileName))
|
||||
return { ok: false, reason: "unsupported_character" };
|
||||
if (WINDOWS_RESERVED_BASENAME.test(fileName))
|
||||
return { ok: false, reason: "reserved_name" };
|
||||
if (/[. ]$/u.test(fileName))
|
||||
return { ok: false, reason: "trailing_dot_or_space" };
|
||||
}
|
||||
|
||||
return { ok: true, fileName };
|
||||
};
|
||||
|
||||
const hasWindowsControl = (fileName: string): boolean => {
|
||||
for (const character of fileName) {
|
||||
const code = character.codePointAt(0)!;
|
||||
if (code > 0 && code < 0x20) return true;
|
||||
}
|
||||
return false;
|
||||
};
|
||||
|
||||
+189
-53
@@ -1,6 +1,7 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import {
|
||||
chmod,
|
||||
lstat,
|
||||
mkdir,
|
||||
mkdtemp,
|
||||
readFile,
|
||||
@@ -52,6 +53,28 @@ const refusedBy = async (
|
||||
return projectAnalysisError(result.error);
|
||||
};
|
||||
|
||||
const namesAliasOnVolume = async (
|
||||
parent: string,
|
||||
firstName: string,
|
||||
secondName: string,
|
||||
): Promise<boolean> => {
|
||||
const probe = join(parent, "destination-name-probe");
|
||||
await mkdir(probe);
|
||||
try {
|
||||
await mkdir(join(probe, firstName));
|
||||
try {
|
||||
await mkdir(join(probe, secondName));
|
||||
return false;
|
||||
} catch (cause) {
|
||||
if (cause instanceof Error && "code" in cause && cause.code === "EEXIST")
|
||||
return true;
|
||||
throw cause;
|
||||
}
|
||||
} finally {
|
||||
await rm(probe, { recursive: true, force: true });
|
||||
}
|
||||
};
|
||||
|
||||
it("inspects the adb client without contacting the server", async () => {
|
||||
const { binary } = await writeAdbStub(
|
||||
() => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; fi`,
|
||||
@@ -176,7 +199,7 @@ const shellStub = (
|
||||
second === undefined
|
||||
? `[ "$4" = "${first}" ]`
|
||||
: `[ "$4" = "${first}" ] && [ "$5" = "${second}" ]`;
|
||||
return ` if ${guard}; then printf '%b' '${output.replace(/'/gu, "'\\''")}'; exit 0; fi`;
|
||||
return ` if ${guard}; then printf '%b' '${output.replaceAll("\\", "\\\\").replaceAll("\0", "\\000").replace(/'/gu, "'\\''")}'; exit 0; fi`;
|
||||
})
|
||||
.join("\n");
|
||||
return () => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; exit 0; fi
|
||||
@@ -283,6 +306,82 @@ it("refuses to reuse an existing package output directory", async () => {
|
||||
expect(result.error.message).toContain("already exists");
|
||||
});
|
||||
|
||||
it.each([
|
||||
["an empty filename", "package:/data/app/x/"],
|
||||
[
|
||||
"duplicate local filenames",
|
||||
"package:/data/app/x/base.apk\npackage:/data/app/y/base.apk",
|
||||
],
|
||||
])("refuses %s before creating output or pulling", async (_, pmOutput) => {
|
||||
const { binary, root } = await writeAdbStub(
|
||||
(root) => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; exit 0; fi
|
||||
if [ "$1" = "-s" ] && [ "$4" = "pm" ]; then printf '%b' '${pmOutput.replaceAll("'", "'\\''").replaceAll("\n", "\\n")}\\n'; exit 0; fi
|
||||
if [ "$1" = "-s" ] && [ "$3" = "pull" ]; then : > '${join(root, "pull-ran")}'; exit 0; fi
|
||||
exit 1`,
|
||||
);
|
||||
const output = join(root, "pulls");
|
||||
const result = await providerOver(binary).execute({
|
||||
operation: "pull_adb_package",
|
||||
input: {
|
||||
serial: "emulator-5554",
|
||||
package: "com.example",
|
||||
output_directory: output,
|
||||
},
|
||||
});
|
||||
expect(result.ok).toBe(false);
|
||||
if (result.ok) return;
|
||||
expect(result.error.capturedOutput?.stdout).toContain(
|
||||
pmOutput.split("\n")[0],
|
||||
);
|
||||
await expect(readFile(join(root, "pull-ran"))).rejects.toThrow();
|
||||
await expect(lstat(output)).rejects.toMatchObject({ code: "ENOENT" });
|
||||
});
|
||||
|
||||
it.each([
|
||||
["case-only names", "base.apk", "BASE.APK"],
|
||||
["normalization-equivalent names", "caf\u00e9.apk", "cafe\u0301.apk"],
|
||||
])(
|
||||
"checks %s against the actual destination filesystem before pulling",
|
||||
async (_, firstName, secondName) => {
|
||||
const { binary, root } = await writeAdbStub((root) =>
|
||||
pullStub(
|
||||
`'package:/data/app/x/${firstName}' 'package:/data/app/y/${secondName}'`,
|
||||
`printf 'pulled\\n' >> '${join(root, "pull.log")}'
|
||||
printf 'apk' > "$5"`,
|
||||
),
|
||||
);
|
||||
const aliases = await namesAliasOnVolume(root, firstName, secondName);
|
||||
const result = await providerOver(binary).execute({
|
||||
operation: "pull_adb_package",
|
||||
input: {
|
||||
serial: "emulator-5554",
|
||||
package: "com.example",
|
||||
output_directory: root,
|
||||
},
|
||||
});
|
||||
const pullLog = join(root, "pull.log");
|
||||
const packageDirectory = join(root, "com.example");
|
||||
if (aliases) {
|
||||
expect(result.ok).toBe(false);
|
||||
if (result.ok) return;
|
||||
expect(result.error.message).toContain("unique local destinations");
|
||||
await expect(readFile(pullLog)).rejects.toThrow();
|
||||
await expect(lstat(packageDirectory)).rejects.toThrow();
|
||||
return;
|
||||
}
|
||||
expect(result.ok).toBe(true);
|
||||
if (!result.ok) throw new Error(result.error.message);
|
||||
expect(
|
||||
(
|
||||
result.value.result as { artifacts: { file_name: string }[] }
|
||||
).artifacts.map((artifact) => artifact.file_name),
|
||||
).toEqual([firstName, secondName]);
|
||||
expect((await readFile(pullLog, "utf8")).trim().split("\n")).toHaveLength(
|
||||
2,
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
it("reports an uninstalled package from empty pm path output", async () => {
|
||||
const { binary } = await writeAdbStub(
|
||||
() => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; exit 0; fi
|
||||
@@ -396,7 +495,7 @@ it("reads a bounded logcat dump with the caller count and buffer", async () => {
|
||||
const { binary } = await writeAdbStub(
|
||||
shellStub({
|
||||
logcat:
|
||||
"08-10 13:00:00.000 I/Tag( 123): first\\n08-10 13:00:01.000 I/Tag( 123): second\\n",
|
||||
"08-10 13:00:00.000 I/Tag( 123): first\n08-10 13:00:01.000 I/Tag( 123): second\n",
|
||||
}),
|
||||
);
|
||||
const result = await evidenceResult(providerOver(binary), {
|
||||
@@ -465,14 +564,14 @@ exit 1`,
|
||||
operation: "pull_adb_file",
|
||||
input: {
|
||||
serial: "emulator-5554",
|
||||
device_path: "/data/local/tmp/notes.txt",
|
||||
device_path: "/data/local/tmp/My Notes:r*sumé.txt",
|
||||
output_directory: output,
|
||||
},
|
||||
});
|
||||
const digest = createHash("sha256").update("file-bytes").digest("hex");
|
||||
expect(result).toMatchObject({
|
||||
device_path: "/data/local/tmp/notes.txt",
|
||||
local_path: join(output, "notes.txt"),
|
||||
device_path: "/data/local/tmp/My Notes:r*sumé.txt",
|
||||
local_path: join(output, "My Notes:r*sumé.txt"),
|
||||
bytes: 10,
|
||||
sha256: digest,
|
||||
});
|
||||
@@ -480,10 +579,7 @@ exit 1`,
|
||||
|
||||
it.each([
|
||||
["/data/local/tmp/..", "dot_segment"],
|
||||
["/data/local/tmp/a\\b", "path_separator"],
|
||||
["/data/local/tmp/CON", "reserved_name"],
|
||||
["/data/local/tmp/report.", "trailing_dot_or_space"],
|
||||
["/data/local/tmp/report:2024", "alternate_data_stream"],
|
||||
["/data/local/tmp/", "empty_file_name"],
|
||||
] as const)(
|
||||
"refuses the unsafe device path %s before pulling",
|
||||
async (devicePath, reason) => {
|
||||
@@ -510,13 +606,19 @@ exit 0`,
|
||||
{
|
||||
path: ["device_path"],
|
||||
reason: "invalid_value",
|
||||
message: expect.stringContaining(reason),
|
||||
message: expect.stringContaining(
|
||||
`${JSON.stringify(devicePath)} does not name`,
|
||||
),
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(
|
||||
(projected.details as { issues: { message: string }[] }).issues[0]!
|
||||
.message,
|
||||
).toContain(reason);
|
||||
// No host directory and no adb invocation are effects of a refused pull.
|
||||
await expect(readFile(join(root, "pull-ran"))).rejects.toThrow();
|
||||
await expect(readFile(output)).rejects.toThrow();
|
||||
await expect(lstat(output)).rejects.toMatchObject({ code: "ENOENT" });
|
||||
},
|
||||
);
|
||||
|
||||
@@ -678,42 +780,74 @@ it("lists processes, features, and services through fixed commands", async () =>
|
||||
]);
|
||||
});
|
||||
|
||||
it("lists a device directory with kinds and symlink targets", async () => {
|
||||
const { binary } = await writeAdbStub(
|
||||
shellStub({
|
||||
ls: "total 4\n-rw-rw---- u0_a1 u0_a1 4096 2026-01-02 11:00 app.db\nlrwxrwxrwx root root 2026-01-01 10:00 lib -> /apex/com.android.art/lib\n",
|
||||
}),
|
||||
);
|
||||
const result = await evidenceResult(providerOver(binary), {
|
||||
operation: "list_adb_directory",
|
||||
input: {
|
||||
serial: "emulator-5554",
|
||||
device_path: "/data/data/com.example.app",
|
||||
},
|
||||
});
|
||||
expect(result.entries).toEqual([
|
||||
{
|
||||
name: "app.db",
|
||||
kind: "file",
|
||||
permissions: "-rw-rw----",
|
||||
owner: "u0_a1",
|
||||
group: "u0_a1",
|
||||
bytes: 4096,
|
||||
date: "2026-01-02 11:00",
|
||||
link_target: null,
|
||||
},
|
||||
{
|
||||
name: "lib",
|
||||
kind: "symlink",
|
||||
permissions: "lrwxrwxrwx",
|
||||
owner: "root",
|
||||
group: "root",
|
||||
bytes: null,
|
||||
date: "2026-01-01 10:00",
|
||||
link_target: "/apex/com.android.art/lib",
|
||||
},
|
||||
]);
|
||||
});
|
||||
it.each(["complete", "truncated"] as const)(
|
||||
"preserves framed directory identities with %s output",
|
||||
async (variant) => {
|
||||
const date = "2026-01-02 11:00:00.000000000 +0000";
|
||||
const names = [
|
||||
"app.db",
|
||||
" app.db ",
|
||||
" ",
|
||||
"one -> two -> three",
|
||||
"line\r",
|
||||
"line\u2028break",
|
||||
"foo\n-rw-rw---- root root 0 2026-01-01 10:00 ghost",
|
||||
"literal\\n",
|
||||
];
|
||||
const links = [
|
||||
["lib", "/apex/com.android.art/lib"],
|
||||
[" lib ", " /apex/lib "],
|
||||
["one -> two\n", "three -> four\n\n"],
|
||||
] as const;
|
||||
const truncated = `-rw-rw----|u0_a1|u0_a1|4096|${date}\0unfinished`;
|
||||
const output = [
|
||||
...names.map(
|
||||
(name) => `-rw-rw----|u0_a1|u0_a1|4096|${date}\0${name}\0\0`,
|
||||
),
|
||||
...links.map(
|
||||
([name, target]) =>
|
||||
`lrwxrwxrwx|root|root|${Buffer.byteLength(target)}|${date}\0${name}\0${target}\0`,
|
||||
),
|
||||
...(variant === "truncated" ? [truncated] : []),
|
||||
].join("");
|
||||
const { binary } = await writeAdbStub(shellStub({ sh: output }));
|
||||
const result = await evidenceResult(providerOver(binary), {
|
||||
operation: "list_adb_directory",
|
||||
input: {
|
||||
serial: "emulator-5554",
|
||||
device_path: "/data/data/com.example.app",
|
||||
},
|
||||
});
|
||||
expect(result.entries).toEqual([
|
||||
...names.map((name) => ({
|
||||
name,
|
||||
kind: "file",
|
||||
permissions: "-rw-rw----",
|
||||
owner: "u0_a1",
|
||||
group: "u0_a1",
|
||||
bytes: 4096,
|
||||
date,
|
||||
link_target: null,
|
||||
})),
|
||||
...links.map(([name, target]) => ({
|
||||
name,
|
||||
kind: "symlink",
|
||||
permissions: "lrwxrwxrwx",
|
||||
owner: "root",
|
||||
group: "root",
|
||||
bytes: Buffer.byteLength(target),
|
||||
date,
|
||||
link_target: target,
|
||||
})),
|
||||
]);
|
||||
expect(result.unparsed_lines).toEqual(
|
||||
variant === "truncated" ? [truncated] : [],
|
||||
);
|
||||
expect(result.coverage).toBe(
|
||||
variant === "truncated" ? "partial" : "complete",
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
it("inspects the display and window focus", async () => {
|
||||
const { binary } = await writeAdbStub(
|
||||
@@ -769,7 +903,7 @@ it("resolves packages by case-insensitive substring", async () => {
|
||||
const { binary } = await writeAdbStub(
|
||||
shellStub({
|
||||
"pm:list":
|
||||
"package:/data/app/x/base.apk=com.whatsapp\\npackage:/data/app/y/base.apk=com.whatsapp.w4b\\npackage:/data/app/z/base.apk=com.example\\n",
|
||||
"package:/data/app/x/base.apk=com.whatsapp\npackage:/data/app/y/base.apk=com.whatsapp.w4b\npackage:/data/app/z/base.apk=com.example\n",
|
||||
}),
|
||||
);
|
||||
const result = await evidenceResult(providerOver(binary), {
|
||||
@@ -977,10 +1111,11 @@ it("quotes caller text before adb joins the remote shell command", async () => {
|
||||
const { binary, root } = await writeAdbStub(
|
||||
(root) => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; exit 0; fi
|
||||
shift 3
|
||||
# Model adb's remote command joining, then execute the resulting shell string.
|
||||
printf '#!/bin/sh\nprintf "%%s" "$@" > "${join(root, "observed")}"\n' > "${join(root, "ls")}"
|
||||
chmod +x "${join(root, "ls")}"
|
||||
PATH="${root}:$PATH" sh -c "$*"
|
||||
# Model adb's remote command joining, then let the device shell parse the
|
||||
# joined string. Its \`sh\` records the arguments it receives.
|
||||
printf '#!/bin/sh\nprintf "%%s\\\\000" "$@" > "${join(root, "observed")}"\n' > "${join(root, "sh")}"
|
||||
chmod +x "${join(root, "sh")}"
|
||||
PATH="${root}:$PATH" /bin/sh -c "$*"
|
||||
`,
|
||||
);
|
||||
const value = `/sdcard/a 'quoted'; touch ${join(root, "injected")}`;
|
||||
@@ -988,7 +1123,8 @@ PATH="${root}:$PATH" sh -c "$*"
|
||||
operation: "list_adb_directory",
|
||||
input: { serial: "device", device_path: value },
|
||||
});
|
||||
expect(await readFile(join(root, "observed"), "utf8")).toContain(value);
|
||||
const observed = (await readFile(join(root, "observed"), "utf8")).split("\0");
|
||||
expect(observed.slice(2, 4)).toEqual(["rea-list-directory", value]);
|
||||
await expect(readFile(join(root, "injected"))).rejects.toThrow();
|
||||
});
|
||||
|
||||
|
||||
+185
-113
@@ -1,7 +1,14 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import { constants as fsConstants } from "node:fs";
|
||||
import { access, lstat, mkdir, rm, stat, writeFile } from "node:fs/promises";
|
||||
import {
|
||||
access,
|
||||
lstat,
|
||||
mkdir,
|
||||
rmdir,
|
||||
rm,
|
||||
stat,
|
||||
writeFile,
|
||||
} from "node:fs/promises";
|
||||
import { join, resolve } from "node:path";
|
||||
|
||||
import type {
|
||||
@@ -30,7 +37,7 @@ import { err, ok, type Result } from "../domain/result.js";
|
||||
import { hashStableFile } from "../filesystem/StableFileHash.js";
|
||||
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
|
||||
import {
|
||||
execFileOutput,
|
||||
execFileBytes,
|
||||
execFileOutputFailure,
|
||||
} from "../process/ExecFileOutput.js";
|
||||
import {
|
||||
@@ -48,12 +55,15 @@ import {
|
||||
parsePmPathOutput,
|
||||
} from "./AdbDeviceOutput.js";
|
||||
import { adbHostFileName } from "./AdbHostFileName.js";
|
||||
import {
|
||||
ADB_DIRECTORY_BATCH_SCRIPT,
|
||||
parseAdbDirectoryListing,
|
||||
} from "./AdbDirectoryListing.js";
|
||||
import {
|
||||
parseAmStartOutput,
|
||||
parseBugreportCopiedPath,
|
||||
parseDumpsysPackageOutput,
|
||||
parseFeatureListOutput,
|
||||
parseLsOutput,
|
||||
parsePackageChangeOutput,
|
||||
parseProcessListOutput,
|
||||
parsePngDimensions,
|
||||
@@ -72,8 +82,6 @@ export const ADB_PROVIDER_IDENTITY = {
|
||||
version: null,
|
||||
} as const;
|
||||
|
||||
const VERSION_TIMEOUT_MS = 10_000;
|
||||
const VERSION_MAX_OUTPUT_BYTES = 64 * 1024;
|
||||
const DEVICES_TIMEOUT_MS = 20_000;
|
||||
const DEVICES_MAX_OUTPUT_BYTES = 1024 * 1024;
|
||||
const SHELL_TIMEOUT_MS = 30_000;
|
||||
@@ -99,7 +107,6 @@ const BUGREPORT_TIMEOUT_MS = 600_000;
|
||||
const BUGREPORT_MAX_OUTPUT_BYTES = 1024 * 1024;
|
||||
const STDERR_EXCERPT_BYTES = 512;
|
||||
const UNPARSED_LINES_REPORTED = 10;
|
||||
const STOP_GRACE_MS = 5_000;
|
||||
|
||||
const hashAdbFile = async (
|
||||
path: string,
|
||||
@@ -138,19 +145,19 @@ interface AdbRunLimits {
|
||||
}
|
||||
|
||||
/** Run one bounded adb invocation and keep its exact failure shape. */
|
||||
const runAdb = async (
|
||||
const runAdbCapture = async (
|
||||
operation: string,
|
||||
targetPath: string,
|
||||
binary: string,
|
||||
arguments_: readonly string[],
|
||||
limits: AdbRunLimits,
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<Result<{ stdout: string; stderr: string }, AnalysisError>> => {
|
||||
): Promise<Result<{ stdout: Buffer; stderr: Buffer }, AnalysisError>> => {
|
||||
if (signal?.aborted === true)
|
||||
return err(new AnalysisCancelledError(operation));
|
||||
try {
|
||||
return ok(
|
||||
await execFileOutput(binary, arguments_, {
|
||||
await execFileBytes(binary, arguments_, {
|
||||
timeout: limits.timeoutMs,
|
||||
maxBuffer: limits.maxOutputBytes,
|
||||
stopSignal: "SIGTERM",
|
||||
@@ -217,61 +224,17 @@ const runAdb = async (
|
||||
}
|
||||
};
|
||||
|
||||
/** Capture one adb invocation's raw binary stdout (screencap PNG). */
|
||||
const runAdbBinary = (
|
||||
operation: string,
|
||||
binary: string,
|
||||
arguments_: readonly string[],
|
||||
limits: AdbRunLimits,
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<Result<Buffer, AnalysisError>> =>
|
||||
new Promise((settle) => {
|
||||
if (signal?.aborted === true)
|
||||
return settle(err(new AnalysisCancelledError(operation)));
|
||||
let stopped: "abort" | "timeout" | undefined;
|
||||
let deadline: NodeJS.Timeout | undefined;
|
||||
let grace: NodeJS.Timeout | undefined;
|
||||
const child = execFile(
|
||||
binary,
|
||||
[...arguments_],
|
||||
{
|
||||
encoding: "buffer" as const,
|
||||
maxBuffer: limits.maxOutputBytes,
|
||||
killSignal: "SIGTERM",
|
||||
timeout: 0,
|
||||
},
|
||||
(error, stdout) => {
|
||||
signal?.removeEventListener("abort", onAbort);
|
||||
clearTimeout(deadline);
|
||||
clearTimeout(grace);
|
||||
if (stopped === "abort")
|
||||
return settle(err(new AnalysisCancelledError(operation)));
|
||||
if (stopped === "timeout")
|
||||
return settle(
|
||||
err(new AnalysisTimeoutError(operation, limits.timeoutMs)),
|
||||
);
|
||||
if (error !== null)
|
||||
return settle(
|
||||
err(
|
||||
new AnalysisProtocolError(
|
||||
`adb failed for ${operation}: ${error.message}`,
|
||||
{ cause: error },
|
||||
),
|
||||
),
|
||||
);
|
||||
settle(ok(stdout as Buffer));
|
||||
},
|
||||
);
|
||||
const stop = (reason: "abort" | "timeout"): void => {
|
||||
if (stopped !== undefined) return;
|
||||
stopped = reason;
|
||||
child.kill("SIGTERM");
|
||||
grace = setTimeout(() => child.kill("SIGKILL"), STOP_GRACE_MS);
|
||||
};
|
||||
const onAbort = (): void => stop("abort");
|
||||
signal?.addEventListener("abort", onAbort, { once: true });
|
||||
deadline = setTimeout(() => stop("timeout"), limits.timeoutMs);
|
||||
/** Decode textual adb protocols after the shared byte capture and error boundary. */
|
||||
const runAdb = async (
|
||||
...arguments_: Parameters<typeof runAdbCapture>
|
||||
): Promise<Result<{ stdout: string; stderr: string }, AnalysisError>> => {
|
||||
const captured = await runAdbCapture(...arguments_);
|
||||
if (!captured.ok) return captured;
|
||||
return ok({
|
||||
stdout: captured.value.stdout.toString("utf8"),
|
||||
stderr: captured.value.stderr.toString("utf8"),
|
||||
});
|
||||
};
|
||||
|
||||
const clientIdentityResult = (
|
||||
selection: AdbBinarySelection,
|
||||
@@ -388,6 +351,68 @@ const removeOwnedPackageDirectory = async (
|
||||
await rm(path, { recursive: true, force: true });
|
||||
};
|
||||
|
||||
const reservePackageDestinations = async (
|
||||
operation: string,
|
||||
packageDirectory: string,
|
||||
entries: readonly { readonly file_name: string }[],
|
||||
capturedOutput: {
|
||||
readonly stdout: string;
|
||||
readonly stderr: string;
|
||||
readonly truncated: boolean;
|
||||
},
|
||||
): Promise<AnalysisOutputError | undefined> => {
|
||||
const reservations: {
|
||||
readonly path: string;
|
||||
readonly identity: PackageDirectoryIdentity;
|
||||
}[] = [];
|
||||
for (const entry of entries) {
|
||||
const path = join(packageDirectory, entry.file_name);
|
||||
try {
|
||||
await mkdir(path);
|
||||
const created = await lstat(path);
|
||||
if (!created.isDirectory() || created.isSymbolicLink())
|
||||
throw new Error(`Could not reserve local filename ${entry.file_name}`);
|
||||
reservations.push({
|
||||
path,
|
||||
identity: {
|
||||
dev: created.dev,
|
||||
ino: created.ino,
|
||||
mode: created.mode,
|
||||
},
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
return new AnalysisOutputError(
|
||||
operation,
|
||||
`Package Manager paths cannot be written to unique local destinations at ${JSON.stringify(entry.file_name)}; refusing before pulling any files`,
|
||||
{ cause, capturedOutput },
|
||||
);
|
||||
}
|
||||
}
|
||||
for (const reservation of reservations) {
|
||||
try {
|
||||
const current = await lstat(reservation.path);
|
||||
if (
|
||||
!current.isDirectory() ||
|
||||
current.isSymbolicLink() ||
|
||||
current.dev !== reservation.identity.dev ||
|
||||
current.ino !== reservation.identity.ino ||
|
||||
current.mode !== reservation.identity.mode
|
||||
)
|
||||
throw new Error(
|
||||
`The local filename reservation changed before release: ${reservation.path}`,
|
||||
);
|
||||
await rmdir(reservation.path);
|
||||
} catch (cause: unknown) {
|
||||
return new AnalysisOutputError(
|
||||
operation,
|
||||
`Could not release a local filename reservation before pulling: ${reservation.path}`,
|
||||
{ cause, capturedOutput },
|
||||
);
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
};
|
||||
|
||||
/**
|
||||
* ADB device inspection and acquisition over a caller-selected adb binary.
|
||||
*
|
||||
@@ -879,12 +904,29 @@ export class AdbProvider {
|
||||
);
|
||||
}
|
||||
case "list_adb_directory": {
|
||||
const ran = await shell(["ls", "-la", request.input.device_path], {
|
||||
timeoutMs: LISTING_TIMEOUT_MS,
|
||||
maxOutputBytes: LISTING_MAX_OUTPUT_BYTES,
|
||||
});
|
||||
const ran = await runAdbCapture(
|
||||
operation,
|
||||
request.input.serial,
|
||||
selection.binary,
|
||||
adbArguments(request.input.serial, [
|
||||
"shell",
|
||||
"sh",
|
||||
"-c",
|
||||
'cd "$1" || exit\nexec find . -mindepth 1 -maxdepth 1 -exec sh -c "$2" rea-list-directory {} +',
|
||||
"rea-list-directory",
|
||||
request.input.device_path,
|
||||
ADB_DIRECTORY_BATCH_SCRIPT,
|
||||
]),
|
||||
{
|
||||
timeoutMs: LISTING_TIMEOUT_MS,
|
||||
maxOutputBytes: LISTING_MAX_OUTPUT_BYTES,
|
||||
},
|
||||
signal,
|
||||
);
|
||||
if (!ran.ok) return ran;
|
||||
const { entries, unparsedLines } = parseLsOutput(ran.value.stdout);
|
||||
const { entries, unparsedLines } = parseAdbDirectoryListing(
|
||||
ran.value.stdout,
|
||||
);
|
||||
return ok(
|
||||
createAnalysisExecution(
|
||||
{
|
||||
@@ -897,7 +939,7 @@ export class AdbProvider {
|
||||
provider,
|
||||
{
|
||||
limitations: [
|
||||
"Entry kinds derive from the permission string's type character",
|
||||
"Entry kinds derive from stat permission mode; directory contents and link targets may change during capture",
|
||||
...unparsedOverflowLimitation(unparsedLines),
|
||||
],
|
||||
},
|
||||
@@ -1156,6 +1198,19 @@ export class AdbProvider {
|
||||
);
|
||||
if (!paths.ok) return paths;
|
||||
const parsed = parsePmPathOutput(paths.value.stdout);
|
||||
const capturedOutput = {
|
||||
stdout: paths.value.stdout,
|
||||
stderr: paths.value.stderr,
|
||||
truncated: false,
|
||||
};
|
||||
if (parsed.rejectedPaths.length > 0)
|
||||
return err(
|
||||
new AnalysisOutputError(
|
||||
operation,
|
||||
`Package Manager reported paths that cannot be represented as host files: ${parsed.rejectedPaths.map((entry) => `${JSON.stringify(entry.device_path)} (${entry.reason})`).join(", ")}`,
|
||||
{ capturedOutput },
|
||||
),
|
||||
);
|
||||
if (parsed.paths.length === 0 && parsed.unparsedLines.length === 0)
|
||||
return err(
|
||||
new AnalysisUnsupportedTargetError(
|
||||
@@ -1168,8 +1223,21 @@ export class AdbProvider {
|
||||
return err(
|
||||
new AnalysisProtocolError(
|
||||
`pm path reported an unrecognized line: ${parsed.unparsedLines[0]}`,
|
||||
{ capturedOutput },
|
||||
),
|
||||
);
|
||||
const destinationNames = new Set<string>();
|
||||
for (const entry of parsed.paths) {
|
||||
if (destinationNames.has(entry.file_name))
|
||||
return err(
|
||||
new AnalysisOutputError(
|
||||
operation,
|
||||
`Package Manager reported multiple paths with the same local destination ${JSON.stringify(entry.file_name)}; refusing before pulling any files`,
|
||||
{ capturedOutput },
|
||||
),
|
||||
);
|
||||
destinationNames.add(entry.file_name);
|
||||
}
|
||||
const directory = await prepareOutputDirectory(
|
||||
operation,
|
||||
input.output_directory,
|
||||
@@ -1228,6 +1296,43 @@ export class AdbProvider {
|
||||
);
|
||||
return err(analysisErrorWithCleanupFailure(primary, cleanup, operation));
|
||||
}
|
||||
const failOwnedPackage = async (
|
||||
primary: AnalysisError,
|
||||
): Promise<Result<AnalysisExecution, AnalysisError>> => {
|
||||
const owner: PendingPackageDirectoryCleanup = {
|
||||
path: packageDirectory,
|
||||
identity: packageDirectoryIdentity,
|
||||
primary,
|
||||
remove: () =>
|
||||
removeOwnedPackageDirectory(
|
||||
packageDirectory,
|
||||
packageDirectoryIdentity,
|
||||
),
|
||||
};
|
||||
try {
|
||||
await owner.remove();
|
||||
} catch (cause: unknown) {
|
||||
this.#pendingPackageDirectoryCleanup.add(owner);
|
||||
const cleanup = new ProviderCleanupError(
|
||||
ADB_PROVIDER_IDENTITY.id,
|
||||
[packageDirectory],
|
||||
{ reason: cause instanceof Error ? cause.message : String(cause) },
|
||||
{ operation, cause },
|
||||
);
|
||||
return err(
|
||||
analysisErrorWithCleanupFailure(primary, cleanup, operation),
|
||||
);
|
||||
}
|
||||
return err(primary);
|
||||
};
|
||||
const destinationError = await reservePackageDestinations(
|
||||
operation,
|
||||
packageDirectory,
|
||||
parsed.paths,
|
||||
capturedOutput,
|
||||
);
|
||||
if (destinationError !== undefined)
|
||||
return failOwnedPackage(destinationError);
|
||||
const artifacts: {
|
||||
device_path: string;
|
||||
file_name: string;
|
||||
@@ -1273,17 +1378,7 @@ export class AdbProvider {
|
||||
});
|
||||
return err(error);
|
||||
};
|
||||
const seenFileNames = new Set<string>();
|
||||
for (const entry of parsed.paths) {
|
||||
if (seenFileNames.has(entry.file_name)) {
|
||||
failures.push({
|
||||
device_path: entry.device_path,
|
||||
stage: "pull",
|
||||
message: `Duplicate file name ${entry.file_name} in the APK set; refusing to overwrite`,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
seenFileNames.add(entry.file_name);
|
||||
const localPath = join(packageDirectory, entry.file_name);
|
||||
const pulled = await runAdb(
|
||||
operation,
|
||||
@@ -1338,33 +1433,7 @@ export class AdbProvider {
|
||||
const primary = new AnalysisProtocolError(
|
||||
`Every APK pull failed for ${packageName} on ${serial}: ${failures[0]!.message}`,
|
||||
);
|
||||
const owner: PendingPackageDirectoryCleanup = {
|
||||
path: packageDirectory,
|
||||
identity: packageDirectoryIdentity,
|
||||
primary,
|
||||
remove: () =>
|
||||
removeOwnedPackageDirectory(
|
||||
packageDirectory,
|
||||
packageDirectoryIdentity,
|
||||
),
|
||||
};
|
||||
try {
|
||||
await owner.remove();
|
||||
} catch (cause: unknown) {
|
||||
this.#pendingPackageDirectoryCleanup.add(owner);
|
||||
const cleanup = new ProviderCleanupError(
|
||||
ADB_PROVIDER_IDENTITY.id,
|
||||
[packageDirectory],
|
||||
{
|
||||
reason: cause instanceof Error ? cause.message : String(cause),
|
||||
},
|
||||
{ operation: operation, cause },
|
||||
);
|
||||
return err(
|
||||
analysisErrorWithCleanupFailure(primary, cleanup, operation),
|
||||
);
|
||||
}
|
||||
return err(primary);
|
||||
return failOwnedPackage(primary);
|
||||
}
|
||||
return ok(
|
||||
createAnalysisExecution(
|
||||
@@ -1407,7 +1476,7 @@ export class AdbProvider {
|
||||
{
|
||||
path: ["device_path"],
|
||||
reason: "invalid_value",
|
||||
message: `The device path does not name a portable host file (${derived.reason}); REA never renames a device-reported file.`,
|
||||
message: `The device path ${JSON.stringify(input.device_path)} does not name a file that can be written on this host (${derived.reason}); REA never renames a device-reported file.`,
|
||||
},
|
||||
]),
|
||||
);
|
||||
@@ -1951,15 +2020,16 @@ export class AdbProvider {
|
||||
const localPath = join(directory.value, "screen.png");
|
||||
const existing = await refuseExistingTarget(operation, localPath);
|
||||
if (existing !== undefined) return err(existing);
|
||||
const captured = await runAdbBinary(
|
||||
const captured = await runAdbCapture(
|
||||
operation,
|
||||
input.serial,
|
||||
selection.binary,
|
||||
adbArguments(input.serial, ["exec-out", "screencap", "-p"]),
|
||||
{ timeoutMs: SCREEN_TIMEOUT_MS, maxOutputBytes: SCREEN_MAX_OUTPUT_BYTES },
|
||||
signal,
|
||||
);
|
||||
if (!captured.ok) return captured;
|
||||
const dimensions = parsePngDimensions(captured.value);
|
||||
const dimensions = parsePngDimensions(captured.value.stdout);
|
||||
if (dimensions.width === null)
|
||||
return err(
|
||||
new AnalysisProtocolError(
|
||||
@@ -1967,7 +2037,7 @@ export class AdbProvider {
|
||||
),
|
||||
);
|
||||
try {
|
||||
await writeFile(localPath, captured.value, { flag: "wx" });
|
||||
await writeFile(localPath, captured.value.stdout, { flag: "wx" });
|
||||
} catch (cause) {
|
||||
return err(
|
||||
new AnalysisOutputError(
|
||||
@@ -1977,14 +2047,16 @@ export class AdbProvider {
|
||||
),
|
||||
);
|
||||
}
|
||||
const sha256 = createHash("sha256").update(captured.value).digest("hex");
|
||||
const sha256 = createHash("sha256")
|
||||
.update(captured.value.stdout)
|
||||
.digest("hex");
|
||||
return ok(
|
||||
createAnalysisExecution(
|
||||
{
|
||||
client,
|
||||
serial: input.serial,
|
||||
local_path: localPath,
|
||||
bytes: captured.value.byteLength,
|
||||
bytes: captured.value.stdout.byteLength,
|
||||
sha256,
|
||||
width: dimensions.width,
|
||||
height: dimensions.height,
|
||||
|
||||
@@ -53,65 +53,6 @@ export const parseProcessListOutput = (
|
||||
return { columns, processes, unparsedLines };
|
||||
};
|
||||
|
||||
export interface AdbDirectoryEntry {
|
||||
readonly name: string;
|
||||
readonly kind: "directory" | "file" | "symlink" | "other";
|
||||
readonly permissions: string | null;
|
||||
readonly owner: string | null;
|
||||
readonly group: string | null;
|
||||
readonly bytes: number | null;
|
||||
readonly date: string | null;
|
||||
readonly link_target: string | null;
|
||||
}
|
||||
|
||||
const LS_LINE =
|
||||
/^([bcdlps-][rwxstST-]{9})\s+(?:(\d+)\s+)?(\S+)\s+(\S+)\s+(?:(\d+)\s+)?([\d-]{10})\s+([\d:]{4,8})\s+(.+)$/u;
|
||||
|
||||
/** Parse `ls -la` output, including `total` headers and symlink targets. */
|
||||
export const parseLsOutput = (
|
||||
stdout: string,
|
||||
): {
|
||||
entries: readonly AdbDirectoryEntry[];
|
||||
unparsedLines: readonly string[];
|
||||
} => {
|
||||
const entries: AdbDirectoryEntry[] = [];
|
||||
const unparsedLines: string[] = [];
|
||||
for (const rawLine of stdout.split(/\r?\n/u)) {
|
||||
const line = rawLine.trimEnd();
|
||||
if (line === "" || line.startsWith("total ")) continue;
|
||||
const match = LS_LINE.exec(line);
|
||||
if (match === null) {
|
||||
if (line.trim() !== "") unparsedLines.push(line);
|
||||
continue;
|
||||
}
|
||||
const permissions = match[1]!;
|
||||
let name = match[8]!.trim();
|
||||
let linkTarget: string | null = null;
|
||||
const linkArrow = name.indexOf(" -> ");
|
||||
if (linkArrow !== -1 && permissions.startsWith("l")) {
|
||||
linkTarget = name.slice(linkArrow + 4);
|
||||
name = name.slice(0, linkArrow);
|
||||
}
|
||||
entries.push({
|
||||
name,
|
||||
kind: permissions.startsWith("d")
|
||||
? "directory"
|
||||
: permissions.startsWith("l")
|
||||
? "symlink"
|
||||
: permissions.startsWith("-")
|
||||
? "file"
|
||||
: "other",
|
||||
permissions,
|
||||
owner: match[3] ?? null,
|
||||
group: match[4] ?? null,
|
||||
bytes: match[5] === undefined ? null : Number(match[5]),
|
||||
date: `${match[6] ?? ""} ${match[7] ?? ""}`.trim() || null,
|
||||
link_target: linkTarget,
|
||||
});
|
||||
}
|
||||
return { entries, unparsedLines };
|
||||
};
|
||||
|
||||
export interface AdbFeatureEntry {
|
||||
readonly name: string;
|
||||
readonly version: number | null;
|
||||
|
||||
@@ -167,16 +167,26 @@ const searchClasses = async (
|
||||
"JADX class pagination is incomplete",
|
||||
);
|
||||
} while (classes.length < total);
|
||||
if (new Set(classes).size !== classes.length || total !== loaded.class_count)
|
||||
if (total !== loaded.class_count)
|
||||
throw new AnalysisOutputError(
|
||||
request.operation,
|
||||
"JADX class inventory is duplicated or differs from the loaded class count",
|
||||
"JADX class inventory count differs from the loaded class count",
|
||||
);
|
||||
const occurrences = new Map<string, number>();
|
||||
for (const name of classes)
|
||||
occurrences.set(name, (occurrences.get(name) ?? 0) + 1);
|
||||
const matchingNames = classes.filter((name) =>
|
||||
name.includes(request.input.query),
|
||||
);
|
||||
const duplicateNames = [...occurrences]
|
||||
.filter(([name, count]) => count > 1 && name.includes(request.input.query))
|
||||
.map(([name, count]) => ({ name, occurrences: count }));
|
||||
return androidResultSchemas.search_android_classes.parse({
|
||||
engine,
|
||||
query: request.input.query,
|
||||
total_classes: total,
|
||||
matches: classes.filter((name) => name.includes(request.input.query)),
|
||||
matches: matchingNames,
|
||||
duplicate_names: duplicateNames,
|
||||
coverage: "complete",
|
||||
});
|
||||
};
|
||||
|
||||
@@ -26,7 +26,6 @@ const REAL_YML = [
|
||||
" ids:",
|
||||
" - 1",
|
||||
" tag: null",
|
||||
"version: 2.7.0-dirty",
|
||||
"versionInfo:",
|
||||
" versionCode: '7'",
|
||||
" versionName: 1.2.3",
|
||||
@@ -44,7 +43,11 @@ describe("parseApktoolYml", () => {
|
||||
});
|
||||
|
||||
it("accepts quoted version values some builds emit", () => {
|
||||
expect(parseApktoolYml("versionName: '2.0.0'\nversionCode: 45\n")).toEqual({
|
||||
expect(
|
||||
parseApktoolYml(
|
||||
'!!brut.androlib.meta.MetaInfo\nversionInfo:\n versionName: "2.0.0"\n versionCode: 45\n',
|
||||
),
|
||||
).toEqual({
|
||||
versionName: "2.0.0",
|
||||
versionCode: "45",
|
||||
minSdkVersion: null,
|
||||
@@ -52,6 +55,49 @@ describe("parseApktoolYml", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("reads only the producer's nested fields and preserves scalar spelling", () => {
|
||||
expect(
|
||||
parseApktoolYml(
|
||||
[
|
||||
"!!brut.androlib.meta.MetaInfo",
|
||||
"versionName: decoy",
|
||||
"versionInfo:",
|
||||
" versionName: 01.2.3",
|
||||
" versionCode: 045",
|
||||
" custom:",
|
||||
" versionName: unrelated",
|
||||
"sdkInfo:",
|
||||
" minSdkVersion: 24",
|
||||
" targetSdkVersion: '35'",
|
||||
"",
|
||||
].join("\n"),
|
||||
),
|
||||
).toEqual({
|
||||
versionName: "01.2.3",
|
||||
versionCode: "045",
|
||||
minSdkVersion: "24",
|
||||
targetSdkVersion: "35",
|
||||
});
|
||||
});
|
||||
|
||||
it("distinguishes YAML nulls from quoted null strings in flow mappings", () => {
|
||||
expect(
|
||||
parseApktoolYml(
|
||||
[
|
||||
"!!brut.androlib.meta.MetaInfo",
|
||||
'versionInfo: { versionName: "null", versionCode: 0045, nested: { versionName: decoy } }',
|
||||
"sdkInfo: { minSdkVersion: null, targetSdkVersion: ~ }",
|
||||
"",
|
||||
].join("\n"),
|
||||
),
|
||||
).toEqual({
|
||||
versionName: "null",
|
||||
versionCode: "0045",
|
||||
minSdkVersion: null,
|
||||
targetSdkVersion: null,
|
||||
});
|
||||
});
|
||||
|
||||
it("reports null facts for unrelated YAML", () => {
|
||||
expect(parseApktoolYml("version: 2.7.0\n")).toEqual({
|
||||
versionName: null,
|
||||
@@ -66,12 +112,26 @@ describe("parseManifestPackage", () => {
|
||||
it("reads the package attribute from a decoded manifest", () => {
|
||||
const manifest =
|
||||
'<?xml version="1.0" encoding="utf-8" standalone="no"?><manifest xmlns:android="http://schemas.android.com/apk/res/android" android:compileSdkVersion="34" package="com.rea.apktool.probe" platformBuildVersionCode="34">';
|
||||
expect(parseManifestPackage(manifest)).toBe("com.rea.apktool.probe");
|
||||
expect(parseManifestPackage(`${manifest}</manifest>`)).toBe(
|
||||
"com.rea.apktool.probe",
|
||||
);
|
||||
});
|
||||
|
||||
it("reads package with reordered or single-quoted XML attributes", () => {
|
||||
expect(
|
||||
parseManifestPackage("<manifest debug='false' package='org.rea.app' />"),
|
||||
).toBe("org.rea.app");
|
||||
});
|
||||
|
||||
it("reports null when no package attribute exists", () => {
|
||||
expect(parseManifestPackage("<manifest>\n</manifest>\n")).toBeNull();
|
||||
});
|
||||
|
||||
it("rejects a well-formed document with the wrong root element", () => {
|
||||
expect(() => parseManifestPackage("<resources/>")).toThrowError(
|
||||
new TypeError("AndroidManifest.xml root must be <manifest>"),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("parseDecodedStrings", () => {
|
||||
@@ -111,4 +171,78 @@ describe("parseDecodedStrings", () => {
|
||||
"a & b <c>",
|
||||
);
|
||||
});
|
||||
|
||||
it("keeps empty, self-closing, and reordered string attributes", () => {
|
||||
expect(
|
||||
parseDecodedStrings(
|
||||
'<resources><string translatable="false" name="empty"/><string name="paired"></string></resources>',
|
||||
),
|
||||
).toEqual({
|
||||
entries: [
|
||||
{ name: "empty", value: "" },
|
||||
{ name: "paired", value: "" },
|
||||
],
|
||||
unparsedCount: 0,
|
||||
});
|
||||
});
|
||||
|
||||
it("ignores comments without mistaking them for nested markup", () => {
|
||||
expect(
|
||||
parseDecodedStrings(
|
||||
'<resources><!-- <string name="fake">bad</string> --><string name="real">before<!-- note -->after</string></resources>',
|
||||
),
|
||||
).toEqual({
|
||||
entries: [{ name: "real", value: "beforeafter" }],
|
||||
unparsedCount: 0,
|
||||
});
|
||||
});
|
||||
|
||||
it("does not include a trailing document comment in the last string", () => {
|
||||
expect(
|
||||
parseDecodedStrings(
|
||||
'<resources><string name="a">one</string></resources><!-- </string> -->',
|
||||
),
|
||||
).toEqual({
|
||||
entries: [{ name: "a", value: "one" }],
|
||||
unparsedCount: 0,
|
||||
});
|
||||
});
|
||||
|
||||
it("preserves source whitespace and entity spelling", () => {
|
||||
expect(
|
||||
parseDecodedStrings(
|
||||
'<resources><string name="raw"> a & b </string></resources>',
|
||||
).entries,
|
||||
).toEqual([{ name: "raw", value: " a & b " }]);
|
||||
});
|
||||
|
||||
it("keeps source slices aligned across CRLF and CR line endings", () => {
|
||||
for (const newline of ["\r\n", "\r"]) {
|
||||
expect(
|
||||
parseDecodedStrings(
|
||||
`<resources>${newline} <string name="raw">x & y</string>${newline}</resources>`,
|
||||
),
|
||||
).toEqual({
|
||||
entries: [{ name: "raw", value: "x & y" }],
|
||||
unparsedCount: 0,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it("counts a string with no name as unsupported instead of emitting an empty name", () => {
|
||||
expect(
|
||||
parseDecodedStrings("<resources><string>value</string></resources>"),
|
||||
).toEqual({ entries: [], unparsedCount: 1 });
|
||||
});
|
||||
|
||||
it("reports malformed XML and unsupported string markup as partial", () => {
|
||||
expect(
|
||||
parseDecodedStrings("<resources><string name='broken'>no</resources>"),
|
||||
).toEqual({ entries: [], unparsedCount: 1 });
|
||||
expect(
|
||||
parseDecodedStrings(
|
||||
'<resources><string name="styled">Hello <b>world</b></string></resources>',
|
||||
),
|
||||
).toEqual({ entries: [], unparsedCount: 1 });
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,11 +1,17 @@
|
||||
/**
|
||||
* Parsers for the decoded workspace `apktool d --no-src` produces.
|
||||
*
|
||||
* Apktool's apktool.yml is a small YAML document; the facts REA projects
|
||||
* are flat enough that line-oriented extraction stays honest without a
|
||||
* YAML dependency, and unrecognized content is ignored rather than guessed.
|
||||
*/
|
||||
import {
|
||||
DOMParser,
|
||||
type Document,
|
||||
type Element,
|
||||
type Node,
|
||||
} from "@xmldom/xmldom";
|
||||
import {
|
||||
isMap,
|
||||
isScalar,
|
||||
parseDocument,
|
||||
type Document as YamlDocument,
|
||||
} from "yaml";
|
||||
|
||||
/** Project only the Apktool metadata fields REA publishes. */
|
||||
export interface ApktoolMetadata {
|
||||
readonly versionName: string | null;
|
||||
readonly versionCode: string | null;
|
||||
@@ -13,43 +19,49 @@ export interface ApktoolMetadata {
|
||||
readonly targetSdkVersion: string | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Project version and SDK facts from apktool.yml. Values keep apktool's
|
||||
* own spelling (version codes can print as `45` or `45.1`).
|
||||
*/
|
||||
export const parseApktoolYml = (yml: string): ApktoolMetadata => {
|
||||
const metadata: {
|
||||
versionName: string | null;
|
||||
versionCode: string | null;
|
||||
minSdkVersion: string | null;
|
||||
targetSdkVersion: string | null;
|
||||
} = {
|
||||
versionName: null,
|
||||
versionCode: null,
|
||||
minSdkVersion: null,
|
||||
targetSdkVersion: null,
|
||||
};
|
||||
for (const line of yml.split(/\r?\n/u)) {
|
||||
const versionName = /^\s*versionName:\s*(?:'([^']*)'|(.*))$/u.exec(line);
|
||||
if (versionName !== null && metadata.versionName === null)
|
||||
metadata.versionName = (versionName[1] ?? versionName[2] ?? "").trim();
|
||||
const versionCode = /^\s*versionCode:\s*(?:'([^']*)'|(.*))$/u.exec(line);
|
||||
if (versionCode !== null && metadata.versionCode === null)
|
||||
metadata.versionCode = (versionCode[1] ?? versionCode[2] ?? "").trim();
|
||||
const minSdk = /minSdkVersion:\s*'?([0-9]+)'?/u.exec(line);
|
||||
if (minSdk !== null && metadata.minSdkVersion === null)
|
||||
metadata.minSdkVersion = minSdk[1]!;
|
||||
const targetSdk = /targetSdkVersion:\s*'?([0-9]+)'?/u.exec(line);
|
||||
if (targetSdk !== null && metadata.targetSdkVersion === null)
|
||||
metadata.targetSdkVersion = targetSdk[1]!;
|
||||
}
|
||||
return metadata;
|
||||
const metaInfoTag = {
|
||||
tag: "tag:yaml.org,2002:brut.androlib.meta.MetaInfo",
|
||||
collection: "map" as const,
|
||||
resolve: (map: unknown) => map,
|
||||
};
|
||||
|
||||
/** Read the package attribute from decoded AndroidManifest.xml text. */
|
||||
/**
|
||||
* Read values from Apktool's tagged metadata document. YAML's core schema
|
||||
* distinguishes null from quoted strings; numeric values use their source
|
||||
* range so version codes retain the spelling Apktool emitted.
|
||||
*/
|
||||
export const parseApktoolYml = (yml: string): ApktoolMetadata => {
|
||||
const document = parseDocument(yml, {
|
||||
schema: "core",
|
||||
customTags: [metaInfoTag],
|
||||
strict: true,
|
||||
uniqueKeys: true,
|
||||
});
|
||||
if (document.errors.length > 0)
|
||||
throw new TypeError(
|
||||
`apktool.yml is not valid Apktool metadata: ${document.errors[0]?.message ?? "YAML parse error"}`,
|
||||
);
|
||||
if (!isMap(document.contents))
|
||||
throw new TypeError("apktool.yml root must be an Apktool metadata mapping");
|
||||
return {
|
||||
versionName: yamlScalar(document, ["versionInfo", "versionName"], yml),
|
||||
versionCode: yamlScalar(document, ["versionInfo", "versionCode"], yml),
|
||||
minSdkVersion: yamlScalar(document, ["sdkInfo", "minSdkVersion"], yml),
|
||||
targetSdkVersion: yamlScalar(
|
||||
document,
|
||||
["sdkInfo", "targetSdkVersion"],
|
||||
yml,
|
||||
),
|
||||
};
|
||||
};
|
||||
|
||||
/** Read the package attribute from the decoded Android manifest root. */
|
||||
export const parseManifestPackage = (manifest: string): string | null => {
|
||||
const match = /<manifest[^>]*\spackage="([^"]+)"/u.exec(manifest);
|
||||
return match === null ? null : match[1]!;
|
||||
const document = parseXml(manifest, "AndroidManifest.xml");
|
||||
const root = document.documentElement;
|
||||
if (root === null || root.nodeName !== "manifest")
|
||||
throw new TypeError("AndroidManifest.xml root must be <manifest>");
|
||||
return root.getAttribute("package") || null;
|
||||
};
|
||||
|
||||
export interface DecodedStringEntry {
|
||||
@@ -58,10 +70,10 @@ export interface DecodedStringEntry {
|
||||
}
|
||||
|
||||
/**
|
||||
* Project `<string name="...">value</string>` entries from one decoded
|
||||
* strings.xml. Values are preserved exactly as apktool printed them —
|
||||
* escaped entities stay escaped — and entries containing nested markup
|
||||
* are counted as unparsed rather than flattened.
|
||||
* Project direct `<string>` children from decoded resources XML. DOM parsing
|
||||
* identifies elements and attributes; text values are sliced from the input
|
||||
* at parser-reported locations so entity spelling and whitespace are kept.
|
||||
* Markup-bearing string values are omitted and counted as partial evidence.
|
||||
*/
|
||||
export const parseDecodedStrings = (
|
||||
stringsXml: string,
|
||||
@@ -69,20 +81,107 @@ export const parseDecodedStrings = (
|
||||
entries: readonly DecodedStringEntry[];
|
||||
unparsedCount: number;
|
||||
} => {
|
||||
let document: Document;
|
||||
try {
|
||||
document = parseXml(stringsXml, "strings.xml");
|
||||
} catch {
|
||||
return { entries: [], unparsedCount: 1 };
|
||||
}
|
||||
const root = document.documentElement;
|
||||
if (root === null || root.nodeName !== "resources")
|
||||
return { entries: [], unparsedCount: 1 };
|
||||
|
||||
const entries: DecodedStringEntry[] = [];
|
||||
let unparsedCount = 0;
|
||||
const pattern = /<string\s+name="([^"]+)"[^>]*>([\s\S]*?)<\/string>/gu;
|
||||
for (
|
||||
let match = pattern.exec(stringsXml);
|
||||
match !== null;
|
||||
match = pattern.exec(stringsXml)
|
||||
) {
|
||||
const value = match[2]!;
|
||||
if (value.includes("<")) {
|
||||
const offsets = sourceLineOffsets(stringsXml);
|
||||
for (const child of childNodes(root)) {
|
||||
if (child.nodeType !== 1) continue;
|
||||
const element = child as Element;
|
||||
if (element.nodeName !== "string") continue;
|
||||
const name = element.getAttribute("name");
|
||||
if (name === null || name.length === 0) {
|
||||
unparsedCount += 1;
|
||||
continue;
|
||||
}
|
||||
entries.push({ name: match[1]!, value: value.trim() });
|
||||
const content = childNodes(element);
|
||||
if (content.some((node) => node.nodeType !== 3 && node.nodeType !== 8)) {
|
||||
unparsedCount += 1;
|
||||
continue;
|
||||
}
|
||||
if (content.every((node) => node.nodeType === 8)) {
|
||||
entries.push({ name, value: "" });
|
||||
continue;
|
||||
}
|
||||
const value = content
|
||||
.filter((node) => node.nodeType === 3)
|
||||
.map((node) => {
|
||||
const start = sourceOffset(node, offsets);
|
||||
if (start === undefined) return null;
|
||||
// A literal '<' cannot appear in well-formed XML text. Ending here
|
||||
// also excludes comments after the document element from this value.
|
||||
const end = stringsXml.indexOf("<", start);
|
||||
if (end < start) return null;
|
||||
return stringsXml.slice(start, end);
|
||||
});
|
||||
if (value.some((part) => part === null)) {
|
||||
unparsedCount += 1;
|
||||
continue;
|
||||
}
|
||||
entries.push({ name, value: value.join("") });
|
||||
}
|
||||
return { entries, unparsedCount };
|
||||
};
|
||||
|
||||
const yamlScalar = (
|
||||
document: YamlDocument,
|
||||
path: readonly string[],
|
||||
source: string,
|
||||
): string | null => {
|
||||
const node = document.getIn(path, true);
|
||||
if (!isScalar(node) || node.value === null) return null;
|
||||
if (typeof node.value === "string") return node.value;
|
||||
const range = node.range;
|
||||
return range === null || range === undefined
|
||||
? String(node.value)
|
||||
: source.slice(range[0], range[1]);
|
||||
};
|
||||
|
||||
const parseXml = (xml: string, label: string) => {
|
||||
const errors: string[] = [];
|
||||
const document = new DOMParser({
|
||||
normalizeLineEndings: (source) => source,
|
||||
onError: (_level, message) => errors.push(message),
|
||||
}).parseFromString(xml, "application/xml");
|
||||
if (errors.length > 0 || document.documentElement === null)
|
||||
throw new TypeError(
|
||||
`${label} is not well-formed XML${errors[0] === undefined ? "" : `: ${errors[0]}`}`,
|
||||
);
|
||||
return document;
|
||||
};
|
||||
|
||||
const childNodes = (node: Node): Node[] => {
|
||||
const children: Node[] = [];
|
||||
for (let child = node.firstChild; child !== null; child = child.nextSibling)
|
||||
children.push(child);
|
||||
return children;
|
||||
};
|
||||
|
||||
const sourceLineOffsets = (source: string): readonly number[] => {
|
||||
const offsets = [0];
|
||||
for (const match of source.matchAll(/\r\n?|\n/gu)) {
|
||||
const index = match.index;
|
||||
offsets.push(index + match[0].length);
|
||||
}
|
||||
return offsets;
|
||||
};
|
||||
|
||||
const sourceOffset = (
|
||||
node: Node,
|
||||
lineOffsets: readonly number[],
|
||||
): number | undefined => {
|
||||
const line = node.lineNumber;
|
||||
const column = node.columnNumber;
|
||||
if (line === undefined || column === undefined) return undefined;
|
||||
const lineStart = lineOffsets[line - 1];
|
||||
return lineStart === undefined ? undefined : lineStart + column - 1;
|
||||
};
|
||||
|
||||
@@ -1,7 +1,14 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { chmod, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import {
|
||||
chmod,
|
||||
mkdtemp,
|
||||
readFile,
|
||||
rm,
|
||||
truncate,
|
||||
writeFile,
|
||||
} from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { expect, onTestFinished, test } from "vitest";
|
||||
import { describe, expect, onTestFinished, test } from "vitest";
|
||||
|
||||
import { ApktoolProvider } from "./ApktoolProvider.js";
|
||||
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
|
||||
@@ -69,7 +76,7 @@ const evidenceResult = async (
|
||||
request: ApktoolRequest,
|
||||
): Promise<Record<string, unknown>> => {
|
||||
const result = await provider.execute(request);
|
||||
if (!result.ok) throw new Error(result.error.message);
|
||||
if (!result.ok) throw result.error;
|
||||
return result.value.result as Record<string, unknown>;
|
||||
};
|
||||
|
||||
@@ -89,19 +96,31 @@ const decodeStub = (
|
||||
readonly withStrings?: boolean;
|
||||
readonly withGerman?: boolean;
|
||||
readonly failDecode?: boolean;
|
||||
readonly manifestFile?: string;
|
||||
readonly stringsFile?: string;
|
||||
readonly stringsXml?: string;
|
||||
} = {},
|
||||
): ((root: string) => string) => {
|
||||
const withStrings = options.withStrings ?? true;
|
||||
const withGerman = options.withGerman ?? true;
|
||||
const stringsXml = options.stringsXml ?? REAL_STRINGS;
|
||||
const manifestWrite =
|
||||
options.manifestFile === undefined
|
||||
? `printf '%s\\n' '${REAL_MANIFEST.replace(/'/gu, "'\\''")}' > "$5/AndroidManifest.xml"`
|
||||
: `cp '${options.manifestFile}' "$5/AndroidManifest.xml"`;
|
||||
const stringsWrite =
|
||||
options.stringsFile === undefined
|
||||
? `printf '%s\\n' '${stringsXml.replace(/'/gu, "'\\''")}' > "$5/res/values/strings.xml"`
|
||||
: `cp '${options.stringsFile}' "$5/res/values/strings.xml"`;
|
||||
return () => `if [ "$1" = "--version" ]; then ${VERSION_BLOCK}; exit 0; fi
|
||||
if [ "$1" = "d" ]; then
|
||||
${
|
||||
options.failDecode === true
|
||||
? `printf 'Exception in thread "main" brut.androlib.AndrolibException: Could not decode the APK\\n' >&2; exit 1;`
|
||||
: `printf '%s\\n' '${REAL_YML.replace(/'/gu, "'\\''")}' > "$5/apktool.yml"
|
||||
printf '%s\\n' '${REAL_MANIFEST.replace(/'/gu, "'\\''")}' > "$5/AndroidManifest.xml"
|
||||
${manifestWrite}
|
||||
mkdir -p "$5/res/values"${withGerman ? ' "$5/res/values-de"' : ""}
|
||||
${withStrings ? `printf '%s\\n' '${REAL_STRINGS.replace(/'/gu, "'\\''")}' > "$5/res/values/strings.xml"` : ""}
|
||||
${withStrings ? stringsWrite : ""}
|
||||
${withGerman ? `printf '%s\\n' '${DE_STRINGS.replace(/'/gu, "'\\''")}' > "$5/res/values-de/strings.xml"` : ""}
|
||||
printf 'I: Using Apktool on stub\\n'
|
||||
exit 0`
|
||||
@@ -173,6 +192,22 @@ it("decodes resources with target identity, metadata, and strings", async () =>
|
||||
expect(result.coverage).toBe("complete");
|
||||
});
|
||||
|
||||
it("preserves empty resource strings and reports markup omissions as partial", async () => {
|
||||
const stringsXml =
|
||||
'<resources><string name="empty"/><string name="raw"> A & B </string><string name="styled">Hello <b>world</b></string></resources>';
|
||||
const { command, root } = await writeApktoolStub(decodeStub({ stringsXml }));
|
||||
const apk = await apkFixture(root);
|
||||
const result = await evidenceResult(providerOver(command), {
|
||||
operation: "decode_android_resources",
|
||||
input: { path: apk.path, include_strings: true },
|
||||
});
|
||||
expect(result.strings).toEqual([
|
||||
{ name: "empty", value: "" },
|
||||
{ name: "raw", value: " A & B " },
|
||||
]);
|
||||
expect(result.coverage).toBe("partial");
|
||||
});
|
||||
|
||||
it("projects one locale's strings on request", async () => {
|
||||
const { command, root } = await writeApktoolStub(decodeStub());
|
||||
const apk = await apkFixture(root);
|
||||
@@ -308,18 +343,16 @@ it("uses an owned APK snapshot and framework cache and resolves regional locales
|
||||
|
||||
it("marks an oversized strings document partial without claiming it is missing", async () => {
|
||||
const { command, root } = await writeApktoolStub((root) =>
|
||||
decodeStub()(root).replace(
|
||||
" printf 'I: Using Apktool on stub",
|
||||
' dd if=/dev/zero of="$5/res/values/strings.xml" bs=1048576 count=9 2>/dev/null\n printf \'I: Using Apktool on stub',
|
||||
),
|
||||
decodeStub({ stringsFile: join(root, "large-strings.xml") })(root),
|
||||
);
|
||||
await writeFile(join(root, "large-strings.xml"), "");
|
||||
await truncate(join(root, "large-strings.xml"), 9 * 1024 * 1024);
|
||||
const apk = await apkFixture(root);
|
||||
const result = await providerOver(command).execute({
|
||||
operation: "decode_android_resources",
|
||||
input: { path: apk.path, include_strings: true },
|
||||
});
|
||||
expect(result.ok).toBe(true);
|
||||
if (!result.ok) return;
|
||||
if (!result.ok) throw result.error;
|
||||
expect(result.value.result).toMatchObject({
|
||||
coverage: "partial",
|
||||
strings: [],
|
||||
@@ -328,6 +361,42 @@ it("marks an oversized strings document partial without claiming it is missing",
|
||||
expect(result.value.limitations.join(" ")).not.toContain("No default");
|
||||
});
|
||||
|
||||
describe("truncated manifest prefixes", () => {
|
||||
it.each([
|
||||
{
|
||||
name: "a retained prefix ending after the root element",
|
||||
manifest: `<manifest package="com.rea.large"></manifest>${" ".repeat(512 * 1024 + 1)}`,
|
||||
},
|
||||
{
|
||||
name: "a retained prefix ending before the root element closes",
|
||||
manifest: `<manifest package="com.rea.large">${" ".repeat(512 * 1024 + 1)}</manifest>`,
|
||||
},
|
||||
])("retains $name without parsing its package", async ({ manifest }) => {
|
||||
const { command, root } = await writeApktoolStub((root) =>
|
||||
decodeStub({ manifestFile: join(root, "large-AndroidManifest.xml") })(
|
||||
root,
|
||||
),
|
||||
);
|
||||
await writeFile(join(root, "large-AndroidManifest.xml"), manifest);
|
||||
const apk = await apkFixture(root);
|
||||
|
||||
const result = await providerOver(command).execute({
|
||||
operation: "decode_android_resources",
|
||||
input: { path: apk.path, include_strings: false },
|
||||
});
|
||||
if (!result.ok) throw result.error;
|
||||
|
||||
expect(result.value.result).toMatchObject({
|
||||
coverage: "partial",
|
||||
manifest: manifest.slice(0, 512 * 1024),
|
||||
metadata: { package_name: null, version_name: "1.2.3" },
|
||||
});
|
||||
expect(result.value.limitations.join(" ")).toContain(
|
||||
"manifest is truncated",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
it("retains failed workspace cleanup and retries it on close", async () => {
|
||||
const { command, root } = await writeApktoolStub(
|
||||
decodeStub({ failDecode: true }),
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import { createWriteStream, constants as fsConstants } from "node:fs";
|
||||
import { createWriteStream } from "node:fs";
|
||||
import {
|
||||
access,
|
||||
chmod,
|
||||
@@ -649,7 +649,9 @@ export class ApktoolProvider {
|
||||
);
|
||||
const manifest = manifestRead.text;
|
||||
const packageName =
|
||||
manifest === null ? null : parseManifestPackage(manifest);
|
||||
manifest === null || manifestRead.truncated
|
||||
? null
|
||||
: parseManifestPackage(manifest);
|
||||
const stringsDirectory =
|
||||
input.locale === undefined
|
||||
? "values"
|
||||
@@ -717,7 +719,7 @@ export class ApktoolProvider {
|
||||
: []),
|
||||
...(parsedStrings.unparsedCount > 0
|
||||
? [
|
||||
`${String(parsedStrings.unparsedCount)} string entries with nested markup were not projected`,
|
||||
`String entries omitted due to malformed XML or unsupported markup: ${String(parsedStrings.unparsedCount)}`,
|
||||
]
|
||||
: []),
|
||||
...(parsedStrings.entries.length > MAX_STRINGS
|
||||
|
||||
@@ -3,12 +3,11 @@ import type {
|
||||
AnalysisOperation,
|
||||
AnalysisOperationPort,
|
||||
} from "./AnalysisProvider.js";
|
||||
import { AnalysisOutputError } from "../domain/analysisErrorCore.js";
|
||||
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import type { JsonValue } from "../domain/jsonValue.js";
|
||||
import { analysisBookmarkSchema } from "../domain/hopperValues.js";
|
||||
import { resolveAnalysisDocument } from "./AnalysisDocument.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { ok, type Result } from "../domain/result.js";
|
||||
|
||||
type Facet =
|
||||
| { readonly state: "available"; readonly value: JsonValue }
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user