Compare commits
14
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
36cbe31e19 | ||
|
|
092de8f08b | ||
|
|
0b7013134c | ||
|
|
e0bc7fd973 | ||
|
|
161377a505 | ||
|
|
732d606609 | ||
|
|
c9bd91d7c3 | ||
|
|
e698c977ed | ||
|
|
31a3cfa7e9 | ||
|
|
829d9a0933 | ||
|
|
9da179425e | ||
|
|
b906defbff | ||
|
|
6e6cce1bf3 | ||
|
|
69761e95b6 |
@@ -82,7 +82,7 @@ jobs:
|
||||
|
||||
test-shard:
|
||||
name: Test shard (${{ matrix.shard }})
|
||||
needs: changes
|
||||
needs: [changes, build]
|
||||
if: github.event_name == 'push' || needs.changes.outputs.code == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
@@ -96,7 +96,11 @@ jobs:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- run: npm run test:ci:shard -- --shard=${{ matrix.shard }}
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: rea-dist
|
||||
path: dist
|
||||
- run: npm run test:ci:shard:run -- --shard=${{ matrix.shard }}
|
||||
- uses: actions/upload-artifact@v4
|
||||
if: ${{ !cancelled() }}
|
||||
with:
|
||||
@@ -107,6 +111,26 @@ jobs:
|
||||
overwrite: true
|
||||
retention-days: 1
|
||||
|
||||
build:
|
||||
name: Build package
|
||||
needs: changes
|
||||
if: github.event_name == 'push' || needs.changes.outputs.code == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- run: npm run build:cached
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: rea-dist
|
||||
path: dist
|
||||
if-no-files-found: error
|
||||
retention-days: 1
|
||||
|
||||
test:
|
||||
name: Test (coverage + timing)
|
||||
needs: [changes, test-shard]
|
||||
|
||||
@@ -37,6 +37,7 @@ coverage/
|
||||
.vitest-reports/
|
||||
test-results.xml
|
||||
.turbo/
|
||||
.cache/
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
{
|
||||
".": "2.3.0"
|
||||
".": "2.4.1"
|
||||
}
|
||||
|
||||
@@ -41,7 +41,7 @@ See [docs/architecture.mermaid](docs/architecture.mermaid) for a visual architec
|
||||
- `tests/`: Vitest suite. `tests/fixtures/` holds reusable provider-process fixtures, source-owned synthetic JavaScript graph and Electron/Webpack/Rspack artifact trees, and the fake launcher, Hopper bridge, and CDP seams.
|
||||
- `scripts/verify-real-hopper.mjs`: real-Hopper end-to-end verifier.
|
||||
- `scripts/verify-real-ghidra.mjs`: real Linux Ghidra verifier for x86-64 debug/stripped ELF, AArch64 ELF, PE, and Mach-O function semantics plus complete cleanup.
|
||||
- `scripts/verify-real-ghidra-windows.mjs`: real Windows x64 Ghidra P0 verifier for the source-owned native PE fixture, all 18 admitted operations, digest linkage, loopback transport, and cleanup.
|
||||
- `scripts/verify-real-ghidra-windows.mjs`: real Windows x64 Ghidra P0 verifier for the source-owned native PE fixture, all 19 admitted operations, digest linkage, loopback transport, and cleanup.
|
||||
- `scripts/verify-real-browser.mjs`: real Chrome end-to-end verifier for the passive CDP provider.
|
||||
- `scripts/print-mcp-config.mjs`: prints an MCP server config with absolute paths filled in (`npm run config:print -- /path/to/binary`).
|
||||
|
||||
@@ -54,9 +54,10 @@ See [docs/architecture.mermaid](docs/architecture.mermaid) for a visual architec
|
||||
- `npm run build`: compile `src/` into `dist/`.
|
||||
- `npm run build:cached`: compile through Turbo and reuse a valid local build cache entry.
|
||||
- `npm test`: restore or build `dist/`, then run the Vitest suite once without coverage or retries locally.
|
||||
- `npm run test:fast`: run the parallel test project without the serial integration group.
|
||||
- `npm run test:fast`: run pure and subprocess test projects without the serial integration group.
|
||||
- `npm run test:changed`: run changed and related non-serial tests once.
|
||||
- `npm run test:integration`: run the serial filesystem, process, and CLI integration group.
|
||||
- `npm run test:watch`: watch tests affected by working-tree changes in the parallel project, without coverage.
|
||||
- `npm run test:watch`: watch changed and related non-serial tests, without coverage.
|
||||
- `npm run test:coverage`: run the complete suite with coverage; CI shards the suite and owns JUnit reporting.
|
||||
- `npm run typecheck`: run strict TypeScript checks without emitting files.
|
||||
- `npm run lint`: apply oxlint rules (complexity, max-lines, unused vars, and TypeScript-specific checks).
|
||||
|
||||
@@ -1,5 +1,31 @@
|
||||
# Changelog
|
||||
|
||||
## [2.4.1](https://github.com/morluto/rea/compare/rea-agents-2.4.0...rea-agents-2.4.1) (2026-07-23)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **hopper:** add bounded analysis recovery ([e0bc7fd](https://github.com/morluto/rea/commit/e0bc7fd973b26448fb2192a2059ab1229bec0817))
|
||||
|
||||
|
||||
### Tests
|
||||
|
||||
* **package:** stabilize fake Hopper ownership ([0b70131](https://github.com/morluto/rea/commit/0b7013134c58085b72177f2954e55534be053ebe))
|
||||
|
||||
## [2.4.0](https://github.com/morluto/rea/compare/rea-agents-2.3.0...rea-agents-2.4.0) (2026-07-23)
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* **artifacts:** support MSIX and AppX packages ([6e6cce1](https://github.com/morluto/rea/commit/6e6cce1bf3cbd9c48799bee9969c53d66c42162e))
|
||||
* expand reactive capture and package analysis ([9da1794](https://github.com/morluto/rea/commit/9da179425e83df9f255597437320c1ab6ff5c5c2))
|
||||
* **process:** add reactive capture scenarios ([69761e9](https://github.com/morluto/rea/commit/69761e95b63ec7617aa161f7a1b4e3c8847a6175))
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **capabilities:** expose typed availability codes ([b906def](https://github.com/morluto/rea/commit/b906defbff28885a1d39f78da7c2b7170a7108c6))
|
||||
|
||||
## [2.3.0](https://github.com/morluto/rea/compare/rea-agents-2.2.0...rea-agents-2.3.0) (2026-07-23)
|
||||
|
||||
|
||||
|
||||
+5
-3
@@ -39,9 +39,11 @@ prerequisite uses Turbo. CI uploads the rendered TypeDoc site as an `api-docs`
|
||||
artifact; the generated HTML is not committed.
|
||||
|
||||
Local `npm test` runs without coverage, retries, or verbose output. Use `npm run
|
||||
test:fast` for the parallel group, `npm run test:integration` for serial
|
||||
filesystem/process/CLI cases, and `npm run test:watch` to watch tests affected by
|
||||
working-tree changes. CI splits the complete suite across two native Vitest
|
||||
test:fast` for pure and subprocess groups, `npm run test:integration` for serial
|
||||
filesystem/process/CLI cases. `npm run test:changed` runs changed and related
|
||||
non-serial tests once; `npm run test:watch` keeps that selection live. Pure tests
|
||||
use Vitest threads, while subprocess tests stay in isolated forks. CI splits the
|
||||
complete suite across two native Vitest
|
||||
shards, then merges their coverage and JUnit reports. `npm run test:ci` runs the
|
||||
equivalent unsharded gate locally. Coverage thresholds remain in
|
||||
`vitest.config.ts`.
|
||||
|
||||
@@ -237,13 +237,13 @@ On Windows, set the same variables in PowerShell and run `rea doctor --json`; au
|
||||
|
||||
REA loads its packaged Java `HeadlessScript` with `-scriptPath`, copies and digest-verifies the target in an ephemeral runtime, enables `-readOnly` and `-deleteProject`, caps auto-analysis at 300 seconds with two CPUs and a 2 GiB Java heap, and authenticates every request. Linux uses a mode-0600 Unix socket. Windows P0 uses token-authenticated IPv4 loopback and a token-free endpoint record because Node path-based IPC does not connect to Java AF_UNIX sockets on Windows. The bridge verifies Ghidra's imported-byte SHA-256 before serving any operation.
|
||||
|
||||
The Ghidra adapter declares 18 direct and enhanced operations. Its ten inventory operations are `list_documents`, `list_procedures`, `list_strings`, `list_names`, `list_segments`, `address_name`, `procedure_address`, `resolve_containing_procedure`, `search_procedures`, and `search_strings`. It also admits `procedure_info`, `procedure_pseudo_code`, `procedure_assembly`, `procedure_callers`, `procedure_callees`, `procedure_references`, `xrefs`, and `analyze_function`. These capabilities enable the shared Swift/Objective-C inventory workflows, `binary_overview`, `batch_decompile`, `get_call_graph`, `find_xrefs_to_name`, `trace_feature`, and complete function dossiers. Default-space addresses are lowercase `0x` hexadecimal. Other spaces, including `EXTERNAL`, use `<percent-encoded-space>:0x<hex>`. Symbol results identify primary, dynamic, external, type, and source facts; procedures distinguish external functions and thunks; strings identify charset, missing-terminator state, byte length, and value truncation; memory-block ends are exclusive and permissions come directly from Ghidra.
|
||||
The Ghidra adapter declares 19 direct and enhanced operations. Its ten inventory operations are `list_documents`, `list_procedures`, `list_strings`, `list_names`, `list_segments`, `address_name`, `procedure_address`, `resolve_containing_procedure`, `search_procedures`, and `search_strings`. It also admits `procedure_info`, `procedure_pseudo_code`, `procedure_assembly`, `read_function_instructions`, `procedure_callers`, `procedure_callees`, `procedure_references`, `xrefs`, and `analyze_function`. `read_function_instructions` is the offset-paginated fast path for raw instruction windows: it does not invoke the decompiler or whole-program name/string inventories, and is also exposed as `rea instructions`. These capabilities enable the shared Swift/Objective-C inventory workflows, `binary_overview`, `batch_decompile`, `get_call_graph`, `find_xrefs_to_name`, `trace_feature`, and complete function dossiers. Default-space addresses are lowercase `0x` hexadecimal. Other spaces, including `EXTERNAL`, use `<percent-encoded-space>:0x<hex>`. Symbol results identify primary, dynamic, external, type, and source facts; procedures distinguish external functions and thunks; strings identify charset, missing-terminator state, byte length, and value truncation; memory-block ends are exclusive and permissions come directly from Ghidra.
|
||||
|
||||
The bridge serves operations only after auto-analysis completes. Each Program owns one persistent `DecompInterface`; a bounded 32-request FIFO serializes Ghidra API access, and every decompile has a 30-second native deadline. Reference results preserve Ghidra's call/jump/data/read/write/indirect/computed/external facts, while unresolved targetless flows remain explicitly unknown. Synthetic entry-point references without actionable memory sources are omitted. Pseudocode and assembly are provider-specific observations, not original source or Hopper-equivalent text. An analysis timeout, scan or inventory safety limit, request timeout, or oversized response fails explicitly instead of returning a partial result labeled complete.
|
||||
|
||||
`npm run verify:ghidra` compiles source-owned x86-64 debug and stripped ELF, AArch64 ELF, x86-64 PE, and x86-64 Mach-O fixtures. Against real Ghidra 12.1.2 it validates every admitted operation, direct and indirect calls, imports/exports/thunks, typed references, strings/xrefs, multi-block CFG, cancellation, deadlines, concurrency, malformed inputs, and complete process/project cleanup. Set `REA_CC`, `REA_CLANG`, or `REA_LLD_LINK` only when the corresponding compiler command is not on `PATH`.
|
||||
|
||||
`npm run verify:ghidra:windows` uses a deterministic source-owned native x86-64 PE application and requires all 18 operations, target/snapshot/import digest linkage, authenticated loopback transport, and cleanup on a controlled Windows x64 Ghidra 12.1.2 runner. This proof does not establish Job Object ownership, private DACLs, or reparse-point-safe authority.
|
||||
`npm run verify:ghidra:windows` uses a deterministic source-owned native x86-64 PE application and requires all 19 operations, target/snapshot/import digest linkage, authenticated loopback transport, and cleanup on a controlled Windows x64 Ghidra 12.1.2 runner. This proof does not establish Job Object ownership, private DACLs, or reparse-point-safe authority.
|
||||
|
||||
To remove only REA-owned MCP registrations and the managed skill:
|
||||
|
||||
@@ -373,10 +373,10 @@ REA handles the app analysis in steps 1–5. The agent performs step 6 with its
|
||||
|
||||
| Tool family | Count | Examples |
|
||||
| ------------------------- | ----: | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Native inspection | 33 | procedures, pseudocode, assembly, strings, names, segments, callers, callees, xrefs, annotations |
|
||||
| Native inspection | 34 | procedures, pseudocode, assembly, strings, names, segments, callers, callees, xrefs, annotations |
|
||||
| Investigation workflows | 10 | `binary_overview`, `analyze_function`, `batch_decompile`, `trace_feature`, call graphs, Swift and Objective-C discovery |
|
||||
| Native macOS utilities | 5 | Mach-O metadata, code signatures, plists, architectures, Swift demangling; Hopper-free and provenance-bearing |
|
||||
| Artifact graph | 2 | deterministic directory, ZIP/APK/IPA, and ASAR inventory; explicitly selected extraction into an absent owned tree |
|
||||
| Artifact graph | 2 | deterministic directory, ZIP/APK/IPA/MSIX/AppX, and ASAR inventory; explicitly selected extraction into an absent owned tree |
|
||||
| Managed PE/CLI | 8 | PE/CLI identity, metadata members, CIL hashes, P/Invoke/native-boundary declarations and verification, application-graph projection, decompiler reconstruction import, token remapping, runtime-correlation plans, and version comparison |
|
||||
| Browser observation | 8 | exact-origin CDP capture, bundle and source-map analysis, WebMCP discovery, session timelines, capture diff, and visual evidence |
|
||||
| Electron analysis | 4 | passive root-confined observation, bounded static application mapping, and evidence-backed static/runtime reconciliation |
|
||||
@@ -479,7 +479,7 @@ Setup detects Claude Code, Claude Desktop, Codex, Cursor, Gemini CLI, Windsurf,
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents@2.3.0", "mcp"]
|
||||
"args": ["-y", "rea-agents@2.4.1", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -592,10 +592,17 @@ and retains one provider-attributed, token-verified observation per started
|
||||
provider. Each observation carries `observed_at` and remains `unavailable` or
|
||||
`verified`; a verified empty descendant list is distinct from both. Snapshots
|
||||
describe bounded observations, not current live state or historical absence.
|
||||
For providers with serial work, `analysis_activity` distinguishes `idle`,
|
||||
`busy`, and `timed_out_busy`, and reports the active operation, elapsed time,
|
||||
caller state, timeout, and queued-request count. A caller timeout therefore
|
||||
does not falsely imply that Hopper's Python thread is available. `close_binary`
|
||||
clears the REA session but returns `cleanup_incomplete` when authenticated
|
||||
document shutdown, owned process cleanup, or private runtime removal cannot be
|
||||
verified.
|
||||
Reopening same target without a selector keeps its binding; runtime failure
|
||||
never selects another provider silently.
|
||||
Ghidra can appear as an available, target-compatible candidate after doctor
|
||||
validates its exact installation. Its capability list contains the 18 admitted
|
||||
validates its exact installation. Its capability list contains the 19 admitted
|
||||
read-only inventory and function-analysis operations; selecting it still does
|
||||
not make Hopper-only GUI or mutation operations available and never triggers a
|
||||
silent fallback.
|
||||
@@ -627,7 +634,11 @@ REA starts Hopper when needed; Hopper does not need to be running first. Hopper'
|
||||
|
||||
REA derives explicit format and architecture arguments to prevent common FAT and ARM selection dialogs. Other Hopper or macOS dialogs may still require a person. REA reports timeouts and remediation through CLI or MCP results instead of attempting to answer UI prompts.
|
||||
|
||||
Closing a REA session shuts down its bridge and removes its private socket directory. It does not quit a Hopper application the user may be using.
|
||||
Hopper bridge calls pass through a bounded serial FIFO because Hopper's Python API runs on one dedicated thread. Queue wait counts against the caller deadline; a timeout settles the caller but does not release the wire slot until Hopper actually replies. `binary_session.analysis_activity` exposes that detached work and MCP progress reports elapsed time. Successful decompilation text is cached per document and procedure, shared by pseudocode and dossier requests, and invalidated after rename or comment mutation.
|
||||
|
||||
Hopper's synchronous public Python calls cannot stream a partial response after the caller deadline. Use `read_function_instructions` or `rea instructions` when raw instruction orientation is enough: this path does not decompile or scan whole-program name/string inventories. If a dossier has already timed out, inspect `analysis_activity`; do not assume a raw request can enter the serial bridge until that detached Hopper call actually returns.
|
||||
|
||||
Closing a REA session shuts down its bridge and removes its private socket directory. It does not quit a Hopper application the user may be using. If shutdown or cleanup cannot be verified, `close_binary` returns `cleanup_incomplete` with the affected local resources.
|
||||
|
||||
## Advanced process-capture setup
|
||||
|
||||
|
||||
+2
-2
@@ -161,7 +161,7 @@ rea uninstall --purge-data
|
||||
|
||||
| عائلة الأدوات | العدد | أمثلة |
|
||||
| --------------------- | ----: | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| فحص الملفات التنفيذية | 33 | الدوال، والشيفرة شبه المصدرية، والتعليمات، والسلاسل، والأسماء، والمراجع، والتعليقات |
|
||||
| فحص الملفات التنفيذية | 34 | الدوال، والشيفرة شبه المصدرية، والتعليمات، والسلاسل، والأسماء، والمراجع، والتعليقات |
|
||||
| التحليل المركب | 10 | النظرة العامة، وفك الترجمة الدفعي، ومخططات الاستدعاء، والمراجع، واكتشاف الأنواع |
|
||||
| أدوات macOS الأصلية | 5 | بيانات Mach-O والتوقيعات وملفات plist والمعماريات وفك رموز Swift دون تشغيل Hopper |
|
||||
| رسم القطع الأثرية | 2 | جرد حتمي للدلائل وZIP/APK/IPA وASAR واستخراج معاملاتي محدد صراحة |
|
||||
@@ -182,7 +182,7 @@ rea uninstall --purge-data
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents@2.3.0", "mcp"]
|
||||
"args": ["-y", "rea-agents@2.4.1", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+2
-2
@@ -151,7 +151,7 @@ REA は手順 1〜5 のバイナリ解析を処理し、手順 6 はエージェ
|
||||
|
||||
| ツール群 | 数 | 例 |
|
||||
| ---------------------- | --: | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| バイナリ調査 | 33 | プロシージャ、疑似コード、アセンブリ、文字列、名前、セグメント、callers、callees、xrefs、注釈 |
|
||||
| バイナリ調査 | 34 | プロシージャ、疑似コード、アセンブリ、文字列、名前、セグメント、callers、callees、xrefs、注釈 |
|
||||
| 合成解析 | 10 | `binary_overview`, `analyze_function`, `batch_decompile`, `get_call_graph`, `find_xrefs_to_name`, Swift / ObjC 検出 |
|
||||
| macOS ネイティブ | 5 | Mach-O メタデータ、署名、plist、アーキテクチャ、Swift デマングル。Hopper 起動不要 |
|
||||
| アーティファクトグラフ | 2 | ディレクトリ、ZIP/APK/IPA、ASAR の決定的一覧と明示選択されたトランザクション抽出 |
|
||||
@@ -172,7 +172,7 @@ Setup は Claude Code、Claude Desktop、Codex、Cursor、Gemini CLI、Windsurf
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents@2.3.0", "mcp"]
|
||||
"args": ["-y", "rea-agents@2.4.1", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+2
-2
@@ -151,7 +151,7 @@ REA는 1–5단계의 바이너리 분석을 처리합니다. 6단계는 에이
|
||||
|
||||
| 도구 그룹 | 수 | 예시 |
|
||||
| ------------------- | --: | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 바이너리 검사 | 33 | 프로시저, 의사 코드, 어셈블리, 문자열, 이름, 세그먼트, callers, callees, xrefs, 주석 |
|
||||
| 바이너리 검사 | 34 | 프로시저, 의사 코드, 어셈블리, 문자열, 이름, 세그먼트, callers, callees, xrefs, 주석 |
|
||||
| 합성 분석 | 10 | `binary_overview`, `analyze_function`, `batch_decompile`, `get_call_graph`, `find_xrefs_to_name`, Swift 및 ObjC 탐색 |
|
||||
| macOS 네이티브 도구 | 5 | Mach-O 메타데이터, 코드 서명, plist, 아키텍처, Swift 디맹글링. Hopper 실행 불필요 |
|
||||
| 아티팩트 그래프 | 2 | 디렉터리, ZIP/APK/IPA, ASAR 결정적 인벤토리와 명시적으로 선택한 트랜잭션 추출 |
|
||||
@@ -172,7 +172,7 @@ Setup은 Claude Code, Claude Desktop, Codex, Cursor, Gemini CLI, Windsurf, Devin
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents@2.3.0", "mcp"]
|
||||
"args": ["-y", "rea-agents@2.4.1", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+2
-2
@@ -151,7 +151,7 @@ REA 负责第 1–5 步中的二进制分析。第 6 步由智能体使用其常
|
||||
|
||||
| 工具类别 | 数量 | 示例 |
|
||||
| -------------- | ---: | --------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 二进制检查 | 33 | 过程、伪代码、汇编、字符串、名称、段、调用者、被调用者、交叉引用、注释 |
|
||||
| 二进制检查 | 34 | 过程、伪代码、汇编、字符串、名称、段、调用者、被调用者、交叉引用、注释 |
|
||||
| 组合分析 | 10 | `binary_overview`, `analyze_function`, `batch_decompile`, `get_call_graph`, `find_xrefs_to_name`、Swift 与 ObjC 发现 |
|
||||
| macOS 原生工具 | 5 | Mach-O 元数据、代码签名、plist、架构与 Swift 符号还原,无需启动 Hopper |
|
||||
| 制品图谱 | 2 | 确定性目录、ZIP/APK/IPA 与 ASAR 清单,以及显式选择的事务式提取 |
|
||||
@@ -172,7 +172,7 @@ Setup 会检测 Claude Code、Claude Desktop、Codex、Cursor、Gemini CLI、Win
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents@2.3.0", "mcp"]
|
||||
"args": ["-y", "rea-agents@2.4.1", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -72,7 +72,7 @@ import ghidra.program.model.symbol.Symbol;
|
||||
import ghidra.program.model.symbol.SymbolIterator;
|
||||
|
||||
public final class ReaGhidraBridge extends HeadlessScript {
|
||||
private static final int BRIDGE_VERSION = 4;
|
||||
private static final int BRIDGE_VERSION = 5;
|
||||
private static final int MAX_DESCRIPTOR_BYTES = 16 * 1024;
|
||||
private static final int MAX_REQUEST_CHARACTERS = 256 * 1024;
|
||||
private static final int MAX_RESPONSE_BYTES = 1024 * 1024;
|
||||
@@ -123,6 +123,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
"procedure_callers",
|
||||
"procedure_info",
|
||||
"procedure_pseudo_code",
|
||||
"read_function_instructions",
|
||||
"procedure_references",
|
||||
"xrefs"
|
||||
};
|
||||
@@ -311,6 +312,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
case "procedure_callers" -> procedureCalls(request.params, true);
|
||||
case "procedure_info" -> procedureInfo(request.params);
|
||||
case "procedure_pseudo_code" -> procedurePseudocode(request.params);
|
||||
case "read_function_instructions" -> readFunctionInstructions(request.params);
|
||||
case "procedure_references" -> procedureReferences(request.params);
|
||||
case "resolve_containing_procedure" -> containingProcedure(request.params);
|
||||
case "search_procedures" -> search(request.params, true);
|
||||
@@ -390,6 +392,51 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return GSON.toJsonTree(renderAssembly(scan.instructions));
|
||||
}
|
||||
|
||||
private JsonObject readFunctionInstructions(JsonObject params) throws Exception {
|
||||
requireKeys(params, Set.of("document", "procedure", "offset", "limit"));
|
||||
requireDocument(params);
|
||||
Function function = resolveProcedure(requireString(params, "procedure"));
|
||||
int offset = requireBoundedInteger(params, "offset", 0, 100_000);
|
||||
int limit = requireBoundedInteger(params, "limit", 1, 500);
|
||||
InstructionScan scan = scanInstructions(function, offset + limit + 1);
|
||||
int start = Math.min(offset, scan.instructions.size());
|
||||
int end = Math.min(scan.instructions.size(), start + limit);
|
||||
JsonArray items = new JsonArray();
|
||||
for (String line : renderAssemblyLines(scan.instructions.subList(start, end))) {
|
||||
items.add(line);
|
||||
}
|
||||
boolean hasMore = scan.truncated || end < scan.instructions.size();
|
||||
JsonObject instructions = new JsonObject();
|
||||
instructions.add("items", items);
|
||||
if (scan.truncated) {
|
||||
instructions.add("total", JsonNull.INSTANCE);
|
||||
}
|
||||
else {
|
||||
instructions.addProperty("total", scan.instructions.size());
|
||||
}
|
||||
instructions.addProperty("returned", items.size());
|
||||
instructions.addProperty("truncated", hasMore);
|
||||
if (hasMore) {
|
||||
instructions.addProperty("next_offset", end);
|
||||
}
|
||||
else {
|
||||
instructions.add("next_offset", JsonNull.INSTANCE);
|
||||
}
|
||||
|
||||
JsonArray limitations = new JsonArray();
|
||||
limitations.add("Instruction text and ordering are Ghidra-specific representations.");
|
||||
limitations.add(
|
||||
"The fast path does not invoke the decompiler or scan whole-program names and strings."
|
||||
);
|
||||
JsonObject result = new JsonObject();
|
||||
result.add("procedure", procedureIdentity(function));
|
||||
result.add("instructions", instructions);
|
||||
result.addProperty("instructions_scanned", scan.instructions.size());
|
||||
result.addProperty("instruction_scan_truncated", scan.truncated);
|
||||
result.add("limitations", limitations);
|
||||
return result;
|
||||
}
|
||||
|
||||
private JsonArray procedureCalls(JsonObject params, boolean callers) throws Exception {
|
||||
requireKeys(params, Set.of("document", "procedure"));
|
||||
requireDocument(params);
|
||||
|
||||
+117
-20
@@ -16,6 +16,7 @@ MAX_LINE_BYTES = 10 * 1024 * 1024
|
||||
BAD_ADDRESSES = (-1, 0xFFFFFFFFFFFFFFFF, None)
|
||||
_selected_document = None
|
||||
_search_inventory_cache = {}
|
||||
_pseudocode_cache = {}
|
||||
MAX_SEARCH_PATTERN_LENGTH = 256
|
||||
MAX_SEARCH_VALUE_LENGTH = 4096
|
||||
|
||||
@@ -225,12 +226,35 @@ def _invalidate_search_inventory(document):
|
||||
del _search_inventory_cache[key]
|
||||
|
||||
|
||||
def _invalidate_pseudocode(document):
|
||||
"""Discard cached decompilation after an analysis annotation changes."""
|
||||
document_id = id(document)
|
||||
for key in list(_pseudocode_cache):
|
||||
if key[0] == document_id:
|
||||
del _pseudocode_cache[key]
|
||||
|
||||
|
||||
def _pseudocode(document, procedure):
|
||||
"""Reuse one provider decompilation for repeated bounded projections."""
|
||||
key = (id(document), procedure.getEntryPoint())
|
||||
if key not in _pseudocode_cache:
|
||||
_pseudocode_cache[key] = procedure.decompile()
|
||||
return _pseudocode_cache[key]
|
||||
|
||||
|
||||
def _search_inventory(document, kind):
|
||||
"""Cache an immutable, address-sorted inventory for an unchanged document."""
|
||||
key = (id(document), kind)
|
||||
inventory = _search_inventory_cache.get(key)
|
||||
if inventory is None:
|
||||
values = _procedure_map(document) if kind == "procedure" else _strings(document)
|
||||
if kind == "procedure":
|
||||
values = _procedure_map(document)
|
||||
elif kind == "string":
|
||||
values = _strings(document)
|
||||
elif kind == "name":
|
||||
values = {_hex(address): value for address, value in _name_map(document).items()}
|
||||
else:
|
||||
raise ValueError("Unknown inventory kind")
|
||||
inventory = tuple(sorted(values.items(), key=lambda item: int(item[0], 16)))
|
||||
_search_inventory_cache[key] = inventory
|
||||
return inventory
|
||||
@@ -507,6 +531,22 @@ def _name_map(document):
|
||||
return result
|
||||
|
||||
|
||||
def _render_instruction(segment, address):
|
||||
instruction = segment.getInstructionAtAddress(address)
|
||||
if instruction is None:
|
||||
return None
|
||||
arguments = [
|
||||
instruction.getFormattedArgument(index)
|
||||
for index in range(instruction.getArgumentCount())
|
||||
]
|
||||
suffix = ", ".join(value for value in arguments if value is not None)
|
||||
return "%s: %s%s" % (
|
||||
_hex(address),
|
||||
instruction.getInstructionString(),
|
||||
(" " + suffix) if suffix else "",
|
||||
)
|
||||
|
||||
|
||||
def _assembly(procedure, limit=None):
|
||||
"""Render bounded assembly while guarding against malformed instruction cycles."""
|
||||
lines = []
|
||||
@@ -520,9 +560,9 @@ def _assembly(procedure, limit=None):
|
||||
instruction = segment.getInstructionAtAddress(address)
|
||||
if instruction is None:
|
||||
break
|
||||
arguments = [instruction.getFormattedArgument(index) for index in range(instruction.getArgumentCount())]
|
||||
suffix = ", ".join(value for value in arguments if value is not None)
|
||||
lines.append("%s: %s%s" % (_hex(address), instruction.getInstructionString(), (" " + suffix) if suffix else ""))
|
||||
line = _render_instruction(segment, address)
|
||||
if line is not None:
|
||||
lines.append(line)
|
||||
if limit is not None and len(lines) >= limit:
|
||||
return "\n".join(lines)
|
||||
length = instruction.getInstructionLength()
|
||||
@@ -532,6 +572,45 @@ def _assembly(procedure, limit=None):
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _read_function_instructions(document, params):
|
||||
"""Read one bounded raw-instruction window without invoking decompilation."""
|
||||
procedure = _procedure(document, params.get("procedure"))
|
||||
offset = _offset(params, "offset")
|
||||
limit = params.get("limit", 64)
|
||||
if offset > 100000:
|
||||
raise ValueError("offset must not exceed 100000")
|
||||
if not isinstance(limit, int) or isinstance(limit, bool) or limit < 1 or limit > 500:
|
||||
raise ValueError("limit must be an integer between 1 and 500")
|
||||
addresses, scan_truncated = _instruction_addresses(
|
||||
procedure, offset + limit + 1
|
||||
)
|
||||
start = min(offset, len(addresses))
|
||||
end = min(len(addresses), start + limit)
|
||||
segment = procedure.getSegment()
|
||||
items = []
|
||||
for address in addresses[start:end]:
|
||||
line = _render_instruction(segment, address)
|
||||
if line is not None:
|
||||
items.append(line)
|
||||
has_more = scan_truncated or end < len(addresses)
|
||||
return {
|
||||
"procedure": _procedure_identity(procedure),
|
||||
"instructions": {
|
||||
"items": items,
|
||||
"total": None if scan_truncated else len(addresses),
|
||||
"returned": len(items),
|
||||
"truncated": has_more,
|
||||
"next_offset": end if has_more else None,
|
||||
},
|
||||
"instructions_scanned": len(addresses),
|
||||
"instruction_scan_truncated": scan_truncated,
|
||||
"limitations": [
|
||||
"Instruction text and ordering are Hopper-specific representations.",
|
||||
"The fast path does not decompile the procedure or scan whole-program names and strings.",
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def _analyze_function(document, params):
|
||||
"""Collect a bounded, single-pass function dossier for agent callers."""
|
||||
procedure = _procedure(document, params.get("procedure"))
|
||||
@@ -560,8 +639,8 @@ def _analyze_function(document, params):
|
||||
"end": _hex(block.getEndingAddress()),
|
||||
"successors": sorted(set(successors), key=lambda value: int(value, 16)),
|
||||
})
|
||||
pseudo = procedure.decompile() or ""
|
||||
assembly_lines = _assembly(procedure).splitlines() if params.get("include_assembly", False) else []
|
||||
pseudo = _pseudocode(document, procedure) or ""
|
||||
assembly_lines = _assembly(procedure, max_instructions).splitlines() if params.get("include_assembly", False) else []
|
||||
callers = sorted((_procedure_identity(item) for item in procedure.getAllCallerProcedures()), key=lambda item: int(item["address"], 16))
|
||||
callees = sorted((_procedure_identity(item) for item in procedure.getAllCalleeProcedures()), key=lambda item: int(item["address"], 16))
|
||||
comments = []
|
||||
@@ -595,8 +674,14 @@ def _analyze_function(document, params):
|
||||
incoming.append(item)
|
||||
if source in procedure_addresses:
|
||||
outgoing.append(item)
|
||||
string_map = {int(address, 16): value for address, value in _strings(document).items()}
|
||||
name_map = _name_map(document)
|
||||
string_map = {
|
||||
int(address, 16): value
|
||||
for address, value in _search_inventory(document, "string")
|
||||
}
|
||||
name_map = {
|
||||
int(address, 16): value
|
||||
for address, value in _search_inventory(document, "name")
|
||||
}
|
||||
referenced_strings = []
|
||||
referenced_names = []
|
||||
for edge in outgoing:
|
||||
@@ -615,7 +700,13 @@ def _analyze_function(document, params):
|
||||
return {
|
||||
"procedure": {"address": _hex(procedure.getEntryPoint()), "name": _procedure_name(procedure), "classification": None, "signature": procedure.signatureString(), "locals": _procedure_locals(procedure)},
|
||||
"pseudocode": {"text": pseudo_text, "total_chars": len(pseudo), "returned_chars": len(pseudo_text), "truncated": pseudo_next < len(pseudo), "next_offset": pseudo_next if pseudo_next < len(pseudo) else None},
|
||||
"assembly": _bounded(assembly_lines, assembly_offset, max_instructions),
|
||||
"assembly": _bounded(
|
||||
assembly_lines,
|
||||
assembly_offset,
|
||||
max_instructions,
|
||||
None,
|
||||
instruction_scan_truncated if params.get("include_assembly", False) else False,
|
||||
),
|
||||
"comments": collection("comments", comments, instruction_scan_truncated),
|
||||
"callers": collection("callers", callers), "callees": collection("callees", callees),
|
||||
"incoming_references": collection("incoming_references", incoming, instruction_scan_truncated),
|
||||
@@ -674,6 +765,8 @@ def _dispatch(method, params):
|
||||
|
||||
if method == "analyze_function":
|
||||
return _analyze_function(document, params)
|
||||
if method == "read_function_instructions":
|
||||
return _read_function_instructions(document, params)
|
||||
if method == "resolve_containing_procedure":
|
||||
address = _address(document, params.get("address"))
|
||||
procedure, reason = _containing_procedure(document, address)
|
||||
@@ -740,19 +833,16 @@ def _dispatch(method, params):
|
||||
})
|
||||
return result
|
||||
if method == "list_procedures":
|
||||
return _page(_procedure_map(document), params.get("offset", 0), params.get("limit", 100))
|
||||
return _page(dict(_search_inventory(document, "procedure")), params.get("offset", 0), params.get("limit", 100))
|
||||
if method == "list_strings":
|
||||
values = _strings(document)
|
||||
values = dict(_search_inventory(document, "string"))
|
||||
requested = params.get("address")
|
||||
if requested is not None:
|
||||
key = _hex(_address(document, requested))
|
||||
values = {key: values[key]} if key in values else {}
|
||||
return _page(values, params.get("offset", 0), params.get("limit", 100))
|
||||
if method == "list_names":
|
||||
result = {}
|
||||
for segment in document.getSegmentsList():
|
||||
for item in segment.getNamedAddresses():
|
||||
result[_hex(item)] = segment.getNameAtAddress(item)
|
||||
result = dict(_search_inventory(document, "name"))
|
||||
requested = params.get("address")
|
||||
if requested is not None:
|
||||
key = _hex(_address(document, requested))
|
||||
@@ -768,7 +858,7 @@ def _dispatch(method, params):
|
||||
if method == "procedure_assembly":
|
||||
return _assembly(procedure)
|
||||
if method == "procedure_pseudo_code":
|
||||
return procedure.decompile()
|
||||
return _pseudocode(document, procedure)
|
||||
if method == "procedure_callers":
|
||||
return sorted((_hex(item.getEntryPoint()) for item in procedure.getAllCallerProcedures()), key=lambda value: int(value, 16))
|
||||
if method == "procedure_callees":
|
||||
@@ -779,12 +869,18 @@ def _dispatch(method, params):
|
||||
return {"name": _procedure_name(procedure), "entrypoint": _hex(procedure.getEntryPoint()), "basicblock_count": procedure.getBasicBlockCount(), "length": length, "signature": procedure.signatureString(), "locals": _procedure_locals(procedure), "classification": None}
|
||||
if method == "set_address_name":
|
||||
address = _address(document, params.get("address"))
|
||||
result = document.setNameAtAddress(address, params["name"])
|
||||
_invalidate_search_inventory(document)
|
||||
try:
|
||||
result = document.setNameAtAddress(address, params["name"])
|
||||
finally:
|
||||
_invalidate_search_inventory(document)
|
||||
_invalidate_pseudocode(document)
|
||||
return result
|
||||
if method == "set_addresses_names":
|
||||
result = {key: document.setNameAtAddress(_address(document, key), value) for key, value in params["names"].items()}
|
||||
_invalidate_search_inventory(document)
|
||||
try:
|
||||
result = {key: document.setNameAtAddress(_address(document, key), value) for key, value in params["names"].items()}
|
||||
finally:
|
||||
_invalidate_search_inventory(document)
|
||||
_invalidate_pseudocode(document)
|
||||
return result
|
||||
if method in ("set_comment", "set_inline_comment"):
|
||||
address = _address(document, params.get("address"))
|
||||
@@ -792,6 +888,7 @@ def _dispatch(method, params):
|
||||
setter = segment.setCommentAtAddress if method == "set_comment" else segment.setInlineCommentAtAddress
|
||||
getter = segment.getCommentAtAddress if method == "set_comment" else segment.getInlineCommentAtAddress
|
||||
setter(address, params["comment"])
|
||||
_invalidate_pseudocode(document)
|
||||
return getter(address) == params["comment"]
|
||||
if method == "list_bookmarks":
|
||||
return [{"address": _hex(item), "name": document.getBookmarkName(item)} for item in document.getBookmarks()]
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
target binding, analysis-profile commitment, and snapshot/Evidence migration
|
||||
are implemented. Ghidra discovery, target/profile resolution, doctor checks,
|
||||
the private headless-session foundation, ten read-only inventory capabilities,
|
||||
and eight function-analysis capabilities are implemented with real
|
||||
and nine function-analysis capabilities are implemented with real
|
||||
cross-format conformance.
|
||||
|
||||
## Context
|
||||
|
||||
@@ -151,7 +151,7 @@ REA's Ghidra provider is bring-your-own and supports Linux x64 with the exact
|
||||
official Ghidra 12.1.2 release and a 64-bit full JDK 21. An experimental
|
||||
Windows x64 P0 supports approved native x86-64 PE applications. It supplies
|
||||
discovery, analysis-profile commitment, an isolated read-only headless session,
|
||||
ten bounded inventory/name/search operations, and eight function-analysis
|
||||
ten bounded inventory/name/search operations, and nine function-analysis
|
||||
operations covering metadata, decompilation, assembly, resolved calls, typed
|
||||
references, xrefs, CFG, and dossiers. GUI state and analysis mutations remain
|
||||
unavailable through Ghidra.
|
||||
|
||||
@@ -115,6 +115,37 @@ redacted from recorded text. The command-shim directory is placed first in the
|
||||
captured process `PATH`; any remaining `PATH` must be explicitly supplied or
|
||||
inherited under operator policy.
|
||||
|
||||
## Reactive process scenarios
|
||||
|
||||
Set `reactive` to a Process Reactive Scenario v1 declaration when interaction
|
||||
must follow observed output instead of guessed delays. The graph declares an
|
||||
initial state, absolute scenario and state deadlines, explicitly prioritized
|
||||
transitions, bounded uses and visits, trigger predicates, ordered actions, and
|
||||
either a next state or `passed` outcome.
|
||||
|
||||
The admitted runtime slice matches decoded UTF-8 terminal literals and exact
|
||||
`terminal_raw`, `interaction`, `process`, `filesystem`, `http`, `websocket`, or
|
||||
`shim` journal events. Process events are sampled observations, not an
|
||||
event-complete lifecycle. Filesystem events identify named checkpoints; shim
|
||||
events identify a command and route. Live predicates receive the same normalized
|
||||
and redacted payload later persisted in Evidence. Predicates may compose with
|
||||
bounded `all`, `any`, `sequence`, and `repeat` nodes and may select a frontier at
|
||||
scenario start, state entry, a prior checkpoint, or an exact event ID. Actions
|
||||
can write PTY input, resize it, signal the root process, or capture a named
|
||||
filesystem checkpoint. `terminal_rendered`, `lifecycle`, and `replay_transition`
|
||||
event triggers and non-root signal selectors fail during preflight before the
|
||||
target launches.
|
||||
|
||||
Reactive results are recorded in `reactive_run`, including the terminal outcome,
|
||||
active state, matched event IDs, action evidence IDs, and deterministic
|
||||
transition journal. Terminal controls also commit the last admitted observation
|
||||
order, so later output cannot be moved ahead of a timeout, cancellation, or
|
||||
target loss during validation. If the process exits before the graph finishes,
|
||||
the outcome is `target_lost`; state/scenario deadlines remain distinct
|
||||
`predicate_timeout`/`scenario_deadline` outcomes. Timed `events` remain
|
||||
available for fixed schedules, but they are not silently translated into the
|
||||
reactive graph.
|
||||
|
||||
## Checkpoints and deterministic shims
|
||||
|
||||
Checkpoint triggers may use an elapsed time, a terminal literal and occurrence
|
||||
|
||||
+30
-13
@@ -2,7 +2,7 @@
|
||||
"catalog_schema_version": 1,
|
||||
"package": {
|
||||
"name": "rea-agents",
|
||||
"version": "2.3.0",
|
||||
"version": "2.4.1",
|
||||
"sdk": {
|
||||
"server": "2.0.0-beta.4",
|
||||
"client": "2.0.0-beta.4",
|
||||
@@ -11,12 +11,12 @@
|
||||
"skill_version": "23"
|
||||
},
|
||||
"tools": {
|
||||
"total": 98,
|
||||
"total": 99,
|
||||
"families": [
|
||||
{
|
||||
"id": "direct",
|
||||
"surface": "official-proxy",
|
||||
"count": 33,
|
||||
"count": 34,
|
||||
"tools": [
|
||||
"address_name",
|
||||
"comment",
|
||||
@@ -40,6 +40,7 @@
|
||||
"procedure_info",
|
||||
"procedure_pseudo_code",
|
||||
"procedure_references",
|
||||
"read_function_instructions",
|
||||
"resolve_containing_procedure",
|
||||
"search_procedures",
|
||||
"search_strings",
|
||||
@@ -86,7 +87,10 @@
|
||||
"id": "artifact",
|
||||
"surface": "artifact-provider",
|
||||
"count": 2,
|
||||
"tools": ["extract_artifact", "inventory_artifact"]
|
||||
"tools": [
|
||||
"extract_artifact",
|
||||
"inventory_artifact"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "managed",
|
||||
@@ -193,6 +197,7 @@
|
||||
"procedure_info",
|
||||
"procedure_pseudo_code",
|
||||
"procedure_references",
|
||||
"read_function_instructions",
|
||||
"resolve_containing_procedure",
|
||||
"search_procedures",
|
||||
"search_strings",
|
||||
@@ -227,6 +232,7 @@
|
||||
"procedure_info",
|
||||
"procedure_pseudo_code",
|
||||
"procedure_references",
|
||||
"read_function_instructions",
|
||||
"resolve_containing_procedure",
|
||||
"search_procedures",
|
||||
"search_strings",
|
||||
@@ -256,7 +262,10 @@
|
||||
"id": "rea-artifact-graph",
|
||||
"name": "REA safe artifact graph provider",
|
||||
"version": "1",
|
||||
"capabilities": ["extract_artifact", "inventory_artifact"]
|
||||
"capabilities": [
|
||||
"extract_artifact",
|
||||
"inventory_artifact"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "rea-cdp-browser",
|
||||
@@ -277,7 +286,10 @@
|
||||
"id": "rea-cdp-electron",
|
||||
"name": "REA Electron file-page CDP observation provider",
|
||||
"version": "1",
|
||||
"capabilities": ["inspect_electron_page", "list_electron_targets"]
|
||||
"capabilities": [
|
||||
"inspect_electron_page",
|
||||
"list_electron_targets"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "rea-dotnet-static",
|
||||
@@ -305,7 +317,9 @@
|
||||
"id": "rea-javascript-application",
|
||||
"name": "REA JavaScript application analyzer",
|
||||
"version": "1",
|
||||
"capabilities": ["analyze_javascript_application"]
|
||||
"capabilities": [
|
||||
"analyze_javascript_application"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "rea-javascript-application-workflows",
|
||||
@@ -327,7 +341,9 @@
|
||||
"id": "rea-javascript-runtime-reconciliation",
|
||||
"name": "REA JavaScript runtime reconciliation",
|
||||
"version": "1",
|
||||
"capabilities": ["reconcile_javascript_runtime"]
|
||||
"capabilities": [
|
||||
"reconcile_javascript_runtime"
|
||||
]
|
||||
}
|
||||
],
|
||||
"setup_clients": [
|
||||
@@ -517,7 +533,7 @@
|
||||
}
|
||||
],
|
||||
"cli": {
|
||||
"primary_count": 58,
|
||||
"primary_count": 59,
|
||||
"commands": [
|
||||
"analyze",
|
||||
"analyze-javascript-application",
|
||||
@@ -553,6 +569,7 @@
|
||||
"inspect-plist",
|
||||
"inspect-signature",
|
||||
"inspect-web-page",
|
||||
"instructions",
|
||||
"inventory-artifact",
|
||||
"investigate-versions",
|
||||
"list-architectures",
|
||||
@@ -587,17 +604,17 @@
|
||||
},
|
||||
"runtime_catalog": {
|
||||
"counts": {
|
||||
"cli_commands": 58,
|
||||
"mcp_tools": 98,
|
||||
"cli_commands": 59,
|
||||
"mcp_tools": 99,
|
||||
"mcp_prompts": 6,
|
||||
"mcp_resources": 2,
|
||||
"mcp_resource_templates": 9
|
||||
},
|
||||
"digests": {
|
||||
"tools_sha256": "501d405efbf8fc3537d13791f87ac81f3b97f3f328b62710ca0885de7213a62f",
|
||||
"tools_sha256": "9159861cda3cc76f876dde306efae886dd310419d4e972f89ab7b85ff5789485",
|
||||
"prompts_sha256": "f6da3ad38a1000c11364d1710f122ac4990551f04efd8c53ffb2e819b5cf5707",
|
||||
"resources_sha256": "94bebbab1a1b3cf47644c38d3de6469de568caa7bd7b842f529ead8ad320a912",
|
||||
"combined_sha256": "71de8a7174176823fd82c2666130233b5a1b4a1613583270d268774775b410bc"
|
||||
"combined_sha256": "e787c57af8337f17b9da72745fcb328634af3b39be1d9ca8bfe1836274351f92"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4,7 +4,7 @@ Status: the Ghidra read-only analysis provider is shipped on Linux x64 and has
|
||||
an experimental Windows x64 P0 for approved native PE applications. It
|
||||
validates an exact bring-your-own Ghidra 12.1.2/JDK 21 environment, resolves a
|
||||
provider/version/profile commitment, runs one isolated read-only headless
|
||||
import, and publishes 18 operation-level capabilities after the authenticated
|
||||
import, and publishes 19 operation-level capabilities after the authenticated
|
||||
post-analysis handshake.
|
||||
|
||||
The provider-neutral target, provider registry, deterministic target binding,
|
||||
@@ -69,14 +69,14 @@ Unix socket. Windows uses authenticated IPv4 loopback with a strict token-free
|
||||
endpoint record. Both transports report actual
|
||||
Ghidra/language/compiler/analysis/import-digest metadata, accept only the exact
|
||||
authenticated `ping`, `shutdown`, and ten
|
||||
inventory plus eight function-analysis methods, and deletes the socket. Close,
|
||||
inventory plus nine function-analysis methods, and deletes the socket. Close,
|
||||
cancellation, timeout,
|
||||
malformed protocol, or process exit stops the owned process resources and
|
||||
removes the entire runtime root. Windows P0 uses bounded `taskkill` tree
|
||||
termination and explicitly does not claim Job Object, private-DACL, or
|
||||
reparse-point authority; see [Windows Ghidra P0](windows-ghidra-p0.md).
|
||||
|
||||
The provider catalog lists only the 18 proved Ghidra operations. GUI cursor,
|
||||
The provider catalog lists only the 19 proved Ghidra operations. GUI cursor,
|
||||
navigation, and mutation operations remain absent; the router therefore
|
||||
reports them unavailable instead of borrowing Hopper semantics or inferring
|
||||
capability from a successful import.
|
||||
@@ -103,13 +103,14 @@ capability from a successful import.
|
||||
| Serialization | A bounded FIFO admits at most 32 active-plus-queued requests and sends one Program request at a time. Queue wait counts against the caller's deadline, and queued cancellation is prompt. This is an adapter safety commitment, not a claim that every Ghidra API is thread-safe. |
|
||||
| Function identity | Every function result carries the entry address and Ghidra FunctionManager classification for external, thunk, and resolved thunk target. These are observations, not proof that unresolved targetless calls have been recovered. |
|
||||
| Assembly and pseudocode | Assembly is bounded Ghidra Listing text; pseudocode is bounded Ghidra decompiler output. Neither is original source, and cross-provider comparison never treats Hopper and Ghidra text as equal or unequal semantic facts. |
|
||||
| Instruction fast path | `read_function_instructions` returns one offset-paginated raw Listing window without invoking the decompiler or whole-program name/string inventories. It reports scan work, exact-or-unknown totals, truncation, and continuation metadata. |
|
||||
| Calls and references | Callers/callees contain only resolved functions. Reference edges preserve exact ReferenceManager type and call/jump/data/read/write/indirect/computed/conditional/terminal/external facts. Targetless computed flow remains unknown. Synthetic entry-point references without actionable memory sources are omitted explicitly. |
|
||||
| CFG | Dossiers use `BasicBlockModel` and retain only non-call successors inside the function body. CFG topology is address-normalized for comparison; provider-specific block construction remains a declared difference. |
|
||||
| Bounds | Direct assembly fails above 100,000 instructions or the wire ceiling. Dossiers scan at most 5,000 requested instructions, return bounded independent pages, mark unknown totals after an incomplete scan, and paginate pseudocode by Unicode code points. |
|
||||
|
||||
`npm run verify:ghidra` compiles the versioned C oracles into x86-64 debug and
|
||||
stripped ELF, AArch64 ELF, x86-64 PE, and x86-64 Mach-O targets. It proves all
|
||||
18 operations, external functions, resolved thunks, exports, stripped-name
|
||||
19 operations, external functions, resolved thunks, exports, stripped-name
|
||||
behavior, direct and targetless indirect calls, typed references, strings/xrefs,
|
||||
multi-block CFG, semantic enhanced workflows, cancellation, deadlines,
|
||||
serialized concurrency, malformed-target rejection, profile identity, and
|
||||
@@ -118,7 +119,7 @@ cover analysis/decompile timeouts, process exit, queue saturation, and malformed
|
||||
wire output.
|
||||
|
||||
`npm run verify:ghidra:windows` generates a deterministic source-owned native
|
||||
x86-64 PE application, proves all 18 operations through the production Windows
|
||||
x86-64 PE application, proves all 19 operations through the production Windows
|
||||
launcher and loopback transport, checks target/snapshot/import SHA-256 linkage,
|
||||
and requires project, endpoint, process, and runtime cleanup. Passing that lane
|
||||
proves only the documented P0 boundary, not the remaining Windows security
|
||||
@@ -146,7 +147,7 @@ only the generic process mechanisms.
|
||||
|
||||
| Provider | License / automation surface | What it brings | REA fit and blockers |
|
||||
| ------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| [Ghidra](https://github.com/NationalSecurityAgency/ghidra) | Apache-2.0 source license; `analyzeHeadless`, Java APIs, and PyGhidra | Broad static analysis, many processors and formats, scripting, and project/database workflows | Read-only analysis shipped: exact BYO checks, isolated state, bounded process and queue, packaged bridge, 18 admitted inventory/function operations, and real ELF/PE/Mach-O x86-64 plus AArch64 conformance. GUI and mutation semantics remain intentionally unavailable. |
|
||||
| [Ghidra](https://github.com/NationalSecurityAgency/ghidra) | Apache-2.0 source license; `analyzeHeadless`, Java APIs, and PyGhidra | Broad static analysis, many processors and formats, scripting, and project/database workflows | Read-only analysis shipped: exact BYO checks, isolated state, bounded process and queue, packaged bridge, 19 admitted inventory/function operations, and real ELF/PE/Mach-O x86-64 plus AArch64 conformance. GUI and mutation semantics remain intentionally unavailable. |
|
||||
| [Rizin](https://github.com/rizinorg/rizin) / [rz-pipe](https://github.com/rizinorg/rz-pipe) | Rizin repository contains LGPL-3.0 and GPL-3.0 components; `rizin`, `rz-bin`, and language bridges through `rzpipe` | Portable CLI analysis, disassembly/debugging, many architectures and file formats, JSON command output | Good candidate for a process-backed Linux provider and fast metadata fallback. License/component inventory must be preserved; command output needs version-pinned parsers and semantic conformance before evidence is trusted. |
|
||||
| [LIEF](https://github.com/lief-project/LIEF) | Apache-2.0; C++, Python, and other bindings | Deterministic parsing and modification of ELF, PE, Mach-O, COFF, and related executable formats; headers, sections, symbols, relocations, and functions | Best near-term complement, not a decompiler replacement. It can cover format metadata and artifact evidence without a long-lived analysis process; function semantics, pseudocode, CFG, and cross-reference parity remain out of scope unless separately demonstrated. |
|
||||
| [Binary Ninja](https://docs.binary.ninja/dev/index.html) | API/documentation components are MIT, while the analysis product is licensed by edition; commercial, Ultimate, or Headless license is required for headless automation | Python/Core/C++/Rust APIs, headless loading, IL layers, function analysis, plugins, and configurable analysis | Strong technical fit for a native provider, especially function dossiers. Commercial licensing, license-secret handling, native runtime packaging, and multithreaded lifecycle rules are material deployment blockers. |
|
||||
|
||||
+2
-2
@@ -11,11 +11,11 @@ additively; Devin is reported but left unchanged.
|
||||
|
||||
The Ghidra foundation supports Linux x64 with exact Ghidra 12.1.2 and a 64-bit
|
||||
full JDK 21. An experimental Windows x64 P0 admits approved native x86-64 PE
|
||||
applications through the same 18-operation Java bridge. Doctor validates those coordinates; approved Linux/macOS setup propagates them
|
||||
applications through the same 19-operation Java bridge. Doctor validates those coordinates; approved Linux/macOS setup propagates them
|
||||
to MCP registrations without installing or modifying either dependency. REA's
|
||||
packaged Java bridge then proves an isolated read-only headless import,
|
||||
post-analysis handshake, complete cleanup, ten admitted read-only inventory
|
||||
operations, and eight function-analysis operations. Real source-owned x86-64
|
||||
operations, and nine function-analysis operations. Real source-owned x86-64
|
||||
debug/stripped ELF, AArch64 ELF, PE, and Mach-O fixtures cover program,
|
||||
procedure, string, symbol, external/thunk, memory, resolution, search,
|
||||
decompilation, assembly, calls, typed references, xrefs, and CFG semantics.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"manifest_id": "ecm_e04a25d07a540df3bec2fd5ba6d8a158e1f582dc94fc59225c7ba329cbabbbf6",
|
||||
"manifest_id": "ecm_0b704bd3b1a8e8f6840d113cf064de935393db1936ca5eba1018a49ccd566cf2",
|
||||
"verifier": {
|
||||
"id": "rea-managed-conformance",
|
||||
"version": "1"
|
||||
@@ -14,7 +14,7 @@
|
||||
"skill_digests": [
|
||||
{
|
||||
"skill_id": "reverse-engineer-anything",
|
||||
"sha256": "a8ffb68f043fc9dfd0eb401caa6e548adce991d1e228b4539333cf90be17a755"
|
||||
"sha256": "fdfbbc813a935b93dbd814c509997024da4a98500b7d12887b0ee6ecf69a9fcb"
|
||||
}
|
||||
],
|
||||
"claims": [
|
||||
|
||||
@@ -13,7 +13,7 @@ The P0 accepts exactly:
|
||||
- an operator-installed official Ghidra 12.1.2 distribution;
|
||||
- a 64-bit full JDK 21;
|
||||
- an explicit native, non-managed, non-DLL x86-64 PE application; and
|
||||
- the existing 18 read-only Ghidra inventory and function-analysis operations.
|
||||
- the existing 19 read-only Ghidra inventory and function-analysis operations.
|
||||
|
||||
Hopper, Ghidra GUI state, mutation, controlled JavaScript replay, process
|
||||
capture, browser/Electron observation, artifact extraction, and general
|
||||
@@ -146,7 +146,7 @@ npm run verify:ghidra:windows
|
||||
|
||||
The verifier creates an ignored deterministic x86-64 PE fixture, checks its
|
||||
fixed SHA-256, opens Ghidra 12.1.2 through the production Java bridge, exercises
|
||||
all 18 admitted operations, verifies the target/snapshot/import digest chain,
|
||||
all 19 admitted operations, verifies the target/snapshot/import digest chain,
|
||||
and checks endpoint, project, process, and runtime cleanup. The GitHub workflow
|
||||
accepts only the fixed `real-ghidra-windows` repository-dispatch event against
|
||||
default-branch `main`, a protected `real-ghidra-windows` environment, and a
|
||||
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "rea-agents",
|
||||
"version": "2.3.0",
|
||||
"version": "2.4.1",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "rea-agents",
|
||||
"version": "2.3.0",
|
||||
"version": "2.4.1",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@babel/parser": "8.0.4",
|
||||
|
||||
+7
-4
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "rea-agents",
|
||||
"version": "2.3.0",
|
||||
"version": "2.4.1",
|
||||
"description": "Reverse engineer anything from your terminal or agent with one CLI and MCP server.",
|
||||
"license": "MIT",
|
||||
"repository": {
|
||||
@@ -80,19 +80,22 @@
|
||||
"test": "vitest run",
|
||||
"test:run": "vitest run",
|
||||
"pretest:fast": "npm run build:cached",
|
||||
"test:fast": "vitest run --project parallel",
|
||||
"test:fast": "vitest run --project parallel --project subprocess",
|
||||
"pretest:changed": "npm run build:cached",
|
||||
"test:changed": "vitest run --project parallel --project subprocess --changed",
|
||||
"pretest:integration": "npm run build:cached",
|
||||
"test:integration": "vitest run --project serial-integration",
|
||||
"pretest:coverage": "npm run build:cached",
|
||||
"test:coverage": "vitest run --coverage.enabled --coverage.provider=v8",
|
||||
"test:ci": "npm run test:coverage -- --reporter=default --reporter=junit --outputFile.junit=test-results.xml",
|
||||
"pretest:ci:shard": "npm run build:cached",
|
||||
"test:ci:shard": "vitest run --coverage.enabled --coverage.provider=v8 --coverage.thresholds.lines=0 --coverage.thresholds.functions=0 --coverage.thresholds.branches=0 --coverage.thresholds.statements=0 --reporter=default --reporter=blob",
|
||||
"test:ci:shard": "npm run test:ci:shard:run",
|
||||
"test:ci:shard:run": "vitest run --coverage.enabled --coverage.provider=v8 --coverage.thresholds.lines=0 --coverage.thresholds.functions=0 --coverage.thresholds.branches=0 --coverage.thresholds.statements=0 --reporter=default --reporter=blob",
|
||||
"test:ci:merge": "vitest --merge-reports --coverage.enabled --coverage.provider=v8 --reporter=default --reporter=junit --outputFile.junit=test-results.xml",
|
||||
"pretest:verbose": "npm run build:cached",
|
||||
"test:verbose": "vitest run --reporter=verbose",
|
||||
"pretest:watch": "npm run build:cached",
|
||||
"test:watch": "vitest --project parallel --changed",
|
||||
"test:watch": "vitest --project parallel --project subprocess --changed",
|
||||
"typecheck": "tsc --noEmit",
|
||||
"verify:test-temp-hygiene": "npm run build:cached && node scripts/verify-test-temp-hygiene.mjs",
|
||||
"verify:hopper": "npm run verify:fixtures && npm run build:cached && node scripts/verify-real-hopper.mjs",
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
import { requireMcpResult } from "./mcp-verifier-results.mjs";
|
||||
|
||||
export const verifyHopperFunctionBasics = async (
|
||||
client,
|
||||
options,
|
||||
procedure,
|
||||
) => {
|
||||
const containment = requireMcpResult(
|
||||
await client.callTool(
|
||||
{
|
||||
name: "resolve_containing_procedure",
|
||||
arguments: { address: procedure },
|
||||
},
|
||||
options,
|
||||
),
|
||||
"resolve_containing_procedure",
|
||||
);
|
||||
if (
|
||||
containment?.found !== true ||
|
||||
containment.procedure?.address !== procedure
|
||||
) {
|
||||
throw new Error(
|
||||
"resolve_containing_procedure returned the wrong procedure",
|
||||
);
|
||||
}
|
||||
|
||||
const references = requireMcpResult(
|
||||
await client.callTool(
|
||||
{
|
||||
name: "procedure_references",
|
||||
arguments: {
|
||||
procedure,
|
||||
direction: "outgoing",
|
||||
limit: 10,
|
||||
max_instructions: 100,
|
||||
},
|
||||
},
|
||||
options,
|
||||
),
|
||||
"procedure_references",
|
||||
);
|
||||
if (!Array.isArray(references?.references?.items)) {
|
||||
throw new Error("procedure_references returned an invalid bounded result");
|
||||
}
|
||||
|
||||
const instructions = requireMcpResult(
|
||||
await client.callTool(
|
||||
{
|
||||
name: "read_function_instructions",
|
||||
arguments: { procedure, limit: 2 },
|
||||
},
|
||||
options,
|
||||
),
|
||||
"read_function_instructions",
|
||||
);
|
||||
if (
|
||||
instructions?.procedure?.address !== procedure ||
|
||||
!Array.isArray(instructions.instructions?.items) ||
|
||||
instructions.instructions.items.length === 0 ||
|
||||
instructions.instructions.items.length > 2
|
||||
) {
|
||||
throw new Error(
|
||||
"read_function_instructions returned an invalid bounded window",
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
outgoingReferenceCount: references.references.items.length,
|
||||
instructionWindowCount: instructions.instructions.items.length,
|
||||
};
|
||||
};
|
||||
@@ -72,10 +72,7 @@ export async function verifyPackageEnvironment({
|
||||
XDG_CONFIG_HOME: join(home, ".config"),
|
||||
PATH: `${fakeBin}:${process.env.PATH ?? ""}`,
|
||||
REA_ANALYSIS_PROVIDER: "auto",
|
||||
HOPPER_LAUNCHER_PATH: process.execPath,
|
||||
HOPPER_LOADER_ARGS_JSON: JSON.stringify([
|
||||
join(root, "tests/fixtures/fakeLauncher.mjs"),
|
||||
]),
|
||||
...packageHopperEnvironment(root),
|
||||
REA_NPX_LOG: npxLog,
|
||||
REA_EVIDENCE_ROOTS_JSON: JSON.stringify([evidenceRoot]),
|
||||
REA_INVESTIGATION_INPUT_ROOTS_JSON: JSON.stringify([workspace]),
|
||||
@@ -94,3 +91,22 @@ export async function verifyPackageEnvironment({
|
||||
environment,
|
||||
};
|
||||
}
|
||||
|
||||
const packageHopperEnvironment = (root) => {
|
||||
const fakeLauncher = join(root, "tests/fixtures/fakeLauncher.mjs");
|
||||
if (process.platform === "win32")
|
||||
return {
|
||||
HOPPER_LAUNCHER_PATH: process.execPath,
|
||||
HOPPER_LOADER_ARGS_JSON: JSON.stringify([fakeLauncher]),
|
||||
};
|
||||
return {
|
||||
HOPPER_LAUNCHER_PATH: "/bin/sh",
|
||||
HOPPER_LOADER_ARGS_JSON: JSON.stringify([
|
||||
"-c",
|
||||
'node_path=$1; shift; "$node_path" "$@"',
|
||||
"rea-package-hopper",
|
||||
process.execPath,
|
||||
fakeLauncher,
|
||||
]),
|
||||
};
|
||||
};
|
||||
|
||||
@@ -44,10 +44,27 @@ export async function verifyDebugFunctionOperations(
|
||||
if (entryString === undefined)
|
||||
throw new Error("Ghidra debug string function probe is unavailable");
|
||||
|
||||
const [info, assembly, pseudocode, callees, callers, outgoing, entryXrefs] =
|
||||
await collectEntryFunctionData(client, entry);
|
||||
const [
|
||||
info,
|
||||
assembly,
|
||||
instructionWindow,
|
||||
pseudocode,
|
||||
callees,
|
||||
callers,
|
||||
outgoing,
|
||||
entryXrefs,
|
||||
] = await collectEntryFunctionData(client, entry);
|
||||
assertLocalProcedureInfo(info, entry.address);
|
||||
assertProviderText(assembly, pseudocode, entry.address);
|
||||
if (
|
||||
instructionWindow.procedure?.address !== entry.address ||
|
||||
instructionWindow.instructions?.items?.length !== 1 ||
|
||||
instructionWindow.instructions.returned !== 1 ||
|
||||
instructionWindow.instructions_scanned < 1
|
||||
)
|
||||
throw new Error(
|
||||
"Ghidra read_function_instructions failed its bounded fast-path contract",
|
||||
);
|
||||
assertEntryCallGraph({ branch, indirect, main, callees, callers });
|
||||
const { xrefOwners, xrefOwnerFailures } = await resolveXrefOwners(
|
||||
client,
|
||||
@@ -83,6 +100,10 @@ export async function verifyDebugFunctionOperations(
|
||||
entry: dossierSummary(entryDossier),
|
||||
branch: dossierSummary(branchDossier),
|
||||
indirect: dossierSummary(indirectDossier),
|
||||
instruction_window: {
|
||||
returned: instructionWindow.instructions.returned,
|
||||
truncated: instructionWindow.instructions.truncated,
|
||||
},
|
||||
cancellation,
|
||||
timeout,
|
||||
concurrency: concurrent,
|
||||
@@ -99,6 +120,12 @@ async function collectEntryFunctionData(client, entry) {
|
||||
document: null,
|
||||
procedure: entry.value,
|
||||
}),
|
||||
functionCall(client, "read_function_instructions", {
|
||||
document: null,
|
||||
procedure: entry.value,
|
||||
offset: 0,
|
||||
limit: 1,
|
||||
}),
|
||||
functionCall(client, "procedure_pseudo_code", {
|
||||
document: null,
|
||||
procedure: entry.value,
|
||||
|
||||
@@ -146,6 +146,7 @@ try {
|
||||
"procedure_callers",
|
||||
"procedure_info",
|
||||
"procedure_pseudo_code",
|
||||
"read_function_instructions",
|
||||
"procedure_references",
|
||||
])
|
||||
await functionOperation(operation, { procedure: procedure.value });
|
||||
|
||||
@@ -26,6 +26,7 @@ import {
|
||||
requireMcpResult,
|
||||
requireWorkflowEvidenceProvider,
|
||||
} from "./lib/mcp-verifier-results.mjs";
|
||||
import { verifyHopperFunctionBasics } from "./lib/real-hopper-function-basics.mjs";
|
||||
import { openAndVerifyLargeFixture } from "./lib/real-hopper-pagination.mjs";
|
||||
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
|
||||
import { startUnrelatedHopperSentinel } from "./lib/unrelated-hopper-sentinel.mjs";
|
||||
@@ -155,42 +156,11 @@ const verifyCurrentTarget = async (client, options) => {
|
||||
"list_procedures",
|
||||
);
|
||||
const firstAddress = firstProcedureAddress(procedures);
|
||||
const containment = requireSuccessfulTool(
|
||||
await client.callTool(
|
||||
{
|
||||
name: "resolve_containing_procedure",
|
||||
arguments: { address: firstAddress },
|
||||
},
|
||||
options,
|
||||
),
|
||||
"resolve_containing_procedure",
|
||||
const basics = await verifyHopperFunctionBasics(
|
||||
client,
|
||||
options,
|
||||
firstAddress,
|
||||
);
|
||||
if (
|
||||
containment?.found !== true ||
|
||||
containment.procedure?.address !== firstAddress
|
||||
) {
|
||||
throw new Error(
|
||||
"resolve_containing_procedure returned the wrong procedure",
|
||||
);
|
||||
}
|
||||
const references = requireSuccessfulTool(
|
||||
await client.callTool(
|
||||
{
|
||||
name: "procedure_references",
|
||||
arguments: {
|
||||
procedure: firstAddress,
|
||||
direction: "outgoing",
|
||||
limit: 10,
|
||||
max_instructions: 100,
|
||||
},
|
||||
},
|
||||
options,
|
||||
),
|
||||
"procedure_references",
|
||||
);
|
||||
if (!Array.isArray(references?.references?.items)) {
|
||||
throw new Error("procedure_references returned an invalid bounded result");
|
||||
}
|
||||
const relationships = await verifyRelationships(
|
||||
client,
|
||||
options,
|
||||
@@ -245,7 +215,8 @@ const verifyCurrentTarget = async (client, options) => {
|
||||
callerCount: relationships.procedure_callers.length,
|
||||
calleeCount: relationships.procedure_callees.length,
|
||||
xrefCount: relationships.xrefs.length,
|
||||
outgoingReferenceCount: references.references.items.length,
|
||||
outgoingReferenceCount: basics.outgoingReferenceCount,
|
||||
instructionWindowCount: basics.instructionWindowCount,
|
||||
};
|
||||
};
|
||||
|
||||
|
||||
@@ -3,8 +3,8 @@ name: reverse-engineer-anything
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, and browser applications with REA. Use shipped-artifact or approved runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
metadata:
|
||||
version: "23"
|
||||
tool_count: 98
|
||||
catalog_digest: "71de8a7174176823fd82c2666130233b5a1b4a1613583270d268774775b410bc"
|
||||
tool_count: 99
|
||||
catalog_digest: "e787c57af8337f17b9da72745fcb328634af3b39be1d9ca8bfe1836274351f92"
|
||||
---
|
||||
|
||||
# REA
|
||||
@@ -22,7 +22,8 @@ Choose the first tool from the target the user supplied. Do not call
|
||||
|
||||
- ASAR or extracted JavaScript/Electron tree:
|
||||
`analyze_javascript_application`.
|
||||
- Archive, application package, ZIP/APK/IPA, or DMG: `inventory_artifact`.
|
||||
- Archive, application package, ZIP/APK/IPA/MSIX/AppX, or DMG:
|
||||
`inventory_artifact`.
|
||||
- Managed PE/CLI assembly: `inspect_managed_artifact`.
|
||||
- User-owned browser page already open: `list_browser_targets`.
|
||||
- User-owned Electron runtime already open: `list_electron_targets`.
|
||||
|
||||
@@ -24,8 +24,8 @@ parser and must not become an implicit setup dependency.
|
||||
## Packages and extraction
|
||||
|
||||
Call `inventory_artifact` before extraction for application bundles, archives,
|
||||
ZIP/APK/IPA, ASAR, or DMG inputs. Continue deterministic occurrence pages from
|
||||
the returned offset only when needed. Cite graph manifest IDs.
|
||||
ZIP/APK/IPA/MSIX/AppX, ASAR, or DMG inputs. Continue deterministic occurrence
|
||||
pages from the returned offset only when needed. Cite graph manifest IDs.
|
||||
|
||||
`extract_artifact` requires explicit approval, an absent absolute output root,
|
||||
and selected occurrence IDs. Never extract every entry implicitly. Symlinks and
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import type { AnalysisError } from "../domain/errors.js";
|
||||
import type { AnalysisClient, ExecutionOptions } from "./AnalysisProvider.js";
|
||||
|
||||
/** Close one provider client and normalize unexpected adapter rejection. */
|
||||
export const closeAnalysisClient = async (
|
||||
client: AnalysisClient,
|
||||
providerId: string,
|
||||
options: Pick<ExecutionOptions, "progress"> = {},
|
||||
): Promise<Result<null, AnalysisError>> => {
|
||||
try {
|
||||
if (client.closeWithOutcome !== undefined)
|
||||
return await client.closeWithOutcome(options);
|
||||
await client.close();
|
||||
return ok(null);
|
||||
} catch (cause: unknown) {
|
||||
return err(
|
||||
new ProviderCleanupError(
|
||||
providerId,
|
||||
["provider-client"],
|
||||
{ reason: "provider client close rejected unexpectedly" },
|
||||
{ cause },
|
||||
),
|
||||
);
|
||||
}
|
||||
};
|
||||
@@ -81,10 +81,29 @@ export interface ProviderRuntimeLineageSnapshot {
|
||||
readonly observation: ProcessLineageObservation;
|
||||
}
|
||||
|
||||
/** One provider's current serial-request activity, including detached callers. */
|
||||
export interface ProviderRequestActivitySnapshot {
|
||||
readonly provider: ProviderIdentity;
|
||||
readonly active: {
|
||||
readonly requestId: number;
|
||||
readonly operation: string;
|
||||
readonly elapsedMs: number;
|
||||
readonly timeoutMs: number;
|
||||
readonly callerState: "waiting" | "timed_out" | "cancelled";
|
||||
} | null;
|
||||
readonly queuedRequests: number;
|
||||
}
|
||||
|
||||
/** Provider session bound to exactly one parsed artifact. */
|
||||
export interface AnalysisClient extends AnalysisOperationPort {
|
||||
/** Retained token-verified process snapshots for dynamic providers this client started. */
|
||||
runtimeLineageSnapshots?(): readonly ProviderRuntimeLineageSnapshot[];
|
||||
/** Current provider work that may outlive a timed-out or cancelled caller. */
|
||||
requestActivitySnapshots?(): readonly ProviderRequestActivitySnapshot[];
|
||||
/** Close with a typed result when the provider can verify cleanup. */
|
||||
closeWithOutcome?(
|
||||
options?: Pick<ExecutionOptions, "progress">,
|
||||
): Promise<Result<null, AnalysisError>>;
|
||||
close(): Promise<void>;
|
||||
}
|
||||
|
||||
@@ -127,8 +146,16 @@ interface CapabilityEffects {
|
||||
}
|
||||
|
||||
type CapabilityAvailability =
|
||||
| { readonly available: true; readonly reason: null }
|
||||
| { readonly available: false; readonly reason: string };
|
||||
| {
|
||||
readonly available: true;
|
||||
readonly reason: null;
|
||||
readonly availabilityCode?: null;
|
||||
}
|
||||
| {
|
||||
readonly available: false;
|
||||
readonly reason: string;
|
||||
readonly availabilityCode?: ProviderRejectionCode;
|
||||
};
|
||||
|
||||
interface CapabilityLimits {
|
||||
readonly maxResults: number | null;
|
||||
|
||||
@@ -269,7 +269,13 @@ const createReader = async (
|
||||
if ((await lstat(path)).isDirectory())
|
||||
return new DirectoryArtifactReader(path);
|
||||
if (format === "asar") return new AsarArtifactReader(path);
|
||||
if (format === "ipa" || format === "apk" || format === "zip")
|
||||
if (
|
||||
format === "ipa" ||
|
||||
format === "apk" ||
|
||||
format === "msix" ||
|
||||
format === "appx" ||
|
||||
format === "zip"
|
||||
)
|
||||
return new ZipArtifactReader(path, format);
|
||||
if (format === "mach-o") return new MachOSliceArtifactReader(path);
|
||||
throw new ArtifactReaderFailure(
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { extname } from "node:path";
|
||||
|
||||
import canonicalize from "canonicalize";
|
||||
|
||||
@@ -13,6 +12,7 @@ import type {
|
||||
ArtifactNode,
|
||||
ArtifactOccurrence,
|
||||
} from "../domain/artifactGraph.js";
|
||||
import { zipPackageFormatForPath } from "../domain/zipPackageFormat.js";
|
||||
|
||||
/** Mutable internal occurrence used until root-bound IDs are known. */
|
||||
export interface MutableOccurrence {
|
||||
@@ -130,7 +130,9 @@ export const createRootNode = (input: {
|
||||
size: input.digest?.bytes ?? 0,
|
||||
kind:
|
||||
input.directory ||
|
||||
["zip", "ipa", "apk", "asar", "dmg", "pkg"].includes(input.format)
|
||||
["zip", "ipa", "apk", "msix", "appx", "asar", "dmg", "pkg"].includes(
|
||||
input.format,
|
||||
)
|
||||
? "container"
|
||||
: classifyArtifactPath(input.path).kind,
|
||||
format: input.format,
|
||||
@@ -264,7 +266,7 @@ export const classifyArtifactPath = (
|
||||
if (/\.(?:m?js|cjs)$/u.test(lower))
|
||||
return { kind: "javascript", format: "javascript-bundle" };
|
||||
if (lower.endsWith(".asar")) return { kind: "container", format: "asar" };
|
||||
const archiveFormat = archiveFormatForExtension(extname(lower));
|
||||
const archiveFormat = zipPackageFormatForPath(lower);
|
||||
if (archiveFormat !== undefined)
|
||||
return { kind: "container", format: archiveFormat };
|
||||
if (/\.framework(?:\/|$)/u.test(lower))
|
||||
@@ -351,19 +353,4 @@ const relationFor = (path: string): ArtifactEdge["relation"] => {
|
||||
return "contains";
|
||||
};
|
||||
|
||||
const archiveFormatForExtension = (
|
||||
extension: string,
|
||||
): "zip" | "ipa" | "apk" | undefined => {
|
||||
switch (extension) {
|
||||
case ".zip":
|
||||
return "zip";
|
||||
case ".ipa":
|
||||
return "ipa";
|
||||
case ".apk":
|
||||
return "apk";
|
||||
default:
|
||||
return undefined;
|
||||
}
|
||||
};
|
||||
|
||||
const depth = (path: string): number => path.split("/").length;
|
||||
|
||||
@@ -2,12 +2,15 @@ import { open } from "node:fs/promises";
|
||||
|
||||
import { classifyArtifactPath } from "../ArtifactGraphConstruction.js";
|
||||
import type { ArtifactNode } from "../../domain/artifactGraph.js";
|
||||
import { zipPackageFormatForPath } from "../../domain/zipPackageFormat.js";
|
||||
|
||||
const classifyContainerExtension = (
|
||||
path: string,
|
||||
): ArtifactNode["format"] | undefined => {
|
||||
const lower = path.toLowerCase();
|
||||
for (const format of ["asar", "ipa", "apk", "zip", "dmg", "pkg"] as const)
|
||||
const zipPackage = zipPackageFormatForPath(lower);
|
||||
if (zipPackage !== undefined) return zipPackage;
|
||||
for (const format of ["asar", "dmg", "pkg"] as const)
|
||||
if (lower.endsWith(`.${format}`)) return format;
|
||||
return undefined;
|
||||
};
|
||||
|
||||
@@ -22,6 +22,8 @@ export const createReader = async (
|
||||
case "zip":
|
||||
case "ipa":
|
||||
case "apk":
|
||||
case "msix":
|
||||
case "appx":
|
||||
return new ZipArtifactReader(path, format);
|
||||
case "asar":
|
||||
return new AsarArtifactReader(path);
|
||||
|
||||
@@ -16,6 +16,7 @@ import type {
|
||||
AnalysisClient,
|
||||
AnalysisExecution,
|
||||
AnalysisClientFactory,
|
||||
ExecutionOptions,
|
||||
AnalysisProfileResolution,
|
||||
AnalysisProfileResolutionOptions,
|
||||
AnalysisOperationPort,
|
||||
@@ -37,6 +38,7 @@ import {
|
||||
type BinarySessionOpenOptions,
|
||||
} from "./BinarySessionOpen.js";
|
||||
import { prepareSessionExecution } from "./BinarySessionExecution.js";
|
||||
import { closeAnalysisClient } from "./AnalysisClientCleanup.js";
|
||||
export type { BinarySessionPort } from "./BinarySessionPort.js";
|
||||
const OFFICIAL_OPERATIONS: ReadonlySet<string> = new Set(
|
||||
OFFICIAL_TOOL_CONTRACTS.map(({ name }) => name),
|
||||
@@ -186,7 +188,17 @@ export class BinarySession
|
||||
await this.#drainCalls();
|
||||
const previous = this.#active;
|
||||
this.#active = undefined;
|
||||
await previous?.client.close();
|
||||
if (previous !== undefined) {
|
||||
const closed = await closeAnalysisClient(
|
||||
previous.client,
|
||||
previous.route.identity.id,
|
||||
);
|
||||
if (!closed.ok) {
|
||||
this.clearSessionRecords();
|
||||
this.#clearRuntimeAvailability();
|
||||
return closed;
|
||||
}
|
||||
}
|
||||
const runId = randomUUID();
|
||||
const client = route.createClient(target, { runId });
|
||||
const started = await client.execute("health", {}, options);
|
||||
@@ -226,15 +238,24 @@ export class BinarySession
|
||||
}
|
||||
|
||||
/** Close the active target, if any. */
|
||||
close(): Promise<Result<null, AnalysisError>> {
|
||||
close(
|
||||
options: Pick<ExecutionOptions, "progress"> = {},
|
||||
): Promise<Result<null, AnalysisError>> {
|
||||
return this.#serialize(async () => {
|
||||
const previous = this.#active;
|
||||
this.#active = undefined;
|
||||
await this.#drainCalls();
|
||||
await previous?.client.close();
|
||||
const closed =
|
||||
previous === undefined
|
||||
? ok(null)
|
||||
: await closeAnalysisClient(
|
||||
previous.client,
|
||||
previous.route.identity.id,
|
||||
options,
|
||||
);
|
||||
this.clearSessionRecords();
|
||||
this.#clearRuntimeAvailability();
|
||||
return ok(null);
|
||||
return closed;
|
||||
});
|
||||
}
|
||||
|
||||
@@ -248,6 +269,8 @@ export class BinarySession
|
||||
runId: this.#active?.runId,
|
||||
runtimeLineageSnapshots:
|
||||
this.#active?.client.runtimeLineageSnapshots?.() ?? [],
|
||||
requestActivitySnapshots:
|
||||
this.#active?.client.requestActivitySnapshots?.() ?? [],
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -23,6 +23,7 @@ import type { ReconstructionCoverageWorkspace } from "../domain/reconstructionCo
|
||||
import type {
|
||||
AnalysisOperation,
|
||||
AnalysisOperationPort,
|
||||
ExecutionOptions,
|
||||
ProviderIdentity,
|
||||
} from "./AnalysisProvider.js";
|
||||
|
||||
@@ -37,7 +38,9 @@ export interface BinarySessionPort extends AnalysisOperationPort {
|
||||
readonly providerId?: AnalysisProviderSelector;
|
||||
},
|
||||
): Promise<Result<BinaryTarget, AnalysisError>>;
|
||||
close(): Promise<Result<null, AnalysisError>>;
|
||||
close(
|
||||
options?: Pick<ExecutionOptions, "progress">,
|
||||
): Promise<Result<null, AnalysisError>>;
|
||||
status(): JsonValue;
|
||||
activeTarget(): BinaryTarget | undefined;
|
||||
recordEvidence(
|
||||
|
||||
@@ -2,6 +2,7 @@ import type { BinaryTarget } from "../domain/binaryTarget.js";
|
||||
import type { JsonValue } from "../domain/jsonValue.js";
|
||||
import type {
|
||||
CapabilityDescriptor,
|
||||
ProviderRequestActivitySnapshot,
|
||||
ProviderRuntimeLineageSnapshot,
|
||||
} from "./AnalysisProvider.js";
|
||||
import type {
|
||||
@@ -19,6 +20,7 @@ interface BinarySessionStatusInput {
|
||||
>;
|
||||
readonly runId: string | undefined;
|
||||
readonly runtimeLineageSnapshots: readonly ProviderRuntimeLineageSnapshot[];
|
||||
readonly requestActivitySnapshots: readonly ProviderRequestActivitySnapshot[];
|
||||
}
|
||||
|
||||
/** Project internal provider routing state into the caller-visible session status. */
|
||||
@@ -29,6 +31,7 @@ export const binarySessionStatus = ({
|
||||
runtimeAvailability,
|
||||
runId,
|
||||
runtimeLineageSnapshots,
|
||||
requestActivitySnapshots,
|
||||
}: BinarySessionStatusInput): JsonValue => {
|
||||
const configuredProvider = router.configuredIdentity();
|
||||
const provider = providerSummary(configuredProvider);
|
||||
@@ -73,6 +76,7 @@ export const binarySessionStatus = ({
|
||||
run_id: runId,
|
||||
process_lineage: processLineageStatus(runtimeLineageSnapshots),
|
||||
},
|
||||
analysis_activity: requestActivityStatus(requestActivitySnapshots),
|
||||
};
|
||||
return target === undefined
|
||||
? { open: false, ...common }
|
||||
@@ -87,6 +91,42 @@ export const binarySessionStatus = ({
|
||||
};
|
||||
};
|
||||
|
||||
const requestActivityStatus = (
|
||||
snapshots: readonly ProviderRequestActivitySnapshot[],
|
||||
) => {
|
||||
const providers = snapshots
|
||||
.slice()
|
||||
.sort((left, right) => left.provider.id.localeCompare(right.provider.id))
|
||||
.map(({ provider, active, queuedRequests }) => ({
|
||||
provider: providerSummary(provider),
|
||||
active:
|
||||
active === null
|
||||
? null
|
||||
: {
|
||||
request_id: active.requestId,
|
||||
operation: active.operation,
|
||||
elapsed_ms: active.elapsedMs,
|
||||
timeout_ms: active.timeoutMs,
|
||||
caller_state: active.callerState,
|
||||
},
|
||||
queued_requests: queuedRequests,
|
||||
}));
|
||||
const active = snapshots.flatMap((snapshot) =>
|
||||
snapshot.active === null ? [] : [snapshot.active],
|
||||
);
|
||||
return {
|
||||
status:
|
||||
snapshots.length === 0
|
||||
? "not_observed"
|
||||
: active.some(({ callerState }) => callerState === "timed_out")
|
||||
? "timed_out_busy"
|
||||
: active.length > 0
|
||||
? "busy"
|
||||
: "idle",
|
||||
providers,
|
||||
};
|
||||
};
|
||||
|
||||
const providerSummary = ({
|
||||
id,
|
||||
name,
|
||||
@@ -147,6 +187,7 @@ const capabilityStatus = (
|
||||
operation: descriptor.operation,
|
||||
available: descriptor.available,
|
||||
reason: descriptor.reason,
|
||||
availability_code: descriptor.availabilityCode ?? null,
|
||||
input_contract_version: descriptor.inputContractVersion,
|
||||
output_contract_version: descriptor.outputContractVersion,
|
||||
pagination: descriptor.pagination,
|
||||
|
||||
@@ -12,6 +12,7 @@ const statusSchema = z.object({
|
||||
operation: z.string(),
|
||||
available: z.boolean(),
|
||||
reason: z.string().nullable(),
|
||||
availability_code: z.string().nullable().optional(),
|
||||
}),
|
||||
),
|
||||
});
|
||||
@@ -29,6 +30,7 @@ type ToolKind = (typeof TOOL_CONTRACTS)[number]["kind"];
|
||||
type ProviderDescriptor = {
|
||||
readonly available: boolean;
|
||||
readonly reason: string | null;
|
||||
readonly availability_code?: string | null;
|
||||
};
|
||||
type AvailabilityPolicy = {
|
||||
readonly processCaptureEnabled: boolean;
|
||||
@@ -86,8 +88,15 @@ export const buildCapabilityInventory = (
|
||||
policy: AvailabilityPolicy,
|
||||
) => {
|
||||
const status = statusSchema.parse(sessionStatus);
|
||||
const descriptors = new Map(
|
||||
status.capabilities.map((descriptor) => [descriptor.operation, descriptor]),
|
||||
const descriptors = new Map<string, ProviderDescriptor>(
|
||||
status.capabilities.map((descriptor) => [
|
||||
descriptor.operation,
|
||||
{
|
||||
available: descriptor.available,
|
||||
reason: descriptor.reason,
|
||||
availability_code: descriptor.availability_code ?? null,
|
||||
},
|
||||
]),
|
||||
);
|
||||
return TOOL_CONTRACTS.map((contract) => {
|
||||
const availability = availabilityFor({
|
||||
@@ -237,7 +246,7 @@ const providerAvailability = ({
|
||||
};
|
||||
if (
|
||||
!descriptor.available &&
|
||||
descriptor.reason?.toLowerCase().includes("require macos") === true
|
||||
descriptor.availability_code === "unsupported_host"
|
||||
)
|
||||
return {
|
||||
reason: "unsupported_host",
|
||||
@@ -254,10 +263,7 @@ const providerAvailability = ({
|
||||
|
||||
const composedAvailability = (
|
||||
name: string,
|
||||
descriptors: ReadonlyMap<
|
||||
string,
|
||||
{ readonly available: boolean; readonly reason: string | null }
|
||||
>,
|
||||
descriptors: ReadonlyMap<string, ProviderDescriptor>,
|
||||
): {
|
||||
readonly reason: ToolAvailabilityReason;
|
||||
readonly remediation: string | null;
|
||||
@@ -280,7 +286,7 @@ const composedAvailability = (
|
||||
if (unavailable !== undefined)
|
||||
return {
|
||||
reason:
|
||||
unavailable.reason?.toLowerCase().includes("require macos") === true
|
||||
unavailable.availability_code === "unsupported_host"
|
||||
? "unsupported_host"
|
||||
: "provider_unavailable",
|
||||
remediation:
|
||||
|
||||
@@ -10,6 +10,7 @@ import type {
|
||||
CapabilityDescriptor,
|
||||
ProviderIdentity,
|
||||
} from "./AnalysisProvider.js";
|
||||
import { closeAnalysisClient } from "./AnalysisClientCleanup.js";
|
||||
|
||||
/** Synthetic compatibility identity for a deterministic provider set. */
|
||||
export const compositeProviderIdentity = (
|
||||
@@ -103,6 +104,16 @@ export class CompositeProvider implements AnalysisProvider {
|
||||
clients.set(provider, created);
|
||||
return created;
|
||||
};
|
||||
const closeWithOutcome: NonNullable<
|
||||
AnalysisClient["closeWithOutcome"]
|
||||
> = async (options) => {
|
||||
const outcomes = await Promise.all(
|
||||
[...clients.entries()].map(([provider, client]) =>
|
||||
closeAnalysisClient(client, provider.identity().id, options),
|
||||
),
|
||||
);
|
||||
return outcomes.find((outcome) => !outcome.ok) ?? ok(null);
|
||||
};
|
||||
return {
|
||||
execute: (operation, parameters, options) => {
|
||||
if (operation === "health")
|
||||
@@ -135,10 +146,15 @@ export class CompositeProvider implements AnalysisProvider {
|
||||
.sort((left, right) =>
|
||||
left.provider.id.localeCompare(right.provider.id),
|
||||
),
|
||||
requestActivitySnapshots: () =>
|
||||
[...clients.values()]
|
||||
.flatMap((client) => client.requestActivitySnapshots?.() ?? [])
|
||||
.sort((left, right) =>
|
||||
left.provider.id.localeCompare(right.provider.id),
|
||||
),
|
||||
closeWithOutcome,
|
||||
close: async () => {
|
||||
await Promise.allSettled(
|
||||
[...clients.values()].map(async (client) => client.close()),
|
||||
);
|
||||
await closeWithOutcome();
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -49,6 +49,7 @@ export {
|
||||
type DirectAnalysisTool =
|
||||
| "binary_overview"
|
||||
| "procedure_pseudo_code"
|
||||
| "read_function_instructions"
|
||||
| "analyze_function"
|
||||
| "search_strings"
|
||||
| "search_procedures"
|
||||
|
||||
@@ -309,22 +309,24 @@ export class EnhancedTools {
|
||||
input: { detail: "concise" | "detailed"; limit: number },
|
||||
signal?: AbortSignal,
|
||||
): EnhancedResult {
|
||||
const [segmentsResult, documentsResult, procedures, stringsResult] =
|
||||
const [segmentsResult, documentsResult, proceduresResult, stringsResult] =
|
||||
await Promise.all([
|
||||
this.#call("list_segments", {}, signal),
|
||||
this.#call("list_documents", {}, signal),
|
||||
this.#allAddressed("list_procedures", signal),
|
||||
this.#call("list_strings", {}, signal),
|
||||
this.#call("list_procedures", { offset: 0, limit: 1 }, signal),
|
||||
this.#call("list_strings", { offset: 0, limit: 1 }, signal),
|
||||
]);
|
||||
if (!segmentsResult.ok) return segmentsResult;
|
||||
if (!documentsResult.ok) return documentsResult;
|
||||
if (!procedures.ok) return procedures;
|
||||
if (!proceduresResult.ok) return proceduresResult;
|
||||
if (!stringsResult.ok) return stringsResult;
|
||||
|
||||
const segments = parseSegments(segmentsResult.value);
|
||||
if (!segments.ok) return segments;
|
||||
const documents = parseDocuments(documentsResult.value);
|
||||
if (!documents.ok) return documents;
|
||||
const procedureCount = parseListCount(proceduresResult.value, "procedures");
|
||||
if (!procedureCount.ok) return procedureCount;
|
||||
const stringCount = parseListCount(stringsResult.value, "strings");
|
||||
if (!stringCount.ok) return stringCount;
|
||||
|
||||
@@ -339,7 +341,7 @@ export class EnhancedTools {
|
||||
: { name, start, end },
|
||||
),
|
||||
segment_count: segments.value.length,
|
||||
procedure_count: procedures.value.length,
|
||||
procedure_count: procedureCount.value,
|
||||
string_count: stringCount.value,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
import type { ProcessScenario } from "../domain/processCapture.js";
|
||||
import type { LoopbackReplay } from "./LoopbackReplay.js";
|
||||
import type { CommandShimReplay } from "./CommandShimReplay.js";
|
||||
|
||||
interface ProcessCaptureEnvironmentOptions {
|
||||
readonly scenario: ProcessScenario;
|
||||
readonly home: string;
|
||||
readonly replay: LoopbackReplay;
|
||||
readonly shimReplay: CommandShimReplay;
|
||||
readonly runId: string;
|
||||
}
|
||||
|
||||
/** Build the explicitly admitted environment for a captured process. */
|
||||
export const makeProcessCaptureEnvironment = (
|
||||
options: ProcessCaptureEnvironmentOptions,
|
||||
): Record<string, string> => {
|
||||
const { scenario, home, replay, shimReplay, runId } = options;
|
||||
const environment: Record<string, string> = {
|
||||
...scenario.environment,
|
||||
HOME: home,
|
||||
TERM: "xterm-256color",
|
||||
REA_PROCESS_RUN_ID: runId,
|
||||
};
|
||||
for (const name of scenario.inherit_environment) {
|
||||
const value = process.env[name];
|
||||
if (value !== undefined) environment[name] = value;
|
||||
}
|
||||
environment.REA_REPLAY_HTTP_URL = replay.httpUrl;
|
||||
environment.REA_REPLAY_WEBSOCKET_URL = replay.websocketUrl;
|
||||
environment.REA_SHIM_LEDGER_URL = shimReplay.url;
|
||||
// PATH is evidence-producing input. Inheriting the host PATH here would let
|
||||
// the authority and reconstruction probe different tools without recording
|
||||
// that difference. Scenarios must opt in through environment or
|
||||
// inherit_environment; deterministic shims always take precedence.
|
||||
environment.PATH = [shimReplay.binPath, environment.PATH ?? ""]
|
||||
.filter((part) => part.length > 0)
|
||||
.join(":");
|
||||
return environment;
|
||||
};
|
||||
@@ -71,6 +71,7 @@ interface CaptureResultOptions {
|
||||
};
|
||||
readonly samples: readonly ProcessSample[];
|
||||
readonly replay: LoopbackReplay;
|
||||
readonly reactiveRun: ProcessCapture["reactive_run"];
|
||||
readonly before: SnapshotResult;
|
||||
readonly after: SnapshotResult;
|
||||
readonly truncated: boolean;
|
||||
@@ -116,6 +117,7 @@ export const buildCaptureResult = (
|
||||
options.scenario,
|
||||
),
|
||||
replay_transitions: options.replay.transitions,
|
||||
reactive_run: options.reactiveRun,
|
||||
event_journal: options.eventJournal,
|
||||
files_before: options.before.files,
|
||||
files_after: options.after.files,
|
||||
|
||||
@@ -106,6 +106,12 @@ export class ProcessCheckpoints {
|
||||
this.#scheduleCapture(name, this.#signal);
|
||||
}
|
||||
|
||||
/** Read one committed checkpoint for synchronous journal projection. */
|
||||
at(index: number): FilesystemCheckpoint | undefined {
|
||||
const checkpoint = this.#captures[index];
|
||||
return checkpoint === undefined ? undefined : structuredClone(checkpoint);
|
||||
}
|
||||
|
||||
#scheduleCapture(name: string, signal: AbortSignal | undefined): void {
|
||||
if (this.#captured.has(name)) return;
|
||||
this.#captured.add(name);
|
||||
|
||||
@@ -15,7 +15,6 @@ import {
|
||||
ProcessCaptureError,
|
||||
processCaptureCancelled,
|
||||
} from "./ProcessCaptureError.js";
|
||||
|
||||
export { ProcessCaptureError } from "./ProcessCaptureError.js";
|
||||
import { startLoopbackReplay, type LoopbackReplay } from "./LoopbackReplay.js";
|
||||
import { startProcessSampler } from "./ProcessSampling.js";
|
||||
@@ -26,7 +25,10 @@ import {
|
||||
startCommandShimReplay,
|
||||
type CommandShimReplay,
|
||||
} from "./CommandShimReplay.js";
|
||||
import { normalizeProcessText } from "./ProcessNormalization.js";
|
||||
import {
|
||||
normalizeProcessShimEvent,
|
||||
normalizeProcessText,
|
||||
} from "./ProcessNormalization.js";
|
||||
import { selectCapturedProcessGroupIds } from "../process/ProcessOwnership.js";
|
||||
import {
|
||||
awaitTerminalExit,
|
||||
@@ -42,45 +44,17 @@ import { assertNotCancelled } from "./ProcessCaptureAuthority.js";
|
||||
import {
|
||||
createProcessCaptureJournal,
|
||||
scheduleScenarioInteractions,
|
||||
type ProcessCaptureJournal,
|
||||
} from "./ProcessCaptureJournal.js";
|
||||
|
||||
import {
|
||||
assertSupportedReactiveScenario,
|
||||
projectProcessReactiveRun,
|
||||
startProcessReactiveHarness,
|
||||
type ProcessReactiveHarness,
|
||||
} from "./ProcessReactiveHarness.js";
|
||||
import { makeProcessCaptureEnvironment } from "./ProcessCaptureEnvironment.js";
|
||||
export { probeProcessCaptureCapability } from "./ProcessCaptureCapability.js";
|
||||
|
||||
interface EnvironmentOptions {
|
||||
readonly scenario: ProcessScenario;
|
||||
readonly home: string;
|
||||
readonly replay: LoopbackReplay;
|
||||
readonly shimReplay: CommandShimReplay;
|
||||
readonly runId: string;
|
||||
}
|
||||
|
||||
const makeEnvironment = (
|
||||
options: EnvironmentOptions,
|
||||
): Record<string, string> => {
|
||||
const { scenario, home, replay, shimReplay, runId } = options;
|
||||
const environment: Record<string, string> = {
|
||||
...scenario.environment,
|
||||
HOME: home,
|
||||
TERM: "xterm-256color",
|
||||
REA_PROCESS_RUN_ID: runId,
|
||||
};
|
||||
for (const name of scenario.inherit_environment) {
|
||||
const value = process.env[name];
|
||||
if (value !== undefined) environment[name] = value;
|
||||
}
|
||||
environment.REA_REPLAY_HTTP_URL = replay.httpUrl;
|
||||
environment.REA_REPLAY_WEBSOCKET_URL = replay.websocketUrl;
|
||||
environment.REA_SHIM_LEDGER_URL = shimReplay.url;
|
||||
// PATH is evidence-producing input. Inheriting the host PATH here would let
|
||||
// the authority and reconstruction probe different tools without recording
|
||||
// that difference. Scenarios must opt in through environment or
|
||||
// inherit_environment; deterministic shims always take precedence.
|
||||
environment.PATH = [shimReplay.binPath, environment.PATH ?? ""]
|
||||
.filter((part) => part.length > 0)
|
||||
.join(":");
|
||||
return environment;
|
||||
};
|
||||
|
||||
interface StartedCaptureRuntime {
|
||||
readonly replay: LoopbackReplay;
|
||||
readonly renderer: TerminalRenderer;
|
||||
@@ -92,6 +66,7 @@ interface StartedCaptureRuntime {
|
||||
readonly lastOutput: () => number;
|
||||
readonly framesTruncated: () => boolean;
|
||||
readonly stopSampler: () => Promise<{ readonly partial: boolean }>;
|
||||
readonly reactive: ProcessReactiveHarness | undefined;
|
||||
}
|
||||
|
||||
const cleanupFailedStartup = async (options: {
|
||||
@@ -136,7 +111,7 @@ const createTerminalRenderer = (
|
||||
recordEvent,
|
||||
});
|
||||
|
||||
const startCaptureRuntime = async (options: {
|
||||
interface StartCaptureRuntimeOptions {
|
||||
readonly scenario: ProcessScenario;
|
||||
readonly home: string;
|
||||
readonly temporaryRoot: string;
|
||||
@@ -148,14 +123,29 @@ const startCaptureRuntime = async (options: {
|
||||
readonly timers: Set<NodeJS.Timeout>;
|
||||
readonly dispatchedEventIndexes: Set<number>;
|
||||
readonly recordEvent: RecordProcessCaptureEvent;
|
||||
readonly journal: ProcessCaptureJournal;
|
||||
readonly signal?: AbortSignal;
|
||||
}): Promise<StartedCaptureRuntime> => {
|
||||
}
|
||||
|
||||
const reactiveCapture = (
|
||||
options: StartCaptureRuntimeOptions,
|
||||
protocolEvents: () => LoopbackReplay["events"],
|
||||
): Parameters<typeof startProcessReactiveHarness>[0]["capture"] => ({
|
||||
...options,
|
||||
processSamples: options.samples,
|
||||
protocolEvents,
|
||||
});
|
||||
|
||||
const startCaptureRuntime = async (
|
||||
options: StartCaptureRuntimeOptions,
|
||||
): Promise<StartedCaptureRuntime> => {
|
||||
const { scenario } = options;
|
||||
let replay: LoopbackReplay | undefined;
|
||||
let renderer: TerminalRenderer | undefined;
|
||||
let checkpoints: ProcessCheckpoints | undefined;
|
||||
let shimReplay: CommandShimReplay | undefined;
|
||||
let terminal: IPty | undefined;
|
||||
let reactive: ProcessReactiveHarness | undefined;
|
||||
try {
|
||||
const { spawn } = await import("@lydell/node-pty");
|
||||
replay = await startLoopbackReplay(scenario, options.recordEvent);
|
||||
@@ -180,7 +170,7 @@ const startCaptureRuntime = async (options: {
|
||||
);
|
||||
terminal = spawn(scenario.executable, [...scenario.arguments], {
|
||||
cwd: scenario.working_directory,
|
||||
env: makeEnvironment({ ...options, replay, shimReplay }),
|
||||
env: makeProcessCaptureEnvironment({ ...options, replay, shimReplay }),
|
||||
cols: scenario.terminal.columns,
|
||||
rows: scenario.terminal.rows,
|
||||
name: "xterm-256color",
|
||||
@@ -193,6 +183,15 @@ const startCaptureRuntime = async (options: {
|
||||
renderer,
|
||||
checkpoints,
|
||||
});
|
||||
reactive = startProcessReactiveHarness({
|
||||
capture: reactiveCapture(options, () => replay?.events ?? []),
|
||||
scenario,
|
||||
terminal: () => terminal,
|
||||
renderer,
|
||||
checkpoints,
|
||||
shimReplay,
|
||||
started,
|
||||
});
|
||||
scheduleScenarioInteractions({
|
||||
...options,
|
||||
getTerminal: () => terminal,
|
||||
@@ -218,8 +217,11 @@ const startCaptureRuntime = async (options: {
|
||||
lastOutput: () => lastOutput,
|
||||
framesTruncated,
|
||||
stopSampler,
|
||||
reactive,
|
||||
};
|
||||
} catch (cause: unknown) {
|
||||
reactive?.unsubscribe();
|
||||
await reactive?.coordinator.close();
|
||||
return cleanupFailedStartup({
|
||||
cause,
|
||||
timers: options.timers,
|
||||
@@ -239,23 +241,14 @@ const normalizedShimEvents = (
|
||||
scenario: ProcessScenario,
|
||||
temporaryRoot: string,
|
||||
): ProcessCapture["shim_events"] =>
|
||||
runtime.shimReplay.events.map((event) => ({
|
||||
...event,
|
||||
arguments: event.arguments.map((argument) =>
|
||||
normalizeProcessText(
|
||||
argument,
|
||||
scenario,
|
||||
temporaryRoot,
|
||||
runtime.terminal.pid,
|
||||
),
|
||||
),
|
||||
working_directory: normalizeProcessText(
|
||||
event.working_directory,
|
||||
runtime.shimReplay.events.map((event) =>
|
||||
normalizeProcessShimEvent(
|
||||
event,
|
||||
scenario,
|
||||
temporaryRoot,
|
||||
runtime.terminal.pid,
|
||||
),
|
||||
}));
|
||||
);
|
||||
|
||||
const finishProcessRun = async (options: {
|
||||
readonly runtime: StartedCaptureRuntime | undefined;
|
||||
@@ -267,6 +260,7 @@ const finishProcessRun = async (options: {
|
||||
readonly capture: ProcessCapture | undefined;
|
||||
readonly executionFailure: unknown;
|
||||
}): Promise<ProcessCapture> => {
|
||||
options.runtime?.reactive?.unsubscribe();
|
||||
await options.stopSampler();
|
||||
const cleanupFailure = await releaseProcessResources({
|
||||
timers: options.timers,
|
||||
@@ -285,10 +279,16 @@ const finishProcessRun = async (options: {
|
||||
options.samples,
|
||||
),
|
||||
});
|
||||
if (cleanupFailure !== undefined)
|
||||
await options.runtime?.reactive?.coordinator.submit({
|
||||
kind: "cleanup_failed",
|
||||
});
|
||||
await options.runtime?.reactive?.coordinator.close();
|
||||
let { capture, executionFailure } = options;
|
||||
if (capture !== undefined) {
|
||||
capture = {
|
||||
...capture,
|
||||
reactive_run: projectProcessReactiveRun(options.runtime?.reactive),
|
||||
settlement: {
|
||||
...capture.settlement,
|
||||
cleanup_outcome:
|
||||
@@ -326,7 +326,17 @@ const completeCapture = async (options: {
|
||||
const { runtime, scenario } = options;
|
||||
runtime.checkpoints.trigger("root_exit");
|
||||
const { reason } = options.exit;
|
||||
if (reason === "cancelled") throw processCaptureCancelled();
|
||||
if (reason === "cancelled") {
|
||||
if (runtime.reactive !== undefined) {
|
||||
await runtime.reactive.coordinator.submit({ kind: "cancelled" });
|
||||
runtime.reactive.unsubscribe();
|
||||
}
|
||||
throw processCaptureCancelled();
|
||||
}
|
||||
if (runtime.reactive !== undefined) {
|
||||
await runtime.reactive.coordinator.submit({ kind: "target_lost" });
|
||||
runtime.reactive.unsubscribe();
|
||||
}
|
||||
const settlement = await observeSettlement(
|
||||
options.runId,
|
||||
selectCapturedProcessGroupIds(runtime.terminal.pid, options.samples),
|
||||
@@ -359,6 +369,7 @@ const completeCapture = async (options: {
|
||||
exit: { ...options.exit, reason },
|
||||
samples: options.samples,
|
||||
replay: runtime.replay,
|
||||
reactiveRun: projectProcessReactiveRun(runtime.reactive),
|
||||
before: options.before,
|
||||
after,
|
||||
truncated,
|
||||
@@ -381,6 +392,7 @@ const runProcessScenario = async (
|
||||
policy: ProcessExecutionPolicy,
|
||||
signal?: AbortSignal,
|
||||
): Promise<ProcessCapture> => {
|
||||
assertSupportedReactiveScenario(scenario);
|
||||
const { temporaryRoot, runId, home, before } = await prepareProcessCapture(
|
||||
scenario,
|
||||
policy,
|
||||
@@ -388,7 +400,8 @@ const runProcessScenario = async (
|
||||
);
|
||||
const frames: TerminalFrame[] = [];
|
||||
const samples: ProcessSample[] = [];
|
||||
const { entries: eventJournal, recordEvent } = createProcessCaptureJournal();
|
||||
const journal = createProcessCaptureJournal();
|
||||
const { entries: eventJournal, recordEvent } = journal;
|
||||
let runtime: StartedCaptureRuntime | undefined;
|
||||
const timers = new Set<NodeJS.Timeout>();
|
||||
let capture: ProcessCapture | undefined;
|
||||
@@ -410,6 +423,7 @@ const runProcessScenario = async (
|
||||
timers,
|
||||
dispatchedEventIndexes,
|
||||
recordEvent,
|
||||
journal,
|
||||
...(signal === undefined ? {} : { signal }),
|
||||
});
|
||||
stopSampler = runtime.stopSampler;
|
||||
|
||||
@@ -2,6 +2,7 @@ import type {
|
||||
ProcessCapture,
|
||||
ProcessSample,
|
||||
ProcessScenario,
|
||||
ShimEvent,
|
||||
} from "../domain/processCapture.js";
|
||||
|
||||
/** Bucket one elapsed process-capture timestamp under scenario normalization. */
|
||||
@@ -91,6 +92,25 @@ export const normalizeProcessSamples = (
|
||||
}));
|
||||
};
|
||||
|
||||
/** Normalize one shim observation identically for live matching and capture. */
|
||||
export const normalizeProcessShimEvent = (
|
||||
event: ShimEvent,
|
||||
scenario: ProcessScenario,
|
||||
temporaryRoot: string,
|
||||
rootPid: number,
|
||||
): ShimEvent => ({
|
||||
...event,
|
||||
arguments: event.arguments.map((argument) =>
|
||||
normalizeProcessText(argument, scenario, temporaryRoot, rootPid),
|
||||
),
|
||||
working_directory: normalizeProcessText(
|
||||
event.working_directory,
|
||||
scenario,
|
||||
temporaryRoot,
|
||||
rootPid,
|
||||
),
|
||||
});
|
||||
|
||||
const normalizePidTokens = (
|
||||
value: string,
|
||||
identifiers: readonly number[],
|
||||
|
||||
@@ -51,7 +51,7 @@ export class ProcessReactiveCoordinator {
|
||||
readonly #now: () => number;
|
||||
readonly #scenarioDeadlineAt: number;
|
||||
readonly #onDecision:
|
||||
| ((decision: ProcessReactiveDecision) => void)
|
||||
| ((decision: ProcessReactiveDecision, input: ProcessReactiveInput) => void)
|
||||
| undefined;
|
||||
#snapshot: ProcessReactiveSnapshot;
|
||||
readonly #pendingInputs: ProcessReactiveInput[] = [];
|
||||
@@ -70,7 +70,10 @@ export class ProcessReactiveCoordinator {
|
||||
readonly executor: ProcessReactiveExecutor;
|
||||
readonly timerHost?: ProcessReactiveTimerHost;
|
||||
readonly now?: () => number;
|
||||
readonly onDecision?: (decision: ProcessReactiveDecision) => void;
|
||||
readonly onDecision?: (
|
||||
decision: ProcessReactiveDecision,
|
||||
input: ProcessReactiveInput,
|
||||
) => void;
|
||||
}) {
|
||||
this.#scenario = options.scenario;
|
||||
this.#executor = options.executor;
|
||||
@@ -206,7 +209,7 @@ export class ProcessReactiveCoordinator {
|
||||
);
|
||||
}
|
||||
this.#snapshot = decision.snapshot;
|
||||
this.#onDecision?.(decision);
|
||||
this.#onDecision?.(decision, input);
|
||||
if (this.#snapshot.status === "finished") this.#clearTimers();
|
||||
else if (
|
||||
this.#accepting &&
|
||||
@@ -216,18 +219,35 @@ export class ProcessReactiveCoordinator {
|
||||
)
|
||||
this.#scheduleStateDeadline();
|
||||
for (const result of effectResults)
|
||||
if (result.status === "succeeded")
|
||||
if (
|
||||
result.status === "succeeded" &&
|
||||
!this.#hasObservation(result.observation.event_id)
|
||||
)
|
||||
this.#enqueueOwned({
|
||||
kind: "observation",
|
||||
observation: result.observation,
|
||||
});
|
||||
}
|
||||
|
||||
#hasObservation(eventId: string): boolean {
|
||||
return (
|
||||
this.#snapshot.observations.some(
|
||||
(observation) => observation.event_id === eventId,
|
||||
) ||
|
||||
this.#pendingInputs.some(
|
||||
(input) =>
|
||||
input.kind === "observation" &&
|
||||
input.observation.event_id === eventId,
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
#interruptsEffects(input: ProcessReactiveInput): boolean {
|
||||
if (
|
||||
input.kind === "cancelled" ||
|
||||
input.kind === "cleanup_failed" ||
|
||||
input.kind === "scenario_deadline"
|
||||
input.kind === "scenario_deadline" ||
|
||||
input.kind === "target_lost"
|
||||
)
|
||||
return true;
|
||||
return (
|
||||
|
||||
@@ -184,9 +184,15 @@ const triggerSources = (
|
||||
): readonly string[] => {
|
||||
if (trigger.kind === "terminal_text") return [];
|
||||
if (trigger.kind === "event")
|
||||
return ["terminal_raw", "interaction", "filesystem"].includes(
|
||||
trigger.source,
|
||||
)
|
||||
return [
|
||||
"terminal_raw",
|
||||
"interaction",
|
||||
"process",
|
||||
"filesystem",
|
||||
"http",
|
||||
"websocket",
|
||||
"shim",
|
||||
].includes(trigger.source)
|
||||
? []
|
||||
: [trigger.source];
|
||||
return trigger.kind === "repeat"
|
||||
|
||||
@@ -0,0 +1,159 @@
|
||||
import type { IPty } from "@lydell/node-pty";
|
||||
|
||||
import type {
|
||||
InteractionEvent,
|
||||
ProcessCapture,
|
||||
ProcessReactiveControlRecord,
|
||||
ProcessSample,
|
||||
ProcessScenario,
|
||||
ProtocolEvent,
|
||||
TerminalFrame,
|
||||
} from "../domain/processCapture.js";
|
||||
import type { ProcessReactiveInput } from "../domain/processReactiveRuntime.js";
|
||||
import { ProcessCaptureError } from "./ProcessCaptureError.js";
|
||||
import type { ProcessCaptureJournal } from "./ProcessCaptureJournal.js";
|
||||
import type { ProcessCheckpoints } from "./ProcessCheckpoints.js";
|
||||
import type { CommandShimReplay } from "./CommandShimReplay.js";
|
||||
import { ProcessReactiveCoordinator } from "./ProcessReactiveCoordinator.js";
|
||||
import {
|
||||
executeProcessReactiveEffects,
|
||||
unsupportedProcessReactiveFeatures,
|
||||
} from "./ProcessReactiveEffects.js";
|
||||
import { subscribeProcessReactiveObservations } from "./ProcessReactiveObservations.js";
|
||||
import {
|
||||
normalizeProcessSamples,
|
||||
normalizeProcessShimEvent,
|
||||
redactProtocolEvents,
|
||||
} from "./ProcessNormalization.js";
|
||||
import type { TerminalRenderer } from "./TerminalRenderer.js";
|
||||
|
||||
/** Coordinator and subscription owned by one live process capture. */
|
||||
export interface ProcessReactiveHarness {
|
||||
readonly coordinator: ProcessReactiveCoordinator;
|
||||
readonly controls: readonly ProcessReactiveControlRecord[];
|
||||
readonly unsubscribe: () => void;
|
||||
}
|
||||
|
||||
const controlKind = (
|
||||
input: ProcessReactiveInput,
|
||||
): ProcessReactiveControlRecord["kind"] | undefined =>
|
||||
input.kind === "observation" ? undefined : input.kind;
|
||||
|
||||
/** Reject reactive declarations the current process adapter cannot execute. */
|
||||
export const assertSupportedReactiveScenario = (
|
||||
scenario: ProcessScenario,
|
||||
): void => {
|
||||
if (scenario.reactive === null) return;
|
||||
const unsupported = unsupportedProcessReactiveFeatures(scenario.reactive);
|
||||
if (unsupported.length === 0) return;
|
||||
throw new ProcessCaptureError(
|
||||
`reactive scenario requests unsupported features: ${unsupported.join(", ")}`,
|
||||
);
|
||||
};
|
||||
|
||||
/**
|
||||
* Attach the serialized reactive runtime once the target PID is available.
|
||||
*
|
||||
* Subscription replays journal entries admitted during launcher startup before
|
||||
* observing new records, so PID-dependent normalization cannot drop them.
|
||||
*/
|
||||
export const startProcessReactiveHarness = (options: {
|
||||
readonly scenario: ProcessScenario;
|
||||
readonly terminal: () => IPty | undefined;
|
||||
readonly renderer: TerminalRenderer;
|
||||
readonly checkpoints: ProcessCheckpoints;
|
||||
readonly shimReplay: CommandShimReplay;
|
||||
readonly started: number;
|
||||
readonly capture: {
|
||||
readonly interactions: InteractionEvent[];
|
||||
readonly journal: ProcessCaptureJournal;
|
||||
readonly frames: readonly TerminalFrame[];
|
||||
readonly processSamples: readonly ProcessSample[];
|
||||
readonly protocolEvents: () => readonly ProtocolEvent[];
|
||||
readonly temporaryRoot: string;
|
||||
};
|
||||
}): ProcessReactiveHarness | undefined => {
|
||||
if (options.scenario.reactive === null) return undefined;
|
||||
const controls: ProcessReactiveControlRecord[] = [];
|
||||
const coordinator = new ProcessReactiveCoordinator({
|
||||
scenario: options.scenario.reactive,
|
||||
executor: {
|
||||
execute: (actions, signal) =>
|
||||
executeProcessReactiveEffects(
|
||||
{
|
||||
terminal: options.terminal,
|
||||
renderer: options.renderer,
|
||||
checkpoints: options.checkpoints,
|
||||
interactions: options.capture.interactions,
|
||||
journal: options.capture.journal,
|
||||
startedAtMs: options.started,
|
||||
now: Date.now,
|
||||
},
|
||||
actions,
|
||||
signal,
|
||||
),
|
||||
},
|
||||
onDecision: (decision, input) => {
|
||||
const kind = controlKind(input);
|
||||
if (kind === undefined || decision.kind !== "finished") return;
|
||||
controls.push({
|
||||
sequence: controls.length,
|
||||
kind,
|
||||
after_capture_order:
|
||||
decision.snapshot.observations.at(-1)?.capture_order ?? -1,
|
||||
});
|
||||
},
|
||||
});
|
||||
return {
|
||||
coordinator,
|
||||
controls,
|
||||
unsubscribe: subscribeProcessReactiveObservations({
|
||||
journal: options.capture.journal,
|
||||
frames: options.capture.frames,
|
||||
interactions: options.capture.interactions,
|
||||
checkpointAt: (index) => options.checkpoints.at(index),
|
||||
processSampleAt: (index) => {
|
||||
const rootPid = options.terminal()?.pid;
|
||||
return rootPid === undefined
|
||||
? undefined
|
||||
: normalizeProcessSamples(
|
||||
options.capture.processSamples,
|
||||
options.scenario,
|
||||
rootPid,
|
||||
)[index];
|
||||
},
|
||||
protocolEventAt: (index) =>
|
||||
redactProtocolEvents(
|
||||
options.capture.protocolEvents().slice(index, index + 1),
|
||||
options.scenario,
|
||||
)[0],
|
||||
shimEventAt: (index) => {
|
||||
const event = options.shimReplay.events[index];
|
||||
const rootPid = options.terminal()?.pid;
|
||||
return event === undefined || rootPid === undefined
|
||||
? undefined
|
||||
: normalizeProcessShimEvent(
|
||||
event,
|
||||
options.scenario,
|
||||
options.capture.temporaryRoot,
|
||||
rootPid,
|
||||
);
|
||||
},
|
||||
sink: coordinator,
|
||||
}),
|
||||
};
|
||||
};
|
||||
|
||||
/** Project internal reducer state into the stable Process Capture shape. */
|
||||
export const projectProcessReactiveRun = (
|
||||
harness: ProcessReactiveHarness | undefined,
|
||||
): ProcessCapture["reactive_run"] =>
|
||||
harness === undefined
|
||||
? null
|
||||
: {
|
||||
status: harness.coordinator.snapshot.status,
|
||||
outcome: harness.coordinator.snapshot.outcome,
|
||||
active_state: harness.coordinator.snapshot.active_state,
|
||||
transitions: harness.coordinator.snapshot.transitions,
|
||||
controls: harness.controls,
|
||||
};
|
||||
@@ -1,8 +1,17 @@
|
||||
import type {
|
||||
InteractionEvent,
|
||||
FilesystemCheckpoint,
|
||||
ProcessCaptureEventJournalEntry,
|
||||
ProcessSample,
|
||||
ProtocolEvent,
|
||||
ShimEvent,
|
||||
TerminalFrame,
|
||||
} from "../domain/processCapture.js";
|
||||
import { createProcessObservation } from "../domain/processObservation.js";
|
||||
import {
|
||||
createProcessObservation,
|
||||
processObservationSubject,
|
||||
type ProcessObservation,
|
||||
} from "../domain/processObservation.js";
|
||||
import type { ProcessReactiveInput } from "../domain/processReactiveRuntime.js";
|
||||
import type { ProcessCaptureJournal } from "./ProcessCaptureJournal.js";
|
||||
|
||||
@@ -11,25 +20,145 @@ export interface ProcessReactiveObservationSink {
|
||||
enqueue(input: ProcessReactiveInput): void;
|
||||
}
|
||||
|
||||
/** Subscribe a reactive coordinator to normalized raw PTY journal records. */
|
||||
export const subscribeProcessReactiveTerminalObservations = (options: {
|
||||
interface ProcessReactiveObservationOptions {
|
||||
readonly journal: ProcessCaptureJournal;
|
||||
readonly frames: readonly TerminalFrame[];
|
||||
readonly interactions: readonly InteractionEvent[];
|
||||
readonly checkpointAt: (index: number) => FilesystemCheckpoint | undefined;
|
||||
readonly processSampleAt: (index: number) => ProcessSample | undefined;
|
||||
readonly protocolEventAt: (index: number) => ProtocolEvent | undefined;
|
||||
readonly shimEventAt: (index: number) => ShimEvent | undefined;
|
||||
readonly sink: ProcessReactiveObservationSink;
|
||||
}): (() => void) =>
|
||||
options.journal.subscribe((entry: ProcessCaptureEventJournalEntry) => {
|
||||
if (entry.collection !== "frames") return;
|
||||
const frame = options.frames[entry.index];
|
||||
if (frame === undefined) return;
|
||||
options.sink.enqueue({
|
||||
kind: "observation",
|
||||
observation: createProcessObservation({
|
||||
}
|
||||
|
||||
const terminalObservation = (
|
||||
options: ProcessReactiveObservationOptions,
|
||||
entry: ProcessCaptureEventJournalEntry,
|
||||
): ProcessObservation | undefined => {
|
||||
const value = options.frames[entry.index];
|
||||
return value === undefined
|
||||
? undefined
|
||||
: createProcessObservation({
|
||||
source: "terminal_raw",
|
||||
source_sequence: frame.sequence,
|
||||
captured_at_ms: frame.at_ms,
|
||||
source_sequence: value.sequence,
|
||||
captured_at_ms: value.at_ms,
|
||||
subject_id: null,
|
||||
location: entry,
|
||||
payload: frame,
|
||||
}),
|
||||
});
|
||||
payload: value,
|
||||
});
|
||||
};
|
||||
|
||||
const interactionObservation = (
|
||||
options: ProcessReactiveObservationOptions,
|
||||
entry: ProcessCaptureEventJournalEntry,
|
||||
): ProcessObservation | undefined => {
|
||||
const value = options.interactions[entry.index];
|
||||
return value === undefined
|
||||
? undefined
|
||||
: createProcessObservation({
|
||||
source: "interaction",
|
||||
source_sequence: value.sequence,
|
||||
captured_at_ms: value.dispatched_at_ms,
|
||||
subject_id: null,
|
||||
location: entry,
|
||||
payload: value,
|
||||
});
|
||||
};
|
||||
|
||||
const filesystemObservation = (
|
||||
options: ProcessReactiveObservationOptions,
|
||||
entry: ProcessCaptureEventJournalEntry,
|
||||
): ProcessObservation | undefined => {
|
||||
const value = options.checkpointAt(entry.index);
|
||||
return value === undefined
|
||||
? undefined
|
||||
: createProcessObservation({
|
||||
source: "filesystem",
|
||||
source_sequence: entry.index,
|
||||
captured_at_ms: value.at_ms,
|
||||
subject_id: processObservationSubject("filesystem", value),
|
||||
location: entry,
|
||||
payload: value,
|
||||
});
|
||||
};
|
||||
|
||||
const processObservation = (
|
||||
options: ProcessReactiveObservationOptions,
|
||||
entry: ProcessCaptureEventJournalEntry,
|
||||
): ProcessObservation | undefined => {
|
||||
const value = options.processSampleAt(entry.index);
|
||||
return value === undefined
|
||||
? undefined
|
||||
: createProcessObservation({
|
||||
source: "process",
|
||||
source_sequence: entry.index,
|
||||
captured_at_ms: value.at_ms,
|
||||
subject_id: processObservationSubject("process", value),
|
||||
location: entry,
|
||||
payload: value,
|
||||
});
|
||||
};
|
||||
|
||||
const protocolObservation = (
|
||||
options: ProcessReactiveObservationOptions,
|
||||
entry: ProcessCaptureEventJournalEntry,
|
||||
): ProcessObservation | undefined => {
|
||||
const value = options.protocolEventAt(entry.index);
|
||||
return value === undefined
|
||||
? undefined
|
||||
: createProcessObservation({
|
||||
source: value.protocol,
|
||||
source_sequence: value.sequence,
|
||||
captured_at_ms: value.at_ms,
|
||||
subject_id: processObservationSubject(value.protocol, value),
|
||||
location: entry,
|
||||
payload: value,
|
||||
});
|
||||
};
|
||||
|
||||
const shimObservation = (
|
||||
options: ProcessReactiveObservationOptions,
|
||||
entry: ProcessCaptureEventJournalEntry,
|
||||
): ProcessObservation | undefined => {
|
||||
const normalized = options.shimEventAt(entry.index);
|
||||
if (normalized === undefined) return undefined;
|
||||
return createProcessObservation({
|
||||
source: "shim",
|
||||
source_sequence: normalized.sequence,
|
||||
captured_at_ms: normalized.at_ms,
|
||||
subject_id: processObservationSubject("shim", normalized),
|
||||
location: entry,
|
||||
payload: normalized,
|
||||
});
|
||||
};
|
||||
|
||||
const observationFor = (
|
||||
options: ProcessReactiveObservationOptions,
|
||||
entry: ProcessCaptureEventJournalEntry,
|
||||
): ProcessObservation | undefined => {
|
||||
if (entry.collection === "frames") return terminalObservation(options, entry);
|
||||
if (entry.collection === "interaction_events")
|
||||
return interactionObservation(options, entry);
|
||||
if (entry.collection === "filesystem_checkpoints")
|
||||
return filesystemObservation(options, entry);
|
||||
if (entry.collection === "process_samples")
|
||||
return processObservation(options, entry);
|
||||
if (entry.collection === "protocol_events")
|
||||
return protocolObservation(options, entry);
|
||||
if (entry.collection === "shim_events")
|
||||
return shimObservation(options, entry);
|
||||
return undefined;
|
||||
};
|
||||
|
||||
/** Subscribe the reactive coordinator to admitted normalized journal records. */
|
||||
export const subscribeProcessReactiveObservations = (
|
||||
options: ProcessReactiveObservationOptions,
|
||||
): (() => void) => {
|
||||
const enqueue = (entry: ProcessCaptureEventJournalEntry): void => {
|
||||
const observation = observationFor(options, entry);
|
||||
if (observation !== undefined)
|
||||
options.sink.enqueue({ kind: "observation", observation });
|
||||
};
|
||||
for (const entry of options.journal.entries) enqueue(entry);
|
||||
return options.journal.subscribe(enqueue);
|
||||
};
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import type { Readable } from "node:stream";
|
||||
import type { ArtifactCommand } from "../domain/artifactGraph.js";
|
||||
import type { ZipPackageFormat } from "../domain/zipPackageFormat.js";
|
||||
|
||||
/** Archive-neutral entry metadata. Reader adapters never choose output paths. */
|
||||
export interface ArtifactEntry {
|
||||
@@ -23,7 +24,7 @@ export interface ArtifactEntry {
|
||||
|
||||
/** Read-only adapter over one directory, archive, or virtual container. */
|
||||
export interface ArtifactReader {
|
||||
readonly format: "directory" | "zip" | "ipa" | "apk" | "asar" | "file";
|
||||
readonly format: "directory" | ZipPackageFormat | "asar" | "file";
|
||||
entries(signal?: AbortSignal): AsyncIterable<ArtifactEntry>;
|
||||
open(entry: ArtifactEntry, signal?: AbortSignal): Promise<Readable>;
|
||||
provenance(): readonly ArtifactCommand[];
|
||||
|
||||
@@ -9,6 +9,7 @@ import {
|
||||
type ArtifactEntry,
|
||||
type ArtifactReader,
|
||||
} from "./ArtifactReader.js";
|
||||
import type { ZipPackageFormat } from "../domain/zipPackageFormat.js";
|
||||
|
||||
class NodeFileReader extends Reader<string> {
|
||||
#handle: FileHandle | undefined;
|
||||
@@ -41,12 +42,12 @@ class NodeFileReader extends Reader<string> {
|
||||
|
||||
/** Lazy Zip64-capable reader with CRC and overlap verification on every read. */
|
||||
export class ZipArtifactReader implements ArtifactReader {
|
||||
readonly format: "zip" | "ipa" | "apk";
|
||||
readonly format: ZipPackageFormat;
|
||||
readonly #source: NodeFileReader;
|
||||
readonly #reader: ZipReader<string>;
|
||||
readonly #entries = new Map<string, Entry>();
|
||||
|
||||
constructor(path: string, format: "zip" | "ipa" | "apk") {
|
||||
constructor(path: string, format: ZipPackageFormat) {
|
||||
this.format = format;
|
||||
this.#source = new NodeFileReader(path);
|
||||
this.#reader = new ZipReader(this.#source, {
|
||||
|
||||
@@ -16,6 +16,7 @@ export const registerCoreAnalysisCommands = (
|
||||
): void => {
|
||||
registerCoreCommands(cli, logger);
|
||||
registerFunctionCommand(cli, logger);
|
||||
registerInstructionsCommand(cli, logger);
|
||||
registerSearchCommand(cli, logger);
|
||||
registerXrefsCommand(cli, logger);
|
||||
registerTraceCommand(cli, logger);
|
||||
@@ -285,6 +286,54 @@ const registerFunctionCommand = (cli: CliInstance, logger: Logger): void => {
|
||||
});
|
||||
};
|
||||
|
||||
const registerInstructionsCommand = (
|
||||
cli: CliInstance,
|
||||
logger: Logger,
|
||||
): void => {
|
||||
cli.command(CLI_COMMANDS.instructions, {
|
||||
description: "Read one bounded raw-instruction window without decompiling",
|
||||
args: z.object({
|
||||
path: z.string().describe("App or program path"),
|
||||
address: z.string().describe("Procedure name or address"),
|
||||
}),
|
||||
options: z.object({
|
||||
offset: z
|
||||
.number()
|
||||
.int()
|
||||
.min(0)
|
||||
.max(100_000)
|
||||
.default(0)
|
||||
.describe("Zero-based instruction offset"),
|
||||
limit: z
|
||||
.number()
|
||||
.int()
|
||||
.min(1)
|
||||
.max(500)
|
||||
.default(64)
|
||||
.describe("Maximum raw instructions to return"),
|
||||
snapshot: z
|
||||
.string()
|
||||
.min(1)
|
||||
.optional()
|
||||
.describe("Load and update a local analysis snapshot"),
|
||||
provider: providerSelectionOption,
|
||||
}),
|
||||
run: ({ args, options }) =>
|
||||
logCliCommand(logger, CLI_COMMANDS.instructions, () =>
|
||||
runDirectAnalysis(
|
||||
args.path,
|
||||
"read_function_instructions",
|
||||
{
|
||||
procedure: args.address,
|
||||
offset: options.offset,
|
||||
limit: options.limit,
|
||||
},
|
||||
directAnalysisOptions(logger, options.snapshot, options.provider),
|
||||
),
|
||||
),
|
||||
});
|
||||
};
|
||||
|
||||
const registerSearchCommand = (cli: CliInstance, logger: Logger): void => {
|
||||
cli.command(CLI_COMMANDS.search, {
|
||||
description: "Search bounded analyzed strings or procedure names",
|
||||
|
||||
@@ -12,6 +12,7 @@ export const CLI_COMMANDS = Object.freeze({
|
||||
capabilities: "capabilities",
|
||||
providers: "providers",
|
||||
function: "function",
|
||||
instructions: "instructions",
|
||||
search: "search",
|
||||
importReferenceSource: "import-reference-source",
|
||||
inventoryArtifact: "inventory-artifact",
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
import { z } from "zod";
|
||||
|
||||
/** Provider-neutral input for one bounded raw function-instruction window. */
|
||||
export const functionInstructionInputSchema = z.object({
|
||||
procedure: z.string().describe("The procedure name or address"),
|
||||
offset: z.number().int().min(0).max(100_000).default(0),
|
||||
limit: z.number().int().min(1).max(500).default(64),
|
||||
document: z.string().optional().describe("The document name"),
|
||||
});
|
||||
@@ -17,6 +17,7 @@ export const TOOL_EXAMPLE_OVERRIDES: Readonly<
|
||||
procedure_callees: { procedure: "main" },
|
||||
procedure_callers: { procedure: "main" },
|
||||
procedure_info: { procedure: "main" },
|
||||
read_function_instructions: { procedure: "main" },
|
||||
procedure_references: { procedure: "main" },
|
||||
procedure_pseudo_code: { procedure: "main" },
|
||||
resolve_containing_procedure: { address: "0x1000" },
|
||||
|
||||
@@ -37,6 +37,7 @@ import {
|
||||
import type { ToolContract, ToolExample } from "./toolContractTypes.js";
|
||||
import { toolContractMetadata } from "./toolEffects.js";
|
||||
import { binarySessionInputSchema } from "./sessionStatusContract.js";
|
||||
import { functionInstructionInputSchema } from "./functionInstructionContract.js";
|
||||
|
||||
export { binarySessionInputSchema } from "./sessionStatusContract.js";
|
||||
|
||||
@@ -240,6 +241,11 @@ export const OFFICIAL_TOOL_CONTRACTS = [
|
||||
"Return bounded metadata for one procedure identified by symbol or address: entrypoint, signature, locals, size, and block count. Follow with decompilation or assembly for behavior.",
|
||||
z.object({ procedure, document }),
|
||||
),
|
||||
official(
|
||||
"read_function_instructions",
|
||||
"Read one offset-paginated window of raw instructions for an analyzed procedure without decompilation, caller discovery, or whole-program string/name scans. Use this fast path for instruction-level orientation; follow next_offset when truncated.",
|
||||
functionInstructionInputSchema,
|
||||
),
|
||||
official(
|
||||
"procedure_references",
|
||||
"Return a bounded set of raw incoming or outgoing reference edges for one procedure. Endpoint procedures are resolved only from provider containment; Ghidra preserves observed reference kinds while providers without kind authority mark them unavailable.",
|
||||
|
||||
@@ -60,6 +60,7 @@ export const TOOL_EFFECTS: Readonly<Record<string, ToolEffects>> = {
|
||||
procedure_callees: evidence,
|
||||
procedure_callers: evidence,
|
||||
procedure_info: evidence,
|
||||
read_function_instructions: evidence,
|
||||
procedure_references: evidence,
|
||||
procedure_pseudo_code: evidence,
|
||||
resolve_containing_procedure: evidence,
|
||||
|
||||
@@ -6,7 +6,10 @@ import {
|
||||
} from "../domain/processCapture.js";
|
||||
import { evidenceBundleSchema } from "../domain/evidenceBundle.js";
|
||||
import { residualUnknownSchema } from "../domain/residualUnknown.js";
|
||||
import { referenceKindSchema } from "../domain/hopperValues.js";
|
||||
import {
|
||||
functionInstructionWindowSchema,
|
||||
referenceKindSchema,
|
||||
} from "../domain/hopperValues.js";
|
||||
import {
|
||||
demangleSwiftSchema,
|
||||
inspectMachoSchema,
|
||||
@@ -39,6 +42,7 @@ import { replayMachineRunOutputSchema } from "../domain/replayMachineRun.js";
|
||||
import { analysisErrorProjectionSchema } from "./errorSchemas.js";
|
||||
import {
|
||||
addressList,
|
||||
analysisActivity,
|
||||
addressedEntry,
|
||||
bounded,
|
||||
containingProcedureResolution,
|
||||
@@ -82,6 +86,7 @@ export const officialOutputSchemas: Readonly<Record<string, z.ZodObject>> = {
|
||||
procedure_callees: resultOf(addressList),
|
||||
procedure_callers: resultOf(addressList),
|
||||
procedure_info: procedureInfoOutput,
|
||||
read_function_instructions: resultOf(functionInstructionWindowSchema),
|
||||
procedure_references: resultOf(
|
||||
z.object({
|
||||
procedure: procedureIdentity,
|
||||
@@ -297,6 +302,7 @@ export const sessionOutputSchemas: Readonly<Record<string, z.ZodObject>> = {
|
||||
reasons: z.array(z.string()),
|
||||
remediation: z.string().nullable(),
|
||||
}),
|
||||
analysis_activity: analysisActivity,
|
||||
recommended_actions: z.array(z.string()),
|
||||
}),
|
||||
z.object({
|
||||
|
||||
@@ -45,6 +45,8 @@ export const targetFormatSchema = z.enum([
|
||||
"zip",
|
||||
"ipa",
|
||||
"apk",
|
||||
"msix",
|
||||
"appx",
|
||||
"asar",
|
||||
"dmg",
|
||||
"pkg",
|
||||
@@ -64,6 +66,7 @@ const providerCapability = z.object({
|
||||
operation: z.string(),
|
||||
available: z.boolean(),
|
||||
reason: z.string().nullable(),
|
||||
availability_code: z.enum(PROVIDER_REJECTION_CODES).nullable(),
|
||||
input_contract_version: z.number().int().min(1),
|
||||
output_contract_version: z.number().int().min(1),
|
||||
pagination: z.enum(["none", "offset", "cursor"]),
|
||||
@@ -137,6 +140,25 @@ const analysisRun = z
|
||||
})
|
||||
.nullable();
|
||||
|
||||
export const analysisActivity = z.object({
|
||||
status: z.enum(["not_observed", "idle", "busy", "timed_out_busy"]),
|
||||
providers: z.array(
|
||||
z.object({
|
||||
provider: providerIdentity,
|
||||
active: z
|
||||
.object({
|
||||
request_id: z.number().int().min(1),
|
||||
operation: z.string().min(1),
|
||||
elapsed_ms: z.number().int().min(0),
|
||||
timeout_ms: z.number().int().min(0),
|
||||
caller_state: z.enum(["waiting", "timed_out", "cancelled"]),
|
||||
})
|
||||
.nullable(),
|
||||
queued_requests: z.number().int().min(0),
|
||||
}),
|
||||
),
|
||||
});
|
||||
|
||||
const providerRejectionCode: z.ZodType<ProviderRejectionCode> = z.enum(
|
||||
PROVIDER_REJECTION_CODES,
|
||||
);
|
||||
@@ -222,6 +244,7 @@ export const sessionProvider = z.object({
|
||||
providers: z.array(providerIdentity),
|
||||
capabilities: z.array(providerCapability),
|
||||
analysis_run: analysisRun,
|
||||
analysis_activity: analysisActivity,
|
||||
analysis_provider_binding: z
|
||||
.object({
|
||||
provider: providerIdentity,
|
||||
|
||||
@@ -60,6 +60,7 @@ export const projectAnalysisError = (
|
||||
};
|
||||
|
||||
const errorCode = (error: AnalysisError): AnalysisErrorProjection["code"] => {
|
||||
if (error.cleanupIncomplete) return "cleanup_incomplete";
|
||||
if (error instanceof ReplayPlanStaleError) return "plan_stale";
|
||||
if (error instanceof PermissionRequiredError) return "permission_required";
|
||||
if (error instanceof ProviderSelectionError)
|
||||
@@ -295,6 +296,12 @@ const providerErrorDetails = (
|
||||
return {
|
||||
provider_id: error.providerId,
|
||||
operation: error.operation,
|
||||
...(error.cleanupIncomplete
|
||||
? {
|
||||
cleanup: "incomplete",
|
||||
resources: [...error.cleanupResources],
|
||||
}
|
||||
: {}),
|
||||
...(error.diagnostics === undefined
|
||||
? {}
|
||||
: { diagnostics: error.diagnostics }),
|
||||
|
||||
@@ -37,6 +37,8 @@ const formatSchema = z.enum([
|
||||
"zip",
|
||||
"ipa",
|
||||
"apk",
|
||||
"msix",
|
||||
"appx",
|
||||
"asar",
|
||||
"dmg",
|
||||
"pkg",
|
||||
|
||||
@@ -52,6 +52,8 @@ const artifactFormatSchema = z.enum([
|
||||
"analysis-database",
|
||||
"ipa",
|
||||
"apk",
|
||||
"msix",
|
||||
"appx",
|
||||
"plist",
|
||||
"entitlements",
|
||||
"directory",
|
||||
|
||||
@@ -14,6 +14,7 @@ import { promisify } from "node:util";
|
||||
|
||||
import { BinaryTargetError } from "./errors.js";
|
||||
import { err, ok, type Result } from "./result.js";
|
||||
import { zipPackageFormatForPath } from "./zipPackageFormat.js";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
@@ -36,6 +37,8 @@ export interface BinaryTarget {
|
||||
| "zip"
|
||||
| "ipa"
|
||||
| "apk"
|
||||
| "msix"
|
||||
| "appx"
|
||||
| "asar"
|
||||
| "dmg"
|
||||
| "pkg"
|
||||
@@ -131,11 +134,7 @@ const detectArtifactFormat = async (
|
||||
[0x03, 0x05, 0x07].includes(magic[2] ?? -1) &&
|
||||
[0x04, 0x06, 0x08].includes(magic[3] ?? -1)
|
||||
) {
|
||||
return lower.endsWith(".ipa")
|
||||
? "ipa"
|
||||
: lower.endsWith(".apk")
|
||||
? "apk"
|
||||
: "zip";
|
||||
return zipPackageFormatForPath(lower) ?? "zip";
|
||||
}
|
||||
const named = namedArtifactFormat(lower);
|
||||
if (named !== undefined) return named;
|
||||
@@ -167,7 +166,7 @@ const namedArtifactFormat = (
|
||||
};
|
||||
|
||||
const isArchiveFormat = (format: BinaryTarget["format"]): boolean =>
|
||||
["zip", "ipa", "apk", "asar", "dmg", "pkg"].includes(format);
|
||||
["zip", "ipa", "apk", "msix", "appx", "asar", "dmg", "pkg"].includes(format);
|
||||
|
||||
const sha256Handle = async (handle: FileHandle): Promise<string> => {
|
||||
const hash = createHash("sha256");
|
||||
|
||||
@@ -29,6 +29,8 @@ const subjectSchema = z.object({
|
||||
"zip",
|
||||
"ipa",
|
||||
"apk",
|
||||
"msix",
|
||||
"appx",
|
||||
"asar",
|
||||
"dmg",
|
||||
"pkg",
|
||||
|
||||
@@ -143,6 +143,18 @@ const boundedSchema = <T extends z.ZodType>(item: T) =>
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
/** Provider-neutral bounded raw-instruction window for one analyzed function. */
|
||||
export const functionInstructionWindowSchema = z
|
||||
.object({
|
||||
procedure: procedureIdentitySchema,
|
||||
instructions: boundedSchema(z.string()),
|
||||
instructions_scanned: z.number().int().min(0),
|
||||
instruction_scan_truncated: z.boolean(),
|
||||
limitations: z.array(z.string()),
|
||||
})
|
||||
.strict();
|
||||
|
||||
const availableReferenceKindSchema = z
|
||||
.object({
|
||||
available: z.literal(true),
|
||||
|
||||
@@ -5,6 +5,11 @@ import {
|
||||
LEGACY_PROCESS_CAPTURE_MESSAGE,
|
||||
normalizationSchema,
|
||||
} from "./processScenario.js";
|
||||
import type {
|
||||
ProcessReactiveOutcome,
|
||||
ProcessReactiveTransitionRecord,
|
||||
} from "./processReactiveRuntime.js";
|
||||
import { PROCESS_REACTIVE_LIMITS } from "./processReactiveScenario.js";
|
||||
import type { ReplayTransitionRecord } from "./replayMachineRuntime.js";
|
||||
|
||||
export * from "./processScenario.js";
|
||||
@@ -133,6 +138,18 @@ export interface ProcessCaptureEventJournalEntry {
|
||||
readonly index: number;
|
||||
}
|
||||
|
||||
/** Ordered control input that terminated or superseded a reactive run. */
|
||||
export interface ProcessReactiveControlRecord {
|
||||
readonly sequence: number;
|
||||
readonly kind:
|
||||
| "state_deadline"
|
||||
| "scenario_deadline"
|
||||
| "target_lost"
|
||||
| "cancelled"
|
||||
| "cleanup_failed";
|
||||
readonly after_capture_order: number;
|
||||
}
|
||||
|
||||
/** Shared observation callback used by every process-capture producer. */
|
||||
export type RecordProcessCaptureEvent = (
|
||||
collection: ProcessCaptureEventCollection,
|
||||
@@ -185,6 +202,13 @@ export interface ProcessCapture {
|
||||
readonly shim_events: readonly ShimEvent[];
|
||||
readonly protocol_events: readonly ProtocolEvent[];
|
||||
readonly replay_transitions: readonly ReplayTransitionRecord[];
|
||||
readonly reactive_run: {
|
||||
readonly status: "running" | "finished";
|
||||
readonly outcome: ProcessReactiveOutcome | null;
|
||||
readonly active_state: string;
|
||||
readonly transitions: readonly ProcessReactiveTransitionRecord[];
|
||||
readonly controls: readonly ProcessReactiveControlRecord[];
|
||||
} | null;
|
||||
/**
|
||||
* Global observation order across independently recorded collections.
|
||||
*
|
||||
@@ -363,6 +387,75 @@ export const processCaptureSchema: z.ZodType<ProcessCapture> = z.object({
|
||||
}),
|
||||
)
|
||||
.default([]),
|
||||
reactive_run: z
|
||||
.object({
|
||||
status: z.enum(["running", "finished"]),
|
||||
outcome: z
|
||||
.enum([
|
||||
"passed",
|
||||
"predicate_timeout",
|
||||
"scenario_deadline",
|
||||
"ambiguous_match",
|
||||
"action_rejected",
|
||||
"target_lost",
|
||||
"capture_incomplete",
|
||||
"cancelled",
|
||||
"cleanup_failed",
|
||||
])
|
||||
.nullable(),
|
||||
active_state: z.string().min(1).max(64),
|
||||
controls: z
|
||||
.array(
|
||||
z.object({
|
||||
sequence: z.number().int().nonnegative(),
|
||||
kind: z.enum([
|
||||
"state_deadline",
|
||||
"scenario_deadline",
|
||||
"target_lost",
|
||||
"cancelled",
|
||||
"cleanup_failed",
|
||||
]),
|
||||
after_capture_order: z.number().int().min(-1),
|
||||
}),
|
||||
)
|
||||
.default([]),
|
||||
transitions: z
|
||||
.array(
|
||||
z.object({
|
||||
sequence: z.number().int().nonnegative(),
|
||||
transition_id: z.string().min(1).max(64),
|
||||
state_before: z.string().min(1).max(64),
|
||||
state_after: z.string().min(1).max(64).nullable(),
|
||||
outcome: z
|
||||
.enum([
|
||||
"passed",
|
||||
"predicate_timeout",
|
||||
"scenario_deadline",
|
||||
"ambiguous_match",
|
||||
"action_rejected",
|
||||
"target_lost",
|
||||
"capture_incomplete",
|
||||
"cancelled",
|
||||
"cleanup_failed",
|
||||
])
|
||||
.nullable(),
|
||||
trigger_event_ids: z
|
||||
.array(z.string().min(1))
|
||||
.max(PROCESS_REACTIVE_LIMITS.predicates),
|
||||
action_event_ids: z
|
||||
.array(z.string().min(1))
|
||||
.max(PROCESS_REACTIVE_LIMITS.actionsPerTransition),
|
||||
action_types: z
|
||||
.array(
|
||||
z.enum(["send_input", "resize", "send_signal", "checkpoint"]),
|
||||
)
|
||||
.max(PROCESS_REACTIVE_LIMITS.actionsPerTransition),
|
||||
}),
|
||||
)
|
||||
.max(PROCESS_REACTIVE_LIMITS.transitions),
|
||||
})
|
||||
.nullable()
|
||||
.default(null),
|
||||
event_journal: z
|
||||
.array(
|
||||
z.object({
|
||||
|
||||
@@ -0,0 +1,273 @@
|
||||
import canonicalize from "canonicalize";
|
||||
|
||||
import type { ProcessCapture } from "./processCapture.js";
|
||||
import {
|
||||
projectProcessObservation,
|
||||
type ProcessObservation,
|
||||
} from "./processObservation.js";
|
||||
import {
|
||||
commitProcessReactiveProposal,
|
||||
createProcessReactiveSnapshot,
|
||||
reduceProcessReactiveScenario,
|
||||
type ProcessReactiveDecision,
|
||||
type ProcessReactiveInput,
|
||||
type ProcessReactiveSnapshot,
|
||||
} from "./processReactiveRuntime.js";
|
||||
import {
|
||||
processReactiveScenarioSchema,
|
||||
type ProcessReactiveScenario,
|
||||
} from "./processReactiveScenario.js";
|
||||
|
||||
type RequireInvariant = (
|
||||
condition: boolean,
|
||||
path: string,
|
||||
message: string,
|
||||
) => void;
|
||||
type ReactiveRun = NonNullable<ProcessCapture["reactive_run"]>;
|
||||
type ReactiveTransition = ReactiveRun["transitions"][number];
|
||||
|
||||
const admittedCollection = (
|
||||
collection: NonNullable<
|
||||
ProcessCapture["event_journal"]
|
||||
>[number]["collection"],
|
||||
): boolean =>
|
||||
[
|
||||
"frames",
|
||||
"interaction_events",
|
||||
"process_samples",
|
||||
"filesystem_checkpoints",
|
||||
"shim_events",
|
||||
"protocol_events",
|
||||
].includes(collection);
|
||||
|
||||
const reactiveObservations = (
|
||||
capture: ProcessCapture,
|
||||
): readonly ProcessObservation[] =>
|
||||
(capture.event_journal ?? []).flatMap((location) => {
|
||||
if (!admittedCollection(location.collection)) return [];
|
||||
const observation = projectProcessObservation(capture, location);
|
||||
return observation === null ? [] : [observation];
|
||||
});
|
||||
|
||||
const validateTransitionDeclaration = (
|
||||
transition: ReactiveTransition,
|
||||
next: ReactiveTransition | undefined,
|
||||
context: {
|
||||
readonly index: number;
|
||||
readonly scenario: ProcessReactiveScenario;
|
||||
readonly require: RequireInvariant;
|
||||
},
|
||||
): void => {
|
||||
const { index, scenario, require } = context;
|
||||
const path = `reactive_run.transitions[${String(index)}]`;
|
||||
const declared = scenario.states
|
||||
.find(({ id }) => id === transition.state_before)
|
||||
?.on.find(({ id }) => id === transition.transition_id);
|
||||
require(declared !==
|
||||
undefined, path, "reactive transition is not declared by its recorded source state");
|
||||
if (declared !== undefined) {
|
||||
const stateAfter =
|
||||
declared.target.kind === "goto" ? declared.target.state : null;
|
||||
const outcome =
|
||||
declared.target.kind === "finish" ? declared.target.outcome : null;
|
||||
require(transition.state_after === stateAfter &&
|
||||
transition.outcome ===
|
||||
outcome, path, "reactive transition target or outcome differs from the committed graph");
|
||||
require(canonicalize(transition.action_types) ===
|
||||
canonicalize(
|
||||
declared.actions.map(({ type }) => type),
|
||||
), `${path}.action_types`, "reactive transition actions differ from the committed graph");
|
||||
}
|
||||
require(transition.sequence ===
|
||||
index, `${path}.sequence`, "reactive transition sequence must be contiguous from zero");
|
||||
if (next !== undefined)
|
||||
require(transition.state_after ===
|
||||
next.state_before, `reactive_run.transitions[${String(index + 1)}].state_before`, "reactive transition states must be contiguous");
|
||||
};
|
||||
|
||||
const validateTransitionDeclarations = (
|
||||
run: ReactiveRun,
|
||||
scenario: ProcessReactiveScenario,
|
||||
require: RequireInvariant,
|
||||
): void => {
|
||||
require(run.transitions[0]?.state_before === scenario.initial_state ||
|
||||
run.transitions.length ===
|
||||
0, "reactive_run.transitions[0].state_before", "reactive transition journal must start at the declared initial state");
|
||||
for (const [index, transition] of run.transitions.entries())
|
||||
validateTransitionDeclaration(transition, run.transitions[index + 1], {
|
||||
index,
|
||||
scenario,
|
||||
require,
|
||||
});
|
||||
};
|
||||
|
||||
const commitRecordedEffects = (options: {
|
||||
readonly scenario: ProcessReactiveScenario;
|
||||
readonly proposal: Extract<
|
||||
ProcessReactiveDecision,
|
||||
{ readonly kind: "proposal" }
|
||||
>;
|
||||
readonly expected: ReactiveTransition | undefined;
|
||||
readonly observations: ReadonlyMap<string, ProcessObservation>;
|
||||
readonly recordedFailure: "action_rejected" | "target_lost" | undefined;
|
||||
}): ProcessReactiveDecision =>
|
||||
commitProcessReactiveProposal(
|
||||
options.scenario,
|
||||
options.proposal,
|
||||
options.recordedFailure === undefined
|
||||
? options.proposal.effects.map((_, index) => {
|
||||
const eventId = options.expected?.action_event_ids[index];
|
||||
const observation =
|
||||
eventId === undefined
|
||||
? undefined
|
||||
: options.observations.get(eventId);
|
||||
return observation === undefined
|
||||
? { status: "rejected" as const }
|
||||
: { status: "succeeded" as const, observation };
|
||||
})
|
||||
: [
|
||||
options.recordedFailure === "target_lost"
|
||||
? { status: "target_lost" as const }
|
||||
: { status: "rejected" as const },
|
||||
],
|
||||
{ committedSensitiveInputs: true },
|
||||
);
|
||||
|
||||
const replayTransitions = (
|
||||
scenario: ProcessReactiveScenario,
|
||||
run: ReactiveRun,
|
||||
observations: readonly ProcessObservation[],
|
||||
require: RequireInvariant,
|
||||
): ProcessReactiveSnapshot => {
|
||||
const observationsById = new Map(
|
||||
observations.map((observation) => [observation.event_id, observation]),
|
||||
);
|
||||
let snapshot = createProcessReactiveSnapshot(scenario);
|
||||
let transitionIndex = 0;
|
||||
const advance = (input: ProcessReactiveInput): void => {
|
||||
let decision = reduceProcessReactiveScenario(scenario, snapshot, input);
|
||||
if (decision.kind === "proposal") {
|
||||
const proposal = decision;
|
||||
decision = commitRecordedEffects({
|
||||
scenario,
|
||||
proposal: decision,
|
||||
expected: run.transitions[transitionIndex],
|
||||
observations: observationsById,
|
||||
recordedFailure:
|
||||
run.transitions[transitionIndex] === undefined &&
|
||||
decision.effects.length > 0 &&
|
||||
(run.outcome === "action_rejected" || run.outcome === "target_lost")
|
||||
? run.outcome
|
||||
: undefined,
|
||||
});
|
||||
require(decision.kind !== "finished" ||
|
||||
decision.outcome !== "action_rejected" ||
|
||||
run.outcome ===
|
||||
"action_rejected", `reactive_run.transitions[${String(transitionIndex)}].action_event_ids`, `recorded action observations do not match proposed effects: ${canonicalize(
|
||||
{
|
||||
effects: proposal.effects,
|
||||
observations: run.transitions[transitionIndex]?.action_event_ids.map(
|
||||
(eventId) => observationsById.get(eventId),
|
||||
),
|
||||
},
|
||||
)}`);
|
||||
}
|
||||
if (decision.kind === "transition") {
|
||||
const expected = run.transitions[transitionIndex];
|
||||
require(expected !== undefined &&
|
||||
canonicalize(decision.record) ===
|
||||
canonicalize(
|
||||
expected,
|
||||
), `reactive_run.transitions[${String(transitionIndex)}]`, "reactive transition differs from deterministic journal replay");
|
||||
transitionIndex += 1;
|
||||
}
|
||||
snapshot = decision.snapshot;
|
||||
};
|
||||
const advanceControl = (
|
||||
control: ReactiveRun["controls"][number],
|
||||
index: number,
|
||||
): void => {
|
||||
require(snapshot.status !== "finished" ||
|
||||
control.kind ===
|
||||
"cleanup_failed", `reactive_run.controls[${String(index)}]`, "reactive control cannot follow a finished run");
|
||||
advance(
|
||||
control.kind === "state_deadline"
|
||||
? {
|
||||
kind: control.kind,
|
||||
state_id: snapshot.active_state,
|
||||
state_entry_capture_order: snapshot.state_entry_capture_order,
|
||||
}
|
||||
: { kind: control.kind },
|
||||
);
|
||||
};
|
||||
let controlIndex = 0;
|
||||
for (const observation of observations) {
|
||||
while (
|
||||
(run.controls[controlIndex]?.after_capture_order ??
|
||||
Number.MAX_SAFE_INTEGER) < observation.capture_order
|
||||
) {
|
||||
const control = run.controls[controlIndex]!;
|
||||
advanceControl(control, controlIndex);
|
||||
controlIndex += 1;
|
||||
}
|
||||
advance({ kind: "observation", observation });
|
||||
}
|
||||
for (; controlIndex < run.controls.length; controlIndex += 1) {
|
||||
const control = run.controls[controlIndex]!;
|
||||
advanceControl(control, controlIndex);
|
||||
}
|
||||
require(transitionIndex ===
|
||||
run.transitions
|
||||
.length, "reactive_run.transitions", `reactive transition journal is not reproduced by deterministic replay (replayed ${String(transitionIndex)} of ${String(run.transitions.length)}; outcome ${snapshot.outcome})`);
|
||||
return snapshot;
|
||||
};
|
||||
|
||||
const validateControls = (
|
||||
run: ReactiveRun,
|
||||
observations: readonly ProcessObservation[],
|
||||
require: RequireInvariant,
|
||||
): void => {
|
||||
const admittedOrders = new Set([
|
||||
-1,
|
||||
...observations.map(({ capture_order }) => capture_order),
|
||||
]);
|
||||
for (const [index, control] of run.controls.entries()) {
|
||||
require(control.sequence ===
|
||||
index, `reactive_run.controls[${String(index)}].sequence`, "reactive control sequence must be contiguous");
|
||||
require(admittedOrders.has(
|
||||
control.after_capture_order,
|
||||
), `reactive_run.controls[${String(index)}].after_capture_order`, "reactive control cutoff must identify the last admitted observation");
|
||||
require(index === 0 ||
|
||||
control.after_capture_order >=
|
||||
run.controls[index - 1]!
|
||||
.after_capture_order, `reactive_run.controls[${String(index)}].after_capture_order`, "reactive control cutoffs must be ordered");
|
||||
}
|
||||
};
|
||||
|
||||
/** Validate one capture's reactive journal by replaying the production reducer. */
|
||||
export const validateProcessCaptureReactiveRun = (
|
||||
capture: ProcessCapture,
|
||||
require: RequireInvariant,
|
||||
): void => {
|
||||
const declared = capture.manifest.scenario["reactive"];
|
||||
const run = capture.reactive_run;
|
||||
const declaredReactive = declared !== undefined && declared !== null;
|
||||
require(declaredReactive ===
|
||||
(run !==
|
||||
null), "reactive_run", "reactive result presence must match the committed scenario");
|
||||
if (run === null) return;
|
||||
const parsed = processReactiveScenarioSchema.safeParse(declared);
|
||||
require(parsed.success, "manifest.scenario.reactive", "committed reactive scenario does not satisfy its declared schema");
|
||||
if (!parsed.success) return;
|
||||
require(run.status === "finished" &&
|
||||
run.outcome !==
|
||||
null, "reactive_run.outcome", "completed capture requires a finished reactive outcome");
|
||||
validateTransitionDeclarations(run, parsed.data, require);
|
||||
const observations = reactiveObservations(capture);
|
||||
validateControls(run, observations, require);
|
||||
const snapshot = replayTransitions(parsed.data, run, observations, require);
|
||||
require(snapshot.outcome ===
|
||||
run.outcome, "reactive_run.outcome", "reactive outcome differs from deterministic journal replay");
|
||||
require(snapshot.active_state ===
|
||||
run.active_state, "reactive_run.active_state", "reactive active state differs from deterministic journal replay");
|
||||
};
|
||||
@@ -1,4 +1,5 @@
|
||||
import type { ProcessCapture } from "./processCapture.js";
|
||||
import { validateProcessCaptureReactiveRun } from "./processCaptureReactiveValidation.js";
|
||||
import { replayMachineSchema, type ReplayMachine } from "./replayMachine.js";
|
||||
import { digestProcessCommitment } from "./processScenario.js";
|
||||
|
||||
@@ -314,5 +315,6 @@ export const validateProcessCapture = (
|
||||
validateLifecycle(capture, require);
|
||||
validateShimEvents(capture, require);
|
||||
validateReplayEvents(capture, require);
|
||||
validateProcessCaptureReactiveRun(capture, require);
|
||||
return issues;
|
||||
};
|
||||
|
||||
@@ -109,16 +109,29 @@ const atMilliseconds = (value: unknown): number | null => {
|
||||
return null;
|
||||
};
|
||||
|
||||
const subjectFor = (
|
||||
/** Derive the stable capture-local identity for one normalized observation. */
|
||||
export const processObservationSubject = (
|
||||
source: ProcessObservationSource,
|
||||
value: JsonValue,
|
||||
value: unknown,
|
||||
): string | null => {
|
||||
if (typeof value !== "object" || value === null || Array.isArray(value))
|
||||
return null;
|
||||
if (source === "process" && typeof value["pid"] === "number")
|
||||
return `process:${String(value["pid"])}`;
|
||||
if (source === "filesystem" && typeof value["name"] === "string")
|
||||
return `checkpoint:${value["name"]}`;
|
||||
if (source === "process" && "pid" in value && typeof value.pid === "number")
|
||||
return `process:${String(value.pid)}`;
|
||||
if (
|
||||
source === "filesystem" &&
|
||||
"name" in value &&
|
||||
typeof value.name === "string"
|
||||
)
|
||||
return `checkpoint:${value.name}`;
|
||||
if (
|
||||
source === "shim" &&
|
||||
"command" in value &&
|
||||
typeof value.command === "string"
|
||||
)
|
||||
return "route_index" in value && typeof value.route_index === "number"
|
||||
? `shim:${value.command}:route:${String(value.route_index)}`
|
||||
: `shim:${value.command}:unmatched`;
|
||||
return null;
|
||||
};
|
||||
|
||||
@@ -188,7 +201,7 @@ const projectedRecord = (
|
||||
return {
|
||||
source: projected.source,
|
||||
captured_at_ms: atMilliseconds(payload),
|
||||
subject_id: subjectFor(projected.source, payload),
|
||||
subject_id: processObservationSubject(projected.source, payload),
|
||||
payload,
|
||||
};
|
||||
};
|
||||
|
||||
@@ -65,6 +65,7 @@ export type ProcessReactiveInput =
|
||||
readonly state_entry_capture_order: number;
|
||||
}
|
||||
| { readonly kind: "scenario_deadline" }
|
||||
| { readonly kind: "target_lost" }
|
||||
| { readonly kind: "cancelled" }
|
||||
| { readonly kind: "cleanup_failed" };
|
||||
|
||||
@@ -123,6 +124,33 @@ export type ProcessReactiveDecision =
|
||||
const counterValue = (counters: readonly Counter[], id: string): number =>
|
||||
counters.find((counter) => counter.id === id)?.count ?? 0;
|
||||
|
||||
type ObservationAdmission =
|
||||
| { readonly kind: "new"; readonly snapshot: ProcessReactiveSnapshot }
|
||||
| { readonly kind: "invalid" };
|
||||
|
||||
const observeInput = (
|
||||
snapshot: ProcessReactiveSnapshot,
|
||||
observation: ProcessObservation,
|
||||
): ObservationAdmission => {
|
||||
const duplicate = snapshot.observations.find(
|
||||
({ event_id }) => event_id === observation.event_id,
|
||||
);
|
||||
if (duplicate !== undefined) return { kind: "invalid" };
|
||||
const prior = snapshot.observations.at(-1);
|
||||
if (
|
||||
(prior !== undefined && observation.capture_order <= prior.capture_order) ||
|
||||
snapshot.observations.length >= PROCESS_REACTIVE_LIMITS.retainedObservations
|
||||
)
|
||||
return { kind: "invalid" };
|
||||
return {
|
||||
kind: "new",
|
||||
snapshot: {
|
||||
...snapshot,
|
||||
observations: [...snapshot.observations, observation],
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
/** Create the initial immutable reducer state for one validated scenario. */
|
||||
export const createProcessReactiveSnapshot = (
|
||||
scenario: ProcessReactiveScenario,
|
||||
@@ -166,6 +194,8 @@ const reduceControlInput = (
|
||||
: { kind: "waiting", snapshot, evaluations: [] };
|
||||
if (input.kind === "scenario_deadline")
|
||||
return finishedDecision(snapshot, "scenario_deadline");
|
||||
if (input.kind === "target_lost")
|
||||
return finishedDecision(snapshot, "target_lost");
|
||||
return finishedDecision(snapshot, "cancelled");
|
||||
};
|
||||
|
||||
@@ -179,23 +209,10 @@ export const reduceProcessReactiveScenario = (
|
||||
if (controlDecision !== null) return controlDecision;
|
||||
if (input.kind !== "observation")
|
||||
return finishedDecision(snapshot, "capture_incomplete");
|
||||
const prior = snapshot.observations.at(-1);
|
||||
if (
|
||||
(prior !== undefined &&
|
||||
input.observation.capture_order <= prior.capture_order) ||
|
||||
snapshot.observations.some(
|
||||
({ event_id }) => event_id === input.observation.event_id,
|
||||
)
|
||||
)
|
||||
const admitted = observeInput(snapshot, input.observation);
|
||||
if (admitted.kind === "invalid")
|
||||
return finishedDecision(snapshot, "capture_incomplete");
|
||||
if (
|
||||
snapshot.observations.length >= PROCESS_REACTIVE_LIMITS.retainedObservations
|
||||
)
|
||||
return finishedDecision(snapshot, "capture_incomplete");
|
||||
const observed = {
|
||||
...snapshot,
|
||||
observations: [...snapshot.observations, input.observation],
|
||||
};
|
||||
const observed = admitted.snapshot;
|
||||
const state = scenario.states.find(({ id }) => id === observed.active_state);
|
||||
if (state === undefined) return finishedDecision(observed, "target_lost");
|
||||
const budget = { remaining: PROCESS_REACTIVE_LIMITS.evaluationWork };
|
||||
|
||||
@@ -124,6 +124,7 @@ const applyProcessReactiveTransition = (input: {
|
||||
const effectObservationMatchesAction = (
|
||||
action: ProcessReactiveScenario["states"][number]["on"][number]["actions"][number],
|
||||
observation: ProcessObservation,
|
||||
committedSensitiveInputs: boolean,
|
||||
): boolean => {
|
||||
const payload =
|
||||
typeof observation.payload === "object" &&
|
||||
@@ -153,9 +154,11 @@ const effectObservationMatchesAction = (
|
||||
return (
|
||||
payload["type"] === "input" &&
|
||||
payload["data"] ===
|
||||
(action.sensitive
|
||||
? `<redacted-input:${String(Buffer.byteLength(action.data))}-bytes>`
|
||||
: action.data)
|
||||
expectedInputEvidence(
|
||||
action.data,
|
||||
action.sensitive,
|
||||
committedSensitiveInputs,
|
||||
)
|
||||
);
|
||||
if (action.type === "resize")
|
||||
return (
|
||||
@@ -165,6 +168,16 @@ const effectObservationMatchesAction = (
|
||||
return payload["type"] === "signal" && payload["data"] === action.signal;
|
||||
};
|
||||
|
||||
const expectedInputEvidence = (
|
||||
data: string,
|
||||
sensitive: boolean,
|
||||
committedSensitiveInput: boolean,
|
||||
): string =>
|
||||
!sensitive ||
|
||||
(committedSensitiveInput && /^<redacted-input:\d+-bytes>$/.test(data))
|
||||
? data
|
||||
: `<redacted-input:${String(Buffer.byteLength(data))}-bytes>`;
|
||||
|
||||
const failedProposal = (
|
||||
proposed: Extract<ProcessReactiveDecision, { readonly kind: "proposal" }>,
|
||||
outcome: "action_rejected" | "target_lost",
|
||||
@@ -180,6 +193,7 @@ export const commitProcessReactiveProposal = (
|
||||
scenario: ProcessReactiveScenario,
|
||||
proposed: Extract<ProcessReactiveDecision, { readonly kind: "proposal" }>,
|
||||
results: readonly ProcessReactiveEffectResult[],
|
||||
options: { readonly committedSensitiveInputs?: boolean } = {},
|
||||
): ProcessReactiveDecision => {
|
||||
if (results.some(({ status }) => status === "target_lost"))
|
||||
return failedProposal(proposed, "target_lost");
|
||||
@@ -227,7 +241,11 @@ export const commitProcessReactiveProposal = (
|
||||
const effectObservation = result.observation;
|
||||
if (
|
||||
action === undefined ||
|
||||
!effectObservationMatchesAction(action, effectObservation) ||
|
||||
!effectObservationMatchesAction(
|
||||
action,
|
||||
effectObservation,
|
||||
options.committedSensitiveInputs ?? false,
|
||||
) ||
|
||||
!Number.isSafeInteger(effectObservation.capture_order) ||
|
||||
effectObservation.capture_order <= previousOrder ||
|
||||
knownEventIds.has(effectObservation.event_id)
|
||||
|
||||
@@ -2,6 +2,10 @@ import { createHash } from "node:crypto";
|
||||
import { resolve } from "node:path";
|
||||
import canonicalize from "canonicalize";
|
||||
import { z } from "zod";
|
||||
import {
|
||||
processReactiveScenarioSchema,
|
||||
type ProcessReactiveScenario,
|
||||
} from "./processReactiveScenario.js";
|
||||
import { replayMachineSchema } from "./replayMachine.js";
|
||||
|
||||
const positiveBudget = z.number().int().positive();
|
||||
@@ -51,6 +55,29 @@ export const digestProcessCommitment = (value: unknown): string => {
|
||||
return createHash("sha256").update(serialized).digest("hex");
|
||||
};
|
||||
|
||||
const reactiveScenarioCommitment = (
|
||||
scenario: ProcessReactiveScenario | null,
|
||||
): Readonly<Record<string, unknown>> | null =>
|
||||
scenario === null
|
||||
? null
|
||||
: {
|
||||
...scenario,
|
||||
states: scenario.states.map((state) => ({
|
||||
...state,
|
||||
on: state.on.map((transition) => ({
|
||||
...transition,
|
||||
actions: transition.actions.map((action) =>
|
||||
action.type === "send_input" && action.sensitive
|
||||
? {
|
||||
...action,
|
||||
data: `<redacted-input:${String(Buffer.byteLength(action.data))}-bytes>`,
|
||||
}
|
||||
: action,
|
||||
),
|
||||
})),
|
||||
})),
|
||||
};
|
||||
|
||||
/** Build the secret-safe scenario projection committed by Process Capture v4. */
|
||||
export const processScenarioCommitment = (
|
||||
scenario: ProcessScenario,
|
||||
@@ -63,6 +90,7 @@ export const processScenarioCommitment = (
|
||||
scenario.secret_aliases.includes(name) ? "<redacted-secret>" : value,
|
||||
]),
|
||||
),
|
||||
reactive: reactiveScenarioCommitment(scenario.reactive),
|
||||
unknown_registry_approved: scenario.unknown_registry_approved === true,
|
||||
executable_sha256: executableSha256 ?? null,
|
||||
});
|
||||
@@ -86,6 +114,7 @@ export const processComparisonContract = (
|
||||
normalization: scenario.normalization,
|
||||
command_shims: scenario.command_shims,
|
||||
replay: scenario.replay,
|
||||
reactive: reactiveScenarioCommitment(scenario.reactive),
|
||||
});
|
||||
|
||||
/**
|
||||
@@ -292,6 +321,7 @@ export const processScenarioSchema = z
|
||||
websocket_messages: [],
|
||||
websocket_connections: [],
|
||||
}),
|
||||
reactive: processReactiveScenarioSchema.nullable().default(null),
|
||||
})
|
||||
.strict()
|
||||
.superRefine((scenario, context) => {
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
import { ProviderAdapterError } from "./errors.js";
|
||||
import type { JsonValue } from "./jsonValue.js";
|
||||
|
||||
/** Provider resources could not be proven closed after bounded cleanup. */
|
||||
export class ProviderCleanupError extends ProviderAdapterError {
|
||||
override readonly cleanupIncomplete = true;
|
||||
override readonly cleanupResources: readonly string[];
|
||||
override readonly userMessage =
|
||||
"Provider cleanup could not be fully confirmed. Review the reported local resources before opening another target.";
|
||||
|
||||
constructor(
|
||||
providerId: string,
|
||||
resources: readonly string[],
|
||||
diagnostics: Readonly<Record<string, JsonValue>>,
|
||||
options?: ErrorOptions,
|
||||
) {
|
||||
super(providerId, "close_binary", { ...options, diagnostics });
|
||||
this.cleanupResources = [...resources];
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
/** ZIP-backed application package families that share the hardened archive reader. */
|
||||
export type ZipPackageFormat = "zip" | "ipa" | "apk" | "msix" | "appx";
|
||||
|
||||
/**
|
||||
* Classify a ZIP-backed package by its complete lower-cased path suffix.
|
||||
*
|
||||
* This is deliberately extension-only. Callers must separately verify ZIP
|
||||
* magic before trusting the classification for a root input.
|
||||
*/
|
||||
export const zipPackageFormatForPath = (
|
||||
path: string,
|
||||
): ZipPackageFormat | undefined => {
|
||||
const lower = path.toLowerCase();
|
||||
if (lower.endsWith(".zip")) return "zip";
|
||||
if (lower.endsWith(".ipa")) return "ipa";
|
||||
if (lower.endsWith(".apk") || lower.endsWith(".aab")) return "apk";
|
||||
if (lower.endsWith(".msix") || lower.endsWith(".msixbundle")) return "msix";
|
||||
if (lower.endsWith(".appx") || lower.endsWith(".appxbundle")) return "appx";
|
||||
return undefined;
|
||||
};
|
||||
@@ -1,7 +1,7 @@
|
||||
/** Generated from package.json and package-lock.json; do not edit. */
|
||||
export const PACKAGE_METADATA = {
|
||||
name: "rea-agents",
|
||||
version: "2.3.0", // x-release-please-version
|
||||
version: "2.4.1", // x-release-please-version
|
||||
serverSdkVersion: "2.0.0-beta.4",
|
||||
clientSdkVersion: "2.0.0-beta.4",
|
||||
coreSdkVersion: "2.0.0-beta.4",
|
||||
|
||||
@@ -17,4 +17,4 @@ export const GHIDRA_MAX_QUEUED_REQUESTS = 32;
|
||||
/** Maximum encoded response line accepted from the Java bridge. */
|
||||
export const GHIDRA_MAX_LINE_BYTES = 1024 * 1024;
|
||||
/** Version of REA's Java bridge handshake and request protocol. */
|
||||
export const GHIDRA_BRIDGE_VERSION = 4;
|
||||
export const GHIDRA_BRIDGE_VERSION = 5;
|
||||
|
||||
@@ -2,6 +2,7 @@ import { z } from "zod";
|
||||
|
||||
import { AnalysisInputError, AnalysisOutputError } from "../domain/errors.js";
|
||||
import {
|
||||
functionInstructionWindowSchema,
|
||||
functionDossierSchema,
|
||||
type FunctionDossier,
|
||||
} from "../domain/hopperValues.js";
|
||||
@@ -24,6 +25,7 @@ export const GHIDRA_FUNCTION_OPERATIONS = [
|
||||
"procedure_callers",
|
||||
"procedure_info",
|
||||
"procedure_pseudo_code",
|
||||
"read_function_instructions",
|
||||
"procedure_references",
|
||||
"xrefs",
|
||||
] as const;
|
||||
@@ -88,6 +90,13 @@ const inputSchemas = {
|
||||
procedure_callers: z.object(directProcedure).strict(),
|
||||
procedure_info: z.object(directProcedure).strict(),
|
||||
procedure_pseudo_code: z.object(directProcedure).strict(),
|
||||
read_function_instructions: z
|
||||
.object({
|
||||
...directProcedure,
|
||||
offset: z.number().int().min(0).max(100_000).default(0),
|
||||
limit: z.number().int().min(1).max(500).default(64),
|
||||
})
|
||||
.strict(),
|
||||
procedure_references: z
|
||||
.object({
|
||||
...directProcedure,
|
||||
@@ -281,6 +290,9 @@ const ghidraFunctionDossier = functionDossierSchema.superRefine(
|
||||
});
|
||||
},
|
||||
);
|
||||
const ghidraFunctionInstructionWindow = functionInstructionWindowSchema
|
||||
.extend({ procedure: procedureIdentity })
|
||||
.strict();
|
||||
|
||||
const resultSchemas = {
|
||||
analyze_function: ghidraFunctionDossier,
|
||||
@@ -289,6 +301,7 @@ const resultSchemas = {
|
||||
procedure_callers: z.array(ghidraCanonicalAddressSchema),
|
||||
procedure_info: procedureInfo,
|
||||
procedure_pseudo_code: z.string().nullable(),
|
||||
read_function_instructions: ghidraFunctionInstructionWindow,
|
||||
procedure_references: procedureReferences,
|
||||
xrefs: z.array(ghidraCanonicalAddressSchema),
|
||||
} satisfies Readonly<Record<GhidraFunctionOperation, z.ZodType>>;
|
||||
|
||||
@@ -49,6 +49,7 @@ const PAGINATED_OPERATIONS: ReadonlySet<string> = new Set([
|
||||
"search_procedures",
|
||||
"search_strings",
|
||||
"procedure_references",
|
||||
"read_function_instructions",
|
||||
"analyze_function",
|
||||
]);
|
||||
const SEARCH_OPERATIONS: ReadonlySet<string> = new Set([
|
||||
@@ -119,6 +120,12 @@ export const limitationsFor = (operation: string): readonly string[] => {
|
||||
"Pseudocode is Ghidra decompiler output, not original source and not text-equivalent to Hopper output; each decompile has a 30-second native deadline.",
|
||||
"External functions and functions without an analyzable body return null; other decompiler failures remain explicit.",
|
||||
];
|
||||
case "read_function_instructions":
|
||||
return [
|
||||
...common,
|
||||
"This bounded fast path reads only the requested function instruction window and does not invoke the decompiler or whole-program string/name inventories.",
|
||||
"Instruction text is Ghidra-specific and does not claim textual equivalence with Hopper output.",
|
||||
];
|
||||
case "procedure_assembly":
|
||||
return [
|
||||
...common,
|
||||
|
||||
@@ -0,0 +1,265 @@
|
||||
import type { Socket } from "node:net";
|
||||
|
||||
import type { ProgressReporter } from "../application/ProgressReporter.js";
|
||||
import type { AnalysisError, HopperError } from "../domain/errors.js";
|
||||
import { HopperProcessError } from "../domain/errors.js";
|
||||
import type { JsonValue } from "../domain/jsonValue.js";
|
||||
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import type { Logger } from "../logger.js";
|
||||
import type { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
|
||||
import type { ProcessCleanupResult } from "../process/ProcessOwnership.js";
|
||||
import type { ProviderProcessSupervisor } from "../process/ProviderProcess.js";
|
||||
import type { BridgeLaunch } from "./BridgeLauncher.js";
|
||||
import {
|
||||
createOwnedHopperShutdownDiagnostic,
|
||||
type HopperDiagnostic,
|
||||
providerCleanupFailure,
|
||||
} from "./HopperDiagnostics.js";
|
||||
import type { HopperRequestActivity } from "./HopperRequestQueue.js";
|
||||
import {
|
||||
isHopperCleanupRequired,
|
||||
isHopperShutdownAcknowledgement,
|
||||
} from "./HopperSessionValues.js";
|
||||
|
||||
interface CleanupIssue {
|
||||
readonly resource: string;
|
||||
readonly reason: string;
|
||||
}
|
||||
|
||||
interface CleanupState {
|
||||
readonly issues: CleanupIssue[];
|
||||
readonly resources: Set<string>;
|
||||
cleanupResult: ProcessCleanupResult | undefined;
|
||||
shutdownConfirmed: boolean;
|
||||
ownedProcessStopped: boolean;
|
||||
}
|
||||
|
||||
export interface HopperCleanupInput {
|
||||
readonly socket: Socket | undefined;
|
||||
readonly launch: BridgeLaunch | undefined;
|
||||
readonly processSupervisor: ProviderProcessSupervisor | undefined;
|
||||
readonly runtimeRoot: PrivateRuntimeRoot | undefined;
|
||||
readonly activeRequest: HopperRequestActivity | null;
|
||||
readonly progress: ProgressReporter | undefined;
|
||||
readonly logger: Logger;
|
||||
readonly onDiagnostic: ((event: HopperDiagnostic) => void) | undefined;
|
||||
request(
|
||||
method: "shutdown" | "shutdown_document",
|
||||
): Promise<Result<JsonValue, HopperError>>;
|
||||
releaseTransport(socket: Socket | undefined): void;
|
||||
}
|
||||
|
||||
/** Close Hopper phases and retain every resource whose cleanup is unverified. */
|
||||
export const cleanupHopperSession = async (
|
||||
input: HopperCleanupInput,
|
||||
): Promise<Result<null, AnalysisError>> => {
|
||||
const state: CleanupState = {
|
||||
issues: [],
|
||||
resources: new Set(),
|
||||
cleanupResult: undefined,
|
||||
shutdownConfirmed: false,
|
||||
ownedProcessStopped: false,
|
||||
};
|
||||
await report(input.progress, 0, "requesting Hopper document shutdown");
|
||||
await requestShutdown(input, state);
|
||||
await report(input.progress, 0.35, "releasing Hopper bridge transport");
|
||||
input.releaseTransport(input.socket);
|
||||
await stopProcess(input, state);
|
||||
recordUnconfirmedDocument(input, state);
|
||||
await report(input.progress, 0.75, "removing Hopper private runtime files");
|
||||
await closeRuntimeRoot(input.runtimeRoot, state);
|
||||
return cleanupOutcome(input, state);
|
||||
};
|
||||
|
||||
const requestShutdown = async (
|
||||
input: HopperCleanupInput,
|
||||
state: CleanupState,
|
||||
): Promise<void> => {
|
||||
if (
|
||||
input.socket === undefined ||
|
||||
input.socket.destroyed ||
|
||||
input.activeRequest !== null
|
||||
) {
|
||||
if (input.activeRequest !== null)
|
||||
input.logger.warn(
|
||||
{
|
||||
method: input.activeRequest.operation,
|
||||
callerState: input.activeRequest.callerState,
|
||||
},
|
||||
"Hopper shutdown skipped because a bridge operation remained active",
|
||||
);
|
||||
return;
|
||||
}
|
||||
const shutdown = await input
|
||||
.request("shutdown")
|
||||
.catch(() => err(new HopperProcessError(null)));
|
||||
if (
|
||||
shutdown.ok &&
|
||||
isHopperCleanupRequired(shutdown.value) &&
|
||||
input.launch?.shutdownByCleanup === true &&
|
||||
input.launch.cleanup !== undefined
|
||||
) {
|
||||
state.cleanupResult = await input.launch
|
||||
.cleanup()
|
||||
.catch(providerCleanupFailure);
|
||||
if (!state.cleanupResult.cleaned) {
|
||||
state.shutdownConfirmed = await fallbackDocumentShutdown(input);
|
||||
state.cleanupResult = await input.launch
|
||||
.cleanup()
|
||||
.catch(providerCleanupFailure);
|
||||
}
|
||||
} else if (shutdown.ok) {
|
||||
state.shutdownConfirmed = isHopperShutdownAcknowledgement(shutdown.value);
|
||||
}
|
||||
if (
|
||||
!shutdown.ok ||
|
||||
(!state.shutdownConfirmed && state.cleanupResult === undefined)
|
||||
)
|
||||
input.logger.warn(
|
||||
{ status: shutdown.ok ? "invalid-acknowledgement" : "failed" },
|
||||
"Hopper document shutdown was not confirmed",
|
||||
);
|
||||
};
|
||||
|
||||
const fallbackDocumentShutdown = async (
|
||||
input: HopperCleanupInput,
|
||||
): Promise<boolean> => {
|
||||
const fallback = await input
|
||||
.request("shutdown_document")
|
||||
.catch(() => err(new HopperProcessError(null)));
|
||||
const confirmed =
|
||||
fallback.ok && isHopperShutdownAcknowledgement(fallback.value);
|
||||
if (!confirmed)
|
||||
input.logger.warn(
|
||||
{ status: fallback.ok ? "invalid-acknowledgement" : "failed" },
|
||||
"Hopper document shutdown fallback was not confirmed",
|
||||
);
|
||||
return confirmed;
|
||||
};
|
||||
|
||||
const stopProcess = async (
|
||||
input: HopperCleanupInput,
|
||||
state: CleanupState,
|
||||
): Promise<void> => {
|
||||
const supervisor = input.processSupervisor;
|
||||
if (supervisor === undefined) return;
|
||||
const stopped = await supervisor.stop(
|
||||
state.cleanupResult === undefined
|
||||
? {}
|
||||
: { cleanupResult: state.cleanupResult },
|
||||
);
|
||||
const diagnostic = createOwnedHopperShutdownDiagnostic(
|
||||
supervisor.launch,
|
||||
stopped,
|
||||
state.cleanupResult,
|
||||
);
|
||||
try {
|
||||
input.onDiagnostic?.(diagnostic);
|
||||
} catch {
|
||||
// Diagnostic consumers cannot change the already-observed cleanup result.
|
||||
}
|
||||
input.logger.info(diagnostic, "Owned Hopper launcher shutdown completed");
|
||||
if (stopped.status !== "incomplete") {
|
||||
state.ownedProcessStopped = stopped.status !== "not-owned";
|
||||
return;
|
||||
}
|
||||
const processGroupId = supervisor.launch.ownership?.processGroupId;
|
||||
const resource =
|
||||
processGroupId === undefined
|
||||
? "hopper-process"
|
||||
: `process-group:${String(processGroupId)}`;
|
||||
state.resources.add(resource);
|
||||
state.issues.push({ resource, reason: stopped.reason });
|
||||
input.logger.warn(
|
||||
{ reason: stopped.reason },
|
||||
"Owned launcher cleanup failed closed",
|
||||
);
|
||||
};
|
||||
|
||||
const recordUnconfirmedDocument = (
|
||||
input: HopperCleanupInput,
|
||||
state: CleanupState,
|
||||
): void => {
|
||||
if (
|
||||
(input.socket === undefined && input.launch === undefined) ||
|
||||
state.shutdownConfirmed ||
|
||||
state.ownedProcessStopped
|
||||
)
|
||||
return;
|
||||
state.resources.add("hopper-document");
|
||||
state.issues.push({
|
||||
resource: "hopper-document",
|
||||
reason:
|
||||
input.activeRequest === null
|
||||
? "authenticated shutdown acknowledgement was not observed"
|
||||
: `bridge operation ${input.activeRequest.operation} remained active after caller ${input.activeRequest.callerState}`,
|
||||
});
|
||||
};
|
||||
|
||||
const closeRuntimeRoot = async (
|
||||
runtimeRoot: PrivateRuntimeRoot | undefined,
|
||||
state: CleanupState,
|
||||
): Promise<void> => {
|
||||
try {
|
||||
await runtimeRoot?.close();
|
||||
} catch (cause: unknown) {
|
||||
const resource = runtimeRoot?.path ?? "hopper-runtime-root";
|
||||
state.resources.add(resource);
|
||||
state.issues.push({
|
||||
resource,
|
||||
reason:
|
||||
cause instanceof Error
|
||||
? cause.message
|
||||
: "private runtime root cleanup failed",
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
const cleanupOutcome = (
|
||||
input: HopperCleanupInput,
|
||||
state: CleanupState,
|
||||
): Result<null, AnalysisError> => {
|
||||
if (state.issues.length === 0) return ok(null);
|
||||
return err(
|
||||
new ProviderCleanupError("hopper", [...state.resources], {
|
||||
issues: cleanupIssueDetails(state.issues),
|
||||
...(input.runtimeRoot === undefined
|
||||
? {}
|
||||
: { runtime_root: input.runtimeRoot.path }),
|
||||
...(input.activeRequest === null
|
||||
? {}
|
||||
: { active_request: activityDetails(input.activeRequest) }),
|
||||
}),
|
||||
);
|
||||
};
|
||||
|
||||
const cleanupIssueDetails = (issues: readonly CleanupIssue[]): JsonValue =>
|
||||
issues.map((issue) => ({
|
||||
resource: issue.resource,
|
||||
reason: issue.reason,
|
||||
}));
|
||||
|
||||
const activityDetails = (activity: HopperRequestActivity): JsonValue => ({
|
||||
request_id: activity.requestId,
|
||||
operation: activity.operation,
|
||||
elapsed_ms: activity.elapsedMs,
|
||||
timeout_ms: activity.timeoutMs,
|
||||
caller_state: activity.callerState,
|
||||
queued_requests: activity.queuedRequests,
|
||||
});
|
||||
|
||||
const report = async (
|
||||
progress: ProgressReporter | undefined,
|
||||
completed: number,
|
||||
message: string,
|
||||
): Promise<void> => {
|
||||
await progress
|
||||
?.report({
|
||||
phase: "hopper_cleanup",
|
||||
completed,
|
||||
total: 1,
|
||||
message,
|
||||
})
|
||||
.catch(() => undefined);
|
||||
};
|
||||
+86
-140
@@ -6,6 +6,7 @@ import { join } from "node:path";
|
||||
|
||||
import {
|
||||
HopperCancelledError,
|
||||
type AnalysisError,
|
||||
type HopperError,
|
||||
HopperProcessError,
|
||||
HopperProtocolError,
|
||||
@@ -16,36 +17,35 @@ import {
|
||||
} from "../domain/errors.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import type { JsonValue } from "../domain/jsonValue.js";
|
||||
import type { ProgressReporter } from "../application/ProgressReporter.js";
|
||||
import { silentLogger, type Logger } from "../logger.js";
|
||||
import { PendingOperations } from "../process/PendingOperations.js";
|
||||
import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
|
||||
import { ProviderStartupDeadline } from "../process/ProviderDeadline.js";
|
||||
import type { ProcessCleanupResult } from "../process/ProcessOwnership.js";
|
||||
import { ProviderRunLineage } from "../process/ProviderRunLineage.js";
|
||||
import {
|
||||
type ProviderProcessDiagnostic,
|
||||
ProviderProcessSupervisor,
|
||||
} from "../process/ProviderProcess.js";
|
||||
import type { BridgeLaunch, BridgeLauncher } from "./BridgeLauncher.js";
|
||||
import type { HopperDiagnostic } from "./HopperDiagnostics.js";
|
||||
import { cleanupHopperSession } from "./HopperCleanup.js";
|
||||
import {
|
||||
createOwnedHopperShutdownDiagnostic,
|
||||
type HopperDiagnostic,
|
||||
providerCleanupFailure,
|
||||
} from "./HopperDiagnostics.js";
|
||||
import {
|
||||
isHopperCleanupRequired,
|
||||
isHopperShutdownAcknowledgement,
|
||||
parseHopperServerInfo,
|
||||
type HopperServerInfo,
|
||||
} from "./HopperSessionValues.js";
|
||||
import { connectHopperSocketOnce } from "./HopperSocketConnection.js";
|
||||
import { hopperLauncherFailureDiagnostic } from "./HopperProcessDiagnostic.js";
|
||||
import { parseResponseLine, responseResult } from "./protocol.js";
|
||||
import {
|
||||
HopperRequestQueue,
|
||||
type HopperRequestActivity,
|
||||
} from "./HopperRequestQueue.js";
|
||||
import { HopperResponseStream } from "./HopperResponseStream.js";
|
||||
import { responseResult } from "./protocol.js";
|
||||
|
||||
export type { HopperServerInfo } from "./HopperSessionValues.js";
|
||||
|
||||
const MAX_LINE_BYTES = 10 * 1024 * 1024;
|
||||
const SHUTDOWN_TIMEOUT_MS = 30_000;
|
||||
const MAX_QUEUED_REQUESTS = 64;
|
||||
const SESSION_ROOT = process.platform === "darwin" ? "/tmp" : tmpdir();
|
||||
|
||||
/** Dependencies, deadlines, and redacted diagnostics for one bridge client. */
|
||||
@@ -72,10 +72,8 @@ export class HopperClient {
|
||||
Pick<HopperClientOptions, "requestTimeoutMs" | "startupTimeoutMs">
|
||||
> &
|
||||
HopperClientOptions;
|
||||
readonly #pending = new PendingOperations<
|
||||
number,
|
||||
Result<JsonValue, HopperError>
|
||||
>();
|
||||
readonly #requests: HopperRequestQueue;
|
||||
readonly #responses: HopperResponseStream;
|
||||
readonly #logger: Logger;
|
||||
#socket: Socket | undefined;
|
||||
#launch: BridgeLaunch | undefined;
|
||||
@@ -84,15 +82,14 @@ export class HopperClient {
|
||||
#token: string | undefined;
|
||||
readonly #lineage = new ProviderRunLineage();
|
||||
#nextId = 1;
|
||||
#buffer = "";
|
||||
#closing = false;
|
||||
#launcherExitCode: number | null | undefined;
|
||||
#launcherFailureDiagnostic: HopperStartupDiagnostic | undefined;
|
||||
#startupController: AbortController | undefined;
|
||||
#startPromise: Promise<Result<HopperServerInfo, HopperError>> | undefined;
|
||||
#closePromise: Promise<void> | undefined;
|
||||
#closePromise: Promise<Result<null, AnalysisError>> | undefined;
|
||||
readonly #onSocketData = (chunk: string): void => {
|
||||
this.#onData(chunk);
|
||||
this.#responses.push(chunk);
|
||||
};
|
||||
readonly #onSocketError = (): void => {
|
||||
this.#failAll(new HopperProcessError(null));
|
||||
@@ -108,6 +105,30 @@ export class HopperClient {
|
||||
requestTimeoutMs: options.requestTimeoutMs ?? 30_000,
|
||||
startupTimeoutMs: options.startupTimeoutMs ?? 120_000,
|
||||
};
|
||||
this.#requests = new HopperRequestQueue(
|
||||
MAX_QUEUED_REQUESTS,
|
||||
({ id, method, params }, failed) => {
|
||||
const socket = this.#socket;
|
||||
const token = this.#token;
|
||||
if (socket === undefined || socket.destroyed || token === undefined) {
|
||||
failed();
|
||||
return;
|
||||
}
|
||||
socket.write(
|
||||
`${JSON.stringify({ id, token, method, params })}\n`,
|
||||
(cause) => {
|
||||
if (cause !== undefined && cause !== null) failed();
|
||||
},
|
||||
);
|
||||
},
|
||||
);
|
||||
this.#responses = new HopperResponseStream({
|
||||
accept: (response) =>
|
||||
this.#requests.accept(response.id, responseResult(response)),
|
||||
hasQueued: (id) => this.#requests.hasQueued(id),
|
||||
nextRequestId: () => this.#nextId,
|
||||
abort: (message, cause) => this.#abortProtocol(message, cause),
|
||||
});
|
||||
}
|
||||
|
||||
/** Latest token-verified launcher lineage for the active run. */
|
||||
@@ -115,6 +136,11 @@ export class HopperClient {
|
||||
return this.#lineage.snapshot();
|
||||
}
|
||||
|
||||
/** Observe a request that still occupies Hopper after caller timeout/cancel. */
|
||||
requestActivity(): HopperRequestActivity | null {
|
||||
return this.#requests.activity();
|
||||
}
|
||||
|
||||
/** Launch the bridge once and complete its authenticated health handshake. */
|
||||
start(signal?: AbortSignal): Promise<Result<HopperServerInfo, HopperError>> {
|
||||
if (this.#closePromise !== undefined) {
|
||||
@@ -251,6 +277,7 @@ export class HopperClient {
|
||||
options: {
|
||||
readonly signal?: AbortSignal;
|
||||
readonly timeoutMs?: number;
|
||||
readonly progress?: ProgressReporter;
|
||||
} = {},
|
||||
): Promise<Result<JsonValue, HopperError>> {
|
||||
const started = await this.start(options.signal);
|
||||
@@ -260,11 +287,18 @@ export class HopperClient {
|
||||
|
||||
/** Stop the bridge, settle outstanding requests, and remove session artifacts. */
|
||||
close(): Promise<void> {
|
||||
return this.closeWithOutcome().then(() => undefined);
|
||||
}
|
||||
|
||||
/** Stop the bridge and report whether every owned resource was verified clean. */
|
||||
closeWithOutcome(
|
||||
options: { readonly progress?: ProgressReporter } = {},
|
||||
): Promise<Result<null, AnalysisError>> {
|
||||
const starting = this.#startPromise;
|
||||
const controller = this.#startupController;
|
||||
controller?.abort();
|
||||
this.#closePromise ??= Promise.resolve().then(() =>
|
||||
this.#close(starting, controller),
|
||||
this.#close(starting, controller, options),
|
||||
);
|
||||
return this.#closePromise;
|
||||
}
|
||||
@@ -272,10 +306,11 @@ export class HopperClient {
|
||||
async #close(
|
||||
starting: Promise<Result<HopperServerInfo, HopperError>> | undefined,
|
||||
controller: AbortController | undefined,
|
||||
): Promise<void> {
|
||||
options: { readonly progress?: ProgressReporter },
|
||||
): Promise<Result<null, AnalysisError>> {
|
||||
try {
|
||||
await starting?.catch(() => undefined);
|
||||
await this.#cleanup();
|
||||
return await this.#cleanup(options);
|
||||
} finally {
|
||||
if (this.#startPromise === starting) this.#startPromise = undefined;
|
||||
if (this.#startupController === controller) {
|
||||
@@ -285,92 +320,41 @@ export class HopperClient {
|
||||
}
|
||||
}
|
||||
|
||||
async #cleanup(): Promise<void> {
|
||||
async #cleanup(
|
||||
options: { readonly progress?: ProgressReporter } = {},
|
||||
): Promise<Result<null, AnalysisError>> {
|
||||
this.#closing = true;
|
||||
try {
|
||||
const socket = this.#socket;
|
||||
let cleanupResult: ProcessCleanupResult | undefined;
|
||||
if (socket !== undefined && !socket.destroyed) {
|
||||
const shutdown = await this.#request(
|
||||
"shutdown",
|
||||
{},
|
||||
{ timeoutMs: SHUTDOWN_TIMEOUT_MS },
|
||||
).catch(() => err(new HopperProcessError(null)));
|
||||
if (
|
||||
shutdown.ok &&
|
||||
isHopperCleanupRequired(shutdown.value) &&
|
||||
this.#launch?.shutdownByCleanup === true &&
|
||||
this.#launch.cleanup !== undefined
|
||||
) {
|
||||
cleanupResult = await this.#launch
|
||||
.cleanup()
|
||||
.catch(providerCleanupFailure);
|
||||
if (!cleanupResult.cleaned) {
|
||||
await this.#fallbackDocumentShutdown();
|
||||
cleanupResult = await this.#launch
|
||||
.cleanup()
|
||||
.catch(providerCleanupFailure);
|
||||
}
|
||||
} else if (
|
||||
!shutdown.ok ||
|
||||
!isHopperShutdownAcknowledgement(shutdown.value)
|
||||
)
|
||||
this.#logger.warn(
|
||||
{ status: shutdown.ok ? "invalid-acknowledgement" : "failed" },
|
||||
"Hopper document shutdown was not confirmed",
|
||||
);
|
||||
}
|
||||
this.#failAll(new HopperProcessError(null));
|
||||
if (socket !== undefined) this.#detachSocket(socket);
|
||||
socket?.destroy();
|
||||
return await cleanupHopperSession({
|
||||
socket: this.#socket,
|
||||
launch: this.#launch,
|
||||
processSupervisor: this.#process,
|
||||
runtimeRoot: this.#runtimeRoot,
|
||||
activeRequest: this.#requests.activity(),
|
||||
progress: options.progress,
|
||||
logger: this.#logger,
|
||||
onDiagnostic: this.#options.onDiagnostic,
|
||||
request: (method) =>
|
||||
this.#request(method, {}, { timeoutMs: SHUTDOWN_TIMEOUT_MS }),
|
||||
releaseTransport: (socket) => {
|
||||
this.#failAll(new HopperProcessError(null));
|
||||
if (socket !== undefined) this.#detachSocket(socket);
|
||||
socket?.destroy();
|
||||
},
|
||||
});
|
||||
} finally {
|
||||
this.#socket = undefined;
|
||||
if (this.#process !== undefined) {
|
||||
const stopped = await this.#process.stop(
|
||||
cleanupResult === undefined ? {} : { cleanupResult },
|
||||
);
|
||||
const diagnostic = createOwnedHopperShutdownDiagnostic(
|
||||
this.#process.launch,
|
||||
stopped,
|
||||
cleanupResult,
|
||||
);
|
||||
this.#options.onDiagnostic?.(diagnostic);
|
||||
this.#logger.info(
|
||||
diagnostic,
|
||||
"Owned Hopper launcher shutdown completed",
|
||||
);
|
||||
if (stopped.status === "incomplete")
|
||||
this.#logger.warn(
|
||||
{ reason: stopped.reason },
|
||||
"Owned launcher cleanup failed closed",
|
||||
);
|
||||
}
|
||||
this.#process = undefined;
|
||||
this.#launch = undefined;
|
||||
const runtimeRoot = this.#runtimeRoot;
|
||||
this.#runtimeRoot = undefined;
|
||||
await runtimeRoot?.close();
|
||||
this.#token = undefined;
|
||||
this.#buffer = "";
|
||||
this.#responses.reset();
|
||||
this.#launcherExitCode = undefined;
|
||||
this.#launcherFailureDiagnostic = undefined;
|
||||
} finally {
|
||||
this.#closing = false;
|
||||
}
|
||||
}
|
||||
|
||||
async #fallbackDocumentShutdown(): Promise<void> {
|
||||
const fallback = await this.#request(
|
||||
"shutdown_document",
|
||||
{},
|
||||
{ timeoutMs: SHUTDOWN_TIMEOUT_MS },
|
||||
).catch(() => err(new HopperProcessError(null)));
|
||||
if (!fallback.ok || !isHopperShutdownAcknowledgement(fallback.value))
|
||||
this.#logger.warn(
|
||||
{ status: fallback.ok ? "invalid-acknowledgement" : "failed" },
|
||||
"Hopper document shutdown fallback was not confirmed",
|
||||
);
|
||||
}
|
||||
|
||||
async #connect(
|
||||
socketPath: string,
|
||||
deadline: ProviderStartupDeadline,
|
||||
@@ -413,7 +397,11 @@ export class HopperClient {
|
||||
async #request(
|
||||
method: string,
|
||||
params: JsonValue,
|
||||
options: { readonly signal?: AbortSignal; readonly timeoutMs?: number },
|
||||
options: {
|
||||
readonly signal?: AbortSignal;
|
||||
readonly timeoutMs?: number;
|
||||
readonly progress?: ProgressReporter;
|
||||
},
|
||||
): Promise<Result<JsonValue, HopperError>> {
|
||||
const socket = this.#socket;
|
||||
const token = this.#token;
|
||||
@@ -425,20 +413,11 @@ export class HopperClient {
|
||||
const id = this.#nextId++;
|
||||
const startedAt = performance.now();
|
||||
const timeoutMs = options.timeoutMs ?? this.#options.requestTimeoutMs;
|
||||
const response = this.#pending.wait(id, {
|
||||
const result = await this.#requests.run(id, method, params, {
|
||||
timeoutMs,
|
||||
...(options.signal === undefined ? {} : { signal: options.signal }),
|
||||
timeoutValue: () => err(new HopperTimeoutError(timeoutMs)),
|
||||
cancelledValue: () => err(new HopperCancelledError()),
|
||||
...(options.progress !== undefined ? { progress: options.progress } : {}),
|
||||
});
|
||||
socket.write(
|
||||
`${JSON.stringify({ id, token, method, params })}\n`,
|
||||
(cause) => {
|
||||
if (cause !== undefined && cause !== null)
|
||||
this.#pending.settle(id, err(new HopperProcessError(null)));
|
||||
},
|
||||
);
|
||||
const result = await response;
|
||||
this.#logger[result.ok ? "debug" : "warn"](
|
||||
{
|
||||
method,
|
||||
@@ -501,46 +480,13 @@ export class HopperClient {
|
||||
);
|
||||
}
|
||||
|
||||
#onData(chunk: string): void {
|
||||
this.#buffer += chunk;
|
||||
let newline = this.#buffer.indexOf("\n");
|
||||
while (newline >= 0) {
|
||||
const line = this.#buffer.slice(0, newline).trim();
|
||||
this.#buffer = this.#buffer.slice(newline + 1);
|
||||
if (Buffer.byteLength(line) > MAX_LINE_BYTES) {
|
||||
this.#abortProtocol("Hopper response exceeded the maximum line size");
|
||||
return;
|
||||
}
|
||||
if (line.length > 0) this.#onLine(line);
|
||||
newline = this.#buffer.indexOf("\n");
|
||||
}
|
||||
if (Buffer.byteLength(this.#buffer) > MAX_LINE_BYTES) {
|
||||
this.#abortProtocol("Hopper response exceeded the maximum line size");
|
||||
}
|
||||
}
|
||||
|
||||
#onLine(line: string): void {
|
||||
const parsed = parseResponseLine(line);
|
||||
if (!parsed.ok) {
|
||||
this.#abortProtocol(parsed.error.message, parsed.error);
|
||||
return;
|
||||
}
|
||||
if (!this.#pending.has(parsed.value.id)) {
|
||||
if (parsed.value.id >= this.#nextId) {
|
||||
this.#abortProtocol("Hopper returned an unknown response id");
|
||||
}
|
||||
return;
|
||||
}
|
||||
this.#pending.settle(parsed.value.id, responseResult(parsed.value));
|
||||
}
|
||||
|
||||
#abortProtocol(message: string, cause?: Error): void {
|
||||
this.#failAll(new HopperProtocolError(message, { cause }));
|
||||
this.#socket?.destroy();
|
||||
}
|
||||
|
||||
#failAll(error: HopperError): void {
|
||||
this.#pending.failAll(() => err(error));
|
||||
this.#requests.failAll(error);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -255,6 +255,26 @@ export class HopperProvider implements AnalysisProviderCandidate {
|
||||
? []
|
||||
: [{ provider: executionProvider, observation }];
|
||||
},
|
||||
requestActivitySnapshots: () => {
|
||||
const activity = client.requestActivity();
|
||||
return [
|
||||
{
|
||||
provider: executionProvider,
|
||||
active:
|
||||
activity === null
|
||||
? null
|
||||
: {
|
||||
requestId: activity.requestId,
|
||||
operation: activity.operation,
|
||||
elapsedMs: activity.elapsedMs,
|
||||
timeoutMs: activity.timeoutMs,
|
||||
callerState: activity.callerState,
|
||||
},
|
||||
queuedRequests: activity?.queuedRequests ?? 0,
|
||||
},
|
||||
];
|
||||
},
|
||||
closeWithOutcome: (options) => client.closeWithOutcome(options),
|
||||
close: () => client.close(),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -0,0 +1,279 @@
|
||||
import { performance } from "node:perf_hooks";
|
||||
|
||||
import type { ProgressReporter } from "../application/ProgressReporter.js";
|
||||
import {
|
||||
HopperCancelledError,
|
||||
type HopperError,
|
||||
HopperProcessError,
|
||||
HopperProtocolError,
|
||||
HopperTimeoutError,
|
||||
} from "../domain/errors.js";
|
||||
import type { JsonValue } from "../domain/jsonValue.js";
|
||||
import { err, type Result } from "../domain/result.js";
|
||||
|
||||
export type HopperRequestResult = Result<JsonValue, HopperError>;
|
||||
|
||||
export interface HopperRequestQueueOptions {
|
||||
readonly signal?: AbortSignal;
|
||||
readonly timeoutMs: number;
|
||||
readonly progress?: ProgressReporter;
|
||||
}
|
||||
|
||||
export interface HopperRequestActivity {
|
||||
readonly requestId: number;
|
||||
readonly operation: string;
|
||||
readonly elapsedMs: number;
|
||||
readonly timeoutMs: number;
|
||||
readonly callerState: "waiting" | "timed_out" | "cancelled";
|
||||
readonly queuedRequests: number;
|
||||
}
|
||||
|
||||
type RequestSender = (
|
||||
request: {
|
||||
readonly id: number;
|
||||
readonly method: string;
|
||||
readonly params: JsonValue;
|
||||
},
|
||||
failed: () => void,
|
||||
) => void;
|
||||
|
||||
interface QueuedRequest {
|
||||
readonly id: number;
|
||||
readonly method: string;
|
||||
readonly params: JsonValue;
|
||||
readonly signal: AbortSignal | undefined;
|
||||
readonly timeoutMs: number;
|
||||
readonly deadline: number;
|
||||
readonly progress: ProgressReporter | undefined;
|
||||
readonly resolve: (result: HopperRequestResult) => void;
|
||||
readonly timer: NodeJS.Timeout;
|
||||
readonly onAbort: (() => void) | undefined;
|
||||
callerState: "waiting" | "timed_out" | "cancelled";
|
||||
callerSettled: boolean;
|
||||
startedAt: number | undefined;
|
||||
heartbeat: NodeJS.Timeout | undefined;
|
||||
}
|
||||
|
||||
/** Bounded FIFO that keeps the wire serialized until Hopper actually replies. */
|
||||
export class HopperRequestQueue {
|
||||
readonly #queue: QueuedRequest[] = [];
|
||||
#active: QueuedRequest | undefined;
|
||||
|
||||
constructor(
|
||||
private readonly maximum: number,
|
||||
private readonly send: RequestSender,
|
||||
) {}
|
||||
|
||||
/** Queue one request, counting queue wait against its caller deadline. */
|
||||
run(
|
||||
id: number,
|
||||
method: string,
|
||||
params: JsonValue,
|
||||
options: HopperRequestQueueOptions,
|
||||
): Promise<HopperRequestResult> {
|
||||
if (options.signal?.aborted === true)
|
||||
return Promise.resolve(err(new HopperCancelledError()));
|
||||
if (options.timeoutMs <= 0)
|
||||
return Promise.resolve(err(new HopperTimeoutError(options.timeoutMs)));
|
||||
if (this.#size() >= this.maximum)
|
||||
return Promise.resolve(
|
||||
err(
|
||||
new HopperProtocolError(
|
||||
`Hopper serial request queue reached its ${String(this.maximum)}-request limit`,
|
||||
),
|
||||
),
|
||||
);
|
||||
return new Promise((resolve) => {
|
||||
let entry: QueuedRequest;
|
||||
const onAbort =
|
||||
options.signal === undefined ? undefined : () => this.#cancel(entry);
|
||||
const timer = setTimeout(() => this.#timeout(entry), options.timeoutMs);
|
||||
entry = {
|
||||
id,
|
||||
method,
|
||||
params,
|
||||
signal: options.signal,
|
||||
timeoutMs: options.timeoutMs,
|
||||
deadline: performance.now() + options.timeoutMs,
|
||||
progress: options.progress,
|
||||
resolve,
|
||||
timer,
|
||||
onAbort,
|
||||
callerState: "waiting",
|
||||
callerSettled: false,
|
||||
startedAt: undefined,
|
||||
heartbeat: undefined,
|
||||
};
|
||||
this.#queue.push(entry);
|
||||
if (onAbort !== undefined)
|
||||
options.signal?.addEventListener("abort", onAbort, { once: true });
|
||||
if (this.#active !== undefined || this.#queue.length > 1)
|
||||
this.#report(
|
||||
entry,
|
||||
`${method} queued behind ${String(this.#size() - 1)} Hopper request(s)`,
|
||||
);
|
||||
this.#drain();
|
||||
});
|
||||
}
|
||||
|
||||
/** Settle the active wire request. False identifies an unexpected response id. */
|
||||
accept(id: number, result: HopperRequestResult): boolean {
|
||||
const entry = this.#active;
|
||||
if (entry === undefined || entry.id !== id) return false;
|
||||
this.#releaseWire(entry);
|
||||
if (!entry.callerSettled) this.#settleCaller(entry, result, "waiting");
|
||||
this.#active = undefined;
|
||||
this.#drain();
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Fail the active wire request and every request still waiting in the FIFO. */
|
||||
failAll(error: HopperError): void {
|
||||
const active = this.#active;
|
||||
this.#active = undefined;
|
||||
if (active !== undefined) {
|
||||
this.#releaseWire(active);
|
||||
if (!active.callerSettled)
|
||||
this.#settleCaller(active, err(error), "waiting");
|
||||
}
|
||||
for (const entry of this.#queue.splice(0))
|
||||
this.#settleCaller(entry, err(error), "waiting");
|
||||
}
|
||||
|
||||
/** Snapshot the one request still occupying Hopper's serial Python thread. */
|
||||
activity(): HopperRequestActivity | null {
|
||||
const active = this.#active;
|
||||
if (active === undefined || active.startedAt === undefined) return null;
|
||||
return {
|
||||
requestId: active.id,
|
||||
operation: active.method,
|
||||
elapsedMs: Math.max(0, Math.round(performance.now() - active.startedAt)),
|
||||
timeoutMs: active.timeoutMs,
|
||||
callerState: active.callerState,
|
||||
queuedRequests: this.#queue.length,
|
||||
};
|
||||
}
|
||||
|
||||
/** Return whether an id belongs to a request that has not crossed the wire. */
|
||||
hasQueued(id: number): boolean {
|
||||
return this.#queue.some((entry) => entry.id === id);
|
||||
}
|
||||
|
||||
/** Number of caller requests waiting behind the active bridge operation. */
|
||||
queuedCount(): number {
|
||||
return this.#queue.length;
|
||||
}
|
||||
|
||||
#size(): number {
|
||||
return this.#queue.length + (this.#active === undefined ? 0 : 1);
|
||||
}
|
||||
|
||||
#drain(): void {
|
||||
if (this.#active !== undefined) return;
|
||||
const entry = this.#queue.shift();
|
||||
if (entry === undefined) return;
|
||||
const remaining = entry.deadline - performance.now();
|
||||
if (entry.signal?.aborted === true || remaining <= 0) {
|
||||
this.#settleCaller(
|
||||
entry,
|
||||
entry.signal?.aborted === true
|
||||
? err(new HopperCancelledError())
|
||||
: err(new HopperTimeoutError(entry.timeoutMs)),
|
||||
entry.signal?.aborted === true ? "cancelled" : "timed_out",
|
||||
);
|
||||
this.#drain();
|
||||
return;
|
||||
}
|
||||
this.#active = entry;
|
||||
entry.startedAt = performance.now();
|
||||
this.#report(entry, `${entry.method} started on Hopper's serial bridge`);
|
||||
entry.heartbeat = setInterval(() => {
|
||||
if (entry.callerSettled || entry.startedAt === undefined) return;
|
||||
const elapsed = Math.round(performance.now() - entry.startedAt);
|
||||
this.#report(
|
||||
entry,
|
||||
`${entry.method} is still running in Hopper (${String(elapsed)} ms elapsed)`,
|
||||
);
|
||||
}, 1_000);
|
||||
try {
|
||||
this.send(
|
||||
{ id: entry.id, method: entry.method, params: entry.params },
|
||||
() => this.accept(entry.id, err(new HopperProcessError(null))),
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
this.accept(
|
||||
entry.id,
|
||||
err(new HopperProtocolError("Hopper socket write failed", { cause })),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#timeout(entry: QueuedRequest): void {
|
||||
if (entry.callerSettled) return;
|
||||
if (this.#active === entry) {
|
||||
this.#settleCaller(
|
||||
entry,
|
||||
err(new HopperTimeoutError(entry.timeoutMs)),
|
||||
"timed_out",
|
||||
);
|
||||
return;
|
||||
}
|
||||
const index = this.#queue.indexOf(entry);
|
||||
if (index < 0) return;
|
||||
this.#queue.splice(index, 1);
|
||||
this.#settleCaller(
|
||||
entry,
|
||||
err(new HopperTimeoutError(entry.timeoutMs)),
|
||||
"timed_out",
|
||||
);
|
||||
}
|
||||
|
||||
#cancel(entry: QueuedRequest): void {
|
||||
if (entry.callerSettled) return;
|
||||
if (this.#active === entry) {
|
||||
this.#settleCaller(entry, err(new HopperCancelledError()), "cancelled");
|
||||
return;
|
||||
}
|
||||
const index = this.#queue.indexOf(entry);
|
||||
if (index < 0) return;
|
||||
this.#queue.splice(index, 1);
|
||||
this.#settleCaller(entry, err(new HopperCancelledError()), "cancelled");
|
||||
}
|
||||
|
||||
#settleCaller(
|
||||
entry: QueuedRequest,
|
||||
result: HopperRequestResult,
|
||||
callerState: QueuedRequest["callerState"],
|
||||
): void {
|
||||
if (entry.callerSettled) return;
|
||||
entry.callerSettled = true;
|
||||
entry.callerState = callerState;
|
||||
if (this.#active === entry && entry.heartbeat !== undefined) {
|
||||
clearInterval(entry.heartbeat);
|
||||
entry.heartbeat = undefined;
|
||||
}
|
||||
clearTimeout(entry.timer);
|
||||
if (entry.signal !== undefined && entry.onAbort !== undefined)
|
||||
entry.signal.removeEventListener("abort", entry.onAbort);
|
||||
entry.resolve(result);
|
||||
}
|
||||
|
||||
#releaseWire(entry: QueuedRequest): void {
|
||||
if (entry.heartbeat !== undefined) clearInterval(entry.heartbeat);
|
||||
entry.heartbeat = undefined;
|
||||
clearTimeout(entry.timer);
|
||||
if (entry.signal !== undefined && entry.onAbort !== undefined)
|
||||
entry.signal.removeEventListener("abort", entry.onAbort);
|
||||
}
|
||||
|
||||
#report(entry: QueuedRequest, message: string): void {
|
||||
void entry.progress
|
||||
?.report({
|
||||
phase: "hopper_request",
|
||||
completed: 0,
|
||||
total: 1,
|
||||
message,
|
||||
})
|
||||
.catch(() => undefined);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,56 @@
|
||||
import { parseResponseLine, type HopperResponse } from "./protocol.js";
|
||||
|
||||
const MAX_LINE_BYTES = 10 * 1024 * 1024;
|
||||
|
||||
export interface HopperResponseStreamOptions {
|
||||
readonly accept: (response: HopperResponse) => boolean;
|
||||
readonly hasQueued: (id: number) => boolean;
|
||||
readonly nextRequestId: () => number;
|
||||
readonly abort: (message: string, cause?: Error) => void;
|
||||
}
|
||||
|
||||
/** Incrementally validates bounded Hopper NDJSON response frames. */
|
||||
export class HopperResponseStream {
|
||||
#buffer = "";
|
||||
|
||||
constructor(readonly options: HopperResponseStreamOptions) {}
|
||||
|
||||
push(chunk: string): void {
|
||||
this.#buffer += chunk;
|
||||
let newline = this.#buffer.indexOf("\n");
|
||||
while (newline >= 0) {
|
||||
const line = this.#buffer.slice(0, newline).trim();
|
||||
this.#buffer = this.#buffer.slice(newline + 1);
|
||||
if (Buffer.byteLength(line) > MAX_LINE_BYTES) {
|
||||
this.options.abort("Hopper response exceeded the maximum line size");
|
||||
return;
|
||||
}
|
||||
if (line.length > 0 && !this.#acceptLine(line)) return;
|
||||
newline = this.#buffer.indexOf("\n");
|
||||
}
|
||||
if (Buffer.byteLength(this.#buffer) > MAX_LINE_BYTES) {
|
||||
this.options.abort("Hopper response exceeded the maximum line size");
|
||||
}
|
||||
}
|
||||
|
||||
reset(): void {
|
||||
this.#buffer = "";
|
||||
}
|
||||
|
||||
#acceptLine(line: string): boolean {
|
||||
const parsed = parseResponseLine(line);
|
||||
if (!parsed.ok) {
|
||||
this.options.abort(parsed.error.message, parsed.error);
|
||||
return false;
|
||||
}
|
||||
if (this.options.accept(parsed.value)) return true;
|
||||
if (
|
||||
parsed.value.id >= this.options.nextRequestId() ||
|
||||
this.options.hasQueued(parsed.value.id)
|
||||
) {
|
||||
this.options.abort("Hopper returned an unknown response id");
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
}
|
||||
@@ -75,6 +75,7 @@ export class NativeMacOSProvider implements AnalysisProvider {
|
||||
? ({ available: true, reason: null } as const)
|
||||
: ({
|
||||
available: false,
|
||||
availabilityCode: "unsupported_host",
|
||||
reason: "Native macOS utilities require macOS.",
|
||||
} as const);
|
||||
return Object.freeze({
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
import type { ProgressReporter } from "../application/ProgressReporter.js";
|
||||
|
||||
/** Report one lifecycle operation start through the shared monotonic boundary. */
|
||||
export const reportLifecycleStart = (
|
||||
progress: ProgressReporter,
|
||||
phase: string,
|
||||
message = "started",
|
||||
): Promise<void> => progress.report({ phase, completed: 0, total: 1, message });
|
||||
|
||||
/** Report one lifecycle operation's terminal disposition. */
|
||||
export const reportLifecycleEnd = (
|
||||
progress: ProgressReporter,
|
||||
phase: string,
|
||||
succeeded: boolean,
|
||||
): Promise<void> =>
|
||||
progress.report({
|
||||
phase,
|
||||
completed: 1,
|
||||
total: 1,
|
||||
message: succeeded ? "completed" : "failed",
|
||||
terminal: true,
|
||||
});
|
||||
@@ -0,0 +1,22 @@
|
||||
import type { PermissionAuthority } from "../application/PermissionAuthority.js";
|
||||
import {
|
||||
AnalysisProtocolError,
|
||||
PermissionRequiredError,
|
||||
type AnalysisError,
|
||||
} from "../domain/errors.js";
|
||||
import { err, type Result } from "../domain/result.js";
|
||||
|
||||
/** Preserve a typed permission denial or contain an unexpected authority error. */
|
||||
export const permissionFailure = (
|
||||
failure: Awaited<ReturnType<PermissionAuthority["authorize"]>>,
|
||||
): Result<never, AnalysisError> => {
|
||||
if (failure.ok)
|
||||
return err(new AnalysisProtocolError("Expected a denied permission"));
|
||||
return err(
|
||||
failure.error instanceof PermissionRequiredError
|
||||
? failure.error
|
||||
: new AnalysisProtocolError(failure.error.message, {
|
||||
cause: failure.error,
|
||||
}),
|
||||
);
|
||||
};
|
||||
@@ -0,0 +1,93 @@
|
||||
import { writeAnalysisSnapshot } from "../application/AnalysisSnapshotFiles.js";
|
||||
import { closeBinaryInputSchema } from "../contracts/sessionLifecycleInputs.js";
|
||||
import { ok } from "../domain/result.js";
|
||||
import {
|
||||
reportLifecycleEnd,
|
||||
reportLifecycleStart,
|
||||
} from "./lifecycleProgress.js";
|
||||
import { logToolExecution } from "./toolLogging.js";
|
||||
import { toCallToolResult } from "./toolResult.js";
|
||||
import { mcpProgressReporter } from "./mcpProgress.js";
|
||||
import { permissionFailure } from "./permissionFailure.js";
|
||||
import type { LifecycleToolRegistration } from "./registerSessionTools.js";
|
||||
import { safeParseToolInput } from "./toolInputValidation.js";
|
||||
import { toolRegistrationOptions } from "./toolRegistrationOptions.js";
|
||||
|
||||
/** Register provider cleanup and optional pre-close snapshot persistence. */
|
||||
export const registerCloseLifecycleTool = ({
|
||||
server,
|
||||
session,
|
||||
logger,
|
||||
contracts: [, closeContract],
|
||||
snapshotFilePolicy,
|
||||
permissionAuthority,
|
||||
}: LifecycleToolRegistration): void => {
|
||||
server.registerTool(
|
||||
closeContract.name,
|
||||
toolRegistrationOptions(closeContract),
|
||||
async (input, context) => {
|
||||
const parsedInput = safeParseToolInput(
|
||||
closeBinaryInputSchema,
|
||||
input,
|
||||
closeContract.name,
|
||||
);
|
||||
if (!parsedInput.ok) return toCallToolResult(parsedInput, closeContract);
|
||||
const parsed = parsedInput.value;
|
||||
const progress = mcpProgressReporter(context);
|
||||
if (parsed.snapshot_path === undefined) {
|
||||
await reportLifecycleStart(progress, closeContract.name);
|
||||
const closed = await logToolExecution(logger, closeContract.name, () =>
|
||||
session.close({ progress }),
|
||||
);
|
||||
await reportLifecycleEnd(progress, closeContract.name, closed.ok);
|
||||
server.sendToolListChanged();
|
||||
return toCallToolResult(closed, closeContract);
|
||||
}
|
||||
if (permissionAuthority !== undefined) {
|
||||
const authorized = await permissionAuthority.authorize(
|
||||
{
|
||||
capability: "snapshot_write",
|
||||
roots: [parsed.snapshot_path],
|
||||
executables: [],
|
||||
environment_names: [],
|
||||
network: "none",
|
||||
mount: false,
|
||||
operation_identity: `close_binary:snapshot:${parsed.snapshot_path}`,
|
||||
},
|
||||
"write",
|
||||
);
|
||||
if (!authorized.ok)
|
||||
return toCallToolResult(permissionFailure(authorized), closeContract);
|
||||
}
|
||||
const snapshot = session.exportAnalysisSnapshot();
|
||||
if (!snapshot.ok) return toCallToolResult(snapshot, closeContract);
|
||||
const written = await writeAnalysisSnapshot(
|
||||
snapshot.value,
|
||||
parsed.snapshot_path,
|
||||
parsed.overwrite,
|
||||
snapshotFilePolicy,
|
||||
);
|
||||
if (!written.ok) return toCallToolResult(written, closeContract);
|
||||
await reportLifecycleStart(
|
||||
progress,
|
||||
closeContract.name,
|
||||
"snapshot written; closing provider",
|
||||
);
|
||||
const closed = await logToolExecution(logger, closeContract.name, () =>
|
||||
session.close({ progress }),
|
||||
);
|
||||
await reportLifecycleEnd(progress, closeContract.name, closed.ok);
|
||||
server.sendToolListChanged();
|
||||
return closed.ok
|
||||
? toCallToolResult(
|
||||
ok({
|
||||
path: written.value.path,
|
||||
bytes: written.value.bytes,
|
||||
entries: snapshot.value.entries.length,
|
||||
}),
|
||||
closeContract,
|
||||
)
|
||||
: toCallToolResult(closed, closeContract);
|
||||
},
|
||||
);
|
||||
};
|
||||
@@ -49,9 +49,8 @@ export const registerEnhancedTools = (
|
||||
analysis: AnalysisOperationPort,
|
||||
options: EnhancedToolRegistration,
|
||||
): void => {
|
||||
const services = new EnhancedTools(analysis);
|
||||
for (const contract of ENHANCED_TOOL_CONTRACTS) {
|
||||
registerEnhancedTool(server, services, contract, {
|
||||
registerEnhancedTool(server, analysis, contract, {
|
||||
logger: options.logger,
|
||||
activeTarget: options.activeTarget,
|
||||
analysisProfile: options.analysisProfile,
|
||||
@@ -63,7 +62,7 @@ export const registerEnhancedTools = (
|
||||
|
||||
const registerEnhancedTool = (
|
||||
server: McpServer,
|
||||
services: EnhancedTools,
|
||||
analysis: AnalysisOperationPort,
|
||||
contract: ToolContract,
|
||||
registration: {
|
||||
readonly logger: Logger;
|
||||
@@ -91,6 +90,13 @@ const registerEnhancedTool = (
|
||||
if (!validatedCall.ok) return toCallToolResult(validatedCall, contract);
|
||||
const parsedInput = validatedCall.value.input;
|
||||
const parameters = jsonParameters(parsedInput);
|
||||
const services = new EnhancedTools({
|
||||
execute: (operation, operationParameters, executionOptions) =>
|
||||
analysis.execute(operation, operationParameters, {
|
||||
...executionOptions,
|
||||
progress,
|
||||
}),
|
||||
});
|
||||
const result = await logToolExecution(registration.logger, name, () =>
|
||||
services.executeValidated(validatedCall.value, context.mcpReq.signal),
|
||||
);
|
||||
|
||||
@@ -148,6 +148,7 @@ const projectSessionStatus = (options: {
|
||||
active_provider: activeProvider,
|
||||
target: open ? targetFrom(options.status) : null,
|
||||
alignment: options.serverIdentity.alignment,
|
||||
analysis_activity: options.status.analysis_activity,
|
||||
recommended_actions: recommendedActions(
|
||||
open,
|
||||
options.serverIdentity.alignment.state,
|
||||
|
||||
@@ -6,7 +6,7 @@ import { SESSION_TOOL_CONTRACTS } from "../contracts/toolContracts.js";
|
||||
import { toCallToolResult } from "./toolResult.js";
|
||||
import type { Logger } from "../logger.js";
|
||||
import { logToolExecution } from "./toolLogging.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { ok, type Result } from "../domain/result.js";
|
||||
import type {
|
||||
ProcessCapture,
|
||||
ProcessExecutionPolicy,
|
||||
@@ -18,16 +18,8 @@ import { processCapturePermissionRequest } from "../application/ProcessCapturePe
|
||||
import type { Evidence } from "../domain/evidence.js";
|
||||
import type { AnalysisSnapshot } from "../domain/analysisSnapshot.js";
|
||||
import type { EvidenceFilePolicy } from "../domain/evidenceBundle.js";
|
||||
import {
|
||||
readAnalysisSnapshot,
|
||||
writeAnalysisSnapshot,
|
||||
} from "../application/AnalysisSnapshotFiles.js";
|
||||
import {
|
||||
AnalysisProtocolError,
|
||||
PermissionRequiredError,
|
||||
UnknownRegistryError,
|
||||
type AnalysisError,
|
||||
} from "../domain/errors.js";
|
||||
import { readAnalysisSnapshot } from "../application/AnalysisSnapshotFiles.js";
|
||||
import { UnknownRegistryError, type AnalysisError } from "../domain/errors.js";
|
||||
import {
|
||||
DENY_EVIDENCE_FILE_POLICY,
|
||||
DENY_PROCESS_POLICY,
|
||||
@@ -40,10 +32,7 @@ import { toolRegistrationOptions } from "./toolRegistrationOptions.js";
|
||||
import { safeParseToolInput } from "./toolInputValidation.js";
|
||||
import { registerBundleComparisonTool } from "./registerBundleComparisonTool.js";
|
||||
import { registerInvestigationTools } from "./registerInvestigationTools.js";
|
||||
import {
|
||||
closeBinaryInputSchema,
|
||||
openBinaryInputSchema,
|
||||
} from "../contracts/sessionLifecycleInputs.js";
|
||||
import { openBinaryInputSchema } from "../contracts/sessionLifecycleInputs.js";
|
||||
import { registerSessionStatusTool } from "./registerSessionStatusTool.js";
|
||||
import type { PermissionAuthority } from "../application/PermissionAuthority.js";
|
||||
import { mcpProgressReporter } from "./mcpProgress.js";
|
||||
@@ -61,20 +50,8 @@ import {
|
||||
sessionAvailabilityPolicy,
|
||||
type SessionAvailability,
|
||||
} from "./sessionAvailabilityPolicy.js";
|
||||
|
||||
const permissionFailure = (
|
||||
failure: Awaited<ReturnType<PermissionAuthority["authorize"]>>,
|
||||
): Result<never, AnalysisError> => {
|
||||
if (failure.ok)
|
||||
return err(new AnalysisProtocolError("Expected a denied permission"));
|
||||
return err(
|
||||
failure.error instanceof PermissionRequiredError
|
||||
? failure.error
|
||||
: new AnalysisProtocolError(failure.error.message, {
|
||||
cause: failure.error,
|
||||
}),
|
||||
);
|
||||
};
|
||||
import { permissionFailure } from "./permissionFailure.js";
|
||||
import { registerCloseLifecycleTool } from "./registerCloseLifecycleTool.js";
|
||||
|
||||
const recordProcessResidualUnknowns = (
|
||||
session: BinarySessionPort,
|
||||
@@ -204,7 +181,7 @@ const registerProcessTools = ({
|
||||
);
|
||||
};
|
||||
|
||||
interface LifecycleToolRegistration {
|
||||
export interface LifecycleToolRegistration {
|
||||
readonly server: McpServer;
|
||||
readonly session: BinarySessionPort;
|
||||
readonly logger: Logger;
|
||||
@@ -314,75 +291,6 @@ const registerOpenLifecycleTool = ({
|
||||
);
|
||||
};
|
||||
|
||||
const registerCloseLifecycleTool = ({
|
||||
server,
|
||||
session,
|
||||
logger,
|
||||
contracts: [, closeContract],
|
||||
snapshotFilePolicy,
|
||||
permissionAuthority,
|
||||
}: LifecycleToolRegistration): void => {
|
||||
server.registerTool(
|
||||
closeContract.name,
|
||||
toolRegistrationOptions(closeContract),
|
||||
async (input) => {
|
||||
const parsedInput = safeParseToolInput(
|
||||
closeBinaryInputSchema,
|
||||
input,
|
||||
closeContract.name,
|
||||
);
|
||||
if (!parsedInput.ok) return toCallToolResult(parsedInput, closeContract);
|
||||
const parsed = parsedInput.value;
|
||||
if (parsed.snapshot_path === undefined) {
|
||||
const closed = await logToolExecution(logger, closeContract.name, () =>
|
||||
session.close(),
|
||||
);
|
||||
if (closed.ok) server.sendToolListChanged();
|
||||
return toCallToolResult(closed, closeContract);
|
||||
}
|
||||
if (permissionAuthority !== undefined) {
|
||||
const authorized = await permissionAuthority.authorize(
|
||||
{
|
||||
capability: "snapshot_write",
|
||||
roots: [parsed.snapshot_path],
|
||||
executables: [],
|
||||
environment_names: [],
|
||||
network: "none",
|
||||
mount: false,
|
||||
operation_identity: `close_binary:snapshot:${parsed.snapshot_path}`,
|
||||
},
|
||||
"write",
|
||||
);
|
||||
if (!authorized.ok)
|
||||
return toCallToolResult(permissionFailure(authorized), closeContract);
|
||||
}
|
||||
const snapshot = session.exportAnalysisSnapshot();
|
||||
if (!snapshot.ok) return toCallToolResult(snapshot, closeContract);
|
||||
const written = await writeAnalysisSnapshot(
|
||||
snapshot.value,
|
||||
parsed.snapshot_path,
|
||||
parsed.overwrite,
|
||||
snapshotFilePolicy,
|
||||
);
|
||||
if (!written.ok) return toCallToolResult(written, closeContract);
|
||||
const closed = await logToolExecution(logger, closeContract.name, () =>
|
||||
session.close(),
|
||||
);
|
||||
if (closed.ok) server.sendToolListChanged();
|
||||
return closed.ok
|
||||
? toCallToolResult(
|
||||
ok({
|
||||
path: written.value.path,
|
||||
bytes: written.value.bytes,
|
||||
entries: snapshot.value.entries.length,
|
||||
}),
|
||||
closeContract,
|
||||
)
|
||||
: toCallToolResult(closed, closeContract);
|
||||
},
|
||||
);
|
||||
};
|
||||
|
||||
/** Register MCP-only target lifecycle operations on a long-lived session. */
|
||||
export interface SessionToolOptions {
|
||||
readonly processPolicy?: ProcessExecutionPolicy;
|
||||
|
||||
@@ -179,6 +179,71 @@ describe("artifact graph provider", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it.each([
|
||||
["fixture.msix", "msix"],
|
||||
["fixture.appxbundle", "appx"],
|
||||
] as const)(
|
||||
"reuses the hardened ZIP reader for %s package inventory",
|
||||
async (name, format) => {
|
||||
const root = await createTestTempDirectory("rea-windows-package-");
|
||||
const path = join(root, name);
|
||||
const writer = new ZipWriter(new Uint8ArrayWriter());
|
||||
await writer.add(
|
||||
"Assets/main.js",
|
||||
new TextReader("export default 'windows';"),
|
||||
);
|
||||
await writer.add(
|
||||
"VFS/ProgramFilesX64/App/addon.node",
|
||||
new TextReader("native"),
|
||||
);
|
||||
await writeFile(path, await writer.close());
|
||||
|
||||
const parsed = await parseBinaryTarget(path);
|
||||
expect(parsed.ok && parsed.value).toMatchObject({
|
||||
kind: "archive",
|
||||
format,
|
||||
});
|
||||
if (!parsed.ok) return;
|
||||
const result = await inventory(parsed.value);
|
||||
expect(result.manifest.root_format).toBe(format);
|
||||
expect(result.nodes.items.map(({ kind }) => kind)).toEqual(
|
||||
expect.arrayContaining(["javascript", "native-addon"]),
|
||||
);
|
||||
const selected = result.occurrences.items.find(
|
||||
({ logical_path: logicalPath }) => logicalPath === "Assets/main.js",
|
||||
);
|
||||
expect(selected).toBeDefined();
|
||||
if (selected === undefined) return;
|
||||
const output = join(root, "output");
|
||||
const extracted = await new ArtifactProvider()
|
||||
.createClient(parsed.value)
|
||||
.execute(
|
||||
"extract_artifact",
|
||||
artifactExtractionInputSchema.parse({
|
||||
approved: true,
|
||||
output_root: output,
|
||||
occurrence_ids: [selected.occurrence_id],
|
||||
}),
|
||||
);
|
||||
expect(extracted.ok).toBe(true);
|
||||
expect(await readFile(join(output, "Assets", "main.js"), "utf8")).toBe(
|
||||
"export default 'windows';",
|
||||
);
|
||||
expect(
|
||||
parseEvidence(
|
||||
await runProviderAnalysis(path, "inventory_artifact", {
|
||||
node_limit: 10,
|
||||
occurrence_limit: 10,
|
||||
edge_limit: 10,
|
||||
}),
|
||||
),
|
||||
).toMatchObject({
|
||||
operation: "inventory_artifact",
|
||||
subject: { format },
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("rejects unsafe, colliding, and over-ratio entries", async () => {
|
||||
expect(() => normalizeArtifactPath("../escape", limits())).toThrow(
|
||||
/normalized/u,
|
||||
|
||||
@@ -14,6 +14,7 @@ import {
|
||||
HopperStartError,
|
||||
ProviderAdapterError,
|
||||
} from "../src/domain/errors.js";
|
||||
import { ProviderCleanupError } from "../src/domain/providerCleanupError.js";
|
||||
import { err, ok as resultOk } from "../src/domain/result.js";
|
||||
import { createEvidenceBundle } from "../src/domain/evidenceBundle.js";
|
||||
import { createAnalysisProfile } from "../src/domain/analysisProfile.js";
|
||||
@@ -123,6 +124,19 @@ describe("binary session", () => {
|
||||
},
|
||||
},
|
||||
],
|
||||
requestActivitySnapshots: () => [
|
||||
{
|
||||
provider: provider.identity(),
|
||||
active: {
|
||||
requestId: 7,
|
||||
operation: "analyze_function",
|
||||
elapsedMs: 31_000,
|
||||
timeoutMs: 30_000,
|
||||
callerState: "timed_out",
|
||||
},
|
||||
queuedRequests: 2,
|
||||
},
|
||||
],
|
||||
close: () => Promise.resolve(),
|
||||
};
|
||||
},
|
||||
@@ -184,6 +198,22 @@ describe("binary session", () => {
|
||||
],
|
||||
},
|
||||
},
|
||||
analysis_activity: {
|
||||
status: "timed_out_busy",
|
||||
providers: [
|
||||
{
|
||||
provider: { id: "fixture", name: "Fixture", version: "1" },
|
||||
active: {
|
||||
request_id: 7,
|
||||
operation: "analyze_function",
|
||||
elapsed_ms: 31_000,
|
||||
timeout_ms: 30_000,
|
||||
caller_state: "timed_out",
|
||||
},
|
||||
queued_requests: 2,
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
expect(operations).toEqual(["health", "address_name"]);
|
||||
await session.close();
|
||||
@@ -724,6 +754,50 @@ describe("binary session", () => {
|
||||
expect(overlapped).toBe(false);
|
||||
await session.close();
|
||||
});
|
||||
|
||||
it("clears session state while preserving a typed provider cleanup failure", async () => {
|
||||
const [first] = await targets();
|
||||
const cleanupError = new ProviderCleanupError(
|
||||
"fixture",
|
||||
["fixture-document"],
|
||||
{ reason: "shutdown acknowledgement missing" },
|
||||
);
|
||||
const session = new BinarySession(() => ({
|
||||
execute: () => Promise.resolve(ok(null)),
|
||||
closeWithOutcome: () => Promise.resolve(err(cleanupError)),
|
||||
close: () => Promise.resolve(),
|
||||
}));
|
||||
expect((await session.open(first)).ok).toBe(true);
|
||||
|
||||
expect(await session.close()).toEqual(err(cleanupError));
|
||||
expect(session.status()).toMatchObject({
|
||||
open: false,
|
||||
analysis_activity: { status: "not_observed", providers: [] },
|
||||
});
|
||||
});
|
||||
|
||||
it("does not start a replacement after the active provider cleanup is unconfirmed", async () => {
|
||||
const [first, second] = await targets();
|
||||
const cleanupError = new ProviderCleanupError(
|
||||
"fixture",
|
||||
["fixture-document"],
|
||||
{ reason: "shutdown acknowledgement missing" },
|
||||
);
|
||||
let created = 0;
|
||||
const session = new BinarySession(() => {
|
||||
created += 1;
|
||||
return {
|
||||
execute: () => Promise.resolve(ok(null)),
|
||||
closeWithOutcome: () => Promise.resolve(err(cleanupError)),
|
||||
close: () => Promise.resolve(),
|
||||
};
|
||||
});
|
||||
expect((await session.open(first)).ok).toBe(true);
|
||||
|
||||
expect(await session.open(second)).toEqual(err(cleanupError));
|
||||
expect(created).toBe(1);
|
||||
expect(session.status()).toMatchObject({ open: false });
|
||||
});
|
||||
});
|
||||
|
||||
const client = (fail = false): AnalysisClient => ({
|
||||
|
||||
@@ -95,18 +95,35 @@ describe("executable header parsing", () => {
|
||||
});
|
||||
|
||||
describe("binary target I/O", () => {
|
||||
it("classifies ZIP profiles and text artifacts without Hopper", async () => {
|
||||
it("classifies ZIP package families and text artifacts without Hopper", async () => {
|
||||
const directory = await createTestTempDirectory("rea-artifact-target-");
|
||||
const zip = join(directory, "fixture.apk");
|
||||
const msix = join(directory, "fixture.msixbundle");
|
||||
const appx = join(directory, "fixture.appx");
|
||||
const script = join(directory, "bundle.js");
|
||||
await writeFile(zip, Buffer.from([0x50, 0x4b, 0x05, 0x06, 0, 0, 0, 0]));
|
||||
const emptyZip = Buffer.from([0x50, 0x4b, 0x05, 0x06, 0, 0, 0, 0]);
|
||||
await Promise.all([
|
||||
writeFile(zip, emptyZip),
|
||||
writeFile(msix, emptyZip),
|
||||
writeFile(appx, emptyZip),
|
||||
]);
|
||||
await writeFile(script, "export default 1;\n");
|
||||
const archive = await parseBinaryTarget(zip);
|
||||
const msixArchive = await parseBinaryTarget(msix);
|
||||
const appxArchive = await parseBinaryTarget(appx);
|
||||
const javascript = await parseBinaryTarget(script);
|
||||
expect(archive.ok && archive.value).toMatchObject({
|
||||
kind: "archive",
|
||||
format: "apk",
|
||||
});
|
||||
expect(msixArchive.ok && msixArchive.value).toMatchObject({
|
||||
kind: "archive",
|
||||
format: "msix",
|
||||
});
|
||||
expect(appxArchive.ok && appxArchive.value).toMatchObject({
|
||||
kind: "archive",
|
||||
format: "appx",
|
||||
});
|
||||
expect(javascript.ok && javascript.value).toMatchObject({
|
||||
kind: "artifact",
|
||||
format: "javascript",
|
||||
|
||||
@@ -55,6 +55,22 @@ describe("Hopper bridge truthfulness", () => {
|
||||
expect(bridgeSource).toContain('params.get("collection_offset", {})');
|
||||
});
|
||||
|
||||
it("provides a raw-instruction fast path without decompilation or global scans", () => {
|
||||
const start = bridgeSource.indexOf("def _read_function_instructions");
|
||||
const end = bridgeSource.indexOf("\ndef _analyze_function", start);
|
||||
const implementation = bridgeSource.slice(start, end);
|
||||
expect(implementation).toContain("_instruction_addresses(");
|
||||
expect(implementation).toContain("offset + limit + 1");
|
||||
expect(implementation).not.toContain("decompile(");
|
||||
expect(implementation).not.toContain("_search_inventory(");
|
||||
expect(implementation).not.toContain("_strings(");
|
||||
expect(implementation).not.toContain("_name_map(");
|
||||
expect(bridgeSource).toContain(
|
||||
"pseudo = _pseudocode(document, procedure) or",
|
||||
);
|
||||
expect(bridgeSource).toContain("return _pseudocode(document, procedure)");
|
||||
});
|
||||
|
||||
it("projects opaque Hopper locals into an exact provenance-bearing shape", () => {
|
||||
expect(bridgeSource).toContain("def _procedure_locals(procedure):");
|
||||
expect(bridgeSource).toContain('"description": str(local)');
|
||||
|
||||
@@ -0,0 +1,166 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { buildCapabilityInventory } from "../src/application/CapabilityInventory.js";
|
||||
|
||||
const enabledPolicy: Parameters<typeof buildCapabilityInventory>[1] = {
|
||||
processCaptureEnabled: true,
|
||||
evidenceFileRoots: 1,
|
||||
investigationInputRoots: 1,
|
||||
browserObservationEnabled: true,
|
||||
electronObservationEnabled: true,
|
||||
javascriptReplayEnabled: true,
|
||||
managedRuntimeEnabled: true,
|
||||
};
|
||||
|
||||
const status = (
|
||||
options: {
|
||||
readonly open?: boolean;
|
||||
readonly kind?: "executable" | "database" | "archive" | "artifact";
|
||||
readonly capabilities?: readonly {
|
||||
readonly operation: string;
|
||||
readonly available: boolean;
|
||||
readonly reason: string | null;
|
||||
readonly availability_code?: string;
|
||||
}[];
|
||||
} = {},
|
||||
) => ({
|
||||
open: options.open ?? false,
|
||||
...(options.kind === undefined ? {} : { kind: options.kind }),
|
||||
capabilities: [...(options.capabilities ?? [])],
|
||||
});
|
||||
|
||||
const entry = (
|
||||
name: string,
|
||||
sessionStatus: ReturnType<typeof status>,
|
||||
policy = enabledPolicy,
|
||||
) => {
|
||||
const found = buildCapabilityInventory(sessionStatus, policy).find(
|
||||
(candidate) => candidate.name === name,
|
||||
);
|
||||
if (found === undefined) throw new Error(`missing capability ${name}`);
|
||||
return found;
|
||||
};
|
||||
|
||||
describe("capability inventory", () => {
|
||||
it.each([
|
||||
{
|
||||
label: "requires a target",
|
||||
name: "current_address",
|
||||
sessionStatus: status(),
|
||||
reason: "target_required",
|
||||
remediation: "open_binary",
|
||||
},
|
||||
{
|
||||
label: "rejects an incompatible target family",
|
||||
name: "current_address",
|
||||
sessionStatus: status({ open: true, kind: "archive" }),
|
||||
reason: "target_unsupported",
|
||||
remediation: "native executable",
|
||||
},
|
||||
{
|
||||
label: "reports a missing provider operation",
|
||||
name: "current_address",
|
||||
sessionStatus: status({ open: true, kind: "executable" }),
|
||||
reason: "provider_missing",
|
||||
remediation: "provider",
|
||||
},
|
||||
{
|
||||
label: "distinguishes an unsupported host",
|
||||
name: "current_address",
|
||||
sessionStatus: status({
|
||||
open: true,
|
||||
kind: "executable",
|
||||
capabilities: [
|
||||
{
|
||||
operation: "current_address",
|
||||
available: false,
|
||||
availability_code: "unsupported_host",
|
||||
reason: "Operation requires macOS",
|
||||
},
|
||||
],
|
||||
}),
|
||||
reason: "unsupported_host",
|
||||
remediation: "macOS",
|
||||
},
|
||||
{
|
||||
label: "retains a provider failure",
|
||||
name: "current_address",
|
||||
sessionStatus: status({
|
||||
open: true,
|
||||
kind: "executable",
|
||||
capabilities: [
|
||||
{
|
||||
operation: "current_address",
|
||||
available: false,
|
||||
reason: "Provider session is unhealthy",
|
||||
},
|
||||
],
|
||||
}),
|
||||
reason: "provider_unavailable",
|
||||
remediation: "unhealthy",
|
||||
},
|
||||
{
|
||||
label: "reports an available provider operation",
|
||||
name: "current_address",
|
||||
sessionStatus: status({
|
||||
open: true,
|
||||
kind: "executable",
|
||||
capabilities: [
|
||||
{
|
||||
operation: "current_address",
|
||||
available: true,
|
||||
reason: null,
|
||||
},
|
||||
],
|
||||
}),
|
||||
reason: "available",
|
||||
remediation: null,
|
||||
},
|
||||
])("$label", ({ name, sessionStatus, reason, remediation }) => {
|
||||
const availability = entry(name, sessionStatus);
|
||||
expect(availability).toMatchObject({
|
||||
reason,
|
||||
available: reason === "available",
|
||||
});
|
||||
if (remediation === null) expect(availability.remediation).toBeNull();
|
||||
else expect(availability.remediation).toContain(remediation);
|
||||
});
|
||||
|
||||
it.each([
|
||||
["capture_process_scenario", { processCaptureEnabled: false }],
|
||||
["inspect_web_page", { browserObservationEnabled: false }],
|
||||
["inspect_electron_page", { electronObservationEnabled: false }],
|
||||
["run_controlled_replay", { javascriptReplayEnabled: false }],
|
||||
["plan_managed_runtime_correlation", { managedRuntimeEnabled: false }],
|
||||
] as const)("reports stable policy denial for %s", (name, override) => {
|
||||
const availability = entry(name, status(), {
|
||||
...enabledPolicy,
|
||||
...override,
|
||||
});
|
||||
expect(availability).toMatchObject({
|
||||
available: false,
|
||||
reason: "policy_disabled",
|
||||
});
|
||||
expect(availability.remediation).toEqual(expect.any(String));
|
||||
expect(availability.remediation?.length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it("gives every unavailable public tool an actionable remediation", () => {
|
||||
const inventory = buildCapabilityInventory(status(), {
|
||||
...enabledPolicy,
|
||||
processCaptureEnabled: false,
|
||||
evidenceFileRoots: 0,
|
||||
investigationInputRoots: 0,
|
||||
browserObservationEnabled: false,
|
||||
electronObservationEnabled: false,
|
||||
javascriptReplayEnabled: false,
|
||||
managedRuntimeEnabled: false,
|
||||
});
|
||||
for (const availability of inventory)
|
||||
if (!availability.available) {
|
||||
expect(availability.reason).not.toBe("available");
|
||||
expect(availability.remediation).toEqual(expect.any(String));
|
||||
expect(availability.remediation?.length).toBeGreaterThan(0);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -1361,7 +1361,9 @@ describe("CdpBrowserProvider", () => {
|
||||
}),
|
||||
{ signal: controller.signal },
|
||||
);
|
||||
await waitForCommand(browser, "Target.attachToTarget");
|
||||
// Wait until the attach response has been consumed and capture has begun.
|
||||
// Seeing the attach request alone races its response under worker load.
|
||||
await waitForCommand(browser, "Page.getFrameTree");
|
||||
controller.abort();
|
||||
const result = await pending;
|
||||
expect(result).toMatchObject({
|
||||
|
||||
@@ -18,7 +18,7 @@ the read-only Ghidra inventory contracts without committing a binary.
|
||||
`scripts/create-ghidra-windows-fixture.mjs` separately emits a deterministic,
|
||||
ignored native x86-64 PE application for the controlled Windows Ghidra P0 lane.
|
||||
The generator and fixed SHA-256 are versioned; the `.exe` is never committed or
|
||||
executed. `npm run verify:ghidra:windows` uses it to prove all 18 operations,
|
||||
executed. `npm run verify:ghidra:windows` uses it to prove all 19 operations,
|
||||
digest linkage, transport, and cleanup on the self-hosted real-Ghidra runner.
|
||||
|
||||
The C fixture fixes a known call chain and strings. The version pair fixes an
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user