Compare commits

..
Author SHA1 Message Date
morluto 36cbe31e19 perf(test): speed up local and CI validation 2026-07-24 03:02:19 +00:00
github-actions[bot]andGitHub 092de8f08b chore(main): release rea-agents 2.4.1 2026-07-23 16:02:38 +00:00
N0zoM1z0andN0zoM1z0 0b7013134c test(package): stabilize fake Hopper ownership 2026-07-24 00:33:29 +09:00
N0zoM1z0andN0zoM1z0 e0bc7fd973 fix(hopper): add bounded analysis recovery 2026-07-24 00:33:29 +09:00
morlutoandGitHub 161377a505 Merge pull request #427 from morluto/release-please--branches--main--components--rea-agents
chore(main): release rea-agents 2.4.0
2026-07-23 16:06:10 +08:00
morluto 732d606609 fix: regenerate release product catalog 2026-07-23 08:00:58 +00:00
morluto c9bd91d7c3 ci: remove release check trigger 2026-07-23 07:53:53 +00:00
morluto e698c977ed ci: trigger release checks 2026-07-23 07:53:03 +00:00
morluto 31a3cfa7e9 chore: trigger release checks 2026-07-23 07:51:35 +00:00
github-actions[bot]andGitHub 829d9a0933 chore(main): release rea-agents 2.4.0 2026-07-23 07:39:29 +00:00
morlutoandGitHub 9da179425e Merge pull request #424 from morluto/codex/reactive-capture-and-package-formats
feat: expand reactive capture and package analysis
2026-07-23 15:36:55 +08:00
morluto b906defbff fix(capabilities): expose typed availability codes 2026-07-23 07:09:16 +00:00
morluto 6e6cce1bf3 feat(artifacts): support MSIX and AppX packages 2026-07-23 07:09:04 +00:00
morluto 69761e95b6 feat(process): add reactive capture scenarios 2026-07-23 07:08:47 +00:00
125 changed files with 4241 additions and 579 deletions
+26 -2
View File
@@ -82,7 +82,7 @@ jobs:
test-shard:
name: Test shard (${{ matrix.shard }})
needs: changes
needs: [changes, build]
if: github.event_name == 'push' || needs.changes.outputs.code == 'true'
runs-on: ubuntu-latest
strategy:
@@ -96,7 +96,11 @@ jobs:
node-version-file: .nvmrc
cache: npm
- run: npm ci
- run: npm run test:ci:shard -- --shard=${{ matrix.shard }}
- uses: actions/download-artifact@v4
with:
name: rea-dist
path: dist
- run: npm run test:ci:shard:run -- --shard=${{ matrix.shard }}
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
@@ -107,6 +111,26 @@ jobs:
overwrite: true
retention-days: 1
build:
name: Build package
needs: changes
if: github.event_name == 'push' || needs.changes.outputs.code == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version-file: .nvmrc
cache: npm
- run: npm ci
- run: npm run build:cached
- uses: actions/upload-artifact@v4
with:
name: rea-dist
path: dist
if-no-files-found: error
retention-days: 1
test:
name: Test (coverage + timing)
needs: [changes, test-shard]
+1
View File
@@ -37,6 +37,7 @@ coverage/
.vitest-reports/
test-results.xml
.turbo/
.cache/
# OS
.DS_Store
+1 -1
View File
@@ -1,3 +1,3 @@
{
".": "2.3.0"
".": "2.4.1"
}
+4 -3
View File
@@ -41,7 +41,7 @@ See [docs/architecture.mermaid](docs/architecture.mermaid) for a visual architec
- `tests/`: Vitest suite. `tests/fixtures/` holds reusable provider-process fixtures, source-owned synthetic JavaScript graph and Electron/Webpack/Rspack artifact trees, and the fake launcher, Hopper bridge, and CDP seams.
- `scripts/verify-real-hopper.mjs`: real-Hopper end-to-end verifier.
- `scripts/verify-real-ghidra.mjs`: real Linux Ghidra verifier for x86-64 debug/stripped ELF, AArch64 ELF, PE, and Mach-O function semantics plus complete cleanup.
- `scripts/verify-real-ghidra-windows.mjs`: real Windows x64 Ghidra P0 verifier for the source-owned native PE fixture, all 18 admitted operations, digest linkage, loopback transport, and cleanup.
- `scripts/verify-real-ghidra-windows.mjs`: real Windows x64 Ghidra P0 verifier for the source-owned native PE fixture, all 19 admitted operations, digest linkage, loopback transport, and cleanup.
- `scripts/verify-real-browser.mjs`: real Chrome end-to-end verifier for the passive CDP provider.
- `scripts/print-mcp-config.mjs`: prints an MCP server config with absolute paths filled in (`npm run config:print -- /path/to/binary`).
@@ -54,9 +54,10 @@ See [docs/architecture.mermaid](docs/architecture.mermaid) for a visual architec
- `npm run build`: compile `src/` into `dist/`.
- `npm run build:cached`: compile through Turbo and reuse a valid local build cache entry.
- `npm test`: restore or build `dist/`, then run the Vitest suite once without coverage or retries locally.
- `npm run test:fast`: run the parallel test project without the serial integration group.
- `npm run test:fast`: run pure and subprocess test projects without the serial integration group.
- `npm run test:changed`: run changed and related non-serial tests once.
- `npm run test:integration`: run the serial filesystem, process, and CLI integration group.
- `npm run test:watch`: watch tests affected by working-tree changes in the parallel project, without coverage.
- `npm run test:watch`: watch changed and related non-serial tests, without coverage.
- `npm run test:coverage`: run the complete suite with coverage; CI shards the suite and owns JUnit reporting.
- `npm run typecheck`: run strict TypeScript checks without emitting files.
- `npm run lint`: apply oxlint rules (complexity, max-lines, unused vars, and TypeScript-specific checks).
+26
View File
@@ -1,5 +1,31 @@
# Changelog
## [2.4.1](https://github.com/morluto/rea/compare/rea-agents-2.4.0...rea-agents-2.4.1) (2026-07-23)
### Bug Fixes
* **hopper:** add bounded analysis recovery ([e0bc7fd](https://github.com/morluto/rea/commit/e0bc7fd973b26448fb2192a2059ab1229bec0817))
### Tests
* **package:** stabilize fake Hopper ownership ([0b70131](https://github.com/morluto/rea/commit/0b7013134c58085b72177f2954e55534be053ebe))
## [2.4.0](https://github.com/morluto/rea/compare/rea-agents-2.3.0...rea-agents-2.4.0) (2026-07-23)
### Features
* **artifacts:** support MSIX and AppX packages ([6e6cce1](https://github.com/morluto/rea/commit/6e6cce1bf3cbd9c48799bee9969c53d66c42162e))
* expand reactive capture and package analysis ([9da1794](https://github.com/morluto/rea/commit/9da179425e83df9f255597437320c1ab6ff5c5c2))
* **process:** add reactive capture scenarios ([69761e9](https://github.com/morluto/rea/commit/69761e95b63ec7617aa161f7a1b4e3c8847a6175))
### Bug Fixes
* **capabilities:** expose typed availability codes ([b906def](https://github.com/morluto/rea/commit/b906defbff28885a1d39f78da7c2b7170a7108c6))
## [2.3.0](https://github.com/morluto/rea/compare/rea-agents-2.2.0...rea-agents-2.3.0) (2026-07-23)
+5 -3
View File
@@ -39,9 +39,11 @@ prerequisite uses Turbo. CI uploads the rendered TypeDoc site as an `api-docs`
artifact; the generated HTML is not committed.
Local `npm test` runs without coverage, retries, or verbose output. Use `npm run
test:fast` for the parallel group, `npm run test:integration` for serial
filesystem/process/CLI cases, and `npm run test:watch` to watch tests affected by
working-tree changes. CI splits the complete suite across two native Vitest
test:fast` for pure and subprocess groups, `npm run test:integration` for serial
filesystem/process/CLI cases. `npm run test:changed` runs changed and related
non-serial tests once; `npm run test:watch` keeps that selection live. Pure tests
use Vitest threads, while subprocess tests stay in isolated forks. CI splits the
complete suite across two native Vitest
shards, then merges their coverage and JUnit reports. `npm run test:ci` runs the
equivalent unsharded gate locally. Coverage thresholds remain in
`vitest.config.ts`.
+18 -7
View File
@@ -237,13 +237,13 @@ On Windows, set the same variables in PowerShell and run `rea doctor --json`; au
REA loads its packaged Java `HeadlessScript` with `-scriptPath`, copies and digest-verifies the target in an ephemeral runtime, enables `-readOnly` and `-deleteProject`, caps auto-analysis at 300 seconds with two CPUs and a 2 GiB Java heap, and authenticates every request. Linux uses a mode-0600 Unix socket. Windows P0 uses token-authenticated IPv4 loopback and a token-free endpoint record because Node path-based IPC does not connect to Java AF_UNIX sockets on Windows. The bridge verifies Ghidra's imported-byte SHA-256 before serving any operation.
The Ghidra adapter declares 18 direct and enhanced operations. Its ten inventory operations are `list_documents`, `list_procedures`, `list_strings`, `list_names`, `list_segments`, `address_name`, `procedure_address`, `resolve_containing_procedure`, `search_procedures`, and `search_strings`. It also admits `procedure_info`, `procedure_pseudo_code`, `procedure_assembly`, `procedure_callers`, `procedure_callees`, `procedure_references`, `xrefs`, and `analyze_function`. These capabilities enable the shared Swift/Objective-C inventory workflows, `binary_overview`, `batch_decompile`, `get_call_graph`, `find_xrefs_to_name`, `trace_feature`, and complete function dossiers. Default-space addresses are lowercase `0x` hexadecimal. Other spaces, including `EXTERNAL`, use `<percent-encoded-space>:0x<hex>`. Symbol results identify primary, dynamic, external, type, and source facts; procedures distinguish external functions and thunks; strings identify charset, missing-terminator state, byte length, and value truncation; memory-block ends are exclusive and permissions come directly from Ghidra.
The Ghidra adapter declares 19 direct and enhanced operations. Its ten inventory operations are `list_documents`, `list_procedures`, `list_strings`, `list_names`, `list_segments`, `address_name`, `procedure_address`, `resolve_containing_procedure`, `search_procedures`, and `search_strings`. It also admits `procedure_info`, `procedure_pseudo_code`, `procedure_assembly`, `read_function_instructions`, `procedure_callers`, `procedure_callees`, `procedure_references`, `xrefs`, and `analyze_function`. `read_function_instructions` is the offset-paginated fast path for raw instruction windows: it does not invoke the decompiler or whole-program name/string inventories, and is also exposed as `rea instructions`. These capabilities enable the shared Swift/Objective-C inventory workflows, `binary_overview`, `batch_decompile`, `get_call_graph`, `find_xrefs_to_name`, `trace_feature`, and complete function dossiers. Default-space addresses are lowercase `0x` hexadecimal. Other spaces, including `EXTERNAL`, use `<percent-encoded-space>:0x<hex>`. Symbol results identify primary, dynamic, external, type, and source facts; procedures distinguish external functions and thunks; strings identify charset, missing-terminator state, byte length, and value truncation; memory-block ends are exclusive and permissions come directly from Ghidra.
The bridge serves operations only after auto-analysis completes. Each Program owns one persistent `DecompInterface`; a bounded 32-request FIFO serializes Ghidra API access, and every decompile has a 30-second native deadline. Reference results preserve Ghidra's call/jump/data/read/write/indirect/computed/external facts, while unresolved targetless flows remain explicitly unknown. Synthetic entry-point references without actionable memory sources are omitted. Pseudocode and assembly are provider-specific observations, not original source or Hopper-equivalent text. An analysis timeout, scan or inventory safety limit, request timeout, or oversized response fails explicitly instead of returning a partial result labeled complete.
`npm run verify:ghidra` compiles source-owned x86-64 debug and stripped ELF, AArch64 ELF, x86-64 PE, and x86-64 Mach-O fixtures. Against real Ghidra 12.1.2 it validates every admitted operation, direct and indirect calls, imports/exports/thunks, typed references, strings/xrefs, multi-block CFG, cancellation, deadlines, concurrency, malformed inputs, and complete process/project cleanup. Set `REA_CC`, `REA_CLANG`, or `REA_LLD_LINK` only when the corresponding compiler command is not on `PATH`.
`npm run verify:ghidra:windows` uses a deterministic source-owned native x86-64 PE application and requires all 18 operations, target/snapshot/import digest linkage, authenticated loopback transport, and cleanup on a controlled Windows x64 Ghidra 12.1.2 runner. This proof does not establish Job Object ownership, private DACLs, or reparse-point-safe authority.
`npm run verify:ghidra:windows` uses a deterministic source-owned native x86-64 PE application and requires all 19 operations, target/snapshot/import digest linkage, authenticated loopback transport, and cleanup on a controlled Windows x64 Ghidra 12.1.2 runner. This proof does not establish Job Object ownership, private DACLs, or reparse-point-safe authority.
To remove only REA-owned MCP registrations and the managed skill:
@@ -373,10 +373,10 @@ REA handles the app analysis in steps 1–5. The agent performs step 6 with its
| Tool family | Count | Examples |
| ------------------------- | ----: | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Native inspection | 33 | procedures, pseudocode, assembly, strings, names, segments, callers, callees, xrefs, annotations |
| Native inspection | 34 | procedures, pseudocode, assembly, strings, names, segments, callers, callees, xrefs, annotations |
| Investigation workflows | 10 | `binary_overview`, `analyze_function`, `batch_decompile`, `trace_feature`, call graphs, Swift and Objective-C discovery |
| Native macOS utilities | 5 | Mach-O metadata, code signatures, plists, architectures, Swift demangling; Hopper-free and provenance-bearing |
| Artifact graph | 2 | deterministic directory, ZIP/APK/IPA, and ASAR inventory; explicitly selected extraction into an absent owned tree |
| Artifact graph | 2 | deterministic directory, ZIP/APK/IPA/MSIX/AppX, and ASAR inventory; explicitly selected extraction into an absent owned tree |
| Managed PE/CLI | 8 | PE/CLI identity, metadata members, CIL hashes, P/Invoke/native-boundary declarations and verification, application-graph projection, decompiler reconstruction import, token remapping, runtime-correlation plans, and version comparison |
| Browser observation | 8 | exact-origin CDP capture, bundle and source-map analysis, WebMCP discovery, session timelines, capture diff, and visual evidence |
| Electron analysis | 4 | passive root-confined observation, bounded static application mapping, and evidence-backed static/runtime reconciliation |
@@ -479,7 +479,7 @@ Setup detects Claude Code, Claude Desktop, Codex, Cursor, Gemini CLI, Windsurf,
"mcpServers": {
"rea": {
"command": "npx",
"args": ["-y", "rea-agents@2.3.0", "mcp"]
"args": ["-y", "rea-agents@2.4.1", "mcp"]
}
}
}
@@ -592,10 +592,17 @@ and retains one provider-attributed, token-verified observation per started
provider. Each observation carries `observed_at` and remains `unavailable` or
`verified`; a verified empty descendant list is distinct from both. Snapshots
describe bounded observations, not current live state or historical absence.
For providers with serial work, `analysis_activity` distinguishes `idle`,
`busy`, and `timed_out_busy`, and reports the active operation, elapsed time,
caller state, timeout, and queued-request count. A caller timeout therefore
does not falsely imply that Hopper's Python thread is available. `close_binary`
clears the REA session but returns `cleanup_incomplete` when authenticated
document shutdown, owned process cleanup, or private runtime removal cannot be
verified.
Reopening same target without a selector keeps its binding; runtime failure
never selects another provider silently.
Ghidra can appear as an available, target-compatible candidate after doctor
validates its exact installation. Its capability list contains the 18 admitted
validates its exact installation. Its capability list contains the 19 admitted
read-only inventory and function-analysis operations; selecting it still does
not make Hopper-only GUI or mutation operations available and never triggers a
silent fallback.
@@ -627,7 +634,11 @@ REA starts Hopper when needed; Hopper does not need to be running first. Hopper'
REA derives explicit format and architecture arguments to prevent common FAT and ARM selection dialogs. Other Hopper or macOS dialogs may still require a person. REA reports timeouts and remediation through CLI or MCP results instead of attempting to answer UI prompts.
Closing a REA session shuts down its bridge and removes its private socket directory. It does not quit a Hopper application the user may be using.
Hopper bridge calls pass through a bounded serial FIFO because Hopper's Python API runs on one dedicated thread. Queue wait counts against the caller deadline; a timeout settles the caller but does not release the wire slot until Hopper actually replies. `binary_session.analysis_activity` exposes that detached work and MCP progress reports elapsed time. Successful decompilation text is cached per document and procedure, shared by pseudocode and dossier requests, and invalidated after rename or comment mutation.
Hopper's synchronous public Python calls cannot stream a partial response after the caller deadline. Use `read_function_instructions` or `rea instructions` when raw instruction orientation is enough: this path does not decompile or scan whole-program name/string inventories. If a dossier has already timed out, inspect `analysis_activity`; do not assume a raw request can enter the serial bridge until that detached Hopper call actually returns.
Closing a REA session shuts down its bridge and removes its private socket directory. It does not quit a Hopper application the user may be using. If shutdown or cleanup cannot be verified, `close_binary` returns `cleanup_incomplete` with the affected local resources.
## Advanced process-capture setup
+2 -2
View File
@@ -161,7 +161,7 @@ rea uninstall --purge-data
| عائلة الأدوات | العدد | أمثلة |
| --------------------- | ----: | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| فحص الملفات التنفيذية | 33 | الدوال، والشيفرة شبه المصدرية، والتعليمات، والسلاسل، والأسماء، والمراجع، والتعليقات |
| فحص الملفات التنفيذية | 34 | الدوال، والشيفرة شبه المصدرية، والتعليمات، والسلاسل، والأسماء، والمراجع، والتعليقات |
| التحليل المركب | 10 | النظرة العامة، وفك الترجمة الدفعي، ومخططات الاستدعاء، والمراجع، واكتشاف الأنواع |
| أدوات macOS الأصلية | 5 | بيانات Mach-O والتوقيعات وملفات plist والمعماريات وفك رموز Swift دون تشغيل Hopper |
| رسم القطع الأثرية | 2 | جرد حتمي للدلائل وZIP/APK/IPA وASAR واستخراج معاملاتي محدد صراحة |
@@ -182,7 +182,7 @@ rea uninstall --purge-data
"mcpServers": {
"rea": {
"command": "npx",
"args": ["-y", "rea-agents@2.3.0", "mcp"]
"args": ["-y", "rea-agents@2.4.1", "mcp"]
}
}
}
+2 -2
View File
@@ -151,7 +151,7 @@ REA は手順 1〜5 のバイナリ解析を処理し、手順 6 はエージェ
| ツール群 | 数 | 例 |
| ---------------------- | --: | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| バイナリ調査 | 33 | プロシージャ、疑似コード、アセンブリ、文字列、名前、セグメント、callers、callees、xrefs、注釈 |
| バイナリ調査 | 34 | プロシージャ、疑似コード、アセンブリ、文字列、名前、セグメント、callers、callees、xrefs、注釈 |
| 合成解析 | 10 | `binary_overview`, `analyze_function`, `batch_decompile`, `get_call_graph`, `find_xrefs_to_name`, Swift / ObjC 検出 |
| macOS ネイティブ | 5 | Mach-O メタデータ、署名、plist、アーキテクチャ、Swift デマングル。Hopper 起動不要 |
| アーティファクトグラフ | 2 | ディレクトリ、ZIP/APK/IPA、ASAR の決定的一覧と明示選択されたトランザクション抽出 |
@@ -172,7 +172,7 @@ Setup は Claude Code、Claude Desktop、Codex、Cursor、Gemini CLI、Windsurf
"mcpServers": {
"rea": {
"command": "npx",
"args": ["-y", "rea-agents@2.3.0", "mcp"]
"args": ["-y", "rea-agents@2.4.1", "mcp"]
}
}
}
+2 -2
View File
@@ -151,7 +151,7 @@ REA는 1–5단계의 바이너리 분석을 처리합니다. 6단계는 에이
| 도구 그룹 | 수 | 예시 |
| ------------------- | --: | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 바이너리 검사 | 33 | 프로시저, 의사 코드, 어셈블리, 문자열, 이름, 세그먼트, callers, callees, xrefs, 주석 |
| 바이너리 검사 | 34 | 프로시저, 의사 코드, 어셈블리, 문자열, 이름, 세그먼트, callers, callees, xrefs, 주석 |
| 합성 분석 | 10 | `binary_overview`, `analyze_function`, `batch_decompile`, `get_call_graph`, `find_xrefs_to_name`, Swift 및 ObjC 탐색 |
| macOS 네이티브 도구 | 5 | Mach-O 메타데이터, 코드 서명, plist, 아키텍처, Swift 디맹글링. Hopper 실행 불필요 |
| 아티팩트 그래프 | 2 | 디렉터리, ZIP/APK/IPA, ASAR 결정적 인벤토리와 명시적으로 선택한 트랜잭션 추출 |
@@ -172,7 +172,7 @@ Setup은 Claude Code, Claude Desktop, Codex, Cursor, Gemini CLI, Windsurf, Devin
"mcpServers": {
"rea": {
"command": "npx",
"args": ["-y", "rea-agents@2.3.0", "mcp"]
"args": ["-y", "rea-agents@2.4.1", "mcp"]
}
}
}
+2 -2
View File
@@ -151,7 +151,7 @@ REA 负责第 1–5 步中的二进制分析。第 6 步由智能体使用其常
| 工具类别 | 数量 | 示例 |
| -------------- | ---: | --------------------------------------------------------------------------------------------------------------------------------------------- |
| 二进制检查 | 33 | 过程、伪代码、汇编、字符串、名称、段、调用者、被调用者、交叉引用、注释 |
| 二进制检查 | 34 | 过程、伪代码、汇编、字符串、名称、段、调用者、被调用者、交叉引用、注释 |
| 组合分析 | 10 | `binary_overview`, `analyze_function`, `batch_decompile`, `get_call_graph`, `find_xrefs_to_name`、Swift 与 ObjC 发现 |
| macOS 原生工具 | 5 | Mach-O 元数据、代码签名、plist、架构与 Swift 符号还原,无需启动 Hopper |
| 制品图谱 | 2 | 确定性目录、ZIP/APK/IPA 与 ASAR 清单,以及显式选择的事务式提取 |
@@ -172,7 +172,7 @@ Setup 会检测 Claude Code、Claude Desktop、Codex、Cursor、Gemini CLI、Win
"mcpServers": {
"rea": {
"command": "npx",
"args": ["-y", "rea-agents@2.3.0", "mcp"]
"args": ["-y", "rea-agents@2.4.1", "mcp"]
}
}
}
+48 -1
View File
@@ -72,7 +72,7 @@ import ghidra.program.model.symbol.Symbol;
import ghidra.program.model.symbol.SymbolIterator;
public final class ReaGhidraBridge extends HeadlessScript {
private static final int BRIDGE_VERSION = 4;
private static final int BRIDGE_VERSION = 5;
private static final int MAX_DESCRIPTOR_BYTES = 16 * 1024;
private static final int MAX_REQUEST_CHARACTERS = 256 * 1024;
private static final int MAX_RESPONSE_BYTES = 1024 * 1024;
@@ -123,6 +123,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
"procedure_callers",
"procedure_info",
"procedure_pseudo_code",
"read_function_instructions",
"procedure_references",
"xrefs"
};
@@ -311,6 +312,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
case "procedure_callers" -> procedureCalls(request.params, true);
case "procedure_info" -> procedureInfo(request.params);
case "procedure_pseudo_code" -> procedurePseudocode(request.params);
case "read_function_instructions" -> readFunctionInstructions(request.params);
case "procedure_references" -> procedureReferences(request.params);
case "resolve_containing_procedure" -> containingProcedure(request.params);
case "search_procedures" -> search(request.params, true);
@@ -390,6 +392,51 @@ public final class ReaGhidraBridge extends HeadlessScript {
return GSON.toJsonTree(renderAssembly(scan.instructions));
}
private JsonObject readFunctionInstructions(JsonObject params) throws Exception {
requireKeys(params, Set.of("document", "procedure", "offset", "limit"));
requireDocument(params);
Function function = resolveProcedure(requireString(params, "procedure"));
int offset = requireBoundedInteger(params, "offset", 0, 100_000);
int limit = requireBoundedInteger(params, "limit", 1, 500);
InstructionScan scan = scanInstructions(function, offset + limit + 1);
int start = Math.min(offset, scan.instructions.size());
int end = Math.min(scan.instructions.size(), start + limit);
JsonArray items = new JsonArray();
for (String line : renderAssemblyLines(scan.instructions.subList(start, end))) {
items.add(line);
}
boolean hasMore = scan.truncated || end < scan.instructions.size();
JsonObject instructions = new JsonObject();
instructions.add("items", items);
if (scan.truncated) {
instructions.add("total", JsonNull.INSTANCE);
}
else {
instructions.addProperty("total", scan.instructions.size());
}
instructions.addProperty("returned", items.size());
instructions.addProperty("truncated", hasMore);
if (hasMore) {
instructions.addProperty("next_offset", end);
}
else {
instructions.add("next_offset", JsonNull.INSTANCE);
}
JsonArray limitations = new JsonArray();
limitations.add("Instruction text and ordering are Ghidra-specific representations.");
limitations.add(
"The fast path does not invoke the decompiler or scan whole-program names and strings."
);
JsonObject result = new JsonObject();
result.add("procedure", procedureIdentity(function));
result.add("instructions", instructions);
result.addProperty("instructions_scanned", scan.instructions.size());
result.addProperty("instruction_scan_truncated", scan.truncated);
result.add("limitations", limitations);
return result;
}
private JsonArray procedureCalls(JsonObject params, boolean callers) throws Exception {
requireKeys(params, Set.of("document", "procedure"));
requireDocument(params);
+117 -20
View File
@@ -16,6 +16,7 @@ MAX_LINE_BYTES = 10 * 1024 * 1024
BAD_ADDRESSES = (-1, 0xFFFFFFFFFFFFFFFF, None)
_selected_document = None
_search_inventory_cache = {}
_pseudocode_cache = {}
MAX_SEARCH_PATTERN_LENGTH = 256
MAX_SEARCH_VALUE_LENGTH = 4096
@@ -225,12 +226,35 @@ def _invalidate_search_inventory(document):
del _search_inventory_cache[key]
def _invalidate_pseudocode(document):
"""Discard cached decompilation after an analysis annotation changes."""
document_id = id(document)
for key in list(_pseudocode_cache):
if key[0] == document_id:
del _pseudocode_cache[key]
def _pseudocode(document, procedure):
"""Reuse one provider decompilation for repeated bounded projections."""
key = (id(document), procedure.getEntryPoint())
if key not in _pseudocode_cache:
_pseudocode_cache[key] = procedure.decompile()
return _pseudocode_cache[key]
def _search_inventory(document, kind):
"""Cache an immutable, address-sorted inventory for an unchanged document."""
key = (id(document), kind)
inventory = _search_inventory_cache.get(key)
if inventory is None:
values = _procedure_map(document) if kind == "procedure" else _strings(document)
if kind == "procedure":
values = _procedure_map(document)
elif kind == "string":
values = _strings(document)
elif kind == "name":
values = {_hex(address): value for address, value in _name_map(document).items()}
else:
raise ValueError("Unknown inventory kind")
inventory = tuple(sorted(values.items(), key=lambda item: int(item[0], 16)))
_search_inventory_cache[key] = inventory
return inventory
@@ -507,6 +531,22 @@ def _name_map(document):
return result
def _render_instruction(segment, address):
instruction = segment.getInstructionAtAddress(address)
if instruction is None:
return None
arguments = [
instruction.getFormattedArgument(index)
for index in range(instruction.getArgumentCount())
]
suffix = ", ".join(value for value in arguments if value is not None)
return "%s: %s%s" % (
_hex(address),
instruction.getInstructionString(),
(" " + suffix) if suffix else "",
)
def _assembly(procedure, limit=None):
"""Render bounded assembly while guarding against malformed instruction cycles."""
lines = []
@@ -520,9 +560,9 @@ def _assembly(procedure, limit=None):
instruction = segment.getInstructionAtAddress(address)
if instruction is None:
break
arguments = [instruction.getFormattedArgument(index) for index in range(instruction.getArgumentCount())]
suffix = ", ".join(value for value in arguments if value is not None)
lines.append("%s: %s%s" % (_hex(address), instruction.getInstructionString(), (" " + suffix) if suffix else ""))
line = _render_instruction(segment, address)
if line is not None:
lines.append(line)
if limit is not None and len(lines) >= limit:
return "\n".join(lines)
length = instruction.getInstructionLength()
@@ -532,6 +572,45 @@ def _assembly(procedure, limit=None):
return "\n".join(lines)
def _read_function_instructions(document, params):
"""Read one bounded raw-instruction window without invoking decompilation."""
procedure = _procedure(document, params.get("procedure"))
offset = _offset(params, "offset")
limit = params.get("limit", 64)
if offset > 100000:
raise ValueError("offset must not exceed 100000")
if not isinstance(limit, int) or isinstance(limit, bool) or limit < 1 or limit > 500:
raise ValueError("limit must be an integer between 1 and 500")
addresses, scan_truncated = _instruction_addresses(
procedure, offset + limit + 1
)
start = min(offset, len(addresses))
end = min(len(addresses), start + limit)
segment = procedure.getSegment()
items = []
for address in addresses[start:end]:
line = _render_instruction(segment, address)
if line is not None:
items.append(line)
has_more = scan_truncated or end < len(addresses)
return {
"procedure": _procedure_identity(procedure),
"instructions": {
"items": items,
"total": None if scan_truncated else len(addresses),
"returned": len(items),
"truncated": has_more,
"next_offset": end if has_more else None,
},
"instructions_scanned": len(addresses),
"instruction_scan_truncated": scan_truncated,
"limitations": [
"Instruction text and ordering are Hopper-specific representations.",
"The fast path does not decompile the procedure or scan whole-program names and strings.",
],
}
def _analyze_function(document, params):
"""Collect a bounded, single-pass function dossier for agent callers."""
procedure = _procedure(document, params.get("procedure"))
@@ -560,8 +639,8 @@ def _analyze_function(document, params):
"end": _hex(block.getEndingAddress()),
"successors": sorted(set(successors), key=lambda value: int(value, 16)),
})
pseudo = procedure.decompile() or ""
assembly_lines = _assembly(procedure).splitlines() if params.get("include_assembly", False) else []
pseudo = _pseudocode(document, procedure) or ""
assembly_lines = _assembly(procedure, max_instructions).splitlines() if params.get("include_assembly", False) else []
callers = sorted((_procedure_identity(item) for item in procedure.getAllCallerProcedures()), key=lambda item: int(item["address"], 16))
callees = sorted((_procedure_identity(item) for item in procedure.getAllCalleeProcedures()), key=lambda item: int(item["address"], 16))
comments = []
@@ -595,8 +674,14 @@ def _analyze_function(document, params):
incoming.append(item)
if source in procedure_addresses:
outgoing.append(item)
string_map = {int(address, 16): value for address, value in _strings(document).items()}
name_map = _name_map(document)
string_map = {
int(address, 16): value
for address, value in _search_inventory(document, "string")
}
name_map = {
int(address, 16): value
for address, value in _search_inventory(document, "name")
}
referenced_strings = []
referenced_names = []
for edge in outgoing:
@@ -615,7 +700,13 @@ def _analyze_function(document, params):
return {
"procedure": {"address": _hex(procedure.getEntryPoint()), "name": _procedure_name(procedure), "classification": None, "signature": procedure.signatureString(), "locals": _procedure_locals(procedure)},
"pseudocode": {"text": pseudo_text, "total_chars": len(pseudo), "returned_chars": len(pseudo_text), "truncated": pseudo_next < len(pseudo), "next_offset": pseudo_next if pseudo_next < len(pseudo) else None},
"assembly": _bounded(assembly_lines, assembly_offset, max_instructions),
"assembly": _bounded(
assembly_lines,
assembly_offset,
max_instructions,
None,
instruction_scan_truncated if params.get("include_assembly", False) else False,
),
"comments": collection("comments", comments, instruction_scan_truncated),
"callers": collection("callers", callers), "callees": collection("callees", callees),
"incoming_references": collection("incoming_references", incoming, instruction_scan_truncated),
@@ -674,6 +765,8 @@ def _dispatch(method, params):
if method == "analyze_function":
return _analyze_function(document, params)
if method == "read_function_instructions":
return _read_function_instructions(document, params)
if method == "resolve_containing_procedure":
address = _address(document, params.get("address"))
procedure, reason = _containing_procedure(document, address)
@@ -740,19 +833,16 @@ def _dispatch(method, params):
})
return result
if method == "list_procedures":
return _page(_procedure_map(document), params.get("offset", 0), params.get("limit", 100))
return _page(dict(_search_inventory(document, "procedure")), params.get("offset", 0), params.get("limit", 100))
if method == "list_strings":
values = _strings(document)
values = dict(_search_inventory(document, "string"))
requested = params.get("address")
if requested is not None:
key = _hex(_address(document, requested))
values = {key: values[key]} if key in values else {}
return _page(values, params.get("offset", 0), params.get("limit", 100))
if method == "list_names":
result = {}
for segment in document.getSegmentsList():
for item in segment.getNamedAddresses():
result[_hex(item)] = segment.getNameAtAddress(item)
result = dict(_search_inventory(document, "name"))
requested = params.get("address")
if requested is not None:
key = _hex(_address(document, requested))
@@ -768,7 +858,7 @@ def _dispatch(method, params):
if method == "procedure_assembly":
return _assembly(procedure)
if method == "procedure_pseudo_code":
return procedure.decompile()
return _pseudocode(document, procedure)
if method == "procedure_callers":
return sorted((_hex(item.getEntryPoint()) for item in procedure.getAllCallerProcedures()), key=lambda value: int(value, 16))
if method == "procedure_callees":
@@ -779,12 +869,18 @@ def _dispatch(method, params):
return {"name": _procedure_name(procedure), "entrypoint": _hex(procedure.getEntryPoint()), "basicblock_count": procedure.getBasicBlockCount(), "length": length, "signature": procedure.signatureString(), "locals": _procedure_locals(procedure), "classification": None}
if method == "set_address_name":
address = _address(document, params.get("address"))
result = document.setNameAtAddress(address, params["name"])
_invalidate_search_inventory(document)
try:
result = document.setNameAtAddress(address, params["name"])
finally:
_invalidate_search_inventory(document)
_invalidate_pseudocode(document)
return result
if method == "set_addresses_names":
result = {key: document.setNameAtAddress(_address(document, key), value) for key, value in params["names"].items()}
_invalidate_search_inventory(document)
try:
result = {key: document.setNameAtAddress(_address(document, key), value) for key, value in params["names"].items()}
finally:
_invalidate_search_inventory(document)
_invalidate_pseudocode(document)
return result
if method in ("set_comment", "set_inline_comment"):
address = _address(document, params.get("address"))
@@ -792,6 +888,7 @@ def _dispatch(method, params):
setter = segment.setCommentAtAddress if method == "set_comment" else segment.setInlineCommentAtAddress
getter = segment.getCommentAtAddress if method == "set_comment" else segment.getInlineCommentAtAddress
setter(address, params["comment"])
_invalidate_pseudocode(document)
return getter(address) == params["comment"]
if method == "list_bookmarks":
return [{"address": _hex(item), "name": document.getBookmarkName(item)} for item in document.getBookmarks()]
@@ -6,7 +6,7 @@
target binding, analysis-profile commitment, and snapshot/Evidence migration
are implemented. Ghidra discovery, target/profile resolution, doctor checks,
the private headless-session foundation, ten read-only inventory capabilities,
and eight function-analysis capabilities are implemented with real
and nine function-analysis capabilities are implemented with real
cross-format conformance.
## Context
+1 -1
View File
@@ -151,7 +151,7 @@ REA's Ghidra provider is bring-your-own and supports Linux x64 with the exact
official Ghidra 12.1.2 release and a 64-bit full JDK 21. An experimental
Windows x64 P0 supports approved native x86-64 PE applications. It supplies
discovery, analysis-profile commitment, an isolated read-only headless session,
ten bounded inventory/name/search operations, and eight function-analysis
ten bounded inventory/name/search operations, and nine function-analysis
operations covering metadata, decompilation, assembly, resolved calls, typed
references, xrefs, CFG, and dossiers. GUI state and analysis mutations remain
unavailable through Ghidra.
+31
View File
@@ -115,6 +115,37 @@ redacted from recorded text. The command-shim directory is placed first in the
captured process `PATH`; any remaining `PATH` must be explicitly supplied or
inherited under operator policy.
## Reactive process scenarios
Set `reactive` to a Process Reactive Scenario v1 declaration when interaction
must follow observed output instead of guessed delays. The graph declares an
initial state, absolute scenario and state deadlines, explicitly prioritized
transitions, bounded uses and visits, trigger predicates, ordered actions, and
either a next state or `passed` outcome.
The admitted runtime slice matches decoded UTF-8 terminal literals and exact
`terminal_raw`, `interaction`, `process`, `filesystem`, `http`, `websocket`, or
`shim` journal events. Process events are sampled observations, not an
event-complete lifecycle. Filesystem events identify named checkpoints; shim
events identify a command and route. Live predicates receive the same normalized
and redacted payload later persisted in Evidence. Predicates may compose with
bounded `all`, `any`, `sequence`, and `repeat` nodes and may select a frontier at
scenario start, state entry, a prior checkpoint, or an exact event ID. Actions
can write PTY input, resize it, signal the root process, or capture a named
filesystem checkpoint. `terminal_rendered`, `lifecycle`, and `replay_transition`
event triggers and non-root signal selectors fail during preflight before the
target launches.
Reactive results are recorded in `reactive_run`, including the terminal outcome,
active state, matched event IDs, action evidence IDs, and deterministic
transition journal. Terminal controls also commit the last admitted observation
order, so later output cannot be moved ahead of a timeout, cancellation, or
target loss during validation. If the process exits before the graph finishes,
the outcome is `target_lost`; state/scenario deadlines remain distinct
`predicate_timeout`/`scenario_deadline` outcomes. Timed `events` remain
available for fixed schedules, but they are not silently translated into the
reactive graph.
## Checkpoints and deterministic shims
Checkpoint triggers may use an elapsed time, a terminal literal and occurrence
+30 -13
View File
@@ -2,7 +2,7 @@
"catalog_schema_version": 1,
"package": {
"name": "rea-agents",
"version": "2.3.0",
"version": "2.4.1",
"sdk": {
"server": "2.0.0-beta.4",
"client": "2.0.0-beta.4",
@@ -11,12 +11,12 @@
"skill_version": "23"
},
"tools": {
"total": 98,
"total": 99,
"families": [
{
"id": "direct",
"surface": "official-proxy",
"count": 33,
"count": 34,
"tools": [
"address_name",
"comment",
@@ -40,6 +40,7 @@
"procedure_info",
"procedure_pseudo_code",
"procedure_references",
"read_function_instructions",
"resolve_containing_procedure",
"search_procedures",
"search_strings",
@@ -86,7 +87,10 @@
"id": "artifact",
"surface": "artifact-provider",
"count": 2,
"tools": ["extract_artifact", "inventory_artifact"]
"tools": [
"extract_artifact",
"inventory_artifact"
]
},
{
"id": "managed",
@@ -193,6 +197,7 @@
"procedure_info",
"procedure_pseudo_code",
"procedure_references",
"read_function_instructions",
"resolve_containing_procedure",
"search_procedures",
"search_strings",
@@ -227,6 +232,7 @@
"procedure_info",
"procedure_pseudo_code",
"procedure_references",
"read_function_instructions",
"resolve_containing_procedure",
"search_procedures",
"search_strings",
@@ -256,7 +262,10 @@
"id": "rea-artifact-graph",
"name": "REA safe artifact graph provider",
"version": "1",
"capabilities": ["extract_artifact", "inventory_artifact"]
"capabilities": [
"extract_artifact",
"inventory_artifact"
]
},
{
"id": "rea-cdp-browser",
@@ -277,7 +286,10 @@
"id": "rea-cdp-electron",
"name": "REA Electron file-page CDP observation provider",
"version": "1",
"capabilities": ["inspect_electron_page", "list_electron_targets"]
"capabilities": [
"inspect_electron_page",
"list_electron_targets"
]
},
{
"id": "rea-dotnet-static",
@@ -305,7 +317,9 @@
"id": "rea-javascript-application",
"name": "REA JavaScript application analyzer",
"version": "1",
"capabilities": ["analyze_javascript_application"]
"capabilities": [
"analyze_javascript_application"
]
},
{
"id": "rea-javascript-application-workflows",
@@ -327,7 +341,9 @@
"id": "rea-javascript-runtime-reconciliation",
"name": "REA JavaScript runtime reconciliation",
"version": "1",
"capabilities": ["reconcile_javascript_runtime"]
"capabilities": [
"reconcile_javascript_runtime"
]
}
],
"setup_clients": [
@@ -517,7 +533,7 @@
}
],
"cli": {
"primary_count": 58,
"primary_count": 59,
"commands": [
"analyze",
"analyze-javascript-application",
@@ -553,6 +569,7 @@
"inspect-plist",
"inspect-signature",
"inspect-web-page",
"instructions",
"inventory-artifact",
"investigate-versions",
"list-architectures",
@@ -587,17 +604,17 @@
},
"runtime_catalog": {
"counts": {
"cli_commands": 58,
"mcp_tools": 98,
"cli_commands": 59,
"mcp_tools": 99,
"mcp_prompts": 6,
"mcp_resources": 2,
"mcp_resource_templates": 9
},
"digests": {
"tools_sha256": "501d405efbf8fc3537d13791f87ac81f3b97f3f328b62710ca0885de7213a62f",
"tools_sha256": "9159861cda3cc76f876dde306efae886dd310419d4e972f89ab7b85ff5789485",
"prompts_sha256": "f6da3ad38a1000c11364d1710f122ac4990551f04efd8c53ffb2e819b5cf5707",
"resources_sha256": "94bebbab1a1b3cf47644c38d3de6469de568caa7bd7b842f529ead8ad320a912",
"combined_sha256": "71de8a7174176823fd82c2666130233b5a1b4a1613583270d268774775b410bc"
"combined_sha256": "e787c57af8337f17b9da72745fcb328634af3b39be1d9ca8bfe1836274351f92"
}
}
}
+7 -6
View File
@@ -4,7 +4,7 @@ Status: the Ghidra read-only analysis provider is shipped on Linux x64 and has
an experimental Windows x64 P0 for approved native PE applications. It
validates an exact bring-your-own Ghidra 12.1.2/JDK 21 environment, resolves a
provider/version/profile commitment, runs one isolated read-only headless
import, and publishes 18 operation-level capabilities after the authenticated
import, and publishes 19 operation-level capabilities after the authenticated
post-analysis handshake.
The provider-neutral target, provider registry, deterministic target binding,
@@ -69,14 +69,14 @@ Unix socket. Windows uses authenticated IPv4 loopback with a strict token-free
endpoint record. Both transports report actual
Ghidra/language/compiler/analysis/import-digest metadata, accept only the exact
authenticated `ping`, `shutdown`, and ten
inventory plus eight function-analysis methods, and deletes the socket. Close,
inventory plus nine function-analysis methods, and deletes the socket. Close,
cancellation, timeout,
malformed protocol, or process exit stops the owned process resources and
removes the entire runtime root. Windows P0 uses bounded `taskkill` tree
termination and explicitly does not claim Job Object, private-DACL, or
reparse-point authority; see [Windows Ghidra P0](windows-ghidra-p0.md).
The provider catalog lists only the 18 proved Ghidra operations. GUI cursor,
The provider catalog lists only the 19 proved Ghidra operations. GUI cursor,
navigation, and mutation operations remain absent; the router therefore
reports them unavailable instead of borrowing Hopper semantics or inferring
capability from a successful import.
@@ -103,13 +103,14 @@ capability from a successful import.
| Serialization | A bounded FIFO admits at most 32 active-plus-queued requests and sends one Program request at a time. Queue wait counts against the caller's deadline, and queued cancellation is prompt. This is an adapter safety commitment, not a claim that every Ghidra API is thread-safe. |
| Function identity | Every function result carries the entry address and Ghidra FunctionManager classification for external, thunk, and resolved thunk target. These are observations, not proof that unresolved targetless calls have been recovered. |
| Assembly and pseudocode | Assembly is bounded Ghidra Listing text; pseudocode is bounded Ghidra decompiler output. Neither is original source, and cross-provider comparison never treats Hopper and Ghidra text as equal or unequal semantic facts. |
| Instruction fast path | `read_function_instructions` returns one offset-paginated raw Listing window without invoking the decompiler or whole-program name/string inventories. It reports scan work, exact-or-unknown totals, truncation, and continuation metadata. |
| Calls and references | Callers/callees contain only resolved functions. Reference edges preserve exact ReferenceManager type and call/jump/data/read/write/indirect/computed/conditional/terminal/external facts. Targetless computed flow remains unknown. Synthetic entry-point references without actionable memory sources are omitted explicitly. |
| CFG | Dossiers use `BasicBlockModel` and retain only non-call successors inside the function body. CFG topology is address-normalized for comparison; provider-specific block construction remains a declared difference. |
| Bounds | Direct assembly fails above 100,000 instructions or the wire ceiling. Dossiers scan at most 5,000 requested instructions, return bounded independent pages, mark unknown totals after an incomplete scan, and paginate pseudocode by Unicode code points. |
`npm run verify:ghidra` compiles the versioned C oracles into x86-64 debug and
stripped ELF, AArch64 ELF, x86-64 PE, and x86-64 Mach-O targets. It proves all
18 operations, external functions, resolved thunks, exports, stripped-name
19 operations, external functions, resolved thunks, exports, stripped-name
behavior, direct and targetless indirect calls, typed references, strings/xrefs,
multi-block CFG, semantic enhanced workflows, cancellation, deadlines,
serialized concurrency, malformed-target rejection, profile identity, and
@@ -118,7 +119,7 @@ cover analysis/decompile timeouts, process exit, queue saturation, and malformed
wire output.
`npm run verify:ghidra:windows` generates a deterministic source-owned native
x86-64 PE application, proves all 18 operations through the production Windows
x86-64 PE application, proves all 19 operations through the production Windows
launcher and loopback transport, checks target/snapshot/import SHA-256 linkage,
and requires project, endpoint, process, and runtime cleanup. Passing that lane
proves only the documented P0 boundary, not the remaining Windows security
@@ -146,7 +147,7 @@ only the generic process mechanisms.
| Provider | License / automation surface | What it brings | REA fit and blockers |
| ------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Ghidra](https://github.com/NationalSecurityAgency/ghidra) | Apache-2.0 source license; `analyzeHeadless`, Java APIs, and PyGhidra | Broad static analysis, many processors and formats, scripting, and project/database workflows | Read-only analysis shipped: exact BYO checks, isolated state, bounded process and queue, packaged bridge, 18 admitted inventory/function operations, and real ELF/PE/Mach-O x86-64 plus AArch64 conformance. GUI and mutation semantics remain intentionally unavailable. |
| [Ghidra](https://github.com/NationalSecurityAgency/ghidra) | Apache-2.0 source license; `analyzeHeadless`, Java APIs, and PyGhidra | Broad static analysis, many processors and formats, scripting, and project/database workflows | Read-only analysis shipped: exact BYO checks, isolated state, bounded process and queue, packaged bridge, 19 admitted inventory/function operations, and real ELF/PE/Mach-O x86-64 plus AArch64 conformance. GUI and mutation semantics remain intentionally unavailable. |
| [Rizin](https://github.com/rizinorg/rizin) / [rz-pipe](https://github.com/rizinorg/rz-pipe) | Rizin repository contains LGPL-3.0 and GPL-3.0 components; `rizin`, `rz-bin`, and language bridges through `rzpipe` | Portable CLI analysis, disassembly/debugging, many architectures and file formats, JSON command output | Good candidate for a process-backed Linux provider and fast metadata fallback. License/component inventory must be preserved; command output needs version-pinned parsers and semantic conformance before evidence is trusted. |
| [LIEF](https://github.com/lief-project/LIEF) | Apache-2.0; C++, Python, and other bindings | Deterministic parsing and modification of ELF, PE, Mach-O, COFF, and related executable formats; headers, sections, symbols, relocations, and functions | Best near-term complement, not a decompiler replacement. It can cover format metadata and artifact evidence without a long-lived analysis process; function semantics, pseudocode, CFG, and cross-reference parity remain out of scope unless separately demonstrated. |
| [Binary Ninja](https://docs.binary.ninja/dev/index.html) | API/documentation components are MIT, while the analysis product is licensed by edition; commercial, Ultimate, or Headless license is required for headless automation | Python/Core/C++/Rust APIs, headless loading, IL layers, function analysis, plugins, and configurable analysis | Strong technical fit for a native provider, especially function dossiers. Commercial licensing, license-secret handling, native runtime packaging, and multithreaded lifecycle rules are material deployment blockers. |
+2 -2
View File
@@ -11,11 +11,11 @@ additively; Devin is reported but left unchanged.
The Ghidra foundation supports Linux x64 with exact Ghidra 12.1.2 and a 64-bit
full JDK 21. An experimental Windows x64 P0 admits approved native x86-64 PE
applications through the same 18-operation Java bridge. Doctor validates those coordinates; approved Linux/macOS setup propagates them
applications through the same 19-operation Java bridge. Doctor validates those coordinates; approved Linux/macOS setup propagates them
to MCP registrations without installing or modifying either dependency. REA's
packaged Java bridge then proves an isolated read-only headless import,
post-analysis handshake, complete cleanup, ten admitted read-only inventory
operations, and eight function-analysis operations. Real source-owned x86-64
operations, and nine function-analysis operations. Real source-owned x86-64
debug/stripped ELF, AArch64 ELF, PE, and Mach-O fixtures cover program,
procedure, string, symbol, external/thunk, memory, resolution, search,
decompilation, assembly, calls, typed references, xrefs, and CFG semantics.
@@ -1,6 +1,6 @@
{
"schema_version": 1,
"manifest_id": "ecm_e04a25d07a540df3bec2fd5ba6d8a158e1f582dc94fc59225c7ba329cbabbbf6",
"manifest_id": "ecm_0b704bd3b1a8e8f6840d113cf064de935393db1936ca5eba1018a49ccd566cf2",
"verifier": {
"id": "rea-managed-conformance",
"version": "1"
@@ -14,7 +14,7 @@
"skill_digests": [
{
"skill_id": "reverse-engineer-anything",
"sha256": "a8ffb68f043fc9dfd0eb401caa6e548adce991d1e228b4539333cf90be17a755"
"sha256": "fdfbbc813a935b93dbd814c509997024da4a98500b7d12887b0ee6ecf69a9fcb"
}
],
"claims": [
+2 -2
View File
@@ -13,7 +13,7 @@ The P0 accepts exactly:
- an operator-installed official Ghidra 12.1.2 distribution;
- a 64-bit full JDK 21;
- an explicit native, non-managed, non-DLL x86-64 PE application; and
- the existing 18 read-only Ghidra inventory and function-analysis operations.
- the existing 19 read-only Ghidra inventory and function-analysis operations.
Hopper, Ghidra GUI state, mutation, controlled JavaScript replay, process
capture, browser/Electron observation, artifact extraction, and general
@@ -146,7 +146,7 @@ npm run verify:ghidra:windows
The verifier creates an ignored deterministic x86-64 PE fixture, checks its
fixed SHA-256, opens Ghidra 12.1.2 through the production Java bridge, exercises
all 18 admitted operations, verifies the target/snapshot/import digest chain,
all 19 admitted operations, verifies the target/snapshot/import digest chain,
and checks endpoint, project, process, and runtime cleanup. The GitHub workflow
accepts only the fixed `real-ghidra-windows` repository-dispatch event against
default-branch `main`, a protected `real-ghidra-windows` environment, and a
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "rea-agents",
"version": "2.3.0",
"version": "2.4.1",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "rea-agents",
"version": "2.3.0",
"version": "2.4.1",
"license": "MIT",
"dependencies": {
"@babel/parser": "8.0.4",
+7 -4
View File
@@ -1,6 +1,6 @@
{
"name": "rea-agents",
"version": "2.3.0",
"version": "2.4.1",
"description": "Reverse engineer anything from your terminal or agent with one CLI and MCP server.",
"license": "MIT",
"repository": {
@@ -80,19 +80,22 @@
"test": "vitest run",
"test:run": "vitest run",
"pretest:fast": "npm run build:cached",
"test:fast": "vitest run --project parallel",
"test:fast": "vitest run --project parallel --project subprocess",
"pretest:changed": "npm run build:cached",
"test:changed": "vitest run --project parallel --project subprocess --changed",
"pretest:integration": "npm run build:cached",
"test:integration": "vitest run --project serial-integration",
"pretest:coverage": "npm run build:cached",
"test:coverage": "vitest run --coverage.enabled --coverage.provider=v8",
"test:ci": "npm run test:coverage -- --reporter=default --reporter=junit --outputFile.junit=test-results.xml",
"pretest:ci:shard": "npm run build:cached",
"test:ci:shard": "vitest run --coverage.enabled --coverage.provider=v8 --coverage.thresholds.lines=0 --coverage.thresholds.functions=0 --coverage.thresholds.branches=0 --coverage.thresholds.statements=0 --reporter=default --reporter=blob",
"test:ci:shard": "npm run test:ci:shard:run",
"test:ci:shard:run": "vitest run --coverage.enabled --coverage.provider=v8 --coverage.thresholds.lines=0 --coverage.thresholds.functions=0 --coverage.thresholds.branches=0 --coverage.thresholds.statements=0 --reporter=default --reporter=blob",
"test:ci:merge": "vitest --merge-reports --coverage.enabled --coverage.provider=v8 --reporter=default --reporter=junit --outputFile.junit=test-results.xml",
"pretest:verbose": "npm run build:cached",
"test:verbose": "vitest run --reporter=verbose",
"pretest:watch": "npm run build:cached",
"test:watch": "vitest --project parallel --changed",
"test:watch": "vitest --project parallel --project subprocess --changed",
"typecheck": "tsc --noEmit",
"verify:test-temp-hygiene": "npm run build:cached && node scripts/verify-test-temp-hygiene.mjs",
"verify:hopper": "npm run verify:fixtures && npm run build:cached && node scripts/verify-real-hopper.mjs",
@@ -0,0 +1,71 @@
import { requireMcpResult } from "./mcp-verifier-results.mjs";
export const verifyHopperFunctionBasics = async (
client,
options,
procedure,
) => {
const containment = requireMcpResult(
await client.callTool(
{
name: "resolve_containing_procedure",
arguments: { address: procedure },
},
options,
),
"resolve_containing_procedure",
);
if (
containment?.found !== true ||
containment.procedure?.address !== procedure
) {
throw new Error(
"resolve_containing_procedure returned the wrong procedure",
);
}
const references = requireMcpResult(
await client.callTool(
{
name: "procedure_references",
arguments: {
procedure,
direction: "outgoing",
limit: 10,
max_instructions: 100,
},
},
options,
),
"procedure_references",
);
if (!Array.isArray(references?.references?.items)) {
throw new Error("procedure_references returned an invalid bounded result");
}
const instructions = requireMcpResult(
await client.callTool(
{
name: "read_function_instructions",
arguments: { procedure, limit: 2 },
},
options,
),
"read_function_instructions",
);
if (
instructions?.procedure?.address !== procedure ||
!Array.isArray(instructions.instructions?.items) ||
instructions.instructions.items.length === 0 ||
instructions.instructions.items.length > 2
) {
throw new Error(
"read_function_instructions returned an invalid bounded window",
);
}
return {
outgoingReferenceCount: references.references.items.length,
instructionWindowCount: instructions.instructions.items.length,
};
};
+20 -4
View File
@@ -72,10 +72,7 @@ export async function verifyPackageEnvironment({
XDG_CONFIG_HOME: join(home, ".config"),
PATH: `${fakeBin}:${process.env.PATH ?? ""}`,
REA_ANALYSIS_PROVIDER: "auto",
HOPPER_LAUNCHER_PATH: process.execPath,
HOPPER_LOADER_ARGS_JSON: JSON.stringify([
join(root, "tests/fixtures/fakeLauncher.mjs"),
]),
...packageHopperEnvironment(root),
REA_NPX_LOG: npxLog,
REA_EVIDENCE_ROOTS_JSON: JSON.stringify([evidenceRoot]),
REA_INVESTIGATION_INPUT_ROOTS_JSON: JSON.stringify([workspace]),
@@ -94,3 +91,22 @@ export async function verifyPackageEnvironment({
environment,
};
}
const packageHopperEnvironment = (root) => {
const fakeLauncher = join(root, "tests/fixtures/fakeLauncher.mjs");
if (process.platform === "win32")
return {
HOPPER_LAUNCHER_PATH: process.execPath,
HOPPER_LOADER_ARGS_JSON: JSON.stringify([fakeLauncher]),
};
return {
HOPPER_LAUNCHER_PATH: "/bin/sh",
HOPPER_LOADER_ARGS_JSON: JSON.stringify([
"-c",
'node_path=$1; shift; "$node_path" "$@"',
"rea-package-hopper",
process.execPath,
fakeLauncher,
]),
};
};
+29 -2
View File
@@ -44,10 +44,27 @@ export async function verifyDebugFunctionOperations(
if (entryString === undefined)
throw new Error("Ghidra debug string function probe is unavailable");
const [info, assembly, pseudocode, callees, callers, outgoing, entryXrefs] =
await collectEntryFunctionData(client, entry);
const [
info,
assembly,
instructionWindow,
pseudocode,
callees,
callers,
outgoing,
entryXrefs,
] = await collectEntryFunctionData(client, entry);
assertLocalProcedureInfo(info, entry.address);
assertProviderText(assembly, pseudocode, entry.address);
if (
instructionWindow.procedure?.address !== entry.address ||
instructionWindow.instructions?.items?.length !== 1 ||
instructionWindow.instructions.returned !== 1 ||
instructionWindow.instructions_scanned < 1
)
throw new Error(
"Ghidra read_function_instructions failed its bounded fast-path contract",
);
assertEntryCallGraph({ branch, indirect, main, callees, callers });
const { xrefOwners, xrefOwnerFailures } = await resolveXrefOwners(
client,
@@ -83,6 +100,10 @@ export async function verifyDebugFunctionOperations(
entry: dossierSummary(entryDossier),
branch: dossierSummary(branchDossier),
indirect: dossierSummary(indirectDossier),
instruction_window: {
returned: instructionWindow.instructions.returned,
truncated: instructionWindow.instructions.truncated,
},
cancellation,
timeout,
concurrency: concurrent,
@@ -99,6 +120,12 @@ async function collectEntryFunctionData(client, entry) {
document: null,
procedure: entry.value,
}),
functionCall(client, "read_function_instructions", {
document: null,
procedure: entry.value,
offset: 0,
limit: 1,
}),
functionCall(client, "procedure_pseudo_code", {
document: null,
procedure: entry.value,
+1
View File
@@ -146,6 +146,7 @@ try {
"procedure_callers",
"procedure_info",
"procedure_pseudo_code",
"read_function_instructions",
"procedure_references",
])
await functionOperation(operation, { procedure: procedure.value });
+7 -36
View File
@@ -26,6 +26,7 @@ import {
requireMcpResult,
requireWorkflowEvidenceProvider,
} from "./lib/mcp-verifier-results.mjs";
import { verifyHopperFunctionBasics } from "./lib/real-hopper-function-basics.mjs";
import { openAndVerifyLargeFixture } from "./lib/real-hopper-pagination.mjs";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
import { startUnrelatedHopperSentinel } from "./lib/unrelated-hopper-sentinel.mjs";
@@ -155,42 +156,11 @@ const verifyCurrentTarget = async (client, options) => {
"list_procedures",
);
const firstAddress = firstProcedureAddress(procedures);
const containment = requireSuccessfulTool(
await client.callTool(
{
name: "resolve_containing_procedure",
arguments: { address: firstAddress },
},
options,
),
"resolve_containing_procedure",
const basics = await verifyHopperFunctionBasics(
client,
options,
firstAddress,
);
if (
containment?.found !== true ||
containment.procedure?.address !== firstAddress
) {
throw new Error(
"resolve_containing_procedure returned the wrong procedure",
);
}
const references = requireSuccessfulTool(
await client.callTool(
{
name: "procedure_references",
arguments: {
procedure: firstAddress,
direction: "outgoing",
limit: 10,
max_instructions: 100,
},
},
options,
),
"procedure_references",
);
if (!Array.isArray(references?.references?.items)) {
throw new Error("procedure_references returned an invalid bounded result");
}
const relationships = await verifyRelationships(
client,
options,
@@ -245,7 +215,8 @@ const verifyCurrentTarget = async (client, options) => {
callerCount: relationships.procedure_callers.length,
calleeCount: relationships.procedure_callees.length,
xrefCount: relationships.xrefs.length,
outgoingReferenceCount: references.references.items.length,
outgoingReferenceCount: basics.outgoingReferenceCount,
instructionWindowCount: basics.instructionWindowCount,
};
};
+4 -3
View File
@@ -3,8 +3,8 @@ name: reverse-engineer-anything
description: Reverse engineer native, managed, Electron/JavaScript, packaged, and browser applications with REA. Use shipped-artifact or approved runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
metadata:
version: "23"
tool_count: 98
catalog_digest: "71de8a7174176823fd82c2666130233b5a1b4a1613583270d268774775b410bc"
tool_count: 99
catalog_digest: "e787c57af8337f17b9da72745fcb328634af3b39be1d9ca8bfe1836274351f92"
---
# REA
@@ -22,7 +22,8 @@ Choose the first tool from the target the user supplied. Do not call
- ASAR or extracted JavaScript/Electron tree:
`analyze_javascript_application`.
- Archive, application package, ZIP/APK/IPA, or DMG: `inventory_artifact`.
- Archive, application package, ZIP/APK/IPA/MSIX/AppX, or DMG:
`inventory_artifact`.
- Managed PE/CLI assembly: `inspect_managed_artifact`.
- User-owned browser page already open: `list_browser_targets`.
- User-owned Electron runtime already open: `list_electron_targets`.
@@ -24,8 +24,8 @@ parser and must not become an implicit setup dependency.
## Packages and extraction
Call `inventory_artifact` before extraction for application bundles, archives,
ZIP/APK/IPA, ASAR, or DMG inputs. Continue deterministic occurrence pages from
the returned offset only when needed. Cite graph manifest IDs.
ZIP/APK/IPA/MSIX/AppX, ASAR, or DMG inputs. Continue deterministic occurrence
pages from the returned offset only when needed. Cite graph manifest IDs.
`extract_artifact` requires explicit approval, an absent absolute output root,
and selected occurrence IDs. Never extract every entry implicitly. Symlinks and
+27
View File
@@ -0,0 +1,27 @@
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import { err, ok, type Result } from "../domain/result.js";
import type { AnalysisError } from "../domain/errors.js";
import type { AnalysisClient, ExecutionOptions } from "./AnalysisProvider.js";
/** Close one provider client and normalize unexpected adapter rejection. */
export const closeAnalysisClient = async (
client: AnalysisClient,
providerId: string,
options: Pick<ExecutionOptions, "progress"> = {},
): Promise<Result<null, AnalysisError>> => {
try {
if (client.closeWithOutcome !== undefined)
return await client.closeWithOutcome(options);
await client.close();
return ok(null);
} catch (cause: unknown) {
return err(
new ProviderCleanupError(
providerId,
["provider-client"],
{ reason: "provider client close rejected unexpectedly" },
{ cause },
),
);
}
};
+29 -2
View File
@@ -81,10 +81,29 @@ export interface ProviderRuntimeLineageSnapshot {
readonly observation: ProcessLineageObservation;
}
/** One provider's current serial-request activity, including detached callers. */
export interface ProviderRequestActivitySnapshot {
readonly provider: ProviderIdentity;
readonly active: {
readonly requestId: number;
readonly operation: string;
readonly elapsedMs: number;
readonly timeoutMs: number;
readonly callerState: "waiting" | "timed_out" | "cancelled";
} | null;
readonly queuedRequests: number;
}
/** Provider session bound to exactly one parsed artifact. */
export interface AnalysisClient extends AnalysisOperationPort {
/** Retained token-verified process snapshots for dynamic providers this client started. */
runtimeLineageSnapshots?(): readonly ProviderRuntimeLineageSnapshot[];
/** Current provider work that may outlive a timed-out or cancelled caller. */
requestActivitySnapshots?(): readonly ProviderRequestActivitySnapshot[];
/** Close with a typed result when the provider can verify cleanup. */
closeWithOutcome?(
options?: Pick<ExecutionOptions, "progress">,
): Promise<Result<null, AnalysisError>>;
close(): Promise<void>;
}
@@ -127,8 +146,16 @@ interface CapabilityEffects {
}
type CapabilityAvailability =
| { readonly available: true; readonly reason: null }
| { readonly available: false; readonly reason: string };
| {
readonly available: true;
readonly reason: null;
readonly availabilityCode?: null;
}
| {
readonly available: false;
readonly reason: string;
readonly availabilityCode?: ProviderRejectionCode;
};
interface CapabilityLimits {
readonly maxResults: number | null;
+7 -1
View File
@@ -269,7 +269,13 @@ const createReader = async (
if ((await lstat(path)).isDirectory())
return new DirectoryArtifactReader(path);
if (format === "asar") return new AsarArtifactReader(path);
if (format === "ipa" || format === "apk" || format === "zip")
if (
format === "ipa" ||
format === "apk" ||
format === "msix" ||
format === "appx" ||
format === "zip"
)
return new ZipArtifactReader(path, format);
if (format === "mach-o") return new MachOSliceArtifactReader(path);
throw new ArtifactReaderFailure(
+5 -18
View File
@@ -1,5 +1,4 @@
import { createHash } from "node:crypto";
import { extname } from "node:path";
import canonicalize from "canonicalize";
@@ -13,6 +12,7 @@ import type {
ArtifactNode,
ArtifactOccurrence,
} from "../domain/artifactGraph.js";
import { zipPackageFormatForPath } from "../domain/zipPackageFormat.js";
/** Mutable internal occurrence used until root-bound IDs are known. */
export interface MutableOccurrence {
@@ -130,7 +130,9 @@ export const createRootNode = (input: {
size: input.digest?.bytes ?? 0,
kind:
input.directory ||
["zip", "ipa", "apk", "asar", "dmg", "pkg"].includes(input.format)
["zip", "ipa", "apk", "msix", "appx", "asar", "dmg", "pkg"].includes(
input.format,
)
? "container"
: classifyArtifactPath(input.path).kind,
format: input.format,
@@ -264,7 +266,7 @@ export const classifyArtifactPath = (
if (/\.(?:m?js|cjs)$/u.test(lower))
return { kind: "javascript", format: "javascript-bundle" };
if (lower.endsWith(".asar")) return { kind: "container", format: "asar" };
const archiveFormat = archiveFormatForExtension(extname(lower));
const archiveFormat = zipPackageFormatForPath(lower);
if (archiveFormat !== undefined)
return { kind: "container", format: archiveFormat };
if (/\.framework(?:\/|$)/u.test(lower))
@@ -351,19 +353,4 @@ const relationFor = (path: string): ArtifactEdge["relation"] => {
return "contains";
};
const archiveFormatForExtension = (
extension: string,
): "zip" | "ipa" | "apk" | undefined => {
switch (extension) {
case ".zip":
return "zip";
case ".ipa":
return "ipa";
case ".apk":
return "apk";
default:
return undefined;
}
};
const depth = (path: string): number => path.split("/").length;
@@ -2,12 +2,15 @@ import { open } from "node:fs/promises";
import { classifyArtifactPath } from "../ArtifactGraphConstruction.js";
import type { ArtifactNode } from "../../domain/artifactGraph.js";
import { zipPackageFormatForPath } from "../../domain/zipPackageFormat.js";
const classifyContainerExtension = (
path: string,
): ArtifactNode["format"] | undefined => {
const lower = path.toLowerCase();
for (const format of ["asar", "ipa", "apk", "zip", "dmg", "pkg"] as const)
const zipPackage = zipPackageFormatForPath(lower);
if (zipPackage !== undefined) return zipPackage;
for (const format of ["asar", "dmg", "pkg"] as const)
if (lower.endsWith(`.${format}`)) return format;
return undefined;
};
@@ -22,6 +22,8 @@ export const createReader = async (
case "zip":
case "ipa":
case "apk":
case "msix":
case "appx":
return new ZipArtifactReader(path, format);
case "asar":
return new AsarArtifactReader(path);
+27 -4
View File
@@ -16,6 +16,7 @@ import type {
AnalysisClient,
AnalysisExecution,
AnalysisClientFactory,
ExecutionOptions,
AnalysisProfileResolution,
AnalysisProfileResolutionOptions,
AnalysisOperationPort,
@@ -37,6 +38,7 @@ import {
type BinarySessionOpenOptions,
} from "./BinarySessionOpen.js";
import { prepareSessionExecution } from "./BinarySessionExecution.js";
import { closeAnalysisClient } from "./AnalysisClientCleanup.js";
export type { BinarySessionPort } from "./BinarySessionPort.js";
const OFFICIAL_OPERATIONS: ReadonlySet<string> = new Set(
OFFICIAL_TOOL_CONTRACTS.map(({ name }) => name),
@@ -186,7 +188,17 @@ export class BinarySession
await this.#drainCalls();
const previous = this.#active;
this.#active = undefined;
await previous?.client.close();
if (previous !== undefined) {
const closed = await closeAnalysisClient(
previous.client,
previous.route.identity.id,
);
if (!closed.ok) {
this.clearSessionRecords();
this.#clearRuntimeAvailability();
return closed;
}
}
const runId = randomUUID();
const client = route.createClient(target, { runId });
const started = await client.execute("health", {}, options);
@@ -226,15 +238,24 @@ export class BinarySession
}
/** Close the active target, if any. */
close(): Promise<Result<null, AnalysisError>> {
close(
options: Pick<ExecutionOptions, "progress"> = {},
): Promise<Result<null, AnalysisError>> {
return this.#serialize(async () => {
const previous = this.#active;
this.#active = undefined;
await this.#drainCalls();
await previous?.client.close();
const closed =
previous === undefined
? ok(null)
: await closeAnalysisClient(
previous.client,
previous.route.identity.id,
options,
);
this.clearSessionRecords();
this.#clearRuntimeAvailability();
return ok(null);
return closed;
});
}
@@ -248,6 +269,8 @@ export class BinarySession
runId: this.#active?.runId,
runtimeLineageSnapshots:
this.#active?.client.runtimeLineageSnapshots?.() ?? [],
requestActivitySnapshots:
this.#active?.client.requestActivitySnapshots?.() ?? [],
});
}
+4 -1
View File
@@ -23,6 +23,7 @@ import type { ReconstructionCoverageWorkspace } from "../domain/reconstructionCo
import type {
AnalysisOperation,
AnalysisOperationPort,
ExecutionOptions,
ProviderIdentity,
} from "./AnalysisProvider.js";
@@ -37,7 +38,9 @@ export interface BinarySessionPort extends AnalysisOperationPort {
readonly providerId?: AnalysisProviderSelector;
},
): Promise<Result<BinaryTarget, AnalysisError>>;
close(): Promise<Result<null, AnalysisError>>;
close(
options?: Pick<ExecutionOptions, "progress">,
): Promise<Result<null, AnalysisError>>;
status(): JsonValue;
activeTarget(): BinaryTarget | undefined;
recordEvidence(
+41
View File
@@ -2,6 +2,7 @@ import type { BinaryTarget } from "../domain/binaryTarget.js";
import type { JsonValue } from "../domain/jsonValue.js";
import type {
CapabilityDescriptor,
ProviderRequestActivitySnapshot,
ProviderRuntimeLineageSnapshot,
} from "./AnalysisProvider.js";
import type {
@@ -19,6 +20,7 @@ interface BinarySessionStatusInput {
>;
readonly runId: string | undefined;
readonly runtimeLineageSnapshots: readonly ProviderRuntimeLineageSnapshot[];
readonly requestActivitySnapshots: readonly ProviderRequestActivitySnapshot[];
}
/** Project internal provider routing state into the caller-visible session status. */
@@ -29,6 +31,7 @@ export const binarySessionStatus = ({
runtimeAvailability,
runId,
runtimeLineageSnapshots,
requestActivitySnapshots,
}: BinarySessionStatusInput): JsonValue => {
const configuredProvider = router.configuredIdentity();
const provider = providerSummary(configuredProvider);
@@ -73,6 +76,7 @@ export const binarySessionStatus = ({
run_id: runId,
process_lineage: processLineageStatus(runtimeLineageSnapshots),
},
analysis_activity: requestActivityStatus(requestActivitySnapshots),
};
return target === undefined
? { open: false, ...common }
@@ -87,6 +91,42 @@ export const binarySessionStatus = ({
};
};
const requestActivityStatus = (
snapshots: readonly ProviderRequestActivitySnapshot[],
) => {
const providers = snapshots
.slice()
.sort((left, right) => left.provider.id.localeCompare(right.provider.id))
.map(({ provider, active, queuedRequests }) => ({
provider: providerSummary(provider),
active:
active === null
? null
: {
request_id: active.requestId,
operation: active.operation,
elapsed_ms: active.elapsedMs,
timeout_ms: active.timeoutMs,
caller_state: active.callerState,
},
queued_requests: queuedRequests,
}));
const active = snapshots.flatMap((snapshot) =>
snapshot.active === null ? [] : [snapshot.active],
);
return {
status:
snapshots.length === 0
? "not_observed"
: active.some(({ callerState }) => callerState === "timed_out")
? "timed_out_busy"
: active.length > 0
? "busy"
: "idle",
providers,
};
};
const providerSummary = ({
id,
name,
@@ -147,6 +187,7 @@ const capabilityStatus = (
operation: descriptor.operation,
available: descriptor.available,
reason: descriptor.reason,
availability_code: descriptor.availabilityCode ?? null,
input_contract_version: descriptor.inputContractVersion,
output_contract_version: descriptor.outputContractVersion,
pagination: descriptor.pagination,
+14 -8
View File
@@ -12,6 +12,7 @@ const statusSchema = z.object({
operation: z.string(),
available: z.boolean(),
reason: z.string().nullable(),
availability_code: z.string().nullable().optional(),
}),
),
});
@@ -29,6 +30,7 @@ type ToolKind = (typeof TOOL_CONTRACTS)[number]["kind"];
type ProviderDescriptor = {
readonly available: boolean;
readonly reason: string | null;
readonly availability_code?: string | null;
};
type AvailabilityPolicy = {
readonly processCaptureEnabled: boolean;
@@ -86,8 +88,15 @@ export const buildCapabilityInventory = (
policy: AvailabilityPolicy,
) => {
const status = statusSchema.parse(sessionStatus);
const descriptors = new Map(
status.capabilities.map((descriptor) => [descriptor.operation, descriptor]),
const descriptors = new Map<string, ProviderDescriptor>(
status.capabilities.map((descriptor) => [
descriptor.operation,
{
available: descriptor.available,
reason: descriptor.reason,
availability_code: descriptor.availability_code ?? null,
},
]),
);
return TOOL_CONTRACTS.map((contract) => {
const availability = availabilityFor({
@@ -237,7 +246,7 @@ const providerAvailability = ({
};
if (
!descriptor.available &&
descriptor.reason?.toLowerCase().includes("require macos") === true
descriptor.availability_code === "unsupported_host"
)
return {
reason: "unsupported_host",
@@ -254,10 +263,7 @@ const providerAvailability = ({
const composedAvailability = (
name: string,
descriptors: ReadonlyMap<
string,
{ readonly available: boolean; readonly reason: string | null }
>,
descriptors: ReadonlyMap<string, ProviderDescriptor>,
): {
readonly reason: ToolAvailabilityReason;
readonly remediation: string | null;
@@ -280,7 +286,7 @@ const composedAvailability = (
if (unavailable !== undefined)
return {
reason:
unavailable.reason?.toLowerCase().includes("require macos") === true
unavailable.availability_code === "unsupported_host"
? "unsupported_host"
: "provider_unavailable",
remediation:
+19 -3
View File
@@ -10,6 +10,7 @@ import type {
CapabilityDescriptor,
ProviderIdentity,
} from "./AnalysisProvider.js";
import { closeAnalysisClient } from "./AnalysisClientCleanup.js";
/** Synthetic compatibility identity for a deterministic provider set. */
export const compositeProviderIdentity = (
@@ -103,6 +104,16 @@ export class CompositeProvider implements AnalysisProvider {
clients.set(provider, created);
return created;
};
const closeWithOutcome: NonNullable<
AnalysisClient["closeWithOutcome"]
> = async (options) => {
const outcomes = await Promise.all(
[...clients.entries()].map(([provider, client]) =>
closeAnalysisClient(client, provider.identity().id, options),
),
);
return outcomes.find((outcome) => !outcome.ok) ?? ok(null);
};
return {
execute: (operation, parameters, options) => {
if (operation === "health")
@@ -135,10 +146,15 @@ export class CompositeProvider implements AnalysisProvider {
.sort((left, right) =>
left.provider.id.localeCompare(right.provider.id),
),
requestActivitySnapshots: () =>
[...clients.values()]
.flatMap((client) => client.requestActivitySnapshots?.() ?? [])
.sort((left, right) =>
left.provider.id.localeCompare(right.provider.id),
),
closeWithOutcome,
close: async () => {
await Promise.allSettled(
[...clients.values()].map(async (client) => client.close()),
);
await closeWithOutcome();
},
};
}
+1
View File
@@ -49,6 +49,7 @@ export {
type DirectAnalysisTool =
| "binary_overview"
| "procedure_pseudo_code"
| "read_function_instructions"
| "analyze_function"
| "search_strings"
| "search_procedures"
+7 -5
View File
@@ -309,22 +309,24 @@ export class EnhancedTools {
input: { detail: "concise" | "detailed"; limit: number },
signal?: AbortSignal,
): EnhancedResult {
const [segmentsResult, documentsResult, procedures, stringsResult] =
const [segmentsResult, documentsResult, proceduresResult, stringsResult] =
await Promise.all([
this.#call("list_segments", {}, signal),
this.#call("list_documents", {}, signal),
this.#allAddressed("list_procedures", signal),
this.#call("list_strings", {}, signal),
this.#call("list_procedures", { offset: 0, limit: 1 }, signal),
this.#call("list_strings", { offset: 0, limit: 1 }, signal),
]);
if (!segmentsResult.ok) return segmentsResult;
if (!documentsResult.ok) return documentsResult;
if (!procedures.ok) return procedures;
if (!proceduresResult.ok) return proceduresResult;
if (!stringsResult.ok) return stringsResult;
const segments = parseSegments(segmentsResult.value);
if (!segments.ok) return segments;
const documents = parseDocuments(documentsResult.value);
if (!documents.ok) return documents;
const procedureCount = parseListCount(proceduresResult.value, "procedures");
if (!procedureCount.ok) return procedureCount;
const stringCount = parseListCount(stringsResult.value, "strings");
if (!stringCount.ok) return stringCount;
@@ -339,7 +341,7 @@ export class EnhancedTools {
: { name, start, end },
),
segment_count: segments.value.length,
procedure_count: procedures.value.length,
procedure_count: procedureCount.value,
string_count: stringCount.value,
});
}
@@ -0,0 +1,39 @@
import type { ProcessScenario } from "../domain/processCapture.js";
import type { LoopbackReplay } from "./LoopbackReplay.js";
import type { CommandShimReplay } from "./CommandShimReplay.js";
interface ProcessCaptureEnvironmentOptions {
readonly scenario: ProcessScenario;
readonly home: string;
readonly replay: LoopbackReplay;
readonly shimReplay: CommandShimReplay;
readonly runId: string;
}
/** Build the explicitly admitted environment for a captured process. */
export const makeProcessCaptureEnvironment = (
options: ProcessCaptureEnvironmentOptions,
): Record<string, string> => {
const { scenario, home, replay, shimReplay, runId } = options;
const environment: Record<string, string> = {
...scenario.environment,
HOME: home,
TERM: "xterm-256color",
REA_PROCESS_RUN_ID: runId,
};
for (const name of scenario.inherit_environment) {
const value = process.env[name];
if (value !== undefined) environment[name] = value;
}
environment.REA_REPLAY_HTTP_URL = replay.httpUrl;
environment.REA_REPLAY_WEBSOCKET_URL = replay.websocketUrl;
environment.REA_SHIM_LEDGER_URL = shimReplay.url;
// PATH is evidence-producing input. Inheriting the host PATH here would let
// the authority and reconstruction probe different tools without recording
// that difference. Scenarios must opt in through environment or
// inherit_environment; deterministic shims always take precedence.
environment.PATH = [shimReplay.binPath, environment.PATH ?? ""]
.filter((part) => part.length > 0)
.join(":");
return environment;
};
@@ -71,6 +71,7 @@ interface CaptureResultOptions {
};
readonly samples: readonly ProcessSample[];
readonly replay: LoopbackReplay;
readonly reactiveRun: ProcessCapture["reactive_run"];
readonly before: SnapshotResult;
readonly after: SnapshotResult;
readonly truncated: boolean;
@@ -116,6 +117,7 @@ export const buildCaptureResult = (
options.scenario,
),
replay_transitions: options.replay.transitions,
reactive_run: options.reactiveRun,
event_journal: options.eventJournal,
files_before: options.before.files,
files_after: options.after.files,
+6
View File
@@ -106,6 +106,12 @@ export class ProcessCheckpoints {
this.#scheduleCapture(name, this.#signal);
}
/** Read one committed checkpoint for synchronous journal projection. */
at(index: number): FilesystemCheckpoint | undefined {
const checkpoint = this.#captures[index];
return checkpoint === undefined ? undefined : structuredClone(checkpoint);
}
#scheduleCapture(name: string, signal: AbortSignal | undefined): void {
if (this.#captured.has(name)) return;
this.#captured.add(name);
+70 -56
View File
@@ -15,7 +15,6 @@ import {
ProcessCaptureError,
processCaptureCancelled,
} from "./ProcessCaptureError.js";
export { ProcessCaptureError } from "./ProcessCaptureError.js";
import { startLoopbackReplay, type LoopbackReplay } from "./LoopbackReplay.js";
import { startProcessSampler } from "./ProcessSampling.js";
@@ -26,7 +25,10 @@ import {
startCommandShimReplay,
type CommandShimReplay,
} from "./CommandShimReplay.js";
import { normalizeProcessText } from "./ProcessNormalization.js";
import {
normalizeProcessShimEvent,
normalizeProcessText,
} from "./ProcessNormalization.js";
import { selectCapturedProcessGroupIds } from "../process/ProcessOwnership.js";
import {
awaitTerminalExit,
@@ -42,45 +44,17 @@ import { assertNotCancelled } from "./ProcessCaptureAuthority.js";
import {
createProcessCaptureJournal,
scheduleScenarioInteractions,
type ProcessCaptureJournal,
} from "./ProcessCaptureJournal.js";
import {
assertSupportedReactiveScenario,
projectProcessReactiveRun,
startProcessReactiveHarness,
type ProcessReactiveHarness,
} from "./ProcessReactiveHarness.js";
import { makeProcessCaptureEnvironment } from "./ProcessCaptureEnvironment.js";
export { probeProcessCaptureCapability } from "./ProcessCaptureCapability.js";
interface EnvironmentOptions {
readonly scenario: ProcessScenario;
readonly home: string;
readonly replay: LoopbackReplay;
readonly shimReplay: CommandShimReplay;
readonly runId: string;
}
const makeEnvironment = (
options: EnvironmentOptions,
): Record<string, string> => {
const { scenario, home, replay, shimReplay, runId } = options;
const environment: Record<string, string> = {
...scenario.environment,
HOME: home,
TERM: "xterm-256color",
REA_PROCESS_RUN_ID: runId,
};
for (const name of scenario.inherit_environment) {
const value = process.env[name];
if (value !== undefined) environment[name] = value;
}
environment.REA_REPLAY_HTTP_URL = replay.httpUrl;
environment.REA_REPLAY_WEBSOCKET_URL = replay.websocketUrl;
environment.REA_SHIM_LEDGER_URL = shimReplay.url;
// PATH is evidence-producing input. Inheriting the host PATH here would let
// the authority and reconstruction probe different tools without recording
// that difference. Scenarios must opt in through environment or
// inherit_environment; deterministic shims always take precedence.
environment.PATH = [shimReplay.binPath, environment.PATH ?? ""]
.filter((part) => part.length > 0)
.join(":");
return environment;
};
interface StartedCaptureRuntime {
readonly replay: LoopbackReplay;
readonly renderer: TerminalRenderer;
@@ -92,6 +66,7 @@ interface StartedCaptureRuntime {
readonly lastOutput: () => number;
readonly framesTruncated: () => boolean;
readonly stopSampler: () => Promise<{ readonly partial: boolean }>;
readonly reactive: ProcessReactiveHarness | undefined;
}
const cleanupFailedStartup = async (options: {
@@ -136,7 +111,7 @@ const createTerminalRenderer = (
recordEvent,
});
const startCaptureRuntime = async (options: {
interface StartCaptureRuntimeOptions {
readonly scenario: ProcessScenario;
readonly home: string;
readonly temporaryRoot: string;
@@ -148,14 +123,29 @@ const startCaptureRuntime = async (options: {
readonly timers: Set<NodeJS.Timeout>;
readonly dispatchedEventIndexes: Set<number>;
readonly recordEvent: RecordProcessCaptureEvent;
readonly journal: ProcessCaptureJournal;
readonly signal?: AbortSignal;
}): Promise<StartedCaptureRuntime> => {
}
const reactiveCapture = (
options: StartCaptureRuntimeOptions,
protocolEvents: () => LoopbackReplay["events"],
): Parameters<typeof startProcessReactiveHarness>[0]["capture"] => ({
...options,
processSamples: options.samples,
protocolEvents,
});
const startCaptureRuntime = async (
options: StartCaptureRuntimeOptions,
): Promise<StartedCaptureRuntime> => {
const { scenario } = options;
let replay: LoopbackReplay | undefined;
let renderer: TerminalRenderer | undefined;
let checkpoints: ProcessCheckpoints | undefined;
let shimReplay: CommandShimReplay | undefined;
let terminal: IPty | undefined;
let reactive: ProcessReactiveHarness | undefined;
try {
const { spawn } = await import("@lydell/node-pty");
replay = await startLoopbackReplay(scenario, options.recordEvent);
@@ -180,7 +170,7 @@ const startCaptureRuntime = async (options: {
);
terminal = spawn(scenario.executable, [...scenario.arguments], {
cwd: scenario.working_directory,
env: makeEnvironment({ ...options, replay, shimReplay }),
env: makeProcessCaptureEnvironment({ ...options, replay, shimReplay }),
cols: scenario.terminal.columns,
rows: scenario.terminal.rows,
name: "xterm-256color",
@@ -193,6 +183,15 @@ const startCaptureRuntime = async (options: {
renderer,
checkpoints,
});
reactive = startProcessReactiveHarness({
capture: reactiveCapture(options, () => replay?.events ?? []),
scenario,
terminal: () => terminal,
renderer,
checkpoints,
shimReplay,
started,
});
scheduleScenarioInteractions({
...options,
getTerminal: () => terminal,
@@ -218,8 +217,11 @@ const startCaptureRuntime = async (options: {
lastOutput: () => lastOutput,
framesTruncated,
stopSampler,
reactive,
};
} catch (cause: unknown) {
reactive?.unsubscribe();
await reactive?.coordinator.close();
return cleanupFailedStartup({
cause,
timers: options.timers,
@@ -239,23 +241,14 @@ const normalizedShimEvents = (
scenario: ProcessScenario,
temporaryRoot: string,
): ProcessCapture["shim_events"] =>
runtime.shimReplay.events.map((event) => ({
...event,
arguments: event.arguments.map((argument) =>
normalizeProcessText(
argument,
scenario,
temporaryRoot,
runtime.terminal.pid,
),
),
working_directory: normalizeProcessText(
event.working_directory,
runtime.shimReplay.events.map((event) =>
normalizeProcessShimEvent(
event,
scenario,
temporaryRoot,
runtime.terminal.pid,
),
}));
);
const finishProcessRun = async (options: {
readonly runtime: StartedCaptureRuntime | undefined;
@@ -267,6 +260,7 @@ const finishProcessRun = async (options: {
readonly capture: ProcessCapture | undefined;
readonly executionFailure: unknown;
}): Promise<ProcessCapture> => {
options.runtime?.reactive?.unsubscribe();
await options.stopSampler();
const cleanupFailure = await releaseProcessResources({
timers: options.timers,
@@ -285,10 +279,16 @@ const finishProcessRun = async (options: {
options.samples,
),
});
if (cleanupFailure !== undefined)
await options.runtime?.reactive?.coordinator.submit({
kind: "cleanup_failed",
});
await options.runtime?.reactive?.coordinator.close();
let { capture, executionFailure } = options;
if (capture !== undefined) {
capture = {
...capture,
reactive_run: projectProcessReactiveRun(options.runtime?.reactive),
settlement: {
...capture.settlement,
cleanup_outcome:
@@ -326,7 +326,17 @@ const completeCapture = async (options: {
const { runtime, scenario } = options;
runtime.checkpoints.trigger("root_exit");
const { reason } = options.exit;
if (reason === "cancelled") throw processCaptureCancelled();
if (reason === "cancelled") {
if (runtime.reactive !== undefined) {
await runtime.reactive.coordinator.submit({ kind: "cancelled" });
runtime.reactive.unsubscribe();
}
throw processCaptureCancelled();
}
if (runtime.reactive !== undefined) {
await runtime.reactive.coordinator.submit({ kind: "target_lost" });
runtime.reactive.unsubscribe();
}
const settlement = await observeSettlement(
options.runId,
selectCapturedProcessGroupIds(runtime.terminal.pid, options.samples),
@@ -359,6 +369,7 @@ const completeCapture = async (options: {
exit: { ...options.exit, reason },
samples: options.samples,
replay: runtime.replay,
reactiveRun: projectProcessReactiveRun(runtime.reactive),
before: options.before,
after,
truncated,
@@ -381,6 +392,7 @@ const runProcessScenario = async (
policy: ProcessExecutionPolicy,
signal?: AbortSignal,
): Promise<ProcessCapture> => {
assertSupportedReactiveScenario(scenario);
const { temporaryRoot, runId, home, before } = await prepareProcessCapture(
scenario,
policy,
@@ -388,7 +400,8 @@ const runProcessScenario = async (
);
const frames: TerminalFrame[] = [];
const samples: ProcessSample[] = [];
const { entries: eventJournal, recordEvent } = createProcessCaptureJournal();
const journal = createProcessCaptureJournal();
const { entries: eventJournal, recordEvent } = journal;
let runtime: StartedCaptureRuntime | undefined;
const timers = new Set<NodeJS.Timeout>();
let capture: ProcessCapture | undefined;
@@ -410,6 +423,7 @@ const runProcessScenario = async (
timers,
dispatchedEventIndexes,
recordEvent,
journal,
...(signal === undefined ? {} : { signal }),
});
stopSampler = runtime.stopSampler;
+20
View File
@@ -2,6 +2,7 @@ import type {
ProcessCapture,
ProcessSample,
ProcessScenario,
ShimEvent,
} from "../domain/processCapture.js";
/** Bucket one elapsed process-capture timestamp under scenario normalization. */
@@ -91,6 +92,25 @@ export const normalizeProcessSamples = (
}));
};
/** Normalize one shim observation identically for live matching and capture. */
export const normalizeProcessShimEvent = (
event: ShimEvent,
scenario: ProcessScenario,
temporaryRoot: string,
rootPid: number,
): ShimEvent => ({
...event,
arguments: event.arguments.map((argument) =>
normalizeProcessText(argument, scenario, temporaryRoot, rootPid),
),
working_directory: normalizeProcessText(
event.working_directory,
scenario,
temporaryRoot,
rootPid,
),
});
const normalizePidTokens = (
value: string,
identifiers: readonly number[],
+25 -5
View File
@@ -51,7 +51,7 @@ export class ProcessReactiveCoordinator {
readonly #now: () => number;
readonly #scenarioDeadlineAt: number;
readonly #onDecision:
| ((decision: ProcessReactiveDecision) => void)
| ((decision: ProcessReactiveDecision, input: ProcessReactiveInput) => void)
| undefined;
#snapshot: ProcessReactiveSnapshot;
readonly #pendingInputs: ProcessReactiveInput[] = [];
@@ -70,7 +70,10 @@ export class ProcessReactiveCoordinator {
readonly executor: ProcessReactiveExecutor;
readonly timerHost?: ProcessReactiveTimerHost;
readonly now?: () => number;
readonly onDecision?: (decision: ProcessReactiveDecision) => void;
readonly onDecision?: (
decision: ProcessReactiveDecision,
input: ProcessReactiveInput,
) => void;
}) {
this.#scenario = options.scenario;
this.#executor = options.executor;
@@ -206,7 +209,7 @@ export class ProcessReactiveCoordinator {
);
}
this.#snapshot = decision.snapshot;
this.#onDecision?.(decision);
this.#onDecision?.(decision, input);
if (this.#snapshot.status === "finished") this.#clearTimers();
else if (
this.#accepting &&
@@ -216,18 +219,35 @@ export class ProcessReactiveCoordinator {
)
this.#scheduleStateDeadline();
for (const result of effectResults)
if (result.status === "succeeded")
if (
result.status === "succeeded" &&
!this.#hasObservation(result.observation.event_id)
)
this.#enqueueOwned({
kind: "observation",
observation: result.observation,
});
}
#hasObservation(eventId: string): boolean {
return (
this.#snapshot.observations.some(
(observation) => observation.event_id === eventId,
) ||
this.#pendingInputs.some(
(input) =>
input.kind === "observation" &&
input.observation.event_id === eventId,
)
);
}
#interruptsEffects(input: ProcessReactiveInput): boolean {
if (
input.kind === "cancelled" ||
input.kind === "cleanup_failed" ||
input.kind === "scenario_deadline"
input.kind === "scenario_deadline" ||
input.kind === "target_lost"
)
return true;
return (
+9 -3
View File
@@ -184,9 +184,15 @@ const triggerSources = (
): readonly string[] => {
if (trigger.kind === "terminal_text") return [];
if (trigger.kind === "event")
return ["terminal_raw", "interaction", "filesystem"].includes(
trigger.source,
)
return [
"terminal_raw",
"interaction",
"process",
"filesystem",
"http",
"websocket",
"shim",
].includes(trigger.source)
? []
: [trigger.source];
return trigger.kind === "repeat"
+159
View File
@@ -0,0 +1,159 @@
import type { IPty } from "@lydell/node-pty";
import type {
InteractionEvent,
ProcessCapture,
ProcessReactiveControlRecord,
ProcessSample,
ProcessScenario,
ProtocolEvent,
TerminalFrame,
} from "../domain/processCapture.js";
import type { ProcessReactiveInput } from "../domain/processReactiveRuntime.js";
import { ProcessCaptureError } from "./ProcessCaptureError.js";
import type { ProcessCaptureJournal } from "./ProcessCaptureJournal.js";
import type { ProcessCheckpoints } from "./ProcessCheckpoints.js";
import type { CommandShimReplay } from "./CommandShimReplay.js";
import { ProcessReactiveCoordinator } from "./ProcessReactiveCoordinator.js";
import {
executeProcessReactiveEffects,
unsupportedProcessReactiveFeatures,
} from "./ProcessReactiveEffects.js";
import { subscribeProcessReactiveObservations } from "./ProcessReactiveObservations.js";
import {
normalizeProcessSamples,
normalizeProcessShimEvent,
redactProtocolEvents,
} from "./ProcessNormalization.js";
import type { TerminalRenderer } from "./TerminalRenderer.js";
/** Coordinator and subscription owned by one live process capture. */
export interface ProcessReactiveHarness {
readonly coordinator: ProcessReactiveCoordinator;
readonly controls: readonly ProcessReactiveControlRecord[];
readonly unsubscribe: () => void;
}
const controlKind = (
input: ProcessReactiveInput,
): ProcessReactiveControlRecord["kind"] | undefined =>
input.kind === "observation" ? undefined : input.kind;
/** Reject reactive declarations the current process adapter cannot execute. */
export const assertSupportedReactiveScenario = (
scenario: ProcessScenario,
): void => {
if (scenario.reactive === null) return;
const unsupported = unsupportedProcessReactiveFeatures(scenario.reactive);
if (unsupported.length === 0) return;
throw new ProcessCaptureError(
`reactive scenario requests unsupported features: ${unsupported.join(", ")}`,
);
};
/**
* Attach the serialized reactive runtime once the target PID is available.
*
* Subscription replays journal entries admitted during launcher startup before
* observing new records, so PID-dependent normalization cannot drop them.
*/
export const startProcessReactiveHarness = (options: {
readonly scenario: ProcessScenario;
readonly terminal: () => IPty | undefined;
readonly renderer: TerminalRenderer;
readonly checkpoints: ProcessCheckpoints;
readonly shimReplay: CommandShimReplay;
readonly started: number;
readonly capture: {
readonly interactions: InteractionEvent[];
readonly journal: ProcessCaptureJournal;
readonly frames: readonly TerminalFrame[];
readonly processSamples: readonly ProcessSample[];
readonly protocolEvents: () => readonly ProtocolEvent[];
readonly temporaryRoot: string;
};
}): ProcessReactiveHarness | undefined => {
if (options.scenario.reactive === null) return undefined;
const controls: ProcessReactiveControlRecord[] = [];
const coordinator = new ProcessReactiveCoordinator({
scenario: options.scenario.reactive,
executor: {
execute: (actions, signal) =>
executeProcessReactiveEffects(
{
terminal: options.terminal,
renderer: options.renderer,
checkpoints: options.checkpoints,
interactions: options.capture.interactions,
journal: options.capture.journal,
startedAtMs: options.started,
now: Date.now,
},
actions,
signal,
),
},
onDecision: (decision, input) => {
const kind = controlKind(input);
if (kind === undefined || decision.kind !== "finished") return;
controls.push({
sequence: controls.length,
kind,
after_capture_order:
decision.snapshot.observations.at(-1)?.capture_order ?? -1,
});
},
});
return {
coordinator,
controls,
unsubscribe: subscribeProcessReactiveObservations({
journal: options.capture.journal,
frames: options.capture.frames,
interactions: options.capture.interactions,
checkpointAt: (index) => options.checkpoints.at(index),
processSampleAt: (index) => {
const rootPid = options.terminal()?.pid;
return rootPid === undefined
? undefined
: normalizeProcessSamples(
options.capture.processSamples,
options.scenario,
rootPid,
)[index];
},
protocolEventAt: (index) =>
redactProtocolEvents(
options.capture.protocolEvents().slice(index, index + 1),
options.scenario,
)[0],
shimEventAt: (index) => {
const event = options.shimReplay.events[index];
const rootPid = options.terminal()?.pid;
return event === undefined || rootPid === undefined
? undefined
: normalizeProcessShimEvent(
event,
options.scenario,
options.capture.temporaryRoot,
rootPid,
);
},
sink: coordinator,
}),
};
};
/** Project internal reducer state into the stable Process Capture shape. */
export const projectProcessReactiveRun = (
harness: ProcessReactiveHarness | undefined,
): ProcessCapture["reactive_run"] =>
harness === undefined
? null
: {
status: harness.coordinator.snapshot.status,
outcome: harness.coordinator.snapshot.outcome,
active_state: harness.coordinator.snapshot.active_state,
transitions: harness.coordinator.snapshot.transitions,
controls: harness.controls,
};
+145 -16
View File
@@ -1,8 +1,17 @@
import type {
InteractionEvent,
FilesystemCheckpoint,
ProcessCaptureEventJournalEntry,
ProcessSample,
ProtocolEvent,
ShimEvent,
TerminalFrame,
} from "../domain/processCapture.js";
import { createProcessObservation } from "../domain/processObservation.js";
import {
createProcessObservation,
processObservationSubject,
type ProcessObservation,
} from "../domain/processObservation.js";
import type { ProcessReactiveInput } from "../domain/processReactiveRuntime.js";
import type { ProcessCaptureJournal } from "./ProcessCaptureJournal.js";
@@ -11,25 +20,145 @@ export interface ProcessReactiveObservationSink {
enqueue(input: ProcessReactiveInput): void;
}
/** Subscribe a reactive coordinator to normalized raw PTY journal records. */
export const subscribeProcessReactiveTerminalObservations = (options: {
interface ProcessReactiveObservationOptions {
readonly journal: ProcessCaptureJournal;
readonly frames: readonly TerminalFrame[];
readonly interactions: readonly InteractionEvent[];
readonly checkpointAt: (index: number) => FilesystemCheckpoint | undefined;
readonly processSampleAt: (index: number) => ProcessSample | undefined;
readonly protocolEventAt: (index: number) => ProtocolEvent | undefined;
readonly shimEventAt: (index: number) => ShimEvent | undefined;
readonly sink: ProcessReactiveObservationSink;
}): (() => void) =>
options.journal.subscribe((entry: ProcessCaptureEventJournalEntry) => {
if (entry.collection !== "frames") return;
const frame = options.frames[entry.index];
if (frame === undefined) return;
options.sink.enqueue({
kind: "observation",
observation: createProcessObservation({
}
const terminalObservation = (
options: ProcessReactiveObservationOptions,
entry: ProcessCaptureEventJournalEntry,
): ProcessObservation | undefined => {
const value = options.frames[entry.index];
return value === undefined
? undefined
: createProcessObservation({
source: "terminal_raw",
source_sequence: frame.sequence,
captured_at_ms: frame.at_ms,
source_sequence: value.sequence,
captured_at_ms: value.at_ms,
subject_id: null,
location: entry,
payload: frame,
}),
});
payload: value,
});
};
const interactionObservation = (
options: ProcessReactiveObservationOptions,
entry: ProcessCaptureEventJournalEntry,
): ProcessObservation | undefined => {
const value = options.interactions[entry.index];
return value === undefined
? undefined
: createProcessObservation({
source: "interaction",
source_sequence: value.sequence,
captured_at_ms: value.dispatched_at_ms,
subject_id: null,
location: entry,
payload: value,
});
};
const filesystemObservation = (
options: ProcessReactiveObservationOptions,
entry: ProcessCaptureEventJournalEntry,
): ProcessObservation | undefined => {
const value = options.checkpointAt(entry.index);
return value === undefined
? undefined
: createProcessObservation({
source: "filesystem",
source_sequence: entry.index,
captured_at_ms: value.at_ms,
subject_id: processObservationSubject("filesystem", value),
location: entry,
payload: value,
});
};
const processObservation = (
options: ProcessReactiveObservationOptions,
entry: ProcessCaptureEventJournalEntry,
): ProcessObservation | undefined => {
const value = options.processSampleAt(entry.index);
return value === undefined
? undefined
: createProcessObservation({
source: "process",
source_sequence: entry.index,
captured_at_ms: value.at_ms,
subject_id: processObservationSubject("process", value),
location: entry,
payload: value,
});
};
const protocolObservation = (
options: ProcessReactiveObservationOptions,
entry: ProcessCaptureEventJournalEntry,
): ProcessObservation | undefined => {
const value = options.protocolEventAt(entry.index);
return value === undefined
? undefined
: createProcessObservation({
source: value.protocol,
source_sequence: value.sequence,
captured_at_ms: value.at_ms,
subject_id: processObservationSubject(value.protocol, value),
location: entry,
payload: value,
});
};
const shimObservation = (
options: ProcessReactiveObservationOptions,
entry: ProcessCaptureEventJournalEntry,
): ProcessObservation | undefined => {
const normalized = options.shimEventAt(entry.index);
if (normalized === undefined) return undefined;
return createProcessObservation({
source: "shim",
source_sequence: normalized.sequence,
captured_at_ms: normalized.at_ms,
subject_id: processObservationSubject("shim", normalized),
location: entry,
payload: normalized,
});
};
const observationFor = (
options: ProcessReactiveObservationOptions,
entry: ProcessCaptureEventJournalEntry,
): ProcessObservation | undefined => {
if (entry.collection === "frames") return terminalObservation(options, entry);
if (entry.collection === "interaction_events")
return interactionObservation(options, entry);
if (entry.collection === "filesystem_checkpoints")
return filesystemObservation(options, entry);
if (entry.collection === "process_samples")
return processObservation(options, entry);
if (entry.collection === "protocol_events")
return protocolObservation(options, entry);
if (entry.collection === "shim_events")
return shimObservation(options, entry);
return undefined;
};
/** Subscribe the reactive coordinator to admitted normalized journal records. */
export const subscribeProcessReactiveObservations = (
options: ProcessReactiveObservationOptions,
): (() => void) => {
const enqueue = (entry: ProcessCaptureEventJournalEntry): void => {
const observation = observationFor(options, entry);
if (observation !== undefined)
options.sink.enqueue({ kind: "observation", observation });
};
for (const entry of options.journal.entries) enqueue(entry);
return options.journal.subscribe(enqueue);
};
+2 -1
View File
@@ -1,5 +1,6 @@
import type { Readable } from "node:stream";
import type { ArtifactCommand } from "../domain/artifactGraph.js";
import type { ZipPackageFormat } from "../domain/zipPackageFormat.js";
/** Archive-neutral entry metadata. Reader adapters never choose output paths. */
export interface ArtifactEntry {
@@ -23,7 +24,7 @@ export interface ArtifactEntry {
/** Read-only adapter over one directory, archive, or virtual container. */
export interface ArtifactReader {
readonly format: "directory" | "zip" | "ipa" | "apk" | "asar" | "file";
readonly format: "directory" | ZipPackageFormat | "asar" | "file";
entries(signal?: AbortSignal): AsyncIterable<ArtifactEntry>;
open(entry: ArtifactEntry, signal?: AbortSignal): Promise<Readable>;
provenance(): readonly ArtifactCommand[];
+3 -2
View File
@@ -9,6 +9,7 @@ import {
type ArtifactEntry,
type ArtifactReader,
} from "./ArtifactReader.js";
import type { ZipPackageFormat } from "../domain/zipPackageFormat.js";
class NodeFileReader extends Reader<string> {
#handle: FileHandle | undefined;
@@ -41,12 +42,12 @@ class NodeFileReader extends Reader<string> {
/** Lazy Zip64-capable reader with CRC and overlap verification on every read. */
export class ZipArtifactReader implements ArtifactReader {
readonly format: "zip" | "ipa" | "apk";
readonly format: ZipPackageFormat;
readonly #source: NodeFileReader;
readonly #reader: ZipReader<string>;
readonly #entries = new Map<string, Entry>();
constructor(path: string, format: "zip" | "ipa" | "apk") {
constructor(path: string, format: ZipPackageFormat) {
this.format = format;
this.#source = new NodeFileReader(path);
this.#reader = new ZipReader(this.#source, {
+49
View File
@@ -16,6 +16,7 @@ export const registerCoreAnalysisCommands = (
): void => {
registerCoreCommands(cli, logger);
registerFunctionCommand(cli, logger);
registerInstructionsCommand(cli, logger);
registerSearchCommand(cli, logger);
registerXrefsCommand(cli, logger);
registerTraceCommand(cli, logger);
@@ -285,6 +286,54 @@ const registerFunctionCommand = (cli: CliInstance, logger: Logger): void => {
});
};
const registerInstructionsCommand = (
cli: CliInstance,
logger: Logger,
): void => {
cli.command(CLI_COMMANDS.instructions, {
description: "Read one bounded raw-instruction window without decompiling",
args: z.object({
path: z.string().describe("App or program path"),
address: z.string().describe("Procedure name or address"),
}),
options: z.object({
offset: z
.number()
.int()
.min(0)
.max(100_000)
.default(0)
.describe("Zero-based instruction offset"),
limit: z
.number()
.int()
.min(1)
.max(500)
.default(64)
.describe("Maximum raw instructions to return"),
snapshot: z
.string()
.min(1)
.optional()
.describe("Load and update a local analysis snapshot"),
provider: providerSelectionOption,
}),
run: ({ args, options }) =>
logCliCommand(logger, CLI_COMMANDS.instructions, () =>
runDirectAnalysis(
args.path,
"read_function_instructions",
{
procedure: args.address,
offset: options.offset,
limit: options.limit,
},
directAnalysisOptions(logger, options.snapshot, options.provider),
),
),
});
};
const registerSearchCommand = (cli: CliInstance, logger: Logger): void => {
cli.command(CLI_COMMANDS.search, {
description: "Search bounded analyzed strings or procedure names",
+1
View File
@@ -12,6 +12,7 @@ export const CLI_COMMANDS = Object.freeze({
capabilities: "capabilities",
providers: "providers",
function: "function",
instructions: "instructions",
search: "search",
importReferenceSource: "import-reference-source",
inventoryArtifact: "inventory-artifact",
@@ -0,0 +1,9 @@
import { z } from "zod";
/** Provider-neutral input for one bounded raw function-instruction window. */
export const functionInstructionInputSchema = z.object({
procedure: z.string().describe("The procedure name or address"),
offset: z.number().int().min(0).max(100_000).default(0),
limit: z.number().int().min(1).max(500).default(64),
document: z.string().optional().describe("The document name"),
});
+1
View File
@@ -17,6 +17,7 @@ export const TOOL_EXAMPLE_OVERRIDES: Readonly<
procedure_callees: { procedure: "main" },
procedure_callers: { procedure: "main" },
procedure_info: { procedure: "main" },
read_function_instructions: { procedure: "main" },
procedure_references: { procedure: "main" },
procedure_pseudo_code: { procedure: "main" },
resolve_containing_procedure: { address: "0x1000" },
+6
View File
@@ -37,6 +37,7 @@ import {
import type { ToolContract, ToolExample } from "./toolContractTypes.js";
import { toolContractMetadata } from "./toolEffects.js";
import { binarySessionInputSchema } from "./sessionStatusContract.js";
import { functionInstructionInputSchema } from "./functionInstructionContract.js";
export { binarySessionInputSchema } from "./sessionStatusContract.js";
@@ -240,6 +241,11 @@ export const OFFICIAL_TOOL_CONTRACTS = [
"Return bounded metadata for one procedure identified by symbol or address: entrypoint, signature, locals, size, and block count. Follow with decompilation or assembly for behavior.",
z.object({ procedure, document }),
),
official(
"read_function_instructions",
"Read one offset-paginated window of raw instructions for an analyzed procedure without decompilation, caller discovery, or whole-program string/name scans. Use this fast path for instruction-level orientation; follow next_offset when truncated.",
functionInstructionInputSchema,
),
official(
"procedure_references",
"Return a bounded set of raw incoming or outgoing reference edges for one procedure. Endpoint procedures are resolved only from provider containment; Ghidra preserves observed reference kinds while providers without kind authority mark them unavailable.",
+1
View File
@@ -60,6 +60,7 @@ export const TOOL_EFFECTS: Readonly<Record<string, ToolEffects>> = {
procedure_callees: evidence,
procedure_callers: evidence,
procedure_info: evidence,
read_function_instructions: evidence,
procedure_references: evidence,
procedure_pseudo_code: evidence,
resolve_containing_procedure: evidence,
+7 -1
View File
@@ -6,7 +6,10 @@ import {
} from "../domain/processCapture.js";
import { evidenceBundleSchema } from "../domain/evidenceBundle.js";
import { residualUnknownSchema } from "../domain/residualUnknown.js";
import { referenceKindSchema } from "../domain/hopperValues.js";
import {
functionInstructionWindowSchema,
referenceKindSchema,
} from "../domain/hopperValues.js";
import {
demangleSwiftSchema,
inspectMachoSchema,
@@ -39,6 +42,7 @@ import { replayMachineRunOutputSchema } from "../domain/replayMachineRun.js";
import { analysisErrorProjectionSchema } from "./errorSchemas.js";
import {
addressList,
analysisActivity,
addressedEntry,
bounded,
containingProcedureResolution,
@@ -82,6 +86,7 @@ export const officialOutputSchemas: Readonly<Record<string, z.ZodObject>> = {
procedure_callees: resultOf(addressList),
procedure_callers: resultOf(addressList),
procedure_info: procedureInfoOutput,
read_function_instructions: resultOf(functionInstructionWindowSchema),
procedure_references: resultOf(
z.object({
procedure: procedureIdentity,
@@ -297,6 +302,7 @@ export const sessionOutputSchemas: Readonly<Record<string, z.ZodObject>> = {
reasons: z.array(z.string()),
remediation: z.string().nullable(),
}),
analysis_activity: analysisActivity,
recommended_actions: z.array(z.string()),
}),
z.object({
@@ -45,6 +45,8 @@ export const targetFormatSchema = z.enum([
"zip",
"ipa",
"apk",
"msix",
"appx",
"asar",
"dmg",
"pkg",
@@ -64,6 +66,7 @@ const providerCapability = z.object({
operation: z.string(),
available: z.boolean(),
reason: z.string().nullable(),
availability_code: z.enum(PROVIDER_REJECTION_CODES).nullable(),
input_contract_version: z.number().int().min(1),
output_contract_version: z.number().int().min(1),
pagination: z.enum(["none", "offset", "cursor"]),
@@ -137,6 +140,25 @@ const analysisRun = z
})
.nullable();
export const analysisActivity = z.object({
status: z.enum(["not_observed", "idle", "busy", "timed_out_busy"]),
providers: z.array(
z.object({
provider: providerIdentity,
active: z
.object({
request_id: z.number().int().min(1),
operation: z.string().min(1),
elapsed_ms: z.number().int().min(0),
timeout_ms: z.number().int().min(0),
caller_state: z.enum(["waiting", "timed_out", "cancelled"]),
})
.nullable(),
queued_requests: z.number().int().min(0),
}),
),
});
const providerRejectionCode: z.ZodType<ProviderRejectionCode> = z.enum(
PROVIDER_REJECTION_CODES,
);
@@ -222,6 +244,7 @@ export const sessionProvider = z.object({
providers: z.array(providerIdentity),
capabilities: z.array(providerCapability),
analysis_run: analysisRun,
analysis_activity: analysisActivity,
analysis_provider_binding: z
.object({
provider: providerIdentity,
+7
View File
@@ -60,6 +60,7 @@ export const projectAnalysisError = (
};
const errorCode = (error: AnalysisError): AnalysisErrorProjection["code"] => {
if (error.cleanupIncomplete) return "cleanup_incomplete";
if (error instanceof ReplayPlanStaleError) return "plan_stale";
if (error instanceof PermissionRequiredError) return "permission_required";
if (error instanceof ProviderSelectionError)
@@ -295,6 +296,12 @@ const providerErrorDetails = (
return {
provider_id: error.providerId,
operation: error.operation,
...(error.cleanupIncomplete
? {
cleanup: "incomplete",
resources: [...error.cleanupResources],
}
: {}),
...(error.diagnostics === undefined
? {}
: { diagnostics: error.diagnostics }),
+2
View File
@@ -37,6 +37,8 @@ const formatSchema = z.enum([
"zip",
"ipa",
"apk",
"msix",
"appx",
"asar",
"dmg",
"pkg",
+2
View File
@@ -52,6 +52,8 @@ const artifactFormatSchema = z.enum([
"analysis-database",
"ipa",
"apk",
"msix",
"appx",
"plist",
"entitlements",
"directory",
+5 -6
View File
@@ -14,6 +14,7 @@ import { promisify } from "node:util";
import { BinaryTargetError } from "./errors.js";
import { err, ok, type Result } from "./result.js";
import { zipPackageFormatForPath } from "./zipPackageFormat.js";
const execFileAsync = promisify(execFile);
@@ -36,6 +37,8 @@ export interface BinaryTarget {
| "zip"
| "ipa"
| "apk"
| "msix"
| "appx"
| "asar"
| "dmg"
| "pkg"
@@ -131,11 +134,7 @@ const detectArtifactFormat = async (
[0x03, 0x05, 0x07].includes(magic[2] ?? -1) &&
[0x04, 0x06, 0x08].includes(magic[3] ?? -1)
) {
return lower.endsWith(".ipa")
? "ipa"
: lower.endsWith(".apk")
? "apk"
: "zip";
return zipPackageFormatForPath(lower) ?? "zip";
}
const named = namedArtifactFormat(lower);
if (named !== undefined) return named;
@@ -167,7 +166,7 @@ const namedArtifactFormat = (
};
const isArchiveFormat = (format: BinaryTarget["format"]): boolean =>
["zip", "ipa", "apk", "asar", "dmg", "pkg"].includes(format);
["zip", "ipa", "apk", "msix", "appx", "asar", "dmg", "pkg"].includes(format);
const sha256Handle = async (handle: FileHandle): Promise<string> => {
const hash = createHash("sha256");
+2
View File
@@ -29,6 +29,8 @@ const subjectSchema = z.object({
"zip",
"ipa",
"apk",
"msix",
"appx",
"asar",
"dmg",
"pkg",
+12
View File
@@ -143,6 +143,18 @@ const boundedSchema = <T extends z.ZodType>(item: T) =>
});
}
});
/** Provider-neutral bounded raw-instruction window for one analyzed function. */
export const functionInstructionWindowSchema = z
.object({
procedure: procedureIdentitySchema,
instructions: boundedSchema(z.string()),
instructions_scanned: z.number().int().min(0),
instruction_scan_truncated: z.boolean(),
limitations: z.array(z.string()),
})
.strict();
const availableReferenceKindSchema = z
.object({
available: z.literal(true),
+93
View File
@@ -5,6 +5,11 @@ import {
LEGACY_PROCESS_CAPTURE_MESSAGE,
normalizationSchema,
} from "./processScenario.js";
import type {
ProcessReactiveOutcome,
ProcessReactiveTransitionRecord,
} from "./processReactiveRuntime.js";
import { PROCESS_REACTIVE_LIMITS } from "./processReactiveScenario.js";
import type { ReplayTransitionRecord } from "./replayMachineRuntime.js";
export * from "./processScenario.js";
@@ -133,6 +138,18 @@ export interface ProcessCaptureEventJournalEntry {
readonly index: number;
}
/** Ordered control input that terminated or superseded a reactive run. */
export interface ProcessReactiveControlRecord {
readonly sequence: number;
readonly kind:
| "state_deadline"
| "scenario_deadline"
| "target_lost"
| "cancelled"
| "cleanup_failed";
readonly after_capture_order: number;
}
/** Shared observation callback used by every process-capture producer. */
export type RecordProcessCaptureEvent = (
collection: ProcessCaptureEventCollection,
@@ -185,6 +202,13 @@ export interface ProcessCapture {
readonly shim_events: readonly ShimEvent[];
readonly protocol_events: readonly ProtocolEvent[];
readonly replay_transitions: readonly ReplayTransitionRecord[];
readonly reactive_run: {
readonly status: "running" | "finished";
readonly outcome: ProcessReactiveOutcome | null;
readonly active_state: string;
readonly transitions: readonly ProcessReactiveTransitionRecord[];
readonly controls: readonly ProcessReactiveControlRecord[];
} | null;
/**
* Global observation order across independently recorded collections.
*
@@ -363,6 +387,75 @@ export const processCaptureSchema: z.ZodType<ProcessCapture> = z.object({
}),
)
.default([]),
reactive_run: z
.object({
status: z.enum(["running", "finished"]),
outcome: z
.enum([
"passed",
"predicate_timeout",
"scenario_deadline",
"ambiguous_match",
"action_rejected",
"target_lost",
"capture_incomplete",
"cancelled",
"cleanup_failed",
])
.nullable(),
active_state: z.string().min(1).max(64),
controls: z
.array(
z.object({
sequence: z.number().int().nonnegative(),
kind: z.enum([
"state_deadline",
"scenario_deadline",
"target_lost",
"cancelled",
"cleanup_failed",
]),
after_capture_order: z.number().int().min(-1),
}),
)
.default([]),
transitions: z
.array(
z.object({
sequence: z.number().int().nonnegative(),
transition_id: z.string().min(1).max(64),
state_before: z.string().min(1).max(64),
state_after: z.string().min(1).max(64).nullable(),
outcome: z
.enum([
"passed",
"predicate_timeout",
"scenario_deadline",
"ambiguous_match",
"action_rejected",
"target_lost",
"capture_incomplete",
"cancelled",
"cleanup_failed",
])
.nullable(),
trigger_event_ids: z
.array(z.string().min(1))
.max(PROCESS_REACTIVE_LIMITS.predicates),
action_event_ids: z
.array(z.string().min(1))
.max(PROCESS_REACTIVE_LIMITS.actionsPerTransition),
action_types: z
.array(
z.enum(["send_input", "resize", "send_signal", "checkpoint"]),
)
.max(PROCESS_REACTIVE_LIMITS.actionsPerTransition),
}),
)
.max(PROCESS_REACTIVE_LIMITS.transitions),
})
.nullable()
.default(null),
event_journal: z
.array(
z.object({
@@ -0,0 +1,273 @@
import canonicalize from "canonicalize";
import type { ProcessCapture } from "./processCapture.js";
import {
projectProcessObservation,
type ProcessObservation,
} from "./processObservation.js";
import {
commitProcessReactiveProposal,
createProcessReactiveSnapshot,
reduceProcessReactiveScenario,
type ProcessReactiveDecision,
type ProcessReactiveInput,
type ProcessReactiveSnapshot,
} from "./processReactiveRuntime.js";
import {
processReactiveScenarioSchema,
type ProcessReactiveScenario,
} from "./processReactiveScenario.js";
type RequireInvariant = (
condition: boolean,
path: string,
message: string,
) => void;
type ReactiveRun = NonNullable<ProcessCapture["reactive_run"]>;
type ReactiveTransition = ReactiveRun["transitions"][number];
const admittedCollection = (
collection: NonNullable<
ProcessCapture["event_journal"]
>[number]["collection"],
): boolean =>
[
"frames",
"interaction_events",
"process_samples",
"filesystem_checkpoints",
"shim_events",
"protocol_events",
].includes(collection);
const reactiveObservations = (
capture: ProcessCapture,
): readonly ProcessObservation[] =>
(capture.event_journal ?? []).flatMap((location) => {
if (!admittedCollection(location.collection)) return [];
const observation = projectProcessObservation(capture, location);
return observation === null ? [] : [observation];
});
const validateTransitionDeclaration = (
transition: ReactiveTransition,
next: ReactiveTransition | undefined,
context: {
readonly index: number;
readonly scenario: ProcessReactiveScenario;
readonly require: RequireInvariant;
},
): void => {
const { index, scenario, require } = context;
const path = `reactive_run.transitions[${String(index)}]`;
const declared = scenario.states
.find(({ id }) => id === transition.state_before)
?.on.find(({ id }) => id === transition.transition_id);
require(declared !==
undefined, path, "reactive transition is not declared by its recorded source state");
if (declared !== undefined) {
const stateAfter =
declared.target.kind === "goto" ? declared.target.state : null;
const outcome =
declared.target.kind === "finish" ? declared.target.outcome : null;
require(transition.state_after === stateAfter &&
transition.outcome ===
outcome, path, "reactive transition target or outcome differs from the committed graph");
require(canonicalize(transition.action_types) ===
canonicalize(
declared.actions.map(({ type }) => type),
), `${path}.action_types`, "reactive transition actions differ from the committed graph");
}
require(transition.sequence ===
index, `${path}.sequence`, "reactive transition sequence must be contiguous from zero");
if (next !== undefined)
require(transition.state_after ===
next.state_before, `reactive_run.transitions[${String(index + 1)}].state_before`, "reactive transition states must be contiguous");
};
const validateTransitionDeclarations = (
run: ReactiveRun,
scenario: ProcessReactiveScenario,
require: RequireInvariant,
): void => {
require(run.transitions[0]?.state_before === scenario.initial_state ||
run.transitions.length ===
0, "reactive_run.transitions[0].state_before", "reactive transition journal must start at the declared initial state");
for (const [index, transition] of run.transitions.entries())
validateTransitionDeclaration(transition, run.transitions[index + 1], {
index,
scenario,
require,
});
};
const commitRecordedEffects = (options: {
readonly scenario: ProcessReactiveScenario;
readonly proposal: Extract<
ProcessReactiveDecision,
{ readonly kind: "proposal" }
>;
readonly expected: ReactiveTransition | undefined;
readonly observations: ReadonlyMap<string, ProcessObservation>;
readonly recordedFailure: "action_rejected" | "target_lost" | undefined;
}): ProcessReactiveDecision =>
commitProcessReactiveProposal(
options.scenario,
options.proposal,
options.recordedFailure === undefined
? options.proposal.effects.map((_, index) => {
const eventId = options.expected?.action_event_ids[index];
const observation =
eventId === undefined
? undefined
: options.observations.get(eventId);
return observation === undefined
? { status: "rejected" as const }
: { status: "succeeded" as const, observation };
})
: [
options.recordedFailure === "target_lost"
? { status: "target_lost" as const }
: { status: "rejected" as const },
],
{ committedSensitiveInputs: true },
);
const replayTransitions = (
scenario: ProcessReactiveScenario,
run: ReactiveRun,
observations: readonly ProcessObservation[],
require: RequireInvariant,
): ProcessReactiveSnapshot => {
const observationsById = new Map(
observations.map((observation) => [observation.event_id, observation]),
);
let snapshot = createProcessReactiveSnapshot(scenario);
let transitionIndex = 0;
const advance = (input: ProcessReactiveInput): void => {
let decision = reduceProcessReactiveScenario(scenario, snapshot, input);
if (decision.kind === "proposal") {
const proposal = decision;
decision = commitRecordedEffects({
scenario,
proposal: decision,
expected: run.transitions[transitionIndex],
observations: observationsById,
recordedFailure:
run.transitions[transitionIndex] === undefined &&
decision.effects.length > 0 &&
(run.outcome === "action_rejected" || run.outcome === "target_lost")
? run.outcome
: undefined,
});
require(decision.kind !== "finished" ||
decision.outcome !== "action_rejected" ||
run.outcome ===
"action_rejected", `reactive_run.transitions[${String(transitionIndex)}].action_event_ids`, `recorded action observations do not match proposed effects: ${canonicalize(
{
effects: proposal.effects,
observations: run.transitions[transitionIndex]?.action_event_ids.map(
(eventId) => observationsById.get(eventId),
),
},
)}`);
}
if (decision.kind === "transition") {
const expected = run.transitions[transitionIndex];
require(expected !== undefined &&
canonicalize(decision.record) ===
canonicalize(
expected,
), `reactive_run.transitions[${String(transitionIndex)}]`, "reactive transition differs from deterministic journal replay");
transitionIndex += 1;
}
snapshot = decision.snapshot;
};
const advanceControl = (
control: ReactiveRun["controls"][number],
index: number,
): void => {
require(snapshot.status !== "finished" ||
control.kind ===
"cleanup_failed", `reactive_run.controls[${String(index)}]`, "reactive control cannot follow a finished run");
advance(
control.kind === "state_deadline"
? {
kind: control.kind,
state_id: snapshot.active_state,
state_entry_capture_order: snapshot.state_entry_capture_order,
}
: { kind: control.kind },
);
};
let controlIndex = 0;
for (const observation of observations) {
while (
(run.controls[controlIndex]?.after_capture_order ??
Number.MAX_SAFE_INTEGER) < observation.capture_order
) {
const control = run.controls[controlIndex]!;
advanceControl(control, controlIndex);
controlIndex += 1;
}
advance({ kind: "observation", observation });
}
for (; controlIndex < run.controls.length; controlIndex += 1) {
const control = run.controls[controlIndex]!;
advanceControl(control, controlIndex);
}
require(transitionIndex ===
run.transitions
.length, "reactive_run.transitions", `reactive transition journal is not reproduced by deterministic replay (replayed ${String(transitionIndex)} of ${String(run.transitions.length)}; outcome ${snapshot.outcome})`);
return snapshot;
};
const validateControls = (
run: ReactiveRun,
observations: readonly ProcessObservation[],
require: RequireInvariant,
): void => {
const admittedOrders = new Set([
-1,
...observations.map(({ capture_order }) => capture_order),
]);
for (const [index, control] of run.controls.entries()) {
require(control.sequence ===
index, `reactive_run.controls[${String(index)}].sequence`, "reactive control sequence must be contiguous");
require(admittedOrders.has(
control.after_capture_order,
), `reactive_run.controls[${String(index)}].after_capture_order`, "reactive control cutoff must identify the last admitted observation");
require(index === 0 ||
control.after_capture_order >=
run.controls[index - 1]!
.after_capture_order, `reactive_run.controls[${String(index)}].after_capture_order`, "reactive control cutoffs must be ordered");
}
};
/** Validate one capture's reactive journal by replaying the production reducer. */
export const validateProcessCaptureReactiveRun = (
capture: ProcessCapture,
require: RequireInvariant,
): void => {
const declared = capture.manifest.scenario["reactive"];
const run = capture.reactive_run;
const declaredReactive = declared !== undefined && declared !== null;
require(declaredReactive ===
(run !==
null), "reactive_run", "reactive result presence must match the committed scenario");
if (run === null) return;
const parsed = processReactiveScenarioSchema.safeParse(declared);
require(parsed.success, "manifest.scenario.reactive", "committed reactive scenario does not satisfy its declared schema");
if (!parsed.success) return;
require(run.status === "finished" &&
run.outcome !==
null, "reactive_run.outcome", "completed capture requires a finished reactive outcome");
validateTransitionDeclarations(run, parsed.data, require);
const observations = reactiveObservations(capture);
validateControls(run, observations, require);
const snapshot = replayTransitions(parsed.data, run, observations, require);
require(snapshot.outcome ===
run.outcome, "reactive_run.outcome", "reactive outcome differs from deterministic journal replay");
require(snapshot.active_state ===
run.active_state, "reactive_run.active_state", "reactive active state differs from deterministic journal replay");
};
+2
View File
@@ -1,4 +1,5 @@
import type { ProcessCapture } from "./processCapture.js";
import { validateProcessCaptureReactiveRun } from "./processCaptureReactiveValidation.js";
import { replayMachineSchema, type ReplayMachine } from "./replayMachine.js";
import { digestProcessCommitment } from "./processScenario.js";
@@ -314,5 +315,6 @@ export const validateProcessCapture = (
validateLifecycle(capture, require);
validateShimEvents(capture, require);
validateReplayEvents(capture, require);
validateProcessCaptureReactiveRun(capture, require);
return issues;
};
+20 -7
View File
@@ -109,16 +109,29 @@ const atMilliseconds = (value: unknown): number | null => {
return null;
};
const subjectFor = (
/** Derive the stable capture-local identity for one normalized observation. */
export const processObservationSubject = (
source: ProcessObservationSource,
value: JsonValue,
value: unknown,
): string | null => {
if (typeof value !== "object" || value === null || Array.isArray(value))
return null;
if (source === "process" && typeof value["pid"] === "number")
return `process:${String(value["pid"])}`;
if (source === "filesystem" && typeof value["name"] === "string")
return `checkpoint:${value["name"]}`;
if (source === "process" && "pid" in value && typeof value.pid === "number")
return `process:${String(value.pid)}`;
if (
source === "filesystem" &&
"name" in value &&
typeof value.name === "string"
)
return `checkpoint:${value.name}`;
if (
source === "shim" &&
"command" in value &&
typeof value.command === "string"
)
return "route_index" in value && typeof value.route_index === "number"
? `shim:${value.command}:route:${String(value.route_index)}`
: `shim:${value.command}:unmatched`;
return null;
};
@@ -188,7 +201,7 @@ const projectedRecord = (
return {
source: projected.source,
captured_at_ms: atMilliseconds(payload),
subject_id: subjectFor(projected.source, payload),
subject_id: processObservationSubject(projected.source, payload),
payload,
};
};
+33 -16
View File
@@ -65,6 +65,7 @@ export type ProcessReactiveInput =
readonly state_entry_capture_order: number;
}
| { readonly kind: "scenario_deadline" }
| { readonly kind: "target_lost" }
| { readonly kind: "cancelled" }
| { readonly kind: "cleanup_failed" };
@@ -123,6 +124,33 @@ export type ProcessReactiveDecision =
const counterValue = (counters: readonly Counter[], id: string): number =>
counters.find((counter) => counter.id === id)?.count ?? 0;
type ObservationAdmission =
| { readonly kind: "new"; readonly snapshot: ProcessReactiveSnapshot }
| { readonly kind: "invalid" };
const observeInput = (
snapshot: ProcessReactiveSnapshot,
observation: ProcessObservation,
): ObservationAdmission => {
const duplicate = snapshot.observations.find(
({ event_id }) => event_id === observation.event_id,
);
if (duplicate !== undefined) return { kind: "invalid" };
const prior = snapshot.observations.at(-1);
if (
(prior !== undefined && observation.capture_order <= prior.capture_order) ||
snapshot.observations.length >= PROCESS_REACTIVE_LIMITS.retainedObservations
)
return { kind: "invalid" };
return {
kind: "new",
snapshot: {
...snapshot,
observations: [...snapshot.observations, observation],
},
};
};
/** Create the initial immutable reducer state for one validated scenario. */
export const createProcessReactiveSnapshot = (
scenario: ProcessReactiveScenario,
@@ -166,6 +194,8 @@ const reduceControlInput = (
: { kind: "waiting", snapshot, evaluations: [] };
if (input.kind === "scenario_deadline")
return finishedDecision(snapshot, "scenario_deadline");
if (input.kind === "target_lost")
return finishedDecision(snapshot, "target_lost");
return finishedDecision(snapshot, "cancelled");
};
@@ -179,23 +209,10 @@ export const reduceProcessReactiveScenario = (
if (controlDecision !== null) return controlDecision;
if (input.kind !== "observation")
return finishedDecision(snapshot, "capture_incomplete");
const prior = snapshot.observations.at(-1);
if (
(prior !== undefined &&
input.observation.capture_order <= prior.capture_order) ||
snapshot.observations.some(
({ event_id }) => event_id === input.observation.event_id,
)
)
const admitted = observeInput(snapshot, input.observation);
if (admitted.kind === "invalid")
return finishedDecision(snapshot, "capture_incomplete");
if (
snapshot.observations.length >= PROCESS_REACTIVE_LIMITS.retainedObservations
)
return finishedDecision(snapshot, "capture_incomplete");
const observed = {
...snapshot,
observations: [...snapshot.observations, input.observation],
};
const observed = admitted.snapshot;
const state = scenario.states.find(({ id }) => id === observed.active_state);
if (state === undefined) return finishedDecision(observed, "target_lost");
const budget = { remaining: PROCESS_REACTIVE_LIMITS.evaluationWork };
+22 -4
View File
@@ -124,6 +124,7 @@ const applyProcessReactiveTransition = (input: {
const effectObservationMatchesAction = (
action: ProcessReactiveScenario["states"][number]["on"][number]["actions"][number],
observation: ProcessObservation,
committedSensitiveInputs: boolean,
): boolean => {
const payload =
typeof observation.payload === "object" &&
@@ -153,9 +154,11 @@ const effectObservationMatchesAction = (
return (
payload["type"] === "input" &&
payload["data"] ===
(action.sensitive
? `<redacted-input:${String(Buffer.byteLength(action.data))}-bytes>`
: action.data)
expectedInputEvidence(
action.data,
action.sensitive,
committedSensitiveInputs,
)
);
if (action.type === "resize")
return (
@@ -165,6 +168,16 @@ const effectObservationMatchesAction = (
return payload["type"] === "signal" && payload["data"] === action.signal;
};
const expectedInputEvidence = (
data: string,
sensitive: boolean,
committedSensitiveInput: boolean,
): string =>
!sensitive ||
(committedSensitiveInput && /^<redacted-input:\d+-bytes>$/.test(data))
? data
: `<redacted-input:${String(Buffer.byteLength(data))}-bytes>`;
const failedProposal = (
proposed: Extract<ProcessReactiveDecision, { readonly kind: "proposal" }>,
outcome: "action_rejected" | "target_lost",
@@ -180,6 +193,7 @@ export const commitProcessReactiveProposal = (
scenario: ProcessReactiveScenario,
proposed: Extract<ProcessReactiveDecision, { readonly kind: "proposal" }>,
results: readonly ProcessReactiveEffectResult[],
options: { readonly committedSensitiveInputs?: boolean } = {},
): ProcessReactiveDecision => {
if (results.some(({ status }) => status === "target_lost"))
return failedProposal(proposed, "target_lost");
@@ -227,7 +241,11 @@ export const commitProcessReactiveProposal = (
const effectObservation = result.observation;
if (
action === undefined ||
!effectObservationMatchesAction(action, effectObservation) ||
!effectObservationMatchesAction(
action,
effectObservation,
options.committedSensitiveInputs ?? false,
) ||
!Number.isSafeInteger(effectObservation.capture_order) ||
effectObservation.capture_order <= previousOrder ||
knownEventIds.has(effectObservation.event_id)
+30
View File
@@ -2,6 +2,10 @@ import { createHash } from "node:crypto";
import { resolve } from "node:path";
import canonicalize from "canonicalize";
import { z } from "zod";
import {
processReactiveScenarioSchema,
type ProcessReactiveScenario,
} from "./processReactiveScenario.js";
import { replayMachineSchema } from "./replayMachine.js";
const positiveBudget = z.number().int().positive();
@@ -51,6 +55,29 @@ export const digestProcessCommitment = (value: unknown): string => {
return createHash("sha256").update(serialized).digest("hex");
};
const reactiveScenarioCommitment = (
scenario: ProcessReactiveScenario | null,
): Readonly<Record<string, unknown>> | null =>
scenario === null
? null
: {
...scenario,
states: scenario.states.map((state) => ({
...state,
on: state.on.map((transition) => ({
...transition,
actions: transition.actions.map((action) =>
action.type === "send_input" && action.sensitive
? {
...action,
data: `<redacted-input:${String(Buffer.byteLength(action.data))}-bytes>`,
}
: action,
),
})),
})),
};
/** Build the secret-safe scenario projection committed by Process Capture v4. */
export const processScenarioCommitment = (
scenario: ProcessScenario,
@@ -63,6 +90,7 @@ export const processScenarioCommitment = (
scenario.secret_aliases.includes(name) ? "<redacted-secret>" : value,
]),
),
reactive: reactiveScenarioCommitment(scenario.reactive),
unknown_registry_approved: scenario.unknown_registry_approved === true,
executable_sha256: executableSha256 ?? null,
});
@@ -86,6 +114,7 @@ export const processComparisonContract = (
normalization: scenario.normalization,
command_shims: scenario.command_shims,
replay: scenario.replay,
reactive: reactiveScenarioCommitment(scenario.reactive),
});
/**
@@ -292,6 +321,7 @@ export const processScenarioSchema = z
websocket_messages: [],
websocket_connections: [],
}),
reactive: processReactiveScenarioSchema.nullable().default(null),
})
.strict()
.superRefine((scenario, context) => {
+20
View File
@@ -0,0 +1,20 @@
import { ProviderAdapterError } from "./errors.js";
import type { JsonValue } from "./jsonValue.js";
/** Provider resources could not be proven closed after bounded cleanup. */
export class ProviderCleanupError extends ProviderAdapterError {
override readonly cleanupIncomplete = true;
override readonly cleanupResources: readonly string[];
override readonly userMessage =
"Provider cleanup could not be fully confirmed. Review the reported local resources before opening another target.";
constructor(
providerId: string,
resources: readonly string[],
diagnostics: Readonly<Record<string, JsonValue>>,
options?: ErrorOptions,
) {
super(providerId, "close_binary", { ...options, diagnostics });
this.cleanupResources = [...resources];
}
}
+20
View File
@@ -0,0 +1,20 @@
/** ZIP-backed application package families that share the hardened archive reader. */
export type ZipPackageFormat = "zip" | "ipa" | "apk" | "msix" | "appx";
/**
* Classify a ZIP-backed package by its complete lower-cased path suffix.
*
* This is deliberately extension-only. Callers must separately verify ZIP
* magic before trusting the classification for a root input.
*/
export const zipPackageFormatForPath = (
path: string,
): ZipPackageFormat | undefined => {
const lower = path.toLowerCase();
if (lower.endsWith(".zip")) return "zip";
if (lower.endsWith(".ipa")) return "ipa";
if (lower.endsWith(".apk") || lower.endsWith(".aab")) return "apk";
if (lower.endsWith(".msix") || lower.endsWith(".msixbundle")) return "msix";
if (lower.endsWith(".appx") || lower.endsWith(".appxbundle")) return "appx";
return undefined;
};
+1 -1
View File
@@ -1,7 +1,7 @@
/** Generated from package.json and package-lock.json; do not edit. */
export const PACKAGE_METADATA = {
name: "rea-agents",
version: "2.3.0", // x-release-please-version
version: "2.4.1", // x-release-please-version
serverSdkVersion: "2.0.0-beta.4",
clientSdkVersion: "2.0.0-beta.4",
coreSdkVersion: "2.0.0-beta.4",
+1 -1
View File
@@ -17,4 +17,4 @@ export const GHIDRA_MAX_QUEUED_REQUESTS = 32;
/** Maximum encoded response line accepted from the Java bridge. */
export const GHIDRA_MAX_LINE_BYTES = 1024 * 1024;
/** Version of REA's Java bridge handshake and request protocol. */
export const GHIDRA_BRIDGE_VERSION = 4;
export const GHIDRA_BRIDGE_VERSION = 5;
+13
View File
@@ -2,6 +2,7 @@ import { z } from "zod";
import { AnalysisInputError, AnalysisOutputError } from "../domain/errors.js";
import {
functionInstructionWindowSchema,
functionDossierSchema,
type FunctionDossier,
} from "../domain/hopperValues.js";
@@ -24,6 +25,7 @@ export const GHIDRA_FUNCTION_OPERATIONS = [
"procedure_callers",
"procedure_info",
"procedure_pseudo_code",
"read_function_instructions",
"procedure_references",
"xrefs",
] as const;
@@ -88,6 +90,13 @@ const inputSchemas = {
procedure_callers: z.object(directProcedure).strict(),
procedure_info: z.object(directProcedure).strict(),
procedure_pseudo_code: z.object(directProcedure).strict(),
read_function_instructions: z
.object({
...directProcedure,
offset: z.number().int().min(0).max(100_000).default(0),
limit: z.number().int().min(1).max(500).default(64),
})
.strict(),
procedure_references: z
.object({
...directProcedure,
@@ -281,6 +290,9 @@ const ghidraFunctionDossier = functionDossierSchema.superRefine(
});
},
);
const ghidraFunctionInstructionWindow = functionInstructionWindowSchema
.extend({ procedure: procedureIdentity })
.strict();
const resultSchemas = {
analyze_function: ghidraFunctionDossier,
@@ -289,6 +301,7 @@ const resultSchemas = {
procedure_callers: z.array(ghidraCanonicalAddressSchema),
procedure_info: procedureInfo,
procedure_pseudo_code: z.string().nullable(),
read_function_instructions: ghidraFunctionInstructionWindow,
procedure_references: procedureReferences,
xrefs: z.array(ghidraCanonicalAddressSchema),
} satisfies Readonly<Record<GhidraFunctionOperation, z.ZodType>>;
+7
View File
@@ -49,6 +49,7 @@ const PAGINATED_OPERATIONS: ReadonlySet<string> = new Set([
"search_procedures",
"search_strings",
"procedure_references",
"read_function_instructions",
"analyze_function",
]);
const SEARCH_OPERATIONS: ReadonlySet<string> = new Set([
@@ -119,6 +120,12 @@ export const limitationsFor = (operation: string): readonly string[] => {
"Pseudocode is Ghidra decompiler output, not original source and not text-equivalent to Hopper output; each decompile has a 30-second native deadline.",
"External functions and functions without an analyzable body return null; other decompiler failures remain explicit.",
];
case "read_function_instructions":
return [
...common,
"This bounded fast path reads only the requested function instruction window and does not invoke the decompiler or whole-program string/name inventories.",
"Instruction text is Ghidra-specific and does not claim textual equivalence with Hopper output.",
];
case "procedure_assembly":
return [
...common,
+265
View File
@@ -0,0 +1,265 @@
import type { Socket } from "node:net";
import type { ProgressReporter } from "../application/ProgressReporter.js";
import type { AnalysisError, HopperError } from "../domain/errors.js";
import { HopperProcessError } from "../domain/errors.js";
import type { JsonValue } from "../domain/jsonValue.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import { err, ok, type Result } from "../domain/result.js";
import type { Logger } from "../logger.js";
import type { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
import type { ProcessCleanupResult } from "../process/ProcessOwnership.js";
import type { ProviderProcessSupervisor } from "../process/ProviderProcess.js";
import type { BridgeLaunch } from "./BridgeLauncher.js";
import {
createOwnedHopperShutdownDiagnostic,
type HopperDiagnostic,
providerCleanupFailure,
} from "./HopperDiagnostics.js";
import type { HopperRequestActivity } from "./HopperRequestQueue.js";
import {
isHopperCleanupRequired,
isHopperShutdownAcknowledgement,
} from "./HopperSessionValues.js";
interface CleanupIssue {
readonly resource: string;
readonly reason: string;
}
interface CleanupState {
readonly issues: CleanupIssue[];
readonly resources: Set<string>;
cleanupResult: ProcessCleanupResult | undefined;
shutdownConfirmed: boolean;
ownedProcessStopped: boolean;
}
export interface HopperCleanupInput {
readonly socket: Socket | undefined;
readonly launch: BridgeLaunch | undefined;
readonly processSupervisor: ProviderProcessSupervisor | undefined;
readonly runtimeRoot: PrivateRuntimeRoot | undefined;
readonly activeRequest: HopperRequestActivity | null;
readonly progress: ProgressReporter | undefined;
readonly logger: Logger;
readonly onDiagnostic: ((event: HopperDiagnostic) => void) | undefined;
request(
method: "shutdown" | "shutdown_document",
): Promise<Result<JsonValue, HopperError>>;
releaseTransport(socket: Socket | undefined): void;
}
/** Close Hopper phases and retain every resource whose cleanup is unverified. */
export const cleanupHopperSession = async (
input: HopperCleanupInput,
): Promise<Result<null, AnalysisError>> => {
const state: CleanupState = {
issues: [],
resources: new Set(),
cleanupResult: undefined,
shutdownConfirmed: false,
ownedProcessStopped: false,
};
await report(input.progress, 0, "requesting Hopper document shutdown");
await requestShutdown(input, state);
await report(input.progress, 0.35, "releasing Hopper bridge transport");
input.releaseTransport(input.socket);
await stopProcess(input, state);
recordUnconfirmedDocument(input, state);
await report(input.progress, 0.75, "removing Hopper private runtime files");
await closeRuntimeRoot(input.runtimeRoot, state);
return cleanupOutcome(input, state);
};
const requestShutdown = async (
input: HopperCleanupInput,
state: CleanupState,
): Promise<void> => {
if (
input.socket === undefined ||
input.socket.destroyed ||
input.activeRequest !== null
) {
if (input.activeRequest !== null)
input.logger.warn(
{
method: input.activeRequest.operation,
callerState: input.activeRequest.callerState,
},
"Hopper shutdown skipped because a bridge operation remained active",
);
return;
}
const shutdown = await input
.request("shutdown")
.catch(() => err(new HopperProcessError(null)));
if (
shutdown.ok &&
isHopperCleanupRequired(shutdown.value) &&
input.launch?.shutdownByCleanup === true &&
input.launch.cleanup !== undefined
) {
state.cleanupResult = await input.launch
.cleanup()
.catch(providerCleanupFailure);
if (!state.cleanupResult.cleaned) {
state.shutdownConfirmed = await fallbackDocumentShutdown(input);
state.cleanupResult = await input.launch
.cleanup()
.catch(providerCleanupFailure);
}
} else if (shutdown.ok) {
state.shutdownConfirmed = isHopperShutdownAcknowledgement(shutdown.value);
}
if (
!shutdown.ok ||
(!state.shutdownConfirmed && state.cleanupResult === undefined)
)
input.logger.warn(
{ status: shutdown.ok ? "invalid-acknowledgement" : "failed" },
"Hopper document shutdown was not confirmed",
);
};
const fallbackDocumentShutdown = async (
input: HopperCleanupInput,
): Promise<boolean> => {
const fallback = await input
.request("shutdown_document")
.catch(() => err(new HopperProcessError(null)));
const confirmed =
fallback.ok && isHopperShutdownAcknowledgement(fallback.value);
if (!confirmed)
input.logger.warn(
{ status: fallback.ok ? "invalid-acknowledgement" : "failed" },
"Hopper document shutdown fallback was not confirmed",
);
return confirmed;
};
const stopProcess = async (
input: HopperCleanupInput,
state: CleanupState,
): Promise<void> => {
const supervisor = input.processSupervisor;
if (supervisor === undefined) return;
const stopped = await supervisor.stop(
state.cleanupResult === undefined
? {}
: { cleanupResult: state.cleanupResult },
);
const diagnostic = createOwnedHopperShutdownDiagnostic(
supervisor.launch,
stopped,
state.cleanupResult,
);
try {
input.onDiagnostic?.(diagnostic);
} catch {
// Diagnostic consumers cannot change the already-observed cleanup result.
}
input.logger.info(diagnostic, "Owned Hopper launcher shutdown completed");
if (stopped.status !== "incomplete") {
state.ownedProcessStopped = stopped.status !== "not-owned";
return;
}
const processGroupId = supervisor.launch.ownership?.processGroupId;
const resource =
processGroupId === undefined
? "hopper-process"
: `process-group:${String(processGroupId)}`;
state.resources.add(resource);
state.issues.push({ resource, reason: stopped.reason });
input.logger.warn(
{ reason: stopped.reason },
"Owned launcher cleanup failed closed",
);
};
const recordUnconfirmedDocument = (
input: HopperCleanupInput,
state: CleanupState,
): void => {
if (
(input.socket === undefined && input.launch === undefined) ||
state.shutdownConfirmed ||
state.ownedProcessStopped
)
return;
state.resources.add("hopper-document");
state.issues.push({
resource: "hopper-document",
reason:
input.activeRequest === null
? "authenticated shutdown acknowledgement was not observed"
: `bridge operation ${input.activeRequest.operation} remained active after caller ${input.activeRequest.callerState}`,
});
};
const closeRuntimeRoot = async (
runtimeRoot: PrivateRuntimeRoot | undefined,
state: CleanupState,
): Promise<void> => {
try {
await runtimeRoot?.close();
} catch (cause: unknown) {
const resource = runtimeRoot?.path ?? "hopper-runtime-root";
state.resources.add(resource);
state.issues.push({
resource,
reason:
cause instanceof Error
? cause.message
: "private runtime root cleanup failed",
});
}
};
const cleanupOutcome = (
input: HopperCleanupInput,
state: CleanupState,
): Result<null, AnalysisError> => {
if (state.issues.length === 0) return ok(null);
return err(
new ProviderCleanupError("hopper", [...state.resources], {
issues: cleanupIssueDetails(state.issues),
...(input.runtimeRoot === undefined
? {}
: { runtime_root: input.runtimeRoot.path }),
...(input.activeRequest === null
? {}
: { active_request: activityDetails(input.activeRequest) }),
}),
);
};
const cleanupIssueDetails = (issues: readonly CleanupIssue[]): JsonValue =>
issues.map((issue) => ({
resource: issue.resource,
reason: issue.reason,
}));
const activityDetails = (activity: HopperRequestActivity): JsonValue => ({
request_id: activity.requestId,
operation: activity.operation,
elapsed_ms: activity.elapsedMs,
timeout_ms: activity.timeoutMs,
caller_state: activity.callerState,
queued_requests: activity.queuedRequests,
});
const report = async (
progress: ProgressReporter | undefined,
completed: number,
message: string,
): Promise<void> => {
await progress
?.report({
phase: "hopper_cleanup",
completed,
total: 1,
message,
})
.catch(() => undefined);
};
+86 -140
View File
@@ -6,6 +6,7 @@ import { join } from "node:path";
import {
HopperCancelledError,
type AnalysisError,
type HopperError,
HopperProcessError,
HopperProtocolError,
@@ -16,36 +17,35 @@ import {
} from "../domain/errors.js";
import { err, ok, type Result } from "../domain/result.js";
import type { JsonValue } from "../domain/jsonValue.js";
import type { ProgressReporter } from "../application/ProgressReporter.js";
import { silentLogger, type Logger } from "../logger.js";
import { PendingOperations } from "../process/PendingOperations.js";
import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
import { ProviderStartupDeadline } from "../process/ProviderDeadline.js";
import type { ProcessCleanupResult } from "../process/ProcessOwnership.js";
import { ProviderRunLineage } from "../process/ProviderRunLineage.js";
import {
type ProviderProcessDiagnostic,
ProviderProcessSupervisor,
} from "../process/ProviderProcess.js";
import type { BridgeLaunch, BridgeLauncher } from "./BridgeLauncher.js";
import type { HopperDiagnostic } from "./HopperDiagnostics.js";
import { cleanupHopperSession } from "./HopperCleanup.js";
import {
createOwnedHopperShutdownDiagnostic,
type HopperDiagnostic,
providerCleanupFailure,
} from "./HopperDiagnostics.js";
import {
isHopperCleanupRequired,
isHopperShutdownAcknowledgement,
parseHopperServerInfo,
type HopperServerInfo,
} from "./HopperSessionValues.js";
import { connectHopperSocketOnce } from "./HopperSocketConnection.js";
import { hopperLauncherFailureDiagnostic } from "./HopperProcessDiagnostic.js";
import { parseResponseLine, responseResult } from "./protocol.js";
import {
HopperRequestQueue,
type HopperRequestActivity,
} from "./HopperRequestQueue.js";
import { HopperResponseStream } from "./HopperResponseStream.js";
import { responseResult } from "./protocol.js";
export type { HopperServerInfo } from "./HopperSessionValues.js";
const MAX_LINE_BYTES = 10 * 1024 * 1024;
const SHUTDOWN_TIMEOUT_MS = 30_000;
const MAX_QUEUED_REQUESTS = 64;
const SESSION_ROOT = process.platform === "darwin" ? "/tmp" : tmpdir();
/** Dependencies, deadlines, and redacted diagnostics for one bridge client. */
@@ -72,10 +72,8 @@ export class HopperClient {
Pick<HopperClientOptions, "requestTimeoutMs" | "startupTimeoutMs">
> &
HopperClientOptions;
readonly #pending = new PendingOperations<
number,
Result<JsonValue, HopperError>
>();
readonly #requests: HopperRequestQueue;
readonly #responses: HopperResponseStream;
readonly #logger: Logger;
#socket: Socket | undefined;
#launch: BridgeLaunch | undefined;
@@ -84,15 +82,14 @@ export class HopperClient {
#token: string | undefined;
readonly #lineage = new ProviderRunLineage();
#nextId = 1;
#buffer = "";
#closing = false;
#launcherExitCode: number | null | undefined;
#launcherFailureDiagnostic: HopperStartupDiagnostic | undefined;
#startupController: AbortController | undefined;
#startPromise: Promise<Result<HopperServerInfo, HopperError>> | undefined;
#closePromise: Promise<void> | undefined;
#closePromise: Promise<Result<null, AnalysisError>> | undefined;
readonly #onSocketData = (chunk: string): void => {
this.#onData(chunk);
this.#responses.push(chunk);
};
readonly #onSocketError = (): void => {
this.#failAll(new HopperProcessError(null));
@@ -108,6 +105,30 @@ export class HopperClient {
requestTimeoutMs: options.requestTimeoutMs ?? 30_000,
startupTimeoutMs: options.startupTimeoutMs ?? 120_000,
};
this.#requests = new HopperRequestQueue(
MAX_QUEUED_REQUESTS,
({ id, method, params }, failed) => {
const socket = this.#socket;
const token = this.#token;
if (socket === undefined || socket.destroyed || token === undefined) {
failed();
return;
}
socket.write(
`${JSON.stringify({ id, token, method, params })}\n`,
(cause) => {
if (cause !== undefined && cause !== null) failed();
},
);
},
);
this.#responses = new HopperResponseStream({
accept: (response) =>
this.#requests.accept(response.id, responseResult(response)),
hasQueued: (id) => this.#requests.hasQueued(id),
nextRequestId: () => this.#nextId,
abort: (message, cause) => this.#abortProtocol(message, cause),
});
}
/** Latest token-verified launcher lineage for the active run. */
@@ -115,6 +136,11 @@ export class HopperClient {
return this.#lineage.snapshot();
}
/** Observe a request that still occupies Hopper after caller timeout/cancel. */
requestActivity(): HopperRequestActivity | null {
return this.#requests.activity();
}
/** Launch the bridge once and complete its authenticated health handshake. */
start(signal?: AbortSignal): Promise<Result<HopperServerInfo, HopperError>> {
if (this.#closePromise !== undefined) {
@@ -251,6 +277,7 @@ export class HopperClient {
options: {
readonly signal?: AbortSignal;
readonly timeoutMs?: number;
readonly progress?: ProgressReporter;
} = {},
): Promise<Result<JsonValue, HopperError>> {
const started = await this.start(options.signal);
@@ -260,11 +287,18 @@ export class HopperClient {
/** Stop the bridge, settle outstanding requests, and remove session artifacts. */
close(): Promise<void> {
return this.closeWithOutcome().then(() => undefined);
}
/** Stop the bridge and report whether every owned resource was verified clean. */
closeWithOutcome(
options: { readonly progress?: ProgressReporter } = {},
): Promise<Result<null, AnalysisError>> {
const starting = this.#startPromise;
const controller = this.#startupController;
controller?.abort();
this.#closePromise ??= Promise.resolve().then(() =>
this.#close(starting, controller),
this.#close(starting, controller, options),
);
return this.#closePromise;
}
@@ -272,10 +306,11 @@ export class HopperClient {
async #close(
starting: Promise<Result<HopperServerInfo, HopperError>> | undefined,
controller: AbortController | undefined,
): Promise<void> {
options: { readonly progress?: ProgressReporter },
): Promise<Result<null, AnalysisError>> {
try {
await starting?.catch(() => undefined);
await this.#cleanup();
return await this.#cleanup(options);
} finally {
if (this.#startPromise === starting) this.#startPromise = undefined;
if (this.#startupController === controller) {
@@ -285,92 +320,41 @@ export class HopperClient {
}
}
async #cleanup(): Promise<void> {
async #cleanup(
options: { readonly progress?: ProgressReporter } = {},
): Promise<Result<null, AnalysisError>> {
this.#closing = true;
try {
const socket = this.#socket;
let cleanupResult: ProcessCleanupResult | undefined;
if (socket !== undefined && !socket.destroyed) {
const shutdown = await this.#request(
"shutdown",
{},
{ timeoutMs: SHUTDOWN_TIMEOUT_MS },
).catch(() => err(new HopperProcessError(null)));
if (
shutdown.ok &&
isHopperCleanupRequired(shutdown.value) &&
this.#launch?.shutdownByCleanup === true &&
this.#launch.cleanup !== undefined
) {
cleanupResult = await this.#launch
.cleanup()
.catch(providerCleanupFailure);
if (!cleanupResult.cleaned) {
await this.#fallbackDocumentShutdown();
cleanupResult = await this.#launch
.cleanup()
.catch(providerCleanupFailure);
}
} else if (
!shutdown.ok ||
!isHopperShutdownAcknowledgement(shutdown.value)
)
this.#logger.warn(
{ status: shutdown.ok ? "invalid-acknowledgement" : "failed" },
"Hopper document shutdown was not confirmed",
);
}
this.#failAll(new HopperProcessError(null));
if (socket !== undefined) this.#detachSocket(socket);
socket?.destroy();
return await cleanupHopperSession({
socket: this.#socket,
launch: this.#launch,
processSupervisor: this.#process,
runtimeRoot: this.#runtimeRoot,
activeRequest: this.#requests.activity(),
progress: options.progress,
logger: this.#logger,
onDiagnostic: this.#options.onDiagnostic,
request: (method) =>
this.#request(method, {}, { timeoutMs: SHUTDOWN_TIMEOUT_MS }),
releaseTransport: (socket) => {
this.#failAll(new HopperProcessError(null));
if (socket !== undefined) this.#detachSocket(socket);
socket?.destroy();
},
});
} finally {
this.#socket = undefined;
if (this.#process !== undefined) {
const stopped = await this.#process.stop(
cleanupResult === undefined ? {} : { cleanupResult },
);
const diagnostic = createOwnedHopperShutdownDiagnostic(
this.#process.launch,
stopped,
cleanupResult,
);
this.#options.onDiagnostic?.(diagnostic);
this.#logger.info(
diagnostic,
"Owned Hopper launcher shutdown completed",
);
if (stopped.status === "incomplete")
this.#logger.warn(
{ reason: stopped.reason },
"Owned launcher cleanup failed closed",
);
}
this.#process = undefined;
this.#launch = undefined;
const runtimeRoot = this.#runtimeRoot;
this.#runtimeRoot = undefined;
await runtimeRoot?.close();
this.#token = undefined;
this.#buffer = "";
this.#responses.reset();
this.#launcherExitCode = undefined;
this.#launcherFailureDiagnostic = undefined;
} finally {
this.#closing = false;
}
}
async #fallbackDocumentShutdown(): Promise<void> {
const fallback = await this.#request(
"shutdown_document",
{},
{ timeoutMs: SHUTDOWN_TIMEOUT_MS },
).catch(() => err(new HopperProcessError(null)));
if (!fallback.ok || !isHopperShutdownAcknowledgement(fallback.value))
this.#logger.warn(
{ status: fallback.ok ? "invalid-acknowledgement" : "failed" },
"Hopper document shutdown fallback was not confirmed",
);
}
async #connect(
socketPath: string,
deadline: ProviderStartupDeadline,
@@ -413,7 +397,11 @@ export class HopperClient {
async #request(
method: string,
params: JsonValue,
options: { readonly signal?: AbortSignal; readonly timeoutMs?: number },
options: {
readonly signal?: AbortSignal;
readonly timeoutMs?: number;
readonly progress?: ProgressReporter;
},
): Promise<Result<JsonValue, HopperError>> {
const socket = this.#socket;
const token = this.#token;
@@ -425,20 +413,11 @@ export class HopperClient {
const id = this.#nextId++;
const startedAt = performance.now();
const timeoutMs = options.timeoutMs ?? this.#options.requestTimeoutMs;
const response = this.#pending.wait(id, {
const result = await this.#requests.run(id, method, params, {
timeoutMs,
...(options.signal === undefined ? {} : { signal: options.signal }),
timeoutValue: () => err(new HopperTimeoutError(timeoutMs)),
cancelledValue: () => err(new HopperCancelledError()),
...(options.progress !== undefined ? { progress: options.progress } : {}),
});
socket.write(
`${JSON.stringify({ id, token, method, params })}\n`,
(cause) => {
if (cause !== undefined && cause !== null)
this.#pending.settle(id, err(new HopperProcessError(null)));
},
);
const result = await response;
this.#logger[result.ok ? "debug" : "warn"](
{
method,
@@ -501,46 +480,13 @@ export class HopperClient {
);
}
#onData(chunk: string): void {
this.#buffer += chunk;
let newline = this.#buffer.indexOf("\n");
while (newline >= 0) {
const line = this.#buffer.slice(0, newline).trim();
this.#buffer = this.#buffer.slice(newline + 1);
if (Buffer.byteLength(line) > MAX_LINE_BYTES) {
this.#abortProtocol("Hopper response exceeded the maximum line size");
return;
}
if (line.length > 0) this.#onLine(line);
newline = this.#buffer.indexOf("\n");
}
if (Buffer.byteLength(this.#buffer) > MAX_LINE_BYTES) {
this.#abortProtocol("Hopper response exceeded the maximum line size");
}
}
#onLine(line: string): void {
const parsed = parseResponseLine(line);
if (!parsed.ok) {
this.#abortProtocol(parsed.error.message, parsed.error);
return;
}
if (!this.#pending.has(parsed.value.id)) {
if (parsed.value.id >= this.#nextId) {
this.#abortProtocol("Hopper returned an unknown response id");
}
return;
}
this.#pending.settle(parsed.value.id, responseResult(parsed.value));
}
#abortProtocol(message: string, cause?: Error): void {
this.#failAll(new HopperProtocolError(message, { cause }));
this.#socket?.destroy();
}
#failAll(error: HopperError): void {
this.#pending.failAll(() => err(error));
this.#requests.failAll(error);
}
}
+20
View File
@@ -255,6 +255,26 @@ export class HopperProvider implements AnalysisProviderCandidate {
? []
: [{ provider: executionProvider, observation }];
},
requestActivitySnapshots: () => {
const activity = client.requestActivity();
return [
{
provider: executionProvider,
active:
activity === null
? null
: {
requestId: activity.requestId,
operation: activity.operation,
elapsedMs: activity.elapsedMs,
timeoutMs: activity.timeoutMs,
callerState: activity.callerState,
},
queuedRequests: activity?.queuedRequests ?? 0,
},
];
},
closeWithOutcome: (options) => client.closeWithOutcome(options),
close: () => client.close(),
};
}
+279
View File
@@ -0,0 +1,279 @@
import { performance } from "node:perf_hooks";
import type { ProgressReporter } from "../application/ProgressReporter.js";
import {
HopperCancelledError,
type HopperError,
HopperProcessError,
HopperProtocolError,
HopperTimeoutError,
} from "../domain/errors.js";
import type { JsonValue } from "../domain/jsonValue.js";
import { err, type Result } from "../domain/result.js";
export type HopperRequestResult = Result<JsonValue, HopperError>;
export interface HopperRequestQueueOptions {
readonly signal?: AbortSignal;
readonly timeoutMs: number;
readonly progress?: ProgressReporter;
}
export interface HopperRequestActivity {
readonly requestId: number;
readonly operation: string;
readonly elapsedMs: number;
readonly timeoutMs: number;
readonly callerState: "waiting" | "timed_out" | "cancelled";
readonly queuedRequests: number;
}
type RequestSender = (
request: {
readonly id: number;
readonly method: string;
readonly params: JsonValue;
},
failed: () => void,
) => void;
interface QueuedRequest {
readonly id: number;
readonly method: string;
readonly params: JsonValue;
readonly signal: AbortSignal | undefined;
readonly timeoutMs: number;
readonly deadline: number;
readonly progress: ProgressReporter | undefined;
readonly resolve: (result: HopperRequestResult) => void;
readonly timer: NodeJS.Timeout;
readonly onAbort: (() => void) | undefined;
callerState: "waiting" | "timed_out" | "cancelled";
callerSettled: boolean;
startedAt: number | undefined;
heartbeat: NodeJS.Timeout | undefined;
}
/** Bounded FIFO that keeps the wire serialized until Hopper actually replies. */
export class HopperRequestQueue {
readonly #queue: QueuedRequest[] = [];
#active: QueuedRequest | undefined;
constructor(
private readonly maximum: number,
private readonly send: RequestSender,
) {}
/** Queue one request, counting queue wait against its caller deadline. */
run(
id: number,
method: string,
params: JsonValue,
options: HopperRequestQueueOptions,
): Promise<HopperRequestResult> {
if (options.signal?.aborted === true)
return Promise.resolve(err(new HopperCancelledError()));
if (options.timeoutMs <= 0)
return Promise.resolve(err(new HopperTimeoutError(options.timeoutMs)));
if (this.#size() >= this.maximum)
return Promise.resolve(
err(
new HopperProtocolError(
`Hopper serial request queue reached its ${String(this.maximum)}-request limit`,
),
),
);
return new Promise((resolve) => {
let entry: QueuedRequest;
const onAbort =
options.signal === undefined ? undefined : () => this.#cancel(entry);
const timer = setTimeout(() => this.#timeout(entry), options.timeoutMs);
entry = {
id,
method,
params,
signal: options.signal,
timeoutMs: options.timeoutMs,
deadline: performance.now() + options.timeoutMs,
progress: options.progress,
resolve,
timer,
onAbort,
callerState: "waiting",
callerSettled: false,
startedAt: undefined,
heartbeat: undefined,
};
this.#queue.push(entry);
if (onAbort !== undefined)
options.signal?.addEventListener("abort", onAbort, { once: true });
if (this.#active !== undefined || this.#queue.length > 1)
this.#report(
entry,
`${method} queued behind ${String(this.#size() - 1)} Hopper request(s)`,
);
this.#drain();
});
}
/** Settle the active wire request. False identifies an unexpected response id. */
accept(id: number, result: HopperRequestResult): boolean {
const entry = this.#active;
if (entry === undefined || entry.id !== id) return false;
this.#releaseWire(entry);
if (!entry.callerSettled) this.#settleCaller(entry, result, "waiting");
this.#active = undefined;
this.#drain();
return true;
}
/** Fail the active wire request and every request still waiting in the FIFO. */
failAll(error: HopperError): void {
const active = this.#active;
this.#active = undefined;
if (active !== undefined) {
this.#releaseWire(active);
if (!active.callerSettled)
this.#settleCaller(active, err(error), "waiting");
}
for (const entry of this.#queue.splice(0))
this.#settleCaller(entry, err(error), "waiting");
}
/** Snapshot the one request still occupying Hopper's serial Python thread. */
activity(): HopperRequestActivity | null {
const active = this.#active;
if (active === undefined || active.startedAt === undefined) return null;
return {
requestId: active.id,
operation: active.method,
elapsedMs: Math.max(0, Math.round(performance.now() - active.startedAt)),
timeoutMs: active.timeoutMs,
callerState: active.callerState,
queuedRequests: this.#queue.length,
};
}
/** Return whether an id belongs to a request that has not crossed the wire. */
hasQueued(id: number): boolean {
return this.#queue.some((entry) => entry.id === id);
}
/** Number of caller requests waiting behind the active bridge operation. */
queuedCount(): number {
return this.#queue.length;
}
#size(): number {
return this.#queue.length + (this.#active === undefined ? 0 : 1);
}
#drain(): void {
if (this.#active !== undefined) return;
const entry = this.#queue.shift();
if (entry === undefined) return;
const remaining = entry.deadline - performance.now();
if (entry.signal?.aborted === true || remaining <= 0) {
this.#settleCaller(
entry,
entry.signal?.aborted === true
? err(new HopperCancelledError())
: err(new HopperTimeoutError(entry.timeoutMs)),
entry.signal?.aborted === true ? "cancelled" : "timed_out",
);
this.#drain();
return;
}
this.#active = entry;
entry.startedAt = performance.now();
this.#report(entry, `${entry.method} started on Hopper's serial bridge`);
entry.heartbeat = setInterval(() => {
if (entry.callerSettled || entry.startedAt === undefined) return;
const elapsed = Math.round(performance.now() - entry.startedAt);
this.#report(
entry,
`${entry.method} is still running in Hopper (${String(elapsed)} ms elapsed)`,
);
}, 1_000);
try {
this.send(
{ id: entry.id, method: entry.method, params: entry.params },
() => this.accept(entry.id, err(new HopperProcessError(null))),
);
} catch (cause: unknown) {
this.accept(
entry.id,
err(new HopperProtocolError("Hopper socket write failed", { cause })),
);
}
}
#timeout(entry: QueuedRequest): void {
if (entry.callerSettled) return;
if (this.#active === entry) {
this.#settleCaller(
entry,
err(new HopperTimeoutError(entry.timeoutMs)),
"timed_out",
);
return;
}
const index = this.#queue.indexOf(entry);
if (index < 0) return;
this.#queue.splice(index, 1);
this.#settleCaller(
entry,
err(new HopperTimeoutError(entry.timeoutMs)),
"timed_out",
);
}
#cancel(entry: QueuedRequest): void {
if (entry.callerSettled) return;
if (this.#active === entry) {
this.#settleCaller(entry, err(new HopperCancelledError()), "cancelled");
return;
}
const index = this.#queue.indexOf(entry);
if (index < 0) return;
this.#queue.splice(index, 1);
this.#settleCaller(entry, err(new HopperCancelledError()), "cancelled");
}
#settleCaller(
entry: QueuedRequest,
result: HopperRequestResult,
callerState: QueuedRequest["callerState"],
): void {
if (entry.callerSettled) return;
entry.callerSettled = true;
entry.callerState = callerState;
if (this.#active === entry && entry.heartbeat !== undefined) {
clearInterval(entry.heartbeat);
entry.heartbeat = undefined;
}
clearTimeout(entry.timer);
if (entry.signal !== undefined && entry.onAbort !== undefined)
entry.signal.removeEventListener("abort", entry.onAbort);
entry.resolve(result);
}
#releaseWire(entry: QueuedRequest): void {
if (entry.heartbeat !== undefined) clearInterval(entry.heartbeat);
entry.heartbeat = undefined;
clearTimeout(entry.timer);
if (entry.signal !== undefined && entry.onAbort !== undefined)
entry.signal.removeEventListener("abort", entry.onAbort);
}
#report(entry: QueuedRequest, message: string): void {
void entry.progress
?.report({
phase: "hopper_request",
completed: 0,
total: 1,
message,
})
.catch(() => undefined);
}
}
+56
View File
@@ -0,0 +1,56 @@
import { parseResponseLine, type HopperResponse } from "./protocol.js";
const MAX_LINE_BYTES = 10 * 1024 * 1024;
export interface HopperResponseStreamOptions {
readonly accept: (response: HopperResponse) => boolean;
readonly hasQueued: (id: number) => boolean;
readonly nextRequestId: () => number;
readonly abort: (message: string, cause?: Error) => void;
}
/** Incrementally validates bounded Hopper NDJSON response frames. */
export class HopperResponseStream {
#buffer = "";
constructor(readonly options: HopperResponseStreamOptions) {}
push(chunk: string): void {
this.#buffer += chunk;
let newline = this.#buffer.indexOf("\n");
while (newline >= 0) {
const line = this.#buffer.slice(0, newline).trim();
this.#buffer = this.#buffer.slice(newline + 1);
if (Buffer.byteLength(line) > MAX_LINE_BYTES) {
this.options.abort("Hopper response exceeded the maximum line size");
return;
}
if (line.length > 0 && !this.#acceptLine(line)) return;
newline = this.#buffer.indexOf("\n");
}
if (Buffer.byteLength(this.#buffer) > MAX_LINE_BYTES) {
this.options.abort("Hopper response exceeded the maximum line size");
}
}
reset(): void {
this.#buffer = "";
}
#acceptLine(line: string): boolean {
const parsed = parseResponseLine(line);
if (!parsed.ok) {
this.options.abort(parsed.error.message, parsed.error);
return false;
}
if (this.options.accept(parsed.value)) return true;
if (
parsed.value.id >= this.options.nextRequestId() ||
this.options.hasQueued(parsed.value.id)
) {
this.options.abort("Hopper returned an unknown response id");
return false;
}
return true;
}
}
+1
View File
@@ -75,6 +75,7 @@ export class NativeMacOSProvider implements AnalysisProvider {
? ({ available: true, reason: null } as const)
: ({
available: false,
availabilityCode: "unsupported_host",
reason: "Native macOS utilities require macOS.",
} as const);
return Object.freeze({
+22
View File
@@ -0,0 +1,22 @@
import type { ProgressReporter } from "../application/ProgressReporter.js";
/** Report one lifecycle operation start through the shared monotonic boundary. */
export const reportLifecycleStart = (
progress: ProgressReporter,
phase: string,
message = "started",
): Promise<void> => progress.report({ phase, completed: 0, total: 1, message });
/** Report one lifecycle operation's terminal disposition. */
export const reportLifecycleEnd = (
progress: ProgressReporter,
phase: string,
succeeded: boolean,
): Promise<void> =>
progress.report({
phase,
completed: 1,
total: 1,
message: succeeded ? "completed" : "failed",
terminal: true,
});
+22
View File
@@ -0,0 +1,22 @@
import type { PermissionAuthority } from "../application/PermissionAuthority.js";
import {
AnalysisProtocolError,
PermissionRequiredError,
type AnalysisError,
} from "../domain/errors.js";
import { err, type Result } from "../domain/result.js";
/** Preserve a typed permission denial or contain an unexpected authority error. */
export const permissionFailure = (
failure: Awaited<ReturnType<PermissionAuthority["authorize"]>>,
): Result<never, AnalysisError> => {
if (failure.ok)
return err(new AnalysisProtocolError("Expected a denied permission"));
return err(
failure.error instanceof PermissionRequiredError
? failure.error
: new AnalysisProtocolError(failure.error.message, {
cause: failure.error,
}),
);
};
+93
View File
@@ -0,0 +1,93 @@
import { writeAnalysisSnapshot } from "../application/AnalysisSnapshotFiles.js";
import { closeBinaryInputSchema } from "../contracts/sessionLifecycleInputs.js";
import { ok } from "../domain/result.js";
import {
reportLifecycleEnd,
reportLifecycleStart,
} from "./lifecycleProgress.js";
import { logToolExecution } from "./toolLogging.js";
import { toCallToolResult } from "./toolResult.js";
import { mcpProgressReporter } from "./mcpProgress.js";
import { permissionFailure } from "./permissionFailure.js";
import type { LifecycleToolRegistration } from "./registerSessionTools.js";
import { safeParseToolInput } from "./toolInputValidation.js";
import { toolRegistrationOptions } from "./toolRegistrationOptions.js";
/** Register provider cleanup and optional pre-close snapshot persistence. */
export const registerCloseLifecycleTool = ({
server,
session,
logger,
contracts: [, closeContract],
snapshotFilePolicy,
permissionAuthority,
}: LifecycleToolRegistration): void => {
server.registerTool(
closeContract.name,
toolRegistrationOptions(closeContract),
async (input, context) => {
const parsedInput = safeParseToolInput(
closeBinaryInputSchema,
input,
closeContract.name,
);
if (!parsedInput.ok) return toCallToolResult(parsedInput, closeContract);
const parsed = parsedInput.value;
const progress = mcpProgressReporter(context);
if (parsed.snapshot_path === undefined) {
await reportLifecycleStart(progress, closeContract.name);
const closed = await logToolExecution(logger, closeContract.name, () =>
session.close({ progress }),
);
await reportLifecycleEnd(progress, closeContract.name, closed.ok);
server.sendToolListChanged();
return toCallToolResult(closed, closeContract);
}
if (permissionAuthority !== undefined) {
const authorized = await permissionAuthority.authorize(
{
capability: "snapshot_write",
roots: [parsed.snapshot_path],
executables: [],
environment_names: [],
network: "none",
mount: false,
operation_identity: `close_binary:snapshot:${parsed.snapshot_path}`,
},
"write",
);
if (!authorized.ok)
return toCallToolResult(permissionFailure(authorized), closeContract);
}
const snapshot = session.exportAnalysisSnapshot();
if (!snapshot.ok) return toCallToolResult(snapshot, closeContract);
const written = await writeAnalysisSnapshot(
snapshot.value,
parsed.snapshot_path,
parsed.overwrite,
snapshotFilePolicy,
);
if (!written.ok) return toCallToolResult(written, closeContract);
await reportLifecycleStart(
progress,
closeContract.name,
"snapshot written; closing provider",
);
const closed = await logToolExecution(logger, closeContract.name, () =>
session.close({ progress }),
);
await reportLifecycleEnd(progress, closeContract.name, closed.ok);
server.sendToolListChanged();
return closed.ok
? toCallToolResult(
ok({
path: written.value.path,
bytes: written.value.bytes,
entries: snapshot.value.entries.length,
}),
closeContract,
)
: toCallToolResult(closed, closeContract);
},
);
};
+9 -3
View File
@@ -49,9 +49,8 @@ export const registerEnhancedTools = (
analysis: AnalysisOperationPort,
options: EnhancedToolRegistration,
): void => {
const services = new EnhancedTools(analysis);
for (const contract of ENHANCED_TOOL_CONTRACTS) {
registerEnhancedTool(server, services, contract, {
registerEnhancedTool(server, analysis, contract, {
logger: options.logger,
activeTarget: options.activeTarget,
analysisProfile: options.analysisProfile,
@@ -63,7 +62,7 @@ export const registerEnhancedTools = (
const registerEnhancedTool = (
server: McpServer,
services: EnhancedTools,
analysis: AnalysisOperationPort,
contract: ToolContract,
registration: {
readonly logger: Logger;
@@ -91,6 +90,13 @@ const registerEnhancedTool = (
if (!validatedCall.ok) return toCallToolResult(validatedCall, contract);
const parsedInput = validatedCall.value.input;
const parameters = jsonParameters(parsedInput);
const services = new EnhancedTools({
execute: (operation, operationParameters, executionOptions) =>
analysis.execute(operation, operationParameters, {
...executionOptions,
progress,
}),
});
const result = await logToolExecution(registration.logger, name, () =>
services.executeValidated(validatedCall.value, context.mcpReq.signal),
);
+1
View File
@@ -148,6 +148,7 @@ const projectSessionStatus = (options: {
active_provider: activeProvider,
target: open ? targetFrom(options.status) : null,
alignment: options.serverIdentity.alignment,
analysis_activity: options.status.analysis_activity,
recommended_actions: recommendedActions(
open,
options.serverIdentity.alignment.state,
+7 -99
View File
@@ -6,7 +6,7 @@ import { SESSION_TOOL_CONTRACTS } from "../contracts/toolContracts.js";
import { toCallToolResult } from "./toolResult.js";
import type { Logger } from "../logger.js";
import { logToolExecution } from "./toolLogging.js";
import { err, ok, type Result } from "../domain/result.js";
import { ok, type Result } from "../domain/result.js";
import type {
ProcessCapture,
ProcessExecutionPolicy,
@@ -18,16 +18,8 @@ import { processCapturePermissionRequest } from "../application/ProcessCapturePe
import type { Evidence } from "../domain/evidence.js";
import type { AnalysisSnapshot } from "../domain/analysisSnapshot.js";
import type { EvidenceFilePolicy } from "../domain/evidenceBundle.js";
import {
readAnalysisSnapshot,
writeAnalysisSnapshot,
} from "../application/AnalysisSnapshotFiles.js";
import {
AnalysisProtocolError,
PermissionRequiredError,
UnknownRegistryError,
type AnalysisError,
} from "../domain/errors.js";
import { readAnalysisSnapshot } from "../application/AnalysisSnapshotFiles.js";
import { UnknownRegistryError, type AnalysisError } from "../domain/errors.js";
import {
DENY_EVIDENCE_FILE_POLICY,
DENY_PROCESS_POLICY,
@@ -40,10 +32,7 @@ import { toolRegistrationOptions } from "./toolRegistrationOptions.js";
import { safeParseToolInput } from "./toolInputValidation.js";
import { registerBundleComparisonTool } from "./registerBundleComparisonTool.js";
import { registerInvestigationTools } from "./registerInvestigationTools.js";
import {
closeBinaryInputSchema,
openBinaryInputSchema,
} from "../contracts/sessionLifecycleInputs.js";
import { openBinaryInputSchema } from "../contracts/sessionLifecycleInputs.js";
import { registerSessionStatusTool } from "./registerSessionStatusTool.js";
import type { PermissionAuthority } from "../application/PermissionAuthority.js";
import { mcpProgressReporter } from "./mcpProgress.js";
@@ -61,20 +50,8 @@ import {
sessionAvailabilityPolicy,
type SessionAvailability,
} from "./sessionAvailabilityPolicy.js";
const permissionFailure = (
failure: Awaited<ReturnType<PermissionAuthority["authorize"]>>,
): Result<never, AnalysisError> => {
if (failure.ok)
return err(new AnalysisProtocolError("Expected a denied permission"));
return err(
failure.error instanceof PermissionRequiredError
? failure.error
: new AnalysisProtocolError(failure.error.message, {
cause: failure.error,
}),
);
};
import { permissionFailure } from "./permissionFailure.js";
import { registerCloseLifecycleTool } from "./registerCloseLifecycleTool.js";
const recordProcessResidualUnknowns = (
session: BinarySessionPort,
@@ -204,7 +181,7 @@ const registerProcessTools = ({
);
};
interface LifecycleToolRegistration {
export interface LifecycleToolRegistration {
readonly server: McpServer;
readonly session: BinarySessionPort;
readonly logger: Logger;
@@ -314,75 +291,6 @@ const registerOpenLifecycleTool = ({
);
};
const registerCloseLifecycleTool = ({
server,
session,
logger,
contracts: [, closeContract],
snapshotFilePolicy,
permissionAuthority,
}: LifecycleToolRegistration): void => {
server.registerTool(
closeContract.name,
toolRegistrationOptions(closeContract),
async (input) => {
const parsedInput = safeParseToolInput(
closeBinaryInputSchema,
input,
closeContract.name,
);
if (!parsedInput.ok) return toCallToolResult(parsedInput, closeContract);
const parsed = parsedInput.value;
if (parsed.snapshot_path === undefined) {
const closed = await logToolExecution(logger, closeContract.name, () =>
session.close(),
);
if (closed.ok) server.sendToolListChanged();
return toCallToolResult(closed, closeContract);
}
if (permissionAuthority !== undefined) {
const authorized = await permissionAuthority.authorize(
{
capability: "snapshot_write",
roots: [parsed.snapshot_path],
executables: [],
environment_names: [],
network: "none",
mount: false,
operation_identity: `close_binary:snapshot:${parsed.snapshot_path}`,
},
"write",
);
if (!authorized.ok)
return toCallToolResult(permissionFailure(authorized), closeContract);
}
const snapshot = session.exportAnalysisSnapshot();
if (!snapshot.ok) return toCallToolResult(snapshot, closeContract);
const written = await writeAnalysisSnapshot(
snapshot.value,
parsed.snapshot_path,
parsed.overwrite,
snapshotFilePolicy,
);
if (!written.ok) return toCallToolResult(written, closeContract);
const closed = await logToolExecution(logger, closeContract.name, () =>
session.close(),
);
if (closed.ok) server.sendToolListChanged();
return closed.ok
? toCallToolResult(
ok({
path: written.value.path,
bytes: written.value.bytes,
entries: snapshot.value.entries.length,
}),
closeContract,
)
: toCallToolResult(closed, closeContract);
},
);
};
/** Register MCP-only target lifecycle operations on a long-lived session. */
export interface SessionToolOptions {
readonly processPolicy?: ProcessExecutionPolicy;
+65
View File
@@ -179,6 +179,71 @@ describe("artifact graph provider", () => {
});
});
it.each([
["fixture.msix", "msix"],
["fixture.appxbundle", "appx"],
] as const)(
"reuses the hardened ZIP reader for %s package inventory",
async (name, format) => {
const root = await createTestTempDirectory("rea-windows-package-");
const path = join(root, name);
const writer = new ZipWriter(new Uint8ArrayWriter());
await writer.add(
"Assets/main.js",
new TextReader("export default 'windows';"),
);
await writer.add(
"VFS/ProgramFilesX64/App/addon.node",
new TextReader("native"),
);
await writeFile(path, await writer.close());
const parsed = await parseBinaryTarget(path);
expect(parsed.ok && parsed.value).toMatchObject({
kind: "archive",
format,
});
if (!parsed.ok) return;
const result = await inventory(parsed.value);
expect(result.manifest.root_format).toBe(format);
expect(result.nodes.items.map(({ kind }) => kind)).toEqual(
expect.arrayContaining(["javascript", "native-addon"]),
);
const selected = result.occurrences.items.find(
({ logical_path: logicalPath }) => logicalPath === "Assets/main.js",
);
expect(selected).toBeDefined();
if (selected === undefined) return;
const output = join(root, "output");
const extracted = await new ArtifactProvider()
.createClient(parsed.value)
.execute(
"extract_artifact",
artifactExtractionInputSchema.parse({
approved: true,
output_root: output,
occurrence_ids: [selected.occurrence_id],
}),
);
expect(extracted.ok).toBe(true);
expect(await readFile(join(output, "Assets", "main.js"), "utf8")).toBe(
"export default 'windows';",
);
expect(
parseEvidence(
await runProviderAnalysis(path, "inventory_artifact", {
node_limit: 10,
occurrence_limit: 10,
edge_limit: 10,
}),
),
).toMatchObject({
operation: "inventory_artifact",
subject: { format },
});
},
);
it("rejects unsafe, colliding, and over-ratio entries", async () => {
expect(() => normalizeArtifactPath("../escape", limits())).toThrow(
/normalized/u,
+74
View File
@@ -14,6 +14,7 @@ import {
HopperStartError,
ProviderAdapterError,
} from "../src/domain/errors.js";
import { ProviderCleanupError } from "../src/domain/providerCleanupError.js";
import { err, ok as resultOk } from "../src/domain/result.js";
import { createEvidenceBundle } from "../src/domain/evidenceBundle.js";
import { createAnalysisProfile } from "../src/domain/analysisProfile.js";
@@ -123,6 +124,19 @@ describe("binary session", () => {
},
},
],
requestActivitySnapshots: () => [
{
provider: provider.identity(),
active: {
requestId: 7,
operation: "analyze_function",
elapsedMs: 31_000,
timeoutMs: 30_000,
callerState: "timed_out",
},
queuedRequests: 2,
},
],
close: () => Promise.resolve(),
};
},
@@ -184,6 +198,22 @@ describe("binary session", () => {
],
},
},
analysis_activity: {
status: "timed_out_busy",
providers: [
{
provider: { id: "fixture", name: "Fixture", version: "1" },
active: {
request_id: 7,
operation: "analyze_function",
elapsed_ms: 31_000,
timeout_ms: 30_000,
caller_state: "timed_out",
},
queued_requests: 2,
},
],
},
});
expect(operations).toEqual(["health", "address_name"]);
await session.close();
@@ -724,6 +754,50 @@ describe("binary session", () => {
expect(overlapped).toBe(false);
await session.close();
});
it("clears session state while preserving a typed provider cleanup failure", async () => {
const [first] = await targets();
const cleanupError = new ProviderCleanupError(
"fixture",
["fixture-document"],
{ reason: "shutdown acknowledgement missing" },
);
const session = new BinarySession(() => ({
execute: () => Promise.resolve(ok(null)),
closeWithOutcome: () => Promise.resolve(err(cleanupError)),
close: () => Promise.resolve(),
}));
expect((await session.open(first)).ok).toBe(true);
expect(await session.close()).toEqual(err(cleanupError));
expect(session.status()).toMatchObject({
open: false,
analysis_activity: { status: "not_observed", providers: [] },
});
});
it("does not start a replacement after the active provider cleanup is unconfirmed", async () => {
const [first, second] = await targets();
const cleanupError = new ProviderCleanupError(
"fixture",
["fixture-document"],
{ reason: "shutdown acknowledgement missing" },
);
let created = 0;
const session = new BinarySession(() => {
created += 1;
return {
execute: () => Promise.resolve(ok(null)),
closeWithOutcome: () => Promise.resolve(err(cleanupError)),
close: () => Promise.resolve(),
};
});
expect((await session.open(first)).ok).toBe(true);
expect(await session.open(second)).toEqual(err(cleanupError));
expect(created).toBe(1);
expect(session.status()).toMatchObject({ open: false });
});
});
const client = (fail = false): AnalysisClient => ({
+19 -2
View File
@@ -95,18 +95,35 @@ describe("executable header parsing", () => {
});
describe("binary target I/O", () => {
it("classifies ZIP profiles and text artifacts without Hopper", async () => {
it("classifies ZIP package families and text artifacts without Hopper", async () => {
const directory = await createTestTempDirectory("rea-artifact-target-");
const zip = join(directory, "fixture.apk");
const msix = join(directory, "fixture.msixbundle");
const appx = join(directory, "fixture.appx");
const script = join(directory, "bundle.js");
await writeFile(zip, Buffer.from([0x50, 0x4b, 0x05, 0x06, 0, 0, 0, 0]));
const emptyZip = Buffer.from([0x50, 0x4b, 0x05, 0x06, 0, 0, 0, 0]);
await Promise.all([
writeFile(zip, emptyZip),
writeFile(msix, emptyZip),
writeFile(appx, emptyZip),
]);
await writeFile(script, "export default 1;\n");
const archive = await parseBinaryTarget(zip);
const msixArchive = await parseBinaryTarget(msix);
const appxArchive = await parseBinaryTarget(appx);
const javascript = await parseBinaryTarget(script);
expect(archive.ok && archive.value).toMatchObject({
kind: "archive",
format: "apk",
});
expect(msixArchive.ok && msixArchive.value).toMatchObject({
kind: "archive",
format: "msix",
});
expect(appxArchive.ok && appxArchive.value).toMatchObject({
kind: "archive",
format: "appx",
});
expect(javascript.ok && javascript.value).toMatchObject({
kind: "artifact",
format: "javascript",
+16
View File
@@ -55,6 +55,22 @@ describe("Hopper bridge truthfulness", () => {
expect(bridgeSource).toContain('params.get("collection_offset", {})');
});
it("provides a raw-instruction fast path without decompilation or global scans", () => {
const start = bridgeSource.indexOf("def _read_function_instructions");
const end = bridgeSource.indexOf("\ndef _analyze_function", start);
const implementation = bridgeSource.slice(start, end);
expect(implementation).toContain("_instruction_addresses(");
expect(implementation).toContain("offset + limit + 1");
expect(implementation).not.toContain("decompile(");
expect(implementation).not.toContain("_search_inventory(");
expect(implementation).not.toContain("_strings(");
expect(implementation).not.toContain("_name_map(");
expect(bridgeSource).toContain(
"pseudo = _pseudocode(document, procedure) or",
);
expect(bridgeSource).toContain("return _pseudocode(document, procedure)");
});
it("projects opaque Hopper locals into an exact provenance-bearing shape", () => {
expect(bridgeSource).toContain("def _procedure_locals(procedure):");
expect(bridgeSource).toContain('"description": str(local)');
+166
View File
@@ -0,0 +1,166 @@
import { describe, expect, it } from "vitest";
import { buildCapabilityInventory } from "../src/application/CapabilityInventory.js";
const enabledPolicy: Parameters<typeof buildCapabilityInventory>[1] = {
processCaptureEnabled: true,
evidenceFileRoots: 1,
investigationInputRoots: 1,
browserObservationEnabled: true,
electronObservationEnabled: true,
javascriptReplayEnabled: true,
managedRuntimeEnabled: true,
};
const status = (
options: {
readonly open?: boolean;
readonly kind?: "executable" | "database" | "archive" | "artifact";
readonly capabilities?: readonly {
readonly operation: string;
readonly available: boolean;
readonly reason: string | null;
readonly availability_code?: string;
}[];
} = {},
) => ({
open: options.open ?? false,
...(options.kind === undefined ? {} : { kind: options.kind }),
capabilities: [...(options.capabilities ?? [])],
});
const entry = (
name: string,
sessionStatus: ReturnType<typeof status>,
policy = enabledPolicy,
) => {
const found = buildCapabilityInventory(sessionStatus, policy).find(
(candidate) => candidate.name === name,
);
if (found === undefined) throw new Error(`missing capability ${name}`);
return found;
};
describe("capability inventory", () => {
it.each([
{
label: "requires a target",
name: "current_address",
sessionStatus: status(),
reason: "target_required",
remediation: "open_binary",
},
{
label: "rejects an incompatible target family",
name: "current_address",
sessionStatus: status({ open: true, kind: "archive" }),
reason: "target_unsupported",
remediation: "native executable",
},
{
label: "reports a missing provider operation",
name: "current_address",
sessionStatus: status({ open: true, kind: "executable" }),
reason: "provider_missing",
remediation: "provider",
},
{
label: "distinguishes an unsupported host",
name: "current_address",
sessionStatus: status({
open: true,
kind: "executable",
capabilities: [
{
operation: "current_address",
available: false,
availability_code: "unsupported_host",
reason: "Operation requires macOS",
},
],
}),
reason: "unsupported_host",
remediation: "macOS",
},
{
label: "retains a provider failure",
name: "current_address",
sessionStatus: status({
open: true,
kind: "executable",
capabilities: [
{
operation: "current_address",
available: false,
reason: "Provider session is unhealthy",
},
],
}),
reason: "provider_unavailable",
remediation: "unhealthy",
},
{
label: "reports an available provider operation",
name: "current_address",
sessionStatus: status({
open: true,
kind: "executable",
capabilities: [
{
operation: "current_address",
available: true,
reason: null,
},
],
}),
reason: "available",
remediation: null,
},
])("$label", ({ name, sessionStatus, reason, remediation }) => {
const availability = entry(name, sessionStatus);
expect(availability).toMatchObject({
reason,
available: reason === "available",
});
if (remediation === null) expect(availability.remediation).toBeNull();
else expect(availability.remediation).toContain(remediation);
});
it.each([
["capture_process_scenario", { processCaptureEnabled: false }],
["inspect_web_page", { browserObservationEnabled: false }],
["inspect_electron_page", { electronObservationEnabled: false }],
["run_controlled_replay", { javascriptReplayEnabled: false }],
["plan_managed_runtime_correlation", { managedRuntimeEnabled: false }],
] as const)("reports stable policy denial for %s", (name, override) => {
const availability = entry(name, status(), {
...enabledPolicy,
...override,
});
expect(availability).toMatchObject({
available: false,
reason: "policy_disabled",
});
expect(availability.remediation).toEqual(expect.any(String));
expect(availability.remediation?.length).toBeGreaterThan(0);
});
it("gives every unavailable public tool an actionable remediation", () => {
const inventory = buildCapabilityInventory(status(), {
...enabledPolicy,
processCaptureEnabled: false,
evidenceFileRoots: 0,
investigationInputRoots: 0,
browserObservationEnabled: false,
electronObservationEnabled: false,
javascriptReplayEnabled: false,
managedRuntimeEnabled: false,
});
for (const availability of inventory)
if (!availability.available) {
expect(availability.reason).not.toBe("available");
expect(availability.remediation).toEqual(expect.any(String));
expect(availability.remediation?.length).toBeGreaterThan(0);
}
});
});
+3 -1
View File
@@ -1361,7 +1361,9 @@ describe("CdpBrowserProvider", () => {
}),
{ signal: controller.signal },
);
await waitForCommand(browser, "Target.attachToTarget");
// Wait until the attach response has been consumed and capture has begun.
// Seeing the attach request alone races its response under worker load.
await waitForCommand(browser, "Page.getFrameTree");
controller.abort();
const result = await pending;
expect(result).toMatchObject({
+1 -1
View File
@@ -18,7 +18,7 @@ the read-only Ghidra inventory contracts without committing a binary.
`scripts/create-ghidra-windows-fixture.mjs` separately emits a deterministic,
ignored native x86-64 PE application for the controlled Windows Ghidra P0 lane.
The generator and fixed SHA-256 are versioned; the `.exe` is never committed or
executed. `npm run verify:ghidra:windows` uses it to prove all 18 operations,
executed. `npm run verify:ghidra:windows` uses it to prove all 19 operations,
digest linkage, transport, and cleanup on the self-hosted real-Ghidra runner.
The C fixture fixes a known call chain and strings. The version pair fixes an

Some files were not shown because too many files have changed in this diff Show More