Merge origin/main into feat/artifact-integrity-policy

Keep main's skill metadata version (27) and regenerate the package, tool
catalog and product metadata for the merged tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kaoru0822
2026-10-07 14:18:28 +00:00
co-authored by Claude Opus 5.5
124 changed files with 6709 additions and 167 deletions
@@ -30,7 +30,7 @@ analyzer, or provider-version differences while appearing target-compatible.
The decision addressed the earlier representations of
[`CompositeProvider`](https://github.com/morluto/rea/blob/main/src/application/binary/CompositeProvider.ts),
[runtime composition](https://github.com/morluto/rea/blob/main/src/application/runtime.ts),
[runtime composition](https://github.com/morluto/rea/blob/main/src/composition/binary.ts),
[`BinarySession`](https://github.com/morluto/rea/blob/main/src/application/binary/BinarySession.ts),
[`BinaryTarget`](https://github.com/morluto/rea/blob/main/src/domain/binaryTarget.ts),
[analysis snapshot](https://github.com/morluto/rea/blob/main/src/domain/analysisSnapshot.ts), and
+14 -5
View File
@@ -5,6 +5,10 @@ graph TD
DISPATCH["scripts/rea.mjs<br/>Executable dispatcher<br/>routes mcp / one-shot"]
end
subgraph "Production Composition"
COMPOSITION["src/composition/<br/>fresh typed provider and session factories<br/>one-shot binding and independent optional loading"]
end
subgraph "Adapters"
SERVER["src/server/<br/>MCP request translation<br/>createServer, tool registration, toolResult"]
HOPPER_BRIDGE["bridge/hopper_bridge.py<br/>Runs inside Hopper<br/>adapts operations to Hopper Python API"]
@@ -18,7 +22,7 @@ graph TD
REGISTRY["AnalysisProviderRegistry<br/>sorted deep candidates<br/>deterministic selection, no fallback"]
RECORDS["src/application/investigation/<br/>composed InvestigationRecords + narrow ports<br/>EvidenceLedger and Unknown ownership"]
SNAPSHOT["BinarySessionRecords + AnalysisSnapshotCache<br/>binary target/profile snapshots<br/>compatibility facade and post-commit notifications"]
JS_RECON["src/application/ + src/artifacts/<br/>JavaScript artifact reconstruction<br/>directory/ASAR reads<br/>AST-only bundle projection"]
JS_RECON["src/application/javascript/ + src/artifacts/javascript/<br/>JavaScript artifact reconstruction<br/>directory/ASAR reads<br/>AST-only bundle projection"]
end
subgraph "Provider Layers"
@@ -27,7 +31,7 @@ graph TD
IDA["src/ida/<br/>Upstream MCP contract adaptation<br/>live GUI binding or owned headless database"]
IDA_MCP["External ida-pro-mcp<br/>GUI proxy or database supervisor<br/>upstream installation and engine lifecycle"]
AUX["src/artifacts/ + src/native/<br/>disjoint artifact and native capabilities"]
BROWSER["src/browser/<br/>CDP / Playwright capture<br/>captured scripts and native module resolution"]
BROWSER["src/browser/<br/>CDP / Playwright capture<br/>captured scripts and native module resolution<br/>execution/: coverage, listener and source attribution"]
JS_CODEC["src/javascript/sourceMaps/<br/>pinned upstream map codec<br/>owned heap-limited Node process, complete point evidence"]
INSPECTOR["src/inspector/<br/>passive V8 Inspector discovery and capture<br/>script and target locations"]
DOTNET["src/dotnet/<br/>Managed static metadata provider<br/>PE/CLI metadata inventory, members, IL"]
@@ -69,6 +73,10 @@ graph TD
MAIN --> SERVER
MAIN --> APP
CLI --> APP
MAIN --> COMPOSITION
CLI --> COMPOSITION
SERVER --> COMPOSITION
COMPOSITION --> SESSION
APP --> FIRMWARE
FIRMWARE --> PROCESS
FIRMWARE --> CONTRACTS
@@ -84,9 +92,10 @@ graph TD
APP -->|record callbacks| RECORDS
SERVER -->|narrow record ports| RECORDS
RECORDS --> DOMAIN
REGISTRY --> HOPPER
REGISTRY --> GHIDRA
REGISTRY --> IDA
COMPOSITION --> HOPPER
COMPOSITION --> GHIDRA
COMPOSITION --> IDA
COMPOSITION --> AUX
IDA --> IDA_MCP
IDA --> CONTRACTS
IDA --> DOMAIN
+38 -19
View File
@@ -6,17 +6,17 @@ there is no new universal provider interface or fixed module manifest.
## Ownership to preserve
| Responsibility | Current owner | Migration direction |
| --------------------------------------------- | ---------------------------------------- | -------------------------------------------------------------- |
| Analyst semantics and Evidence representation | domain | Keep pure and provider-neutral |
| Named public input/output contracts | contracts | Keep exact CLI/MCP meaning and complete discovery |
| Producer parsing and external tool protocols | provider adapters | Keep behind the relevant typed port |
| Shared analyst workflows | application | Share between CLI and MCP |
| Concrete provider construction | binary runtime, composition, MCP startup | Consolidate one capability at a time in production composition |
| Investigation Evidence and Unknowns | InvestigationRecords/EvidenceLedger | Consume narrow read/write/atomic record ports |
| Target/profile snapshots and invalidation | BinarySessionRecords/BinarySession | Keep binary-owned |
| Subprocess ownership and host primitives | process/windows | Reuse; preserve cancellation and cleanup |
| Public translation | CLI/MCP adapters | Delegate to shared workflows and named contracts |
| Responsibility | Current owner | Migration direction |
| --------------------------------------------- | ----------------------------------- | ----------------------------------------------------- |
| Analyst semantics and Evidence representation | domain | Keep pure and provider-neutral |
| Named public input/output contracts | contracts | Keep exact CLI/MCP meaning and complete discovery |
| Producer parsing and external tool protocols | provider adapters | Keep behind the relevant typed port |
| Shared analyst workflows | application | Share between CLI and MCP |
| Concrete provider construction | composition, MCP startup | Keep fresh typed factories at the production boundary |
| Investigation Evidence and Unknowns | InvestigationRecords/EvidenceLedger | Consume narrow read/write/atomic record ports |
| Target/profile snapshots and invalidation | BinarySessionRecords/BinarySession | Keep binary-owned |
| Subprocess ownership and host primitives | process/windows | Reuse; preserve cancellation and cleanup |
| Public translation | CLI/MCP adapters | Delegate to shared workflows and named contracts |
Retain exact Evidence IDs, provenance, detached reads, atomic Evidence/Unknown
mutations and optimistic revisions. Preserve current close and failed-cleanup
@@ -69,8 +69,8 @@ paths removed during a move. Pass exact source paths for a focused check.
Pure source and its tests keep inward dependencies. Application/server provider
construction guards cover the migrated deep, Android, firmware, JavaScript recovery and observation
implementations. The existing binary runtime is an exact temporary composition
exception; shared browser capture/export helpers remain admitted while their
implementations. Application workflows cannot import composition, including the
former binary runtime entrypoint; shared browser capture/export helpers remain admitted while their
ownership is reviewed. Test lane restrictions still use Oxlint. Producer-dependent
browser and managed comparison tests live in boundary lanes with their original
fixtures and assertions. This is a development guard, not runtime authorization.
@@ -104,8 +104,8 @@ inside the provider. The existing launcher seams remain available to boundary
fixtures, which now exercise the production factories.
Application workflows and MCP registration must not import these concrete
providers. Composition may import their implementations. Binary composition is
still at its existing entrypoint while its direct-analysis callers are investigated.
providers. Composition may import their implementations. Binary composition also
uses this outer boundary after its direct-analysis callers were migrated.
This concrete pilot does not introduce a shared provider interface or module
manifest.
@@ -329,10 +329,9 @@ records still have their separate owner in `src/application/investigation/`.
`AnalysisProvider.ts` stays shared because its identity/execution types also serve
nonbinary ports. `BinaryTargetResolver.ts` stays shared because Android, firmware,
managed and artifact workflows use its target parsing. The existing `runtime.ts`
remains the exact temporary production wiring entrypoint while direct-analysis
callers are migrated deliberately; application workflows gain no outward
composition dependency.
managed and artifact workflows use its target parsing. Production wiring lives
in `src/composition/binary.ts`; application workflows gain no outward composition
dependency.
The three colocated application tests move with their owner. Composition cases
remain in `tests/composition/analysis-sessions/`; filesystem and SDK cases retain
@@ -385,3 +384,23 @@ Portable decoder and injected producer tests establish their format/boundary
claims. Actual Swift/Xcode, assetutil, native NIB, bundle signing and DMG lifecycle
claims require the real Apple lane. Native binary API/value-flow semantics and
shared cross-domain contract aggregates keep their separate owners.
## Binary production composition closure
`src/composition/binary.ts` is the shared production owner for deep candidate
construction, lazy auxiliary composition and the managed-only session factory.
It preserves the existing constructor bodies, selection policy and fresh session
per invocation. Factory registration acquires no engine or target.
`DirectAnalysis` and `DirectAnalysisStatus` receive these two existing factories
through `DirectAnalysisDependencies`. They retain configuration parsing, selected
status environment, cancellation, snapshot replay/binding, Evidence projection
and finally-close. The dependency uses the existing BinarySession type, including
its replay policy; it adds no universal backend or lifecycle contract.
`src/composition/directAnalysis.ts` binds the production factories for CLI and
one-shot managed MCP callers. Binding creates no session and stores no retained
analysis state. MCP startup constructs its connection session through the same
binary factory. Source and compiled fixtures use these entrypoints. The former
application runtime path is removed, and the source guard no longer admits its
outward provider/composition imports.
+15 -9
View File
@@ -7,10 +7,10 @@
"client": "2.3.1",
"core": "2.3.1"
},
"skill_version": "26"
"skill_version": "27"
},
"tools": {
"total": 131,
"total": 133,
"families": [
{
"id": "direct",
@@ -142,7 +142,7 @@
{
"id": "browser",
"surface": "browser-provider",
"count": 9,
"count": 11,
"tools": [
"analyze_web_bundle",
"capture_browser_scenario",
@@ -150,8 +150,10 @@
"compare_web_captures",
"compare_web_screenshots",
"discover_webmcp_tools",
"inspect_web_event_listeners",
"inspect_web_page",
"list_browser_targets",
"observe_web_execution",
"observe_web_session"
]
},
@@ -389,8 +391,10 @@
"compare_web_captures",
"compare_web_screenshots",
"discover_webmcp_tools",
"inspect_web_event_listeners",
"inspect_web_page",
"list_browser_targets",
"observe_web_execution",
"observe_web_session"
]
},
@@ -566,7 +570,7 @@
}
],
"cli": {
"primary_count": 86,
"primary_count": 88,
"commands": [
"address-to-file-offset",
"analyze",
@@ -622,6 +626,7 @@
"inspect-native-load-image",
"inspect-plist",
"inspect-signature",
"inspect-web-event-listeners",
"inspect-web-page",
"instructions",
"list-architectures",
@@ -630,6 +635,7 @@
"list-javascript-runtime-targets",
"observe-javascript-runtime",
"observe-native-ui",
"observe-web-execution",
"observe-web-session",
"project-android-application-graph",
"project-apple-application-graph",
@@ -664,15 +670,15 @@
},
"runtime_catalog": {
"counts": {
"cli_commands": 86,
"mcp_tools": 131,
"cli_commands": 88,
"mcp_tools": 133,
"mcp_prompts": 6
},
"digests": {
"tools_sha256": "68769748b624a288a5f4c57374de2d6fa5c2b5f5586fbbdef109795c907b7b7b",
"tools_sha256": "4e652b6689a76b0d4c9c64f1ed819b9fdf2bfc995df23be8cfb4fa1334b4224f",
"prompts_sha256": "ef2de8814390b340623c32001610aa2709d90bed51537b6e562ab3abf144ab4d",
"combined_sha256": "44b30e8892c84cf27766d5739228268b8a85f885c63002157bc134ed8d636bb8",
"providers_sha256": "a7d885531ee25d42c51385444faec544d34cd7eba6250a7dc75c4faedd4454a3"
"combined_sha256": "63c52e101898418ed77fed2c9c4fc83f743f945581e35637683ffdd282919525",
"providers_sha256": "aea15f517c822d80918ccaba57a4fa06557f484cf27b8acbaed4e1a660fac896"
}
}
}
+16
View File
@@ -543,3 +543,19 @@ checks an installed package after building the verifier dependencies. The separa
conditional `real-web-source-map` CI job supplies Chrome and an isolated pinned
fixture compiler; static/unit checks do not acquire a browser. See
[the source location guide](web-source-location.md) for the verified decoder profile.
### Website runtime attribution lane
`npm run verify:browser:runtime` uses caller-supplied
`REA_BROWSER_EXECUTABLE` and an owned synthetic site/profile. It exercises public
CLI and stdio MCP for precise execution and native listener source locations,
including actual armed progress, Unicode/CRLF digests, repeated source URLs with
distinct script IDs, zero branches and function-only unknowns on repeated
coverage, request initiators and an externally owned page that remains open.
An optional entrypoint argument to `scripts/verify-browser-runtime.mjs` runs the
same checks through an isolated installed package. The conditional
`real-web-runtime` CI job runs only for relevant changes and needs no fixture
compiler. Ordinary unit/static gates acquire no browser. See
[website runtime attribution](web-runtime.md) for effects, resource bounds and
coverage limits.
@@ -1,5 +1,5 @@
{
"manifest_id": "ecm_b3da2d10df0f03a654528fe4101f68084d4b9476bba072508c30564c5468e349",
"manifest_id": "ecm_2b2934cec1e1a386eed57f0a8f9cbd385ae22eaa63331179676482fc3a94f0a7",
"verifier": {
"id": "rea-managed-conformance",
"version": "1"
@@ -13,7 +13,7 @@
"skill_digests": [
{
"skill_id": "reverse-engineer-anything",
"sha256": "10c7b7d80d72e8ec4faa3f17a57e4563907ecac5533d22645c43ba932f9dcc13"
"sha256": "ba6f172d69b5c92132be05df0260e545f6e99b5d8e6105d2e6c2ab270a9ae269"
}
],
"claims": [
+61
View File
@@ -0,0 +1,61 @@
# Website runtime source attribution
REA integrates native Chrome DevTools Protocol inspection and precise V8 coverage through two composable tools. Supply an existing loopback CDP endpoint and a page target from `list_browser_targets`. With omitted origins, REA selects the page's current HTTP(S) origin. The browser and page remain externally owned.
## Observe execution
```bash
rea observe-web-execution http://127.0.0.1:9222 TARGET_ID \
--observation-ms 10000 --json
```
MCP: `observe_web_execution` with `cdp_endpoint`, `target_id`, optional `allowed_origins` and optional `observation_ms` (default 10000). The operation itself selects instrumentation; no extra approval flag is needed.
Starting precise coverage resets counters and prevents optimized execution until REA stops it. This changes timing/JIT behavior. REA requests accurate call counts and detailed ranges, takes one resetting sample, stops coverage, disables enabled domains and detaches its own connection. It does not navigate, click, evaluate selected application code or close the page.
Wait for the actual armed progress event before performing an action. MCP emits negotiated progress notifications; CLI emits `rea_progress` JSON on stderr while the Evidence result stays on stdout. Use ordinary commands, scripts or user actions to trigger behavior. CLI SIGINT/SIGTERM and MCP cancellation await instrumentation cleanup; unconfirmed cleanup is an error with the target, failed commands and previous failure.
Results retain:
- The selected target, frame and document loader identity; local armed/end timestamps and separate backend monotonic coverage timestamps.
- Every admitted producer function/block range, exclusive end offset and execution count in UTF-16 code units, with verified bounds when exact text is available. Nested/overlapping counts are preserved and must not be summed.
- Main-document script inventory, exact session script IDs, resource start coordinates, sourceURL/source-map declarations, producer hash and independently computed UTF-8 text digest.
- Armed-window requests from the selected main frame and selected origins, complete reported initiator objects (including stack hierarchy/descriptions), derived callsites, exact script-ID associations and unresolved async parent IDs. Source association does not establish UI causality.
The coverage counter interval starts at backend acceptance before the armed receipt and ends at the resetting sample after the local timer. Counts may include execution during these command intervals; request events are restricted to the locally armed window. The result retains both backend timestamps and the distinct local clock.
Source metadata remains collected through receipt of the resetting sample, including scripts parsed during the final command. REA then freezes this inventory for source joins. Fatal protocol parsing failures terminate the observer even when no command is outstanding; listener inspection checks producer failures after its final document assertion.
A document replacement ends the window; final coverage then remains unavailable. Missing coverage is unknown: an already loaded script may be absent from a sample. A detailed request also does not guarantee block granularity for already compiled functions. If `is_block_coverage` is false, the function-only ranges are observed and branch execution remains unknown. Native Chrome verification reproduced block coverage in the first window and function-only coverage in a subsequent window, including without REA.
Same-document navigation such as `history.pushState()` remains valid when the frame and known loader identity stay stable and the current origin is allowed. Unknown loader identity retains a conservative URL check; newly reported or replaced loaders are rejected. Malformed producer events end an armed window immediately and trigger cleanup, independently of the selected duration.
Target detachment ends the window even when the browser's root connection stays open. When the closed session cannot acknowledge cleanup, the operation returns a failure with the selected target, failed commands and original producer reason; it does not claim successful cleanup.
Child frames, workers, service workers, WebAssembly, chronological execution order and framework-specific causal interpretation are outside this operation. Script ownership comes from frame/context metadata, never URL equality. Repeated URLs and synthetic names therefore retain separate identities.
Observed script metadata remains available when ownership is unknown; source text and source association remain excluded or unknown. Request/initiator resource URLs omit transport userinfo. Stack and script URLs can also be inert `sourceURL` names: REA preserves declared names and unknown representations, and only removes script URL userinfo when stable producer metadata explicitly reports `hasSourceURL: false`. This policy also applies when metadata arrives after a request. Unknown producer extension fields and source-map declarations are preserved.
## Inspect a node's listeners
```bash
rea inspect-web-event-listeners http://127.0.0.1:9222 TARGET_ID '#submit' --json
```
MCP: `inspect_web_event_listeners` with the same scope and an explicit CSS `selector`. Native DOM selection inspects the first matching node in the main document. A missing match is an explicit empty result.
The result contains directly registered listener types/flags, callback coordinates, source identity/text/digest and execution unknowns. REA uses native DOM/DOMDebugger operations and releases its own remote object group. It never invokes a listener or dispatches an event. Delegated ancestor listeners, descendants, closed shadow roots and framework registries are not enumerated.
## Bounds and verification
Each protocol message has a 64 MiB budget; script/context metadata and request metadata each have an 8 MiB retained budget. Complete retained source text has a 32 MiB budget. Oversized or malformed evidence returns no partial successful result. Each command and the complete inspection/source-join phase have a 20-second deadline; cleanup has a separate 5-second command deadline plus transport closure. The selected observation duration must fit the host timer's positive 32-bit millisecond range. REA does not control the externally owned browser's CPU or memory.
`npm run verify:browser:runtime` uses caller-supplied `REA_BROWSER_EXECUTABLE` and the native `ps` command on Linux/macOS against an owned synthetic fixture, through real public CLI and stdio MCP. It checks exact Unicode/CRLF text/digests, same-URL identities, same-document navigation, inert declared names, executed and zero-count branches, function-only unknowns on repeated observation, listener selection, initiator association, actual arming and preservation of the external page. A separate fixture page closes during an armed MCP window to verify prompt termination and inline cleanup diagnostics. The optional positional entrypoint to `scripts/verify-browser-runtime.mjs` permits the same checks against an isolated installed tarball.
Real workflow coverage is Linux x64 / Chrome 153 / Node 24. Other browser engines and host workflows remain unverified. The conditional `real-web-runtime` CI lane acquires Chrome only for relevant changes; ordinary unit/static checks need no browser.
Primary protocol references: [Profiler and Debugger](https://raw.githubusercontent.com/ChromeDevTools/devtools-protocol/master/pdl/js_protocol.pdl), [DOMDebugger](https://raw.githubusercontent.com/ChromeDevTools/devtools-protocol/master/pdl/domains/DOMDebugger.pdl).
Source ownership requires both the selected main frame and producer evidence for its default execution world. Extension/automation isolated worlds, worker worlds and unknown worlds cannot establish website source joins. Contradictory world metadata stays unowned. Referenced source records retain the reported execution context and script auxiliary context metadata; source text is read only for proven main-document default worlds.
Known script/context identities are monitored through asynchronous source completion after the sample cutoff. A changed identity excludes its retained text, frame attribution, coverage and request/source joins. Chromium's specific native CSS syntax rejection returns an input error for `selector`, including the selected target; unrelated node, command and transport failures retain their original category.