Merge origin/main into feat/artifact-integrity-policy
Keep main's skill metadata version (27) and regenerate the package, tool catalog and product metadata for the merged tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -30,7 +30,7 @@ analyzer, or provider-version differences while appearing target-compatible.
|
||||
|
||||
The decision addressed the earlier representations of
|
||||
[`CompositeProvider`](https://github.com/morluto/rea/blob/main/src/application/binary/CompositeProvider.ts),
|
||||
[runtime composition](https://github.com/morluto/rea/blob/main/src/application/runtime.ts),
|
||||
[runtime composition](https://github.com/morluto/rea/blob/main/src/composition/binary.ts),
|
||||
[`BinarySession`](https://github.com/morluto/rea/blob/main/src/application/binary/BinarySession.ts),
|
||||
[`BinaryTarget`](https://github.com/morluto/rea/blob/main/src/domain/binaryTarget.ts),
|
||||
[analysis snapshot](https://github.com/morluto/rea/blob/main/src/domain/analysisSnapshot.ts), and
|
||||
|
||||
@@ -5,6 +5,10 @@ graph TD
|
||||
DISPATCH["scripts/rea.mjs<br/>Executable dispatcher<br/>routes mcp / one-shot"]
|
||||
end
|
||||
|
||||
subgraph "Production Composition"
|
||||
COMPOSITION["src/composition/<br/>fresh typed provider and session factories<br/>one-shot binding and independent optional loading"]
|
||||
end
|
||||
|
||||
subgraph "Adapters"
|
||||
SERVER["src/server/<br/>MCP request translation<br/>createServer, tool registration, toolResult"]
|
||||
HOPPER_BRIDGE["bridge/hopper_bridge.py<br/>Runs inside Hopper<br/>adapts operations to Hopper Python API"]
|
||||
@@ -18,7 +22,7 @@ graph TD
|
||||
REGISTRY["AnalysisProviderRegistry<br/>sorted deep candidates<br/>deterministic selection, no fallback"]
|
||||
RECORDS["src/application/investigation/<br/>composed InvestigationRecords + narrow ports<br/>EvidenceLedger and Unknown ownership"]
|
||||
SNAPSHOT["BinarySessionRecords + AnalysisSnapshotCache<br/>binary target/profile snapshots<br/>compatibility facade and post-commit notifications"]
|
||||
JS_RECON["src/application/ + src/artifacts/<br/>JavaScript artifact reconstruction<br/>directory/ASAR reads<br/>AST-only bundle projection"]
|
||||
JS_RECON["src/application/javascript/ + src/artifacts/javascript/<br/>JavaScript artifact reconstruction<br/>directory/ASAR reads<br/>AST-only bundle projection"]
|
||||
end
|
||||
|
||||
subgraph "Provider Layers"
|
||||
@@ -27,7 +31,7 @@ graph TD
|
||||
IDA["src/ida/<br/>Upstream MCP contract adaptation<br/>live GUI binding or owned headless database"]
|
||||
IDA_MCP["External ida-pro-mcp<br/>GUI proxy or database supervisor<br/>upstream installation and engine lifecycle"]
|
||||
AUX["src/artifacts/ + src/native/<br/>disjoint artifact and native capabilities"]
|
||||
BROWSER["src/browser/<br/>CDP / Playwright capture<br/>captured scripts and native module resolution"]
|
||||
BROWSER["src/browser/<br/>CDP / Playwright capture<br/>captured scripts and native module resolution<br/>execution/: coverage, listener and source attribution"]
|
||||
JS_CODEC["src/javascript/sourceMaps/<br/>pinned upstream map codec<br/>owned heap-limited Node process, complete point evidence"]
|
||||
INSPECTOR["src/inspector/<br/>passive V8 Inspector discovery and capture<br/>script and target locations"]
|
||||
DOTNET["src/dotnet/<br/>Managed static metadata provider<br/>PE/CLI metadata inventory, members, IL"]
|
||||
@@ -69,6 +73,10 @@ graph TD
|
||||
MAIN --> SERVER
|
||||
MAIN --> APP
|
||||
CLI --> APP
|
||||
MAIN --> COMPOSITION
|
||||
CLI --> COMPOSITION
|
||||
SERVER --> COMPOSITION
|
||||
COMPOSITION --> SESSION
|
||||
APP --> FIRMWARE
|
||||
FIRMWARE --> PROCESS
|
||||
FIRMWARE --> CONTRACTS
|
||||
@@ -84,9 +92,10 @@ graph TD
|
||||
APP -->|record callbacks| RECORDS
|
||||
SERVER -->|narrow record ports| RECORDS
|
||||
RECORDS --> DOMAIN
|
||||
REGISTRY --> HOPPER
|
||||
REGISTRY --> GHIDRA
|
||||
REGISTRY --> IDA
|
||||
COMPOSITION --> HOPPER
|
||||
COMPOSITION --> GHIDRA
|
||||
COMPOSITION --> IDA
|
||||
COMPOSITION --> AUX
|
||||
IDA --> IDA_MCP
|
||||
IDA --> CONTRACTS
|
||||
IDA --> DOMAIN
|
||||
|
||||
@@ -6,17 +6,17 @@ there is no new universal provider interface or fixed module manifest.
|
||||
|
||||
## Ownership to preserve
|
||||
|
||||
| Responsibility | Current owner | Migration direction |
|
||||
| --------------------------------------------- | ---------------------------------------- | -------------------------------------------------------------- |
|
||||
| Analyst semantics and Evidence representation | domain | Keep pure and provider-neutral |
|
||||
| Named public input/output contracts | contracts | Keep exact CLI/MCP meaning and complete discovery |
|
||||
| Producer parsing and external tool protocols | provider adapters | Keep behind the relevant typed port |
|
||||
| Shared analyst workflows | application | Share between CLI and MCP |
|
||||
| Concrete provider construction | binary runtime, composition, MCP startup | Consolidate one capability at a time in production composition |
|
||||
| Investigation Evidence and Unknowns | InvestigationRecords/EvidenceLedger | Consume narrow read/write/atomic record ports |
|
||||
| Target/profile snapshots and invalidation | BinarySessionRecords/BinarySession | Keep binary-owned |
|
||||
| Subprocess ownership and host primitives | process/windows | Reuse; preserve cancellation and cleanup |
|
||||
| Public translation | CLI/MCP adapters | Delegate to shared workflows and named contracts |
|
||||
| Responsibility | Current owner | Migration direction |
|
||||
| --------------------------------------------- | ----------------------------------- | ----------------------------------------------------- |
|
||||
| Analyst semantics and Evidence representation | domain | Keep pure and provider-neutral |
|
||||
| Named public input/output contracts | contracts | Keep exact CLI/MCP meaning and complete discovery |
|
||||
| Producer parsing and external tool protocols | provider adapters | Keep behind the relevant typed port |
|
||||
| Shared analyst workflows | application | Share between CLI and MCP |
|
||||
| Concrete provider construction | composition, MCP startup | Keep fresh typed factories at the production boundary |
|
||||
| Investigation Evidence and Unknowns | InvestigationRecords/EvidenceLedger | Consume narrow read/write/atomic record ports |
|
||||
| Target/profile snapshots and invalidation | BinarySessionRecords/BinarySession | Keep binary-owned |
|
||||
| Subprocess ownership and host primitives | process/windows | Reuse; preserve cancellation and cleanup |
|
||||
| Public translation | CLI/MCP adapters | Delegate to shared workflows and named contracts |
|
||||
|
||||
Retain exact Evidence IDs, provenance, detached reads, atomic Evidence/Unknown
|
||||
mutations and optimistic revisions. Preserve current close and failed-cleanup
|
||||
@@ -69,8 +69,8 @@ paths removed during a move. Pass exact source paths for a focused check.
|
||||
|
||||
Pure source and its tests keep inward dependencies. Application/server provider
|
||||
construction guards cover the migrated deep, Android, firmware, JavaScript recovery and observation
|
||||
implementations. The existing binary runtime is an exact temporary composition
|
||||
exception; shared browser capture/export helpers remain admitted while their
|
||||
implementations. Application workflows cannot import composition, including the
|
||||
former binary runtime entrypoint; shared browser capture/export helpers remain admitted while their
|
||||
ownership is reviewed. Test lane restrictions still use Oxlint. Producer-dependent
|
||||
browser and managed comparison tests live in boundary lanes with their original
|
||||
fixtures and assertions. This is a development guard, not runtime authorization.
|
||||
@@ -104,8 +104,8 @@ inside the provider. The existing launcher seams remain available to boundary
|
||||
fixtures, which now exercise the production factories.
|
||||
|
||||
Application workflows and MCP registration must not import these concrete
|
||||
providers. Composition may import their implementations. Binary composition is
|
||||
still at its existing entrypoint while its direct-analysis callers are investigated.
|
||||
providers. Composition may import their implementations. Binary composition also
|
||||
uses this outer boundary after its direct-analysis callers were migrated.
|
||||
This concrete pilot does not introduce a shared provider interface or module
|
||||
manifest.
|
||||
|
||||
@@ -329,10 +329,9 @@ records still have their separate owner in `src/application/investigation/`.
|
||||
|
||||
`AnalysisProvider.ts` stays shared because its identity/execution types also serve
|
||||
nonbinary ports. `BinaryTargetResolver.ts` stays shared because Android, firmware,
|
||||
managed and artifact workflows use its target parsing. The existing `runtime.ts`
|
||||
remains the exact temporary production wiring entrypoint while direct-analysis
|
||||
callers are migrated deliberately; application workflows gain no outward
|
||||
composition dependency.
|
||||
managed and artifact workflows use its target parsing. Production wiring lives
|
||||
in `src/composition/binary.ts`; application workflows gain no outward composition
|
||||
dependency.
|
||||
|
||||
The three colocated application tests move with their owner. Composition cases
|
||||
remain in `tests/composition/analysis-sessions/`; filesystem and SDK cases retain
|
||||
@@ -385,3 +384,23 @@ Portable decoder and injected producer tests establish their format/boundary
|
||||
claims. Actual Swift/Xcode, assetutil, native NIB, bundle signing and DMG lifecycle
|
||||
claims require the real Apple lane. Native binary API/value-flow semantics and
|
||||
shared cross-domain contract aggregates keep their separate owners.
|
||||
|
||||
## Binary production composition closure
|
||||
|
||||
`src/composition/binary.ts` is the shared production owner for deep candidate
|
||||
construction, lazy auxiliary composition and the managed-only session factory.
|
||||
It preserves the existing constructor bodies, selection policy and fresh session
|
||||
per invocation. Factory registration acquires no engine or target.
|
||||
|
||||
`DirectAnalysis` and `DirectAnalysisStatus` receive these two existing factories
|
||||
through `DirectAnalysisDependencies`. They retain configuration parsing, selected
|
||||
status environment, cancellation, snapshot replay/binding, Evidence projection
|
||||
and finally-close. The dependency uses the existing BinarySession type, including
|
||||
its replay policy; it adds no universal backend or lifecycle contract.
|
||||
|
||||
`src/composition/directAnalysis.ts` binds the production factories for CLI and
|
||||
one-shot managed MCP callers. Binding creates no session and stores no retained
|
||||
analysis state. MCP startup constructs its connection session through the same
|
||||
binary factory. Source and compiled fixtures use these entrypoints. The former
|
||||
application runtime path is removed, and the source guard no longer admits its
|
||||
outward provider/composition imports.
|
||||
|
||||
@@ -7,10 +7,10 @@
|
||||
"client": "2.3.1",
|
||||
"core": "2.3.1"
|
||||
},
|
||||
"skill_version": "26"
|
||||
"skill_version": "27"
|
||||
},
|
||||
"tools": {
|
||||
"total": 131,
|
||||
"total": 133,
|
||||
"families": [
|
||||
{
|
||||
"id": "direct",
|
||||
@@ -142,7 +142,7 @@
|
||||
{
|
||||
"id": "browser",
|
||||
"surface": "browser-provider",
|
||||
"count": 9,
|
||||
"count": 11,
|
||||
"tools": [
|
||||
"analyze_web_bundle",
|
||||
"capture_browser_scenario",
|
||||
@@ -150,8 +150,10 @@
|
||||
"compare_web_captures",
|
||||
"compare_web_screenshots",
|
||||
"discover_webmcp_tools",
|
||||
"inspect_web_event_listeners",
|
||||
"inspect_web_page",
|
||||
"list_browser_targets",
|
||||
"observe_web_execution",
|
||||
"observe_web_session"
|
||||
]
|
||||
},
|
||||
@@ -389,8 +391,10 @@
|
||||
"compare_web_captures",
|
||||
"compare_web_screenshots",
|
||||
"discover_webmcp_tools",
|
||||
"inspect_web_event_listeners",
|
||||
"inspect_web_page",
|
||||
"list_browser_targets",
|
||||
"observe_web_execution",
|
||||
"observe_web_session"
|
||||
]
|
||||
},
|
||||
@@ -566,7 +570,7 @@
|
||||
}
|
||||
],
|
||||
"cli": {
|
||||
"primary_count": 86,
|
||||
"primary_count": 88,
|
||||
"commands": [
|
||||
"address-to-file-offset",
|
||||
"analyze",
|
||||
@@ -622,6 +626,7 @@
|
||||
"inspect-native-load-image",
|
||||
"inspect-plist",
|
||||
"inspect-signature",
|
||||
"inspect-web-event-listeners",
|
||||
"inspect-web-page",
|
||||
"instructions",
|
||||
"list-architectures",
|
||||
@@ -630,6 +635,7 @@
|
||||
"list-javascript-runtime-targets",
|
||||
"observe-javascript-runtime",
|
||||
"observe-native-ui",
|
||||
"observe-web-execution",
|
||||
"observe-web-session",
|
||||
"project-android-application-graph",
|
||||
"project-apple-application-graph",
|
||||
@@ -664,15 +670,15 @@
|
||||
},
|
||||
"runtime_catalog": {
|
||||
"counts": {
|
||||
"cli_commands": 86,
|
||||
"mcp_tools": 131,
|
||||
"cli_commands": 88,
|
||||
"mcp_tools": 133,
|
||||
"mcp_prompts": 6
|
||||
},
|
||||
"digests": {
|
||||
"tools_sha256": "68769748b624a288a5f4c57374de2d6fa5c2b5f5586fbbdef109795c907b7b7b",
|
||||
"tools_sha256": "4e652b6689a76b0d4c9c64f1ed819b9fdf2bfc995df23be8cfb4fa1334b4224f",
|
||||
"prompts_sha256": "ef2de8814390b340623c32001610aa2709d90bed51537b6e562ab3abf144ab4d",
|
||||
"combined_sha256": "44b30e8892c84cf27766d5739228268b8a85f885c63002157bc134ed8d636bb8",
|
||||
"providers_sha256": "a7d885531ee25d42c51385444faec544d34cd7eba6250a7dc75c4faedd4454a3"
|
||||
"combined_sha256": "63c52e101898418ed77fed2c9c4fc83f743f945581e35637683ffdd282919525",
|
||||
"providers_sha256": "aea15f517c822d80918ccaba57a4fa06557f484cf27b8acbaed4e1a660fac896"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -543,3 +543,19 @@ checks an installed package after building the verifier dependencies. The separa
|
||||
conditional `real-web-source-map` CI job supplies Chrome and an isolated pinned
|
||||
fixture compiler; static/unit checks do not acquire a browser. See
|
||||
[the source location guide](web-source-location.md) for the verified decoder profile.
|
||||
|
||||
### Website runtime attribution lane
|
||||
|
||||
`npm run verify:browser:runtime` uses caller-supplied
|
||||
`REA_BROWSER_EXECUTABLE` and an owned synthetic site/profile. It exercises public
|
||||
CLI and stdio MCP for precise execution and native listener source locations,
|
||||
including actual armed progress, Unicode/CRLF digests, repeated source URLs with
|
||||
distinct script IDs, zero branches and function-only unknowns on repeated
|
||||
coverage, request initiators and an externally owned page that remains open.
|
||||
|
||||
An optional entrypoint argument to `scripts/verify-browser-runtime.mjs` runs the
|
||||
same checks through an isolated installed package. The conditional
|
||||
`real-web-runtime` CI job runs only for relevant changes and needs no fixture
|
||||
compiler. Ordinary unit/static gates acquire no browser. See
|
||||
[website runtime attribution](web-runtime.md) for effects, resource bounds and
|
||||
coverage limits.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"manifest_id": "ecm_b3da2d10df0f03a654528fe4101f68084d4b9476bba072508c30564c5468e349",
|
||||
"manifest_id": "ecm_2b2934cec1e1a386eed57f0a8f9cbd385ae22eaa63331179676482fc3a94f0a7",
|
||||
"verifier": {
|
||||
"id": "rea-managed-conformance",
|
||||
"version": "1"
|
||||
@@ -13,7 +13,7 @@
|
||||
"skill_digests": [
|
||||
{
|
||||
"skill_id": "reverse-engineer-anything",
|
||||
"sha256": "10c7b7d80d72e8ec4faa3f17a57e4563907ecac5533d22645c43ba932f9dcc13"
|
||||
"sha256": "ba6f172d69b5c92132be05df0260e545f6e99b5d8e6105d2e6c2ab270a9ae269"
|
||||
}
|
||||
],
|
||||
"claims": [
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
# Website runtime source attribution
|
||||
|
||||
REA integrates native Chrome DevTools Protocol inspection and precise V8 coverage through two composable tools. Supply an existing loopback CDP endpoint and a page target from `list_browser_targets`. With omitted origins, REA selects the page's current HTTP(S) origin. The browser and page remain externally owned.
|
||||
|
||||
## Observe execution
|
||||
|
||||
```bash
|
||||
rea observe-web-execution http://127.0.0.1:9222 TARGET_ID \
|
||||
--observation-ms 10000 --json
|
||||
```
|
||||
|
||||
MCP: `observe_web_execution` with `cdp_endpoint`, `target_id`, optional `allowed_origins` and optional `observation_ms` (default 10000). The operation itself selects instrumentation; no extra approval flag is needed.
|
||||
|
||||
Starting precise coverage resets counters and prevents optimized execution until REA stops it. This changes timing/JIT behavior. REA requests accurate call counts and detailed ranges, takes one resetting sample, stops coverage, disables enabled domains and detaches its own connection. It does not navigate, click, evaluate selected application code or close the page.
|
||||
|
||||
Wait for the actual armed progress event before performing an action. MCP emits negotiated progress notifications; CLI emits `rea_progress` JSON on stderr while the Evidence result stays on stdout. Use ordinary commands, scripts or user actions to trigger behavior. CLI SIGINT/SIGTERM and MCP cancellation await instrumentation cleanup; unconfirmed cleanup is an error with the target, failed commands and previous failure.
|
||||
|
||||
Results retain:
|
||||
|
||||
- The selected target, frame and document loader identity; local armed/end timestamps and separate backend monotonic coverage timestamps.
|
||||
- Every admitted producer function/block range, exclusive end offset and execution count in UTF-16 code units, with verified bounds when exact text is available. Nested/overlapping counts are preserved and must not be summed.
|
||||
- Main-document script inventory, exact session script IDs, resource start coordinates, sourceURL/source-map declarations, producer hash and independently computed UTF-8 text digest.
|
||||
- Armed-window requests from the selected main frame and selected origins, complete reported initiator objects (including stack hierarchy/descriptions), derived callsites, exact script-ID associations and unresolved async parent IDs. Source association does not establish UI causality.
|
||||
|
||||
The coverage counter interval starts at backend acceptance before the armed receipt and ends at the resetting sample after the local timer. Counts may include execution during these command intervals; request events are restricted to the locally armed window. The result retains both backend timestamps and the distinct local clock.
|
||||
|
||||
Source metadata remains collected through receipt of the resetting sample, including scripts parsed during the final command. REA then freezes this inventory for source joins. Fatal protocol parsing failures terminate the observer even when no command is outstanding; listener inspection checks producer failures after its final document assertion.
|
||||
|
||||
A document replacement ends the window; final coverage then remains unavailable. Missing coverage is unknown: an already loaded script may be absent from a sample. A detailed request also does not guarantee block granularity for already compiled functions. If `is_block_coverage` is false, the function-only ranges are observed and branch execution remains unknown. Native Chrome verification reproduced block coverage in the first window and function-only coverage in a subsequent window, including without REA.
|
||||
|
||||
Same-document navigation such as `history.pushState()` remains valid when the frame and known loader identity stay stable and the current origin is allowed. Unknown loader identity retains a conservative URL check; newly reported or replaced loaders are rejected. Malformed producer events end an armed window immediately and trigger cleanup, independently of the selected duration.
|
||||
|
||||
Target detachment ends the window even when the browser's root connection stays open. When the closed session cannot acknowledge cleanup, the operation returns a failure with the selected target, failed commands and original producer reason; it does not claim successful cleanup.
|
||||
|
||||
Child frames, workers, service workers, WebAssembly, chronological execution order and framework-specific causal interpretation are outside this operation. Script ownership comes from frame/context metadata, never URL equality. Repeated URLs and synthetic names therefore retain separate identities.
|
||||
|
||||
Observed script metadata remains available when ownership is unknown; source text and source association remain excluded or unknown. Request/initiator resource URLs omit transport userinfo. Stack and script URLs can also be inert `sourceURL` names: REA preserves declared names and unknown representations, and only removes script URL userinfo when stable producer metadata explicitly reports `hasSourceURL: false`. This policy also applies when metadata arrives after a request. Unknown producer extension fields and source-map declarations are preserved.
|
||||
|
||||
## Inspect a node's listeners
|
||||
|
||||
```bash
|
||||
rea inspect-web-event-listeners http://127.0.0.1:9222 TARGET_ID '#submit' --json
|
||||
```
|
||||
|
||||
MCP: `inspect_web_event_listeners` with the same scope and an explicit CSS `selector`. Native DOM selection inspects the first matching node in the main document. A missing match is an explicit empty result.
|
||||
|
||||
The result contains directly registered listener types/flags, callback coordinates, source identity/text/digest and execution unknowns. REA uses native DOM/DOMDebugger operations and releases its own remote object group. It never invokes a listener or dispatches an event. Delegated ancestor listeners, descendants, closed shadow roots and framework registries are not enumerated.
|
||||
|
||||
## Bounds and verification
|
||||
|
||||
Each protocol message has a 64 MiB budget; script/context metadata and request metadata each have an 8 MiB retained budget. Complete retained source text has a 32 MiB budget. Oversized or malformed evidence returns no partial successful result. Each command and the complete inspection/source-join phase have a 20-second deadline; cleanup has a separate 5-second command deadline plus transport closure. The selected observation duration must fit the host timer's positive 32-bit millisecond range. REA does not control the externally owned browser's CPU or memory.
|
||||
|
||||
`npm run verify:browser:runtime` uses caller-supplied `REA_BROWSER_EXECUTABLE` and the native `ps` command on Linux/macOS against an owned synthetic fixture, through real public CLI and stdio MCP. It checks exact Unicode/CRLF text/digests, same-URL identities, same-document navigation, inert declared names, executed and zero-count branches, function-only unknowns on repeated observation, listener selection, initiator association, actual arming and preservation of the external page. A separate fixture page closes during an armed MCP window to verify prompt termination and inline cleanup diagnostics. The optional positional entrypoint to `scripts/verify-browser-runtime.mjs` permits the same checks against an isolated installed tarball.
|
||||
|
||||
Real workflow coverage is Linux x64 / Chrome 153 / Node 24. Other browser engines and host workflows remain unverified. The conditional `real-web-runtime` CI lane acquires Chrome only for relevant changes; ordinary unit/static checks need no browser.
|
||||
|
||||
Primary protocol references: [Profiler and Debugger](https://raw.githubusercontent.com/ChromeDevTools/devtools-protocol/master/pdl/js_protocol.pdl), [DOMDebugger](https://raw.githubusercontent.com/ChromeDevTools/devtools-protocol/master/pdl/domains/DOMDebugger.pdl).
|
||||
|
||||
Source ownership requires both the selected main frame and producer evidence for its default execution world. Extension/automation isolated worlds, worker worlds and unknown worlds cannot establish website source joins. Contradictory world metadata stays unowned. Referenced source records retain the reported execution context and script auxiliary context metadata; source text is read only for proven main-document default worlds.
|
||||
|
||||
Known script/context identities are monitored through asynchronous source completion after the sample cutoff. A changed identity excludes its retained text, frame attribution, coverage and request/source joins. Chromium's specific native CSS syntax rejection returns an input error for `selector`, including the selected target; unrelated node, command and transport failures retain their original category.
|
||||
Reference in New Issue
Block a user