Compare commits
166
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
7f348657e6 | ||
|
|
fd768524d8 | ||
|
|
c02fb0ea80 | ||
|
|
96fc84f1fb | ||
|
|
b69d20a300 | ||
|
|
11f8b76257 | ||
|
|
11f1d952e7 | ||
|
|
a0317cf732 | ||
|
|
67c7a24e60 | ||
|
|
c7a890d129 | ||
|
|
b955b825fc | ||
|
|
9cf49c7a78 | ||
|
|
c923fb0dc9 | ||
|
|
ded199abcd | ||
|
|
ed881c6d2e | ||
|
|
eaa9616585 | ||
|
|
bf34d0d2fc | ||
|
|
e802441bce | ||
|
|
e7882b34f4 | ||
|
|
9049dbaf0c | ||
|
|
7fc0b5ec9c | ||
|
|
6a05248d46 | ||
|
|
20f51d7286 | ||
|
|
6bd2b28161 | ||
|
|
d9e9fb2273 | ||
|
|
b953f96c0a | ||
|
|
4276aaf18b | ||
|
|
fe8d0bdd36 | ||
|
|
9c34c091b3 | ||
|
|
b9ee921560 | ||
|
|
15463bdbe6 | ||
|
|
55e21a2eec | ||
|
|
f6775a3669 | ||
|
|
a07ceaec36 | ||
|
|
6a04f922e1 | ||
|
|
e74e4e951b | ||
|
|
68ecf3df8c | ||
|
|
ab62ee9dcf | ||
|
|
2c6a4fac4f | ||
|
|
c6acd19b6d | ||
|
|
1a1b67c888 | ||
|
|
12303812a8 | ||
|
|
f6556ad2bc | ||
|
|
5f0f25ad61 | ||
|
|
d73ed6fce2 | ||
|
|
60f771a11e | ||
|
|
650237efb0 | ||
|
|
26bffb94bf | ||
|
|
7017c4d68c | ||
|
|
141d2095f5 | ||
|
|
1b953021ea | ||
|
|
e1cd292aa6 | ||
|
|
18831491cf | ||
|
|
18d56f5321 | ||
|
|
91886982fc | ||
|
|
3ae826ceba | ||
|
|
73dd8d6ed5 | ||
|
|
ec0b081371 | ||
|
|
2187e16f14 | ||
|
|
bb96afdbe3 | ||
|
|
1991542e59 | ||
|
|
1f7f808c4f | ||
|
|
3c57e1a8b2 | ||
|
|
63e796d50e | ||
|
|
16f5ded446 | ||
|
|
1447fd2f99 | ||
|
|
7a2cd1ea0a | ||
|
|
8b181716ac | ||
|
|
10935b7b11 | ||
|
|
acf75e6700 | ||
|
|
c105391db5 | ||
|
|
d867136cec | ||
|
|
a73a55f53c | ||
|
|
9581e067ad | ||
|
|
c74142e6bd | ||
|
|
042dcaa1ee | ||
|
|
b045df100e | ||
|
|
fa4b828865 | ||
|
|
45984a4319 | ||
|
|
bb540826d6 | ||
|
|
2c7240cf9d | ||
|
|
4303dcdcf6 | ||
|
|
a86deecac5 | ||
|
|
a6efa4fcef | ||
|
|
172407787f | ||
|
|
23de4f77c7 | ||
|
|
bb154d1067 | ||
|
|
a2d8b94dd5 | ||
|
|
f254f5ac2b | ||
|
|
ca2d15b3ee | ||
|
|
4773c152bc | ||
|
|
eeba61826c | ||
|
|
4f4f6cd7b5 | ||
|
|
ee87111009 | ||
|
|
37b7550ecc | ||
|
|
c462a4ebd3 | ||
|
|
ebc22d99d6 | ||
|
|
9a18251426 | ||
|
|
34a4bb6508 | ||
|
|
f09bb8a997 | ||
|
|
9435c7aae2 | ||
|
|
d8ceb3f276 | ||
|
|
951b095e11 | ||
|
|
82cfcee19f | ||
|
|
f43fc02475 | ||
|
|
dc038d16bb | ||
|
|
653067d799 | ||
|
|
3b0ea590fc | ||
|
|
ae8051302b | ||
|
|
8fc2043eb8 | ||
|
|
fdd8f82602 | ||
|
|
c59fbf5e9c | ||
|
|
81cca2e0e3 | ||
|
|
1dd20ce1ff | ||
|
|
7bc57e1e64 | ||
|
|
5c2956587a | ||
|
|
85e124253a | ||
|
|
af29247a78 | ||
|
|
d59ab6d977 | ||
|
|
86fab11fbe | ||
|
|
6ce5903e7a | ||
|
|
91b8f587cd | ||
|
|
5f79c852af | ||
|
|
ea2f94d4de | ||
|
|
7b15762d09 | ||
|
|
3ee6ff43c1 | ||
|
|
4dd44ddf16 | ||
|
|
8224aeba9d | ||
|
|
ac57c88efe | ||
|
|
fd9588089d | ||
|
|
b7169c8b74 | ||
|
|
049132ab43 | ||
|
|
c89972c877 | ||
|
|
5ab0cbb48e | ||
|
|
921bb95755 | ||
|
|
c339f9ebe7 | ||
|
|
1c37914306 | ||
|
|
ed485a3dc8 | ||
|
|
8d4d631606 | ||
|
|
7fb097d72c | ||
|
|
c9fed50691 | ||
|
|
f611d8ce72 | ||
|
|
570c40cde5 | ||
|
|
221c856481 | ||
|
|
02cedc3796 | ||
|
|
2397137615 | ||
|
|
cecc6ecbff | ||
|
|
b123b2e51c | ||
|
|
a97b713f74 | ||
|
|
d1aaeee400 | ||
|
|
f76163c246 | ||
|
|
bb26622670 | ||
|
|
0365454a57 | ||
|
|
e20994a8ea | ||
|
|
0043d9b134 | ||
|
|
aeee5074d1 | ||
|
|
916ac97bf9 | ||
|
|
fd985dbc5b | ||
|
|
47a51b08f4 | ||
|
|
f941968eb1 | ||
|
|
ec2d251c0a | ||
|
|
2e45b4d7a5 | ||
|
|
11fe5a13f8 | ||
|
|
ad08510a11 | ||
|
|
d96550f2f6 | ||
|
|
8cbd57cafa |
@@ -2,7 +2,7 @@
|
||||
name: reverse-engineer-anything
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
metadata:
|
||||
version: "40"
|
||||
version: "41"
|
||||
---
|
||||
|
||||
# REA
|
||||
@@ -61,12 +61,47 @@ explicit path or endpoint and do not need it.
|
||||
consult the NativeAOT workflow in `references/native-and-artifacts.md` for
|
||||
optional metadata recovery and its supported host/layout boundary.
|
||||
- User-owned browser page already open: `list_browser_targets`.
|
||||
- Retained HAR/native mitmproxy capture: `inspect_web_network_capture`. Select
|
||||
- Fiddler Everywhere traffic or a native `.saz` capture: read the
|
||||
[Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging).
|
||||
Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`;
|
||||
connect Everywhere's separate MCP server for live traffic.
|
||||
- Retained HAR/SAZ/native mitmproxy/aioquic qlog capture: `inspect_web_network_capture`. Select
|
||||
original record ordinals when useful; inspect producer fields and byte/number
|
||||
sidecars without fetching recorded URLs or claiming live attribution. See
|
||||
[historical captures](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md) for the exact upstream
|
||||
profile and credential exclusions.
|
||||
profile and credential exclusions. Qlog support starts with aioquic 1.5.0
|
||||
JSON 0.3; stream coverage and endpoint loss declarations do not recover bodies
|
||||
or establish network loss. Prefer detail summary, then kind qlog trace/stream
|
||||
pages; select one trace, stream ID and direction for contribution/range pages.
|
||||
Facts retain all-recorded-events-before-selection scope. Other qlog layouts
|
||||
and PCAP correlation are unsupported.
|
||||
- User-owned Electron runtime already open: `list_electron_targets`.
|
||||
- Packet-field discovery: `inspect_packet_fields` searches installed protocols
|
||||
and field names/types before choosing fields for `inspect_packet_capture`.
|
||||
Query/page the relevant records; registrations do not establish observed traffic.
|
||||
- Recorded packet captures: `inspect_packet_capture` with selected TShark fields,
|
||||
display filters and decode-as rules. Requires Linux, util-linux prlimit and `REA_TSHARK_COMMAND`; see
|
||||
[capture guide](https://github.com/morluto/rea/blob/main/docs/packet-captures.md).
|
||||
Packet limits count input packets before filtering. Output defaults to 50 matching
|
||||
frames; use `match_offset` / `match_limit` for later pages, or explicit null
|
||||
limit for complete matching output. Preserve the source digest across pages.
|
||||
This does not perform live capture.
|
||||
For address/DNS/SNI/protocol summaries use `inspect_pcap_capture` with MCP
|
||||
detail summary, then retained kind pcap pages. Requires Wireshark 4.x tshark
|
||||
and capinfos, with Linux prlimit or Windows Job Objects; see
|
||||
[packet summaries](https://github.com/morluto/rea/blob/main/docs/pcap-analysis.md).
|
||||
Select an exact DNS question for address pages; page exhaustion does not prove
|
||||
network completeness. This reads recordings and does not acquire traffic.
|
||||
- Decoder discovery: `inspect_signal_decoders` searches installed decoder IDs
|
||||
or selected decoder help, including channels and options. Query and page the
|
||||
relevant details before choosing settings; this does not scan hardware.
|
||||
- Recorded logic/signal traces: `decode_signal_recording` with explicit decoder,
|
||||
channel and protocol settings. On Linux with util-linux prlimit, supply sigrok-cli 0.7.2 through
|
||||
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/signal-recordings.md).
|
||||
Output defaults to 50 literal lines; select annotation classes and page with
|
||||
`line_offset` / `line_limit`, or explicit null for all remaining output.
|
||||
Window counts are display lines, not annotations or samples. Preserve source digests across pages.
|
||||
This reads existing files; it does not acquire hardware signals.
|
||||
- Explicit local WebAssembly artifact: `inspect_wasm_artifact`. Supply WABT 1.0.42
|
||||
through `REA_WABT_BIN_DIRECTORY`; retain exact byte/tool identity and decoded WAT.
|
||||
Select JavaScript glue and candidate paths explicitly and preserve URL/path
|
||||
@@ -75,7 +110,11 @@ explicit path or endpoint and do not need it.
|
||||
- Explicit local EVM bytecode carrier: `inspect_evm_interface` with caller-selected
|
||||
`raw` or `hex` encoding. Preserve carrier/decoded digests and treat selectors,
|
||||
argument strings and mutability as inferred candidates; this performs no chain
|
||||
lookup or target execution. See the
|
||||
lookup or target execution. For the complete inferred interface without hex or
|
||||
duplicate stdout, discover support and select `detail: "summary"`; complete
|
||||
Evidence stays retained under its original identity. Use the returned
|
||||
`parent_evidence_id` for exact function or `bytecode_hex`/stdout/stderr pages.
|
||||
Candidate coverage does not establish interface completeness. See the
|
||||
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/evm-bytecode.md).
|
||||
- Embedded Go compiler, module or build-setting metadata: `inspect_go_binary`
|
||||
with the explicit local file. This built-in target-free inspection requires no
|
||||
@@ -83,6 +122,15 @@ explicit path or endpoint and do not need it.
|
||||
absent metadata does not prove a binary is not Go, and unparsed text records
|
||||
remain explicit. See the
|
||||
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/go-binaries.md).
|
||||
|
||||
- Explicit local SQLite database snapshot: `inspect_sqlite_database`. Inspect
|
||||
with Node.js 24.x >=24.15 or 26+ providing native SQLite limits. Read schema
|
||||
first (compact inventory by default), or select an exact ordinary table
|
||||
and `columns` when rows are needed. Defaults are selected-table schema and
|
||||
20 rows; request `schema_scope: "database"` for the complete stored schema. Preserve both database and existing sibling WAL identity; row samples
|
||||
report truncation and have unspecified order. Use a quiescent snapshot rather
|
||||
than assuming a live database/WAL pair was copied atomically. See the
|
||||
[SQLite guide](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
|
||||
- Explicit Linux ELF file for offline layout, symbols, relocations or static
|
||||
mitigation evidence: `inspect_binary_layout`. This target-free operation uses
|
||||
caller-supplied pwntools without opening a disassembler database. Preserve its
|
||||
|
||||
@@ -11,6 +11,21 @@ unparsed module-text line. A missing record is an unknown, not proof of a
|
||||
non-Go binary; observed metadata does not authenticate its claims. Function,
|
||||
source and type recovery remain a separate native-analysis question.
|
||||
|
||||
## SQLite snapshots
|
||||
|
||||
Use `inspect_sqlite_database` with an explicit local database snapshot for its
|
||||
schema and optional rows from one exact ordinary table. It requires Node.js
|
||||
24.x >=24.15 or 26+ with the native `DatabaseSync.limits` API. It also consumes the
|
||||
existing sibling `-wal`; retain both original digests when citing its result.
|
||||
Default output is a compact inventory; a selected table expands only its own
|
||||
schema and samples 20 rows. Select `columns` to avoid irrelevant or large cells,
|
||||
and request `schema_scope: "database"` only for complete schema details.
|
||||
Choose `row_limit` when a finite sample is useful and keep truncation and
|
||||
unspecified row order explicit. Virtual tables, shadow tables and views are
|
||||
schema facts, not row-read targets. REA inspects private copies and leaves
|
||||
the source files untouched; obtain a coherent quiescent database/WAL snapshot
|
||||
before inspection. See [SQLite snapshots](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
|
||||
|
||||
## Recorded Linux crashes
|
||||
|
||||
Use `inspect_recorded_crash` with an explicit Linux x86-64 ELF core path to
|
||||
@@ -68,6 +83,14 @@ names, the native API record, and value-flow summaries. Pseudocode offsets and l
|
||||
code units; use the returned `next_offset` for continuation. Provider limitations
|
||||
and unavailable facts remain explicit; the complete parent Evidence is retained.
|
||||
|
||||
`inspect_macho` on a large framework returns megabytes of imports and exports.
|
||||
Select `detail: "summary"` for slice identity, UUID and per-collection counts,
|
||||
then page rows with `view: {kind: "macho", facet: "exports", offset: 0, limit: 64}`
|
||||
and `normalized_result.parent_evidence_id`. Facets are entrypoints,
|
||||
architectures, build_metadata, load_commands, dependencies, imports, exports,
|
||||
segments and provenance. `source_exhaustive: false` means unreturned symbols
|
||||
may exist.
|
||||
|
||||
To see which functions or Objective-C methods a Mach-O actually calls in one
|
||||
run, use `observe_native_calls` with explicit breakpoints and a bounded
|
||||
`duration_ms`/`max_events`. It launches a new process under LLDB, so the target
|
||||
@@ -120,6 +143,18 @@ assembly. Keep managed/native boundaries and unavailable reconstruction facts
|
||||
explicit. A bring-your-own reconstruction oracle is separate from the canonical
|
||||
parser and must not become an implicit setup dependency.
|
||||
|
||||
For identity/classification questions, `inspect_managed_artifact` can explicitly
|
||||
select `detail: "summary"` on the first MCP call. It retains complete metadata
|
||||
and returns a linked overview with full classification/evidence, nullable
|
||||
module/MVID, frameworks, metadata coverage/issues/limitations and returned-row
|
||||
counts. Default complete requests remain unchanged. Read original rows with
|
||||
`inspect_analysis_view`, source `normalized_result.parent_evidence_id`, and
|
||||
`view: {kind: "managed", collection: "attributes", offset: 0, limit: 16}`.
|
||||
Collections are `references`, `resources` and `attributes`. Paging does not
|
||||
repeat parsing. Returned-row exhaustion
|
||||
does not prove complete metadata or absence under partial/unavailable coverage;
|
||||
preserve build-local tokens and their artifact/MVID scope.
|
||||
|
||||
### .NET NativeAOT
|
||||
|
||||
NativeAOT binaries contain native machine code, so ordinary CIL decompilers
|
||||
@@ -167,8 +202,9 @@ every executable in the bundle. It also lists missing or weak loads and earlier
|
||||
`architecture`. System paths stay undetermined because the dyld shared cache
|
||||
provides them.
|
||||
|
||||
Use `extract_artifact` when materialized files are needed. It takes no arguments
|
||||
and materializes all regular files into a fresh temporary directory chosen by
|
||||
Use `extract_artifact` when materialized files are needed. It extracts the active
|
||||
target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/mcp-contracts.md).
|
||||
It materializes all regular files into a fresh temporary directory chosen by
|
||||
REA. Symlinks and encrypted entries are inventory facts, not extractable files.
|
||||
The result reports the materialized directory; callers do not choose the
|
||||
destination. No separate permission grant is needed.
|
||||
|
||||
@@ -26,3 +26,18 @@ passive browser/Electron Evidence exist. Prefer captured-source digest identity;
|
||||
report path/digest disagreement as a mismatch and preserve ambiguity. Explicit
|
||||
path mappings are inference inputs and never broaden filesystem or CDP authority.
|
||||
A bundle observed at runtime does not prove every contained module executed.
|
||||
|
||||
## Fiddler Everywhere traffic debugging
|
||||
|
||||
For large saved captures, request `detail: "summary"`, then page retained Evidence
|
||||
with `inspect_analysis_view` and `view: {kind: "capture", detail: "preview", offset: 0, limit: 20}`.
|
||||
Select complete record pages only for needed headers, metadata or payloads; views
|
||||
read the historical parent without repeating capture decoding.
|
||||
|
||||
Inspect native `.saz` snapshots with `inspect_web_network_capture` and
|
||||
`format: "saz"`; encrypted archives are unsupported. For live
|
||||
traffic analysis, connect Everywhere's separate MCP server and use its
|
||||
advertised session tools with an explicit `sessionsSource` (`LiveTraffic` or
|
||||
`AgentCalls`). Preserve existing sessions and filters when inspecting a
|
||||
recorded run. See the [capture and MCP guide](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md)
|
||||
for the workflow and Telerik's public setup and traffic-debugging skills.
|
||||
|
||||
@@ -3,3 +3,6 @@
|
||||
|
||||
# Windows command fixtures retain their native line endings.
|
||||
*.cmd text eol=crlf
|
||||
|
||||
# Preserve whitespace in original upstream licensing texts.
|
||||
licenses/npm-runtime-notices.txt whitespace=-blank-at-eol
|
||||
|
||||
@@ -77,7 +77,7 @@ jobs:
|
||||
static:
|
||||
name: Static checks
|
||||
needs: changes
|
||||
if: ${{ always() }}
|
||||
if: ${{ !cancelled() }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Require successful change classification
|
||||
@@ -99,6 +99,15 @@ jobs:
|
||||
run: npm run test:python
|
||||
- if: needs.changes.outputs.code != 'true'
|
||||
run: npm run format:check
|
||||
- name: Upload complete static diagnostics
|
||||
if: always() && needs.changes.outputs.code == 'true'
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: static-check-reports
|
||||
path: .cache/static-checks/
|
||||
include-hidden-files: true
|
||||
if-no-files-found: ignore
|
||||
retention-days: 7
|
||||
|
||||
test-shard:
|
||||
name: Test shard (${{ matrix.shard }})
|
||||
@@ -200,10 +209,15 @@ jobs:
|
||||
if-no-files-found: error
|
||||
retention-days: 7
|
||||
|
||||
sqlite-runtime:
|
||||
needs: [changes, build]
|
||||
if: needs.changes.outputs.code == 'true'
|
||||
uses: ./.github/workflows/sqlite-runtime.yml
|
||||
|
||||
test:
|
||||
name: Test (coverage + timing)
|
||||
needs: [changes, test-shard]
|
||||
if: ${{ always() }}
|
||||
if: ${{ !cancelled() }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Require successful change classification
|
||||
@@ -495,6 +509,7 @@ jobs:
|
||||
tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts
|
||||
tests/boundary/filesystem/setupUnavailableProvider.test.ts
|
||||
tests/boundary/filesystem/windowsNpxRegistration.test.ts
|
||||
tests/acceptance/setup/homeDirectoryFallback.test.ts
|
||||
tests/boundary/filesystem/webScriptManifestPaths.test.ts
|
||||
tests/boundary/process/ProcessEvidence.host.test.ts
|
||||
tests/acceptance/webModuleCli.test.ts
|
||||
@@ -642,7 +657,7 @@ jobs:
|
||||
|
||||
required:
|
||||
name: CI required
|
||||
if: ${{ always() }}
|
||||
if: ${{ !cancelled() }}
|
||||
needs:
|
||||
[
|
||||
changes,
|
||||
@@ -650,6 +665,7 @@ jobs:
|
||||
static,
|
||||
build,
|
||||
test-shard,
|
||||
sqlite-runtime,
|
||||
test,
|
||||
conformance-fixtures,
|
||||
windows-native-build,
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
name: Real packet capture
|
||||
on:
|
||||
workflow_dispatch:
|
||||
pull_request:
|
||||
paths:
|
||||
- "tests/boundary/captures/**"
|
||||
- "tests/boundary/cli/captureCleanup.test.ts"
|
||||
- "tests/boundary/mcp/packetCaptureMcp.test.ts"
|
||||
- "src/captures/**"
|
||||
- "src/pcap/**"
|
||||
- "src/domain/pcap/**"
|
||||
- "src/domain/analysisView/pcapCaptureView.ts"
|
||||
- "src/application/pcap/**"
|
||||
- "src/contracts/pcap/**"
|
||||
- "src/server/registerPcapTools.ts"
|
||||
- "tests/boundary/pcap/**"
|
||||
- "src/domain/captures/**"
|
||||
- "src/application/captures/**"
|
||||
- "src/contracts/captures/**"
|
||||
- "src/composition/captures.ts"
|
||||
- "src/cli/captureCommands.ts"
|
||||
- "src/server/registerCaptureTools.ts"
|
||||
- "src/server/createServer.ts"
|
||||
- "src/server/sessionAvailabilityPolicy.ts"
|
||||
- "src/application/CapabilityInventory.ts"
|
||||
- "src/contracts/toolContracts.ts"
|
||||
- "src/contracts/toolEffects.ts"
|
||||
- "src/cli.ts"
|
||||
- "src/cliCommandNames.ts"
|
||||
- "src/process/**"
|
||||
- "src/artifacts/readStableArtifact.ts"
|
||||
- "scripts/verify-packet-capture.mjs"
|
||||
- ".github/workflows/real-packet-capture.yml"
|
||||
- "package*.json"
|
||||
- ".nvmrc"
|
||||
permissions:
|
||||
contents: read
|
||||
concurrency:
|
||||
group: real-packet-capture-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
jobs:
|
||||
tshark:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- run: sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y tshark util-linux python3
|
||||
- run: npm ci
|
||||
- run: npm run build:cached
|
||||
- run: npm run test:focused -- tests/boundary/captures/fieldDiscovery.test.ts src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts
|
||||
- run: node scripts/verify-packet-capture.mjs
|
||||
env:
|
||||
REA_TSHARK_COMMAND: /usr/bin/tshark
|
||||
REA_CAPINFOS_COMMAND: /usr/bin/capinfos
|
||||
@@ -0,0 +1,50 @@
|
||||
name: Real signal recording
|
||||
on:
|
||||
workflow_dispatch:
|
||||
pull_request:
|
||||
paths:
|
||||
- "tests/boundary/signals/**"
|
||||
- "tests/boundary/cli/signalCleanup.test.ts"
|
||||
- "tests/boundary/mcp/signalRecordingMcp.test.ts"
|
||||
- "src/signals/**"
|
||||
- "src/domain/signals/**"
|
||||
- "src/application/signals/**"
|
||||
- "src/contracts/signals/**"
|
||||
- "src/composition/signals.ts"
|
||||
- "src/cli/signalCommands.ts"
|
||||
- "src/server/registerSignalTools.ts"
|
||||
- "src/server/createServer.ts"
|
||||
- "src/server/sessionAvailabilityPolicy.ts"
|
||||
- "src/application/CapabilityInventory.ts"
|
||||
- "src/contracts/toolContracts.ts"
|
||||
- "src/contracts/toolEffects.ts"
|
||||
- "src/cli.ts"
|
||||
- "src/cliCommandNames.ts"
|
||||
- "src/process/**"
|
||||
- "src/artifacts/readStableArtifact.ts"
|
||||
- "scripts/verify-signal-recording.mjs"
|
||||
- ".github/workflows/real-signal-recording.yml"
|
||||
- "package*.json"
|
||||
- ".nvmrc"
|
||||
permissions:
|
||||
contents: read
|
||||
concurrency:
|
||||
group: real-signal-recording-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
jobs:
|
||||
sigrok:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- run: sudo apt-get update && sudo apt-get install -y sigrok-cli util-linux python3
|
||||
- run: npm ci
|
||||
- run: npm run build:cached
|
||||
- run: npm run test:focused -- tests/boundary/signals/decoderDiscovery.test.ts src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts
|
||||
- run: node scripts/verify-signal-recording.mjs
|
||||
env:
|
||||
REA_SIGROK_COMMAND: /usr/bin/sigrok-cli
|
||||
@@ -0,0 +1,37 @@
|
||||
name: SQLite runtime compatibility
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
|
||||
jobs:
|
||||
sqlite:
|
||||
name: SQLite runtime compatibility (Node ${{ matrix.node }})
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
# Exercise unavailable-runtime diagnostics, the first supported release,
|
||||
# and new patches on both supported lines without hand-editing versions.
|
||||
# The repository-pinned Node 24 version also runs the complete test suite.
|
||||
node: ["22.x", "24.15.0", "24.x", "26.x"]
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: ${{ matrix.node }}
|
||||
check-latest: true
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: rea-test-runtime
|
||||
path: .
|
||||
- name: Verify SQLite discovery, inspection and cleanup
|
||||
run: npm run test:sqlite:run
|
||||
@@ -72,3 +72,5 @@ Thumbs.db
|
||||
|
||||
# Complete static website output, including generated language editions.
|
||||
website/_site/
|
||||
# Temporary translation output prepared before replacing the complete site.
|
||||
website/.i18n-*/
|
||||
|
||||
+1
-2
@@ -1,6 +1,5 @@
|
||||
{
|
||||
"_comment": "Test files remain included so the gate measures duplication across both src/ and tests/. The 3.0% limit leaves headroom for ordinary changes while keeping duplication visible.",
|
||||
"threshold": 3.0,
|
||||
"threshold": 3.5,
|
||||
"reporters": ["consoleFull"],
|
||||
"ignore": [
|
||||
"node_modules/**",
|
||||
|
||||
@@ -24,6 +24,8 @@ Assess brittleness in touched code during implementation, debugging, review, and
|
||||
|
||||
When fixing a bug, inspect the producing boundary and affected consumers for duplicated interpretation or unclear ownership. Complete any directly related refactor in the same change and remove superseded code. Passing regression tests alone does not establish that the brittleness is resolved. If related cleanup remains, explain what remains and why.
|
||||
|
||||
When addressing a review thread, inspect the producing boundary and affected consumers for brittleness. Complete directly related refactoring in the same change. Before resolving the thread, explain the root cause, structural fix, validation, and any remaining cleanup.
|
||||
|
||||
Parse unknown boundary values and model expected failures with the tagged error algebra and `Result`. Preserve partial facts and meaningful failure reasons. Keep identity and lookup values separate from display formatting; preserve source values and explicit unknowns when normalization loses information. Absolute paths, file URLs, and HTTP paths have different semantics.
|
||||
|
||||
Validate advertised JSON Schemas after SDK conversion against their declared dialect and actual clients. Bind handlers to named contracts with exact types; catalog ordering must never select an operation or schema.
|
||||
|
||||
@@ -157,10 +157,11 @@ Additional tools and host support depend on the target:
|
||||
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
|
||||
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/evm-bytecode.md) |
|
||||
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
|
||||
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/recorded-crashes.md) |
|
||||
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; [application analysis](https://rea.tools/guides/javascript/) |
|
||||
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](https://rea.tools/guides/browser/) |
|
||||
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR; mitmdump on Linux for native mitmproxy captures; [capture guide](docs/web-network-captures.md) |
|
||||
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/web-network-captures.md) |
|
||||
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/managed-code-analysis.md) |
|
||||
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/android-analysis.md) |
|
||||
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/adb-device-analysis.md) |
|
||||
@@ -168,6 +169,9 @@ Additional tools and host support depend on the target:
|
||||
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/apktool-resource-analysis.md) |
|
||||
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/flutter-analysis.md) |
|
||||
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/firmware-analysis.md) |
|
||||
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/packet-captures.md) |
|
||||
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/pcap-analysis.md) |
|
||||
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/signal-recordings.md) |
|
||||
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/javascript-artifact-reconstruction.md), [Apple applications](docs/apple-application-analysis.md) |
|
||||
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/process-capture.md) |
|
||||
|
||||
@@ -179,7 +183,7 @@ using your user permissions; each runtime guide describes its effects.
|
||||
|
||||
Native formats and host support vary by provider. See
|
||||
[Hopper and Ghidra setup](docs/installation.md#hopper), the
|
||||
[IDA guide](docs/ida-provider.md), and
|
||||
[IDA guide](docs/ida-provider.md) (including an optional native MCP bridge), and
|
||||
[experimental Windows Ghidra support](docs/windows-ghidra-p0.md).
|
||||
Ghidra also supports [16-bit DOS analysis](docs/ghidra-dos.md).
|
||||
For large binaries, raise the provider's startup deadline with
|
||||
@@ -318,7 +322,7 @@ Report vulnerabilities through [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Star history
|
||||
|
||||
🎉 **110,000 GitHub stars — thank you!**
|
||||
🎉 **170,000 GitHub stars — thank you!**
|
||||
|
||||
Thanks to everyone using REA, reporting bugs, requesting features, testing builds, and contributing fixes.
|
||||
|
||||
|
||||
+4
-1
@@ -139,6 +139,7 @@ npx rea-agents@latest setup
|
||||
| JavaScript / Electron | الوحدات، وعمليات الاستيراد، وخرائط المصدر، والمسارات، وIPC، وعلاقات إضافات الشيفرة الأصلية | Node.js وnpm؛ [تحليل التطبيقات](https://rea.tools/ar/guides/javascript/) |
|
||||
| المواقع | بنية الصفحة، والسكربتات، وملاحظات الشبكة، ولقطات الشاشة المطلوبة | متصفح من عائلة Chrome؛ [تحليل المتصفح](https://rea.tools/ar/guides/browser/) |
|
||||
| تسجيلات الشبكة المحفوظة | الطلبات، والاستجابات، ومحتويات الحمولة المتاحة، ومواضعها في المصدر | HAR؛ وmitmdump على Linux لتسجيلات mitmproxy بصيغتها الأصلية؛ [دليل تسجيلات الشبكة](docs/web-network-captures.md) |
|
||||
| التقاط الحزم | نقاط النهاية وDNS وTLS SNI وتوزيع البروتوكولات وطلبات HTTP | tshark وcapinfos يوفّرهما المُستدعي؛ [دليل pcap](docs/pcap-analysis.md) |
|
||||
| تجميعات .NET | البيانات الوصفية، وتعليمات CIL، واعتماديات الشيفرة الأصلية المعلنة، ومقارنات البناء | فحص ثابت؛ [دليل الشيفرة المُدارة](docs/managed-code-analysis.md) |
|
||||
| حزم Android APK | تصريحات ملف manifest، والأصناف، والدوال الناتجة عن فك الترجمة، والمراجع | JADX دون واجهة رسومية وJDK كامل على Linux/macOS/Windows x64؛ [دليل Android](docs/android-analysis.md) |
|
||||
| أجهزة Android | الأجهزة والحزم والعمليات والسجلات والدامبات ولقطات الشاشة ونقل APK/الملفات | adb يوفّره المُستدعي (محاكي أو جهاز)؛ [دليل ADB](docs/adb-device-analysis.md) |
|
||||
@@ -146,6 +147,8 @@ npx rea-agents@latest setup
|
||||
| موارد Android | البيان المفكوك الترميز وجداول السلاسل واللغات وحقائق الإصدار | مُشغّل Apktool يوفّره المُستدعي؛ [دليل الموارد](docs/apktool-resource-analysis.md) |
|
||||
| تطبيقات Flutter | بصمة لقطة Dart وهشوش المكتبات لكل ABI وأصل بناء المحرك | لا شيء (تحليل APK ساكن)؛ [دليل Flutter](docs/flutter-analysis.md) |
|
||||
| البرامج الثابتة | المناطق، ونتائج الاستخراج، وما يُمرّر إلى تحليل الشيفرة الأصلية | Binwalk / Unblob على Linux؛ [دليل البرامج الثابتة](docs/firmware-analysis.md) |
|
||||
| الحزم المسجلة | حقول مختارة وقيم متكررة ومرشحات وقواعد فك الترميز | Linux: TShark 4.x + prlimit; [دليل التقاط الحزم](docs/packet-captures.md) |
|
||||
| الإشارات المنطقية المسجلة | تعليقات مفكك الترميز ونطاقات العينات ومعلومات التشخيص | Linux: sigrok-cli 0.7.2 + prlimit; [دليل الإشارات](docs/signal-recordings.md) |
|
||||
| الحزم والموارد | قوائم الملفات، والبصمات، وملفات plist، وبنية حزم Apple، والموارد المستخرجة | [دليل تحليل الملفات وJavaScript](docs/javascript-artifact-reconstruction.md)، [تطبيقات Apple](docs/apple-application-analysis.md) |
|
||||
| سلوك العمليات | مخرجات الطرفية، والتفاعلات، وملاحظات الخروج ونظام الملفات، ومقارنات التشغيل | Linux/macOS مع PTY أصلي؛ [تسجيل سلوك العمليات](docs/process-capture.md) |
|
||||
|
||||
@@ -261,7 +264,7 @@ npx rea-agents@latest setup
|
||||
|
||||
## سجل النجوم
|
||||
|
||||
🎉 **110,000 نجمة على GitHub — شكرًا لكم!**
|
||||
🎉 **170,000 نجمة على GitHub — شكرًا لكم!**
|
||||
|
||||
شكرًا لكل من يستخدم REA، ويبلّغ عن الأخطاء، ويقترح ميزات، ويختبر إصدارات البناء، ويساهم بالإصلاحات.
|
||||
|
||||
|
||||
+4
-1
@@ -139,6 +139,7 @@ REA benötigt Node.js 22.x (>=22.19), 24.x (>=24.11) oder 26+ sowie npm. Zusätz
|
||||
| JavaScript / Electron | Module, Imports, Source Maps, Routen, IPC und Beziehungen zu nativen Add-ons | Node.js und npm; [Anwendungsanalyse](https://rea.tools/de/guides/javascript/) |
|
||||
| Websites | Seitenstruktur, Skripte, Netzwerkbeobachtungen und angeforderte Screenshots | Ein Browser aus der Chrome-Familie; [Browseranalyse](https://rea.tools/de/guides/browser/) |
|
||||
| Gespeicherte Netzwerkaufzeichnungen | Anfragen, Antworten, zugängliche Nutzdaten und Quellpositionen | HAR; mitmdump unter Linux für Aufzeichnungen im nativen mitmproxy-Format; [Leitfaden zu Netzwerkaufzeichnungen](docs/web-network-captures.md) |
|
||||
| Paketerfassungen | Endpunkte, DNS, TLS SNI, Protokollverteilung und HTTP-Anfragen | Vom Aufrufer bereitgestellte tshark- und capinfos-Binaries; [pcap-Leitfaden](docs/pcap-analysis.md) |
|
||||
| .NET-Assemblies | Metadaten, CIL-Anweisungen, deklarierte native Abhängigkeiten und Build-Vergleiche | Statische Untersuchung; [Leitfaden für verwalteten Code](docs/managed-code-analysis.md) |
|
||||
| Android-APKs | Manifest-Deklarationen, Klassen, dekompilierte Methoden und Verweise | JADX ohne grafische Oberfläche und ein vollständiges JDK unter Linux/macOS/Windows x64; [Android-Leitfaden](docs/android-analysis.md) |
|
||||
| Android-Geräte | Geräte, Pakete, Prozesse, Logs, Dumps, Bildschirmaufnahmen sowie APK- und Dateiübertragungen | Vom Aufrufer bereitgestelltes adb (Emulator oder Gerät); [ADB-Leitfaden](docs/adb-device-analysis.md) |
|
||||
@@ -146,6 +147,8 @@ REA benötigt Node.js 22.x (>=22.19), 24.x (>=24.11) oder 26+ sowie npm. Zusätz
|
||||
| Android-Ressourcen | Dekodiertes Manifest, String-Tabellen, Sprachen und Versionsfakten | Vom Aufrufer bereitgestellter Apktool-Starter; [Ressourcen-Leitfaden](docs/apktool-resource-analysis.md) |
|
||||
| Flutter-Apps | Dart-Snapshot-Hash, Per-ABI-Bibliotheksprüfsummen und Engine-Build-Provenienz | Keine (statische APK-Analyse); [Flutter-Leitfaden](docs/flutter-analysis.md) |
|
||||
| Firmware | Bereiche, Extraktionsergebnisse und Übergaben an die native Analyse | Binwalk / Unblob unter Linux; [Firmware-Leitfaden](docs/firmware-analysis.md) |
|
||||
| Aufgezeichnete Pakete | Ausgewählte Felder, wiederholte Werte, Filter und Decode-as-Regeln | Linux: TShark 4.x + prlimit; [Mitschnitt-Anleitung](docs/packet-captures.md) |
|
||||
| Aufgezeichnete Logiksignale | Decoder-Anmerkungen, Sample-Bereiche und Diagnosen | Linux: sigrok-cli 0.7.2 + prlimit; [Signalaufzeichnungen](docs/signal-recordings.md) |
|
||||
| Pakete und Ressourcen | Dateiinventare, Hashwerte, plists, Aufbau von Apple-Bundles und extrahierte Ressourcen | [Artefakt- und JavaScript-Leitfaden](docs/javascript-artifact-reconstruction.md), [Apple-Anwendungen](docs/apple-application-analysis.md) |
|
||||
| Prozessverhalten | Terminalausgabe, Interaktionen, Beobachtungen zu Prozessende und Dateisystem sowie Laufvergleiche | Linux/macOS mit nativer PTY; [Prozessaufzeichnung](docs/process-capture.md) |
|
||||
|
||||
@@ -259,7 +262,7 @@ Melde Sicherheitslücken gemäß [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Sternverlauf
|
||||
|
||||
🎉 **110.000 GitHub-Sterne – vielen Dank!**
|
||||
🎉 **170.000 GitHub-Sterne – vielen Dank!**
|
||||
|
||||
Danke an alle, die REA nutzen, Fehler melden, Funktionswünsche äußern, Builds testen und Fehlerbehebungen beitragen.
|
||||
|
||||
|
||||
+4
-1
@@ -139,6 +139,7 @@ REA requiere Node.js 22.x (>=22.19), 24.x (>=24.11) o 26+, además de npm. Las h
|
||||
| JavaScript / Electron | Módulos, importaciones, mapas de código fuente, rutas, IPC y relaciones con complementos nativos | Node.js y npm; [análisis de aplicaciones](https://rea.tools/es/guides/javascript/) |
|
||||
| Sitios web | Estructura de la página, scripts, observaciones de red y capturas de pantalla solicitadas | Un navegador de la familia Chrome; [análisis del navegador](https://rea.tools/es/guides/browser/) |
|
||||
| Capturas de red guardadas | Solicitudes, respuestas, contenido accesible de las cargas útiles y ubicaciones de origen | HAR; mitmdump en Linux para capturas en formato nativo de mitmproxy; [guía de capturas](docs/web-network-captures.md) |
|
||||
| Capturas de paquetes | Endpoints, DNS, TLS SNI, distribución de protocolos y solicitudes HTTP | tshark y capinfos suministrados por el autor de la llamada; [guía pcap](docs/pcap-analysis.md) |
|
||||
| Ensamblados .NET | Metadatos, instrucciones CIL, dependencias nativas declaradas y comparaciones de compilaciones | Inspección estática; [guía de código administrado](docs/managed-code-analysis.md) |
|
||||
| APK de Android | Declaraciones del manifiesto, clases, métodos descompilados y referencias | JADX sin interfaz gráfica y un JDK completo en Linux/macOS/Windows x64; [guía de Android](docs/android-analysis.md) |
|
||||
| Dispositivos Android | Dispositivos, paquetes, procesos, registros, volcados, capturas de pantalla y transferencias de APK/archivos | adb suministrado por el autor de la llamada (emulador o dispositivo); [guía de ADB](docs/adb-device-analysis.md) |
|
||||
@@ -146,6 +147,8 @@ REA requiere Node.js 22.x (>=22.19), 24.x (>=24.11) o 26+, además de npm. Las h
|
||||
| Recursos de Android | Manifiesto decodificado, tablas de cadenas, idiomas y datos de versión | Lanzador de Apktool suministrado por el autor de la llamada; [guía de recursos](docs/apktool-resource-analysis.md) |
|
||||
| Aplicaciones Flutter | Hash de snapshot de Dart, resúmenes de bibliotecas por ABI y procedencia de compilación del motor | Ninguna (análisis APK estático); [guía de Flutter](docs/flutter-analysis.md) |
|
||||
| Firmware | Regiones, resultados de extracción y derivaciones al análisis nativo | Binwalk / Unblob en Linux; [guía de firmware](docs/firmware-analysis.md) |
|
||||
| Paquetes grabados | Campos seleccionados, valores repetidos, filtros y reglas de decodificación | Linux: TShark 4.x + prlimit; [guía de capturas](docs/packet-captures.md) |
|
||||
| Señales lógicas grabadas | Anotaciones del decodificador, rangos de muestras y diagnósticos | Linux: sigrok-cli 0.7.2 + prlimit; [guía de señales](docs/signal-recordings.md) |
|
||||
| Paquetes y recursos | Inventarios de archivos, resúmenes criptográficos, plists, estructura de paquetes de Apple y recursos extraídos | [Guía de artefactos y JavaScript](docs/javascript-artifact-reconstruction.md), [aplicaciones de Apple](docs/apple-application-analysis.md) |
|
||||
| Comportamiento de procesos | Salida de la terminal, interacciones, observaciones de salida y del sistema de archivos, y comparaciones entre ejecuciones | Linux/macOS con una PTY nativa; [captura de procesos](docs/process-capture.md) |
|
||||
|
||||
@@ -259,7 +262,7 @@ Informa de vulnerabilidades siguiendo [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Historial de estrellas
|
||||
|
||||
🎉 **110.000 estrellas en GitHub: ¡gracias!**
|
||||
🎉 **170.000 estrellas en GitHub: ¡gracias!**
|
||||
|
||||
Gracias a quienes usan REA, informan de errores, solicitan funciones, prueban compilaciones y contribuyen con correcciones.
|
||||
|
||||
|
||||
+22
-19
@@ -122,24 +122,27 @@ npx rea-agents@latest setup
|
||||
|
||||
REA به Node.js 22.x (>=22.19)، 24.x (>=24.11) یا 26+، بههمراه npm نیاز دارد. ابزارهای اضافی و پشتیبانی سیستم میزبان به نوع هدف بستگی دارند:
|
||||
|
||||
| هدف | خروجی REA | پیشنیازها و راهنما |
|
||||
| --------------------- | ----------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| باینریهای بومی | شبهکد، اسمبلی، رشتهها، نمادها، فراخوانیها و ارجاعات | Hopper، Ghidra یا IDA؛ [تحلیل بومی](https://rea.tools/fa/guides/native/) |
|
||||
| ساختار ELF آفلاین | بخشها، سگمنتها، نمادها و relocationهای اصلی و گزینههای احتمالی کاهش آسیبپذیری ایستا | pwntools تأمینشده توسط فراخواننده روی Linux x64؛ [تشخیص باینری](docs/binary-diagnostics.md) |
|
||||
| بایتکد EVM | selectorهای dispatch، offsetهای بایتی، آرگومانهای استنباطشده و قابلیت تغییر وضعیت | دادهٔ خام/hex محلی روی Linux x64؛ [راهنمای EVM آفلاین](docs/evm-bytecode.md) |
|
||||
| کرشهای ثبتشدهٔ لینوکس | یادداشتهای خام، رجیسترها/سیگنالهای تمام threadهای ثبتشده و گزینههای احتمالی نگاشت | pwntools تأمینشده توسط فراخواننده روی Linux x64؛ GDB/pwndbg اختیاری؛ [کرشهای ثبتشده](docs/recorded-crashes.md) |
|
||||
| JavaScript / Electron | ماژولها، importها، source mapها، مسیرها، IPC و ارتباط افزونههای بومی | Node.js و npm؛ [تحلیل برنامه](https://rea.tools/fa/guides/javascript/) |
|
||||
| وبسایتها | ساختار صفحه، اسکریپتها، مشاهدات شبکه و اسکرینشاتهای درخواستی | مرورگر مبتنی بر Chrome؛ [تحلیل مرورگر](https://rea.tools/fa/guides/browser/) |
|
||||
| دادههای ضبطشدهٔ شبکه | درخواستها، پاسخها، payloadهای آشکارشده و مکان آنها در منبع | HAR؛ ابزار mitmdump در لینوکس برای ضبطهای بومی mitmproxy؛ [راهنمای ضبط شبکه](docs/web-network-captures.md) |
|
||||
| اسمبلیهای .NET | فراداده، دستورالعملهای CIL، وابستگیهای بومی اعلامشده و مقایسهٔ buildها | بررسی ایستا؛ [راهنمای کد مدیریتشده](docs/managed-code-analysis.md) |
|
||||
| فایلهای APK اندروید | تعاریف Manifest، کلاسها، متدهای دیکامپایلشده و ارجاعات | JADX بدون رابط گرافیکی و JDK کامل در Linux/macOS/Windows x64؛ [راهنمای اندروید](docs/android-analysis.md) |
|
||||
| دستگاههای اندروید | دستگاهها، بستهها، فرایندها، لاگها، دامپها، اسکرینشاتها و انتقال APK/فایل | adb تأمینشده توسط فراخوانکننده (شبیهساز یا دستگاه)؛ [راهنمای ADB](docs/adb-device-analysis.md) |
|
||||
| پروژههای JEB | واحدهای پروژه از موتور، هشهای مصنوع و شبهکدِ نوع/متد | یک کلاینت JEB که توسط فراخوانکننده اجرا شده و MCP ارائه میدهد و با REA_JEB_MCP_URL انتخاب میشود؛ [راهنمای JEB](docs/jeb-analysis.md) |
|
||||
| منابع اندروید | مانیفست رمزگشاییشده، جداول رشتهها، زبانها و اطلاعات نسخه | راهانداز Apktool تأمینشده توسط فراخوانکننده؛ [راهنمای منابع](docs/apktool-resource-analysis.md) |
|
||||
| برنامههای فلاتر | هش اسنپشات دارت، هش کتابخانهها به تفکیک ABI و اصالت بیلد موتور | هیچ (تحلیل ایستا APK)؛ [راهنمای فلاتر](docs/flutter-analysis.md) |
|
||||
| Firmware | ناحیهها، نتایج استخراج و تحویل به ابزارهای تحلیل بومی | Binwalk / Unblob در لینوکس؛ [راهنمای Firmware](docs/firmware-analysis.md) |
|
||||
| بستهها و منابع | فهرست فایلها، digestها، فایلهای plist، ساختار bundle اپل و منابع استخراجشده | [راهنمای آرتیفکت و JavaScript](docs/javascript-artifact-reconstruction.md)، [برنامههای اپل](docs/apple-application-analysis.md) |
|
||||
| رفتار فرایند | خروجی ترمینال، تعاملات، وضعیت خروج، مشاهدات سیستم فایل و مقایسهٔ اجراها | Linux/macOS با PTY بومی؛ [ضبط فرایند](docs/process-capture.md) |
|
||||
| هدف | خروجی REA | پیشنیازها و راهنما |
|
||||
| ---------------------- | ----------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| باینریهای بومی | شبهکد، اسمبلی، رشتهها، نمادها، فراخوانیها و ارجاعات | Hopper، Ghidra یا IDA؛ [تحلیل بومی](https://rea.tools/fa/guides/native/) |
|
||||
| ساختار ELF آفلاین | بخشها، سگمنتها، نمادها و relocationهای اصلی و گزینههای احتمالی کاهش آسیبپذیری ایستا | pwntools تأمینشده توسط فراخواننده روی Linux x64؛ [تشخیص باینری](docs/binary-diagnostics.md) |
|
||||
| بایتکد EVM | selectorهای dispatch، offsetهای بایتی، آرگومانهای استنباطشده و قابلیت تغییر وضعیت | دادهٔ خام/hex محلی روی Linux x64؛ [راهنمای EVM آفلاین](docs/evm-bytecode.md) |
|
||||
| کرشهای ثبتشدهٔ لینوکس | یادداشتهای خام، رجیسترها/سیگنالهای تمام threadهای ثبتشده و گزینههای احتمالی نگاشت | pwntools تأمینشده توسط فراخواننده روی Linux x64؛ GDB/pwndbg اختیاری؛ [کرشهای ثبتشده](docs/recorded-crashes.md) |
|
||||
| JavaScript / Electron | ماژولها، importها، source mapها، مسیرها، IPC و ارتباط افزونههای بومی | Node.js و npm؛ [تحلیل برنامه](https://rea.tools/fa/guides/javascript/) |
|
||||
| وبسایتها | ساختار صفحه، اسکریپتها، مشاهدات شبکه و اسکرینشاتهای درخواستی | مرورگر مبتنی بر Chrome؛ [تحلیل مرورگر](https://rea.tools/fa/guides/browser/) |
|
||||
| دادههای ضبطشدهٔ شبکه | درخواستها، پاسخها، payloadهای آشکارشده و مکان آنها در منبع | HAR؛ ابزار mitmdump در لینوکس برای ضبطهای بومی mitmproxy؛ [راهنمای ضبط شبکه](docs/web-network-captures.md) |
|
||||
| اسمبلیهای .NET | فراداده، دستورالعملهای CIL، وابستگیهای بومی اعلامشده و مقایسهٔ buildها | بررسی ایستا؛ [راهنمای کد مدیریتشده](docs/managed-code-analysis.md) |
|
||||
| فایلهای APK اندروید | تعاریف Manifest، کلاسها، متدهای دیکامپایلشده و ارجاعات | JADX بدون رابط گرافیکی و JDK کامل در Linux/macOS/Windows x64؛ [راهنمای اندروید](docs/android-analysis.md) |
|
||||
| دستگاههای اندروید | دستگاهها، بستهها، فرایندها، لاگها، دامپها، اسکرینشاتها و انتقال APK/فایل | adb تأمینشده توسط فراخوانکننده (شبیهساز یا دستگاه)؛ [راهنمای ADB](docs/adb-device-analysis.md) |
|
||||
| پروژههای JEB | واحدهای پروژه از موتور، هشهای مصنوع و شبهکدِ نوع/متد | یک کلاینت JEB که توسط فراخوانکننده اجرا شده و MCP ارائه میدهد و با REA_JEB_MCP_URL انتخاب میشود؛ [راهنمای JEB](docs/jeb-analysis.md) |
|
||||
| منابع اندروید | مانیفست رمزگشاییشده، جداول رشتهها، زبانها و اطلاعات نسخه | راهانداز Apktool تأمینشده توسط فراخوانکننده؛ [راهنمای منابع](docs/apktool-resource-analysis.md) |
|
||||
| برنامههای فلاتر | هش اسنپشات دارت، هش کتابخانهها به تفکیک ABI و اصالت بیلد موتور | هیچ (تحلیل ایستا APK)؛ [راهنمای فلاتر](docs/flutter-analysis.md) |
|
||||
| Firmware | ناحیهها، نتایج استخراج و تحویل به ابزارهای تحلیل بومی | Binwalk / Unblob در لینوکس؛ [راهنمای Firmware](docs/firmware-analysis.md) |
|
||||
| بستههای ضبطشده | فیلدهای منتخب، مقادیر تکراری، فیلترها و قواعد رمزگشایی | Linux: TShark 4.x + prlimit; [راهنمای ضبط بسته](docs/packet-captures.md) |
|
||||
| ضبط بستهها | نقاط پایانی، DNS، TLS SNI، توزیع پروتکل و درخواستهای HTTP | tshark و capinfos تأمینشده توسط فراخوانکننده؛ [راهنمای pcap](docs/pcap-analysis.md) |
|
||||
| سیگنالهای منطقی ضبطشده | توضیحات رمزگشا، بازههای نمونه و اطلاعات عیبیابی | Linux: sigrok-cli 0.7.2 + prlimit; [راهنمای سیگنالها](docs/signal-recordings.md) |
|
||||
| بستهها و منابع | فهرست فایلها، digestها، فایلهای plist، ساختار bundle اپل و منابع استخراجشده | [راهنمای آرتیفکت و JavaScript](docs/javascript-artifact-reconstruction.md)، [برنامههای اپل](docs/apple-application-analysis.md) |
|
||||
| رفتار فرایند | خروجی ترمینال، تعاملات، وضعیت خروج، مشاهدات سیستم فایل و مقایسهٔ اجراها | Linux/macOS با PTY بومی؛ [ضبط فرایند](docs/process-capture.md) |
|
||||
|
||||
بررسی ایستای JavaScript و .NET فایلهای ارائهشده را بدون اجرای برنامه میخواند. ضبط زمان اجرا، هدف انتخابشده را با مجوزهای کاربری شما اجرا میکند یا با آن تعامل دارد؛ آثار این عملیات در راهنمای مربوط به هر نوع ضبط توضیح داده شدهاند.
|
||||
|
||||
@@ -244,7 +247,7 @@ npx rea-agents@latest setup
|
||||
|
||||
## تاریخچهٔ ستارهها
|
||||
|
||||
🎉 **۱۱۰٬۰۰۰ ستارهٔ GitHub — سپاسگزاریم!**
|
||||
🎉 **۱۷۰٬۰۰۰ ستارهٔ GitHub — سپاسگزاریم!**
|
||||
|
||||
از همهٔ کسانی که از REA استفاده میکنند، خطاها را گزارش میدهند، قابلیتهای جدید پیشنهاد میکنند، buildها را آزمایش میکنند و در بهبود پروژه مشارکت دارند سپاسگزاریم.
|
||||
|
||||
|
||||
+4
-1
@@ -143,6 +143,7 @@ REA nécessite Node.js 22.x (>=22.19), 24.x (>=24.11) ou 26+, ainsi que npm. Les
|
||||
| JavaScript / Electron | Modules, imports, source maps, routes, IPC et relations avec les modules natifs | Node.js et npm ; [analyse d'applications](https://rea.tools/fr/guides/javascript/) |
|
||||
| Sites web | Structure des pages, scripts, observations réseau et captures d'écran demandées | Un navigateur de la famille Chrome ; [analyse du navigateur](https://rea.tools/fr/guides/browser/) |
|
||||
| Captures réseau enregistrées | Requêtes, réponses, payloads exposés et emplacements dans le code source | HAR ; mitmdump sous Linux pour les captures mitmproxy natives ; [guide des captures](docs/web-network-captures.md) |
|
||||
| Captures de paquets | Points de terminaison, DNS, TLS SNI, distribution des protocoles et requêtes HTTP | tshark et capinfos fournis par l'appelant ; [guide pcap](docs/pcap-analysis.md) |
|
||||
| Assemblys .NET | Métadonnées, instructions CIL, dépendances natives déclarées et comparaisons de builds | Inspection statique ; [guide du code managé](docs/managed-code-analysis.md) |
|
||||
| APK Android | Déclarations du manifeste, classes, méthodes décompilées et références | JADX headless et un JDK complet sous Linux/macOS/Windows x64 ; [guide Android](docs/android-analysis.md) |
|
||||
| Appareils Android | Appareils, paquets, processus, journaux, dumps, captures d'écran et transferts d'APK/fichiers | adb fourni par l'appelant (émulateur ou appareil) ; [guide ADB](docs/adb-device-analysis.md) |
|
||||
@@ -150,6 +151,8 @@ REA nécessite Node.js 22.x (>=22.19), 24.x (>=24.11) ou 26+, ainsi que npm. Les
|
||||
| Ressources Android | Manifeste décodé, tables de chaînes, langues et faits de version | Lanceur Apktool fourni par l'appelant ; [guide des ressources](docs/apktool-resource-analysis.md) |
|
||||
| Applications Flutter | Empreinte de snapshot Dart, condensés de bibliothèques par ABI et provenance de build du moteur | Aucune (analyse APK statique) ; [guide Flutter](docs/flutter-analysis.md) |
|
||||
| Firmware | Régions, résultats d'extraction et passage de relais vers l'analyse native | Binwalk / Unblob sous Linux ; [guide firmware](docs/firmware-analysis.md) |
|
||||
| Paquets enregistrés | Champs sélectionnés, valeurs répétées, filtres et règles de décodage | Linux: TShark 4.x + prlimit; [guide des captures](docs/packet-captures.md) |
|
||||
| Signaux logiques enregistrés | Annotations du décodeur, plages d’échantillons et diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [guide des signaux](docs/signal-recordings.md) |
|
||||
| Paquets et ressources | Inventaires de fichiers, empreintes, plists, anatomie des bundles Apple et ressources extraites | [Guide artefacts et JavaScript](docs/javascript-artifact-reconstruction.md), [applications Apple](docs/apple-application-analysis.md) |
|
||||
| Comportement des processus | Sortie du terminal, interactions, code de sortie, observations du système de fichiers et comparaisons | Linux/macOS avec un PTY natif ; [capture de processus](docs/process-capture.md) |
|
||||
|
||||
@@ -263,7 +266,7 @@ Signalez les vulnérabilités via [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Historique des étoiles
|
||||
|
||||
🎉 **110 000 étoiles GitHub — merci !**
|
||||
🎉 **170 000 étoiles GitHub — merci !**
|
||||
|
||||
Merci à toutes les personnes qui utilisent REA, signalent des bugs, proposent des fonctionnalités, testent les builds et contribuent des correctifs.
|
||||
|
||||
|
||||
+4
-1
@@ -146,6 +146,7 @@ Alat tambahan dan dukungan sistem host bergantung pada target yang dianalisis:
|
||||
| JavaScript / Electron | Modul, impor, source map, rute, IPC, dan hubungan add-on native | Node.js dan npm; [analisis aplikasi](https://rea.tools/id/guides/javascript/) |
|
||||
| Situs web | Struktur halaman, skrip, pengamatan jaringan, dan tangkapan layar yang diminta | Peramban berbasis Chrome; [analisis peramban](https://rea.tools/id/guides/browser/) |
|
||||
| Rekaman jaringan tersimpan | Permintaan, respons, payload yang terekspos, dan lokasi sumber | HAR; mitmdump di Linux untuk rekaman native mitmproxy; [panduan rekaman](docs/web-network-captures.md) |
|
||||
| Capturan paket | Endpoint, DNS, TLS SNI, distribusi protokol, dan permintaan HTTP | tshark dan capinfos buatan pemanggil; [panduan pcap](docs/pcap-analysis.md) |
|
||||
| Assembly .NET | Metadata, instruksi CIL, dependensi native yang dideklarasikan, dan perbandingan build | Tidak ada (hanya analisis statis); [panduan kode terkelola](docs/managed-code-analysis.md) |
|
||||
| APK Android | Deklarasi manifest, kelas, metode hasil dekompilasi, dan referensi | JADX mode headless dan JDK lengkap di Linux/macOS/Windows x64; [panduan Android](docs/android-analysis.md) |
|
||||
| Perangkat Android | Perangkat, paket, proses, log, dump, tangkapan layar, serta transfer APK/berkas | adb buatan pemanggil (emulator atau perangkat); [panduan ADB](docs/adb-device-analysis.md) |
|
||||
@@ -153,6 +154,8 @@ Alat tambahan dan dukungan sistem host bergantung pada target yang dianalisis:
|
||||
| Resource Android | Manifest terdekode, tabel string, lokal, dan fakta versi | Peluncur Apktool buatan pemanggil; [panduan resource](docs/apktool-resource-analysis.md) |
|
||||
| Aplikasi Flutter | Hash snapshot Dart, hash pustaka per ABI, dan asal build engine | Tidak ada (parsing APK statis); [panduan Flutter](docs/flutter-analysis.md) |
|
||||
| Firmware | Wilayah, hasil ekstraksi, dan pengalihan ke analisis native | Binwalk / Unblob di Linux; [panduan firmware](docs/firmware-analysis.md) |
|
||||
| Paket rekaman | Kolom terpilih, nilai berulang, filter, dan aturan decode-as | Linux: TShark 4.x + prlimit; [panduan tangkapan](docs/packet-captures.md) |
|
||||
| Sinyal logika terekam | Anotasi dekoder, rentang sampel, dan diagnostik | Linux: sigrok-cli 0.7.2 + prlimit; [panduan sinyal](docs/signal-recordings.md) |
|
||||
| Paket dan sumber daya | Inventaris berkas, digest, plist, struktur paket Apple, dan sumber daya yang diekstrak | [Panduan artefak dan JavaScript](docs/javascript-artifact-reconstruction.md), [aplikasi Apple](docs/apple-application-analysis.md) |
|
||||
| Perilaku proses | Keluaran terminal, interaksi, pengamatan proses keluar dan sistem berkas, serta perbandingan eksekusi | Linux/macOS dengan PTY native; [perekaman proses](docs/process-capture.md) |
|
||||
|
||||
@@ -284,7 +287,7 @@ Laporkan kerentanan melalui [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Riwayat Bintang
|
||||
|
||||
🎉 **110.000 bintang GitHub, terima kasih!**
|
||||
🎉 **170.000 bintang GitHub, terima kasih!**
|
||||
|
||||
Terima kasih kepada semua orang yang menggunakan REA, melaporkan bug, meminta fitur, menguji build, dan berkontribusi dalam perbaikan.
|
||||
|
||||
|
||||
+4
-1
@@ -143,6 +143,7 @@ REA richiede Node.js 22.x (>=22.19), 24.x (>=24.11) o 26+, oltre a npm. Strument
|
||||
| JavaScript / Electron | Moduli, import, source map, route, IPC e relazioni con add-on nativi | Node.js e npm; [analisi delle applicazioni](https://rea.tools/it/guides/javascript/) |
|
||||
| Siti web | Struttura della pagina, script, osservazioni di rete e screenshot richiesti | Un browser della famiglia Chrome; [analisi browser](https://rea.tools/it/guides/browser/) |
|
||||
| Acquisizioni di rete salvate | Richieste, risposte, payload esposti e posizioni sorgente | HAR; mitmdump su Linux per acquisizioni mitmproxy native; [guida alle acquisizioni](docs/web-network-captures.md) |
|
||||
| Captured di pacchetti | Endpoint, DNS, TLS SNI, distribuzione dei protocolli e richieste HTTP | tshark e capinfos forniti dal chiamante; [guida pcap](docs/pcap-analysis.md) |
|
||||
| Assembly .NET | Metadati, istruzioni CIL, dipendenze native dichiarate e confronti di build | Nessuno (solo statico); [guida managed-code](docs/managed-code-analysis.md) |
|
||||
| APK Android | Dichiarazioni manifest, classi, metodi decompilati e riferimenti | JADX headless e un JDK completo su Linux/macOS/Windows x64; [guida Android](docs/android-analysis.md) |
|
||||
| Dispositivi Android | Dispositivi, pacchetti, processi, log, dump, screenshot e trasferimenti APK/file | adb fornito dal chiamante (emulatore o hardware); [guida ADB](docs/adb-device-analysis.md) |
|
||||
@@ -150,6 +151,8 @@ REA richiede Node.js 22.x (>=22.19), 24.x (>=24.11) o 26+, oltre a npm. Strument
|
||||
| Risorse Android | Manifest decodificato, tabelle stringhe, lingue e dati di versione | Launcher Apktool fornito dal chiamante; [guida alle risorse](docs/apktool-resource-analysis.md) |
|
||||
| App Flutter | Hash dello snapshot Dart, digest delle librerie per ABI e provenienza della build del motore | Nessuno (analisi APK statica); [guida Flutter](docs/flutter-analysis.md) |
|
||||
| Firmware | Regioni, risultati di estrazione e passaggi all'analisi nativa | Binwalk / Unblob su Linux; [guida firmware](docs/firmware-analysis.md) |
|
||||
| Pacchetti registrati | Campi selezionati, valori ripetuti, filtri e regole di decodifica | Linux: TShark 4.x + prlimit; [guida alle catture](docs/packet-captures.md) |
|
||||
| Segnali logici registrati | Annotazioni del decoder, intervalli di campioni e diagnostica | Linux: sigrok-cli 0.7.2 + prlimit; [guida ai segnali](docs/signal-recordings.md) |
|
||||
| Pacchetti e risorse | Inventari file, digest, plist, anatomia dei bundle Apple e risorse estratte | [Guida artifact e JavaScript](docs/javascript-artifact-reconstruction.md), [Applicazioni Apple](docs/apple-application-analysis.md) |
|
||||
| Comportamento dei processi | Output del terminale, interazioni, osservazioni su uscita e filesystem ed confronti di esecuzione | Linux/macOS con un PTY nativo; [cattura dei processi](docs/process-capture.md) |
|
||||
|
||||
@@ -263,7 +266,7 @@ Segnala vulnerabilità tramite [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Cronologia delle stelle
|
||||
|
||||
🎉 **110.000 stelle su GitHub — grazie!**
|
||||
🎉 **170.000 stelle su GitHub — grazie!**
|
||||
|
||||
Grazie a tutti coloro che usano REA, segnalano bug, propongono funzionalità, testano le build e contribuiscono con correzioni.
|
||||
|
||||
|
||||
+4
-1
@@ -139,6 +139,7 @@ REA には Node.js 22.x(>=22.19)、24.x(>=24.11)、または 26+ と npm
|
||||
| JavaScript / Electron | モジュール、インポート、ソースマップ、ルート、IPC、ネイティブアドオンの関係 | Node.js と npm;[アプリケーション分析](https://rea.tools/ja/guides/javascript/) |
|
||||
| ウェブサイト | ページ構造、スクリプト、ネットワークの観測結果、依頼されたスクリーンショット | Chrome 系ブラウザー;[ブラウザー分析](https://rea.tools/ja/guides/browser/) |
|
||||
| 保存済みのネットワークキャプチャ | リクエスト、レスポンス、取得可能なペイロード、ソース内の位置 | HAR;mitmproxy ネイティブ形式のキャプチャには Linux 上の mitmdump;[キャプチャガイド](docs/web-network-captures.md) |
|
||||
| パケットキャプチャ | エンドポイント、DNS、TLS SNI、プロトコル分布、HTTPリクエスト | 呼び出し側が用意した tshark と capinfos;[pcapガイド](docs/pcap-analysis.md) |
|
||||
| .NET アセンブリ | メタデータ、CIL 命令、宣言されたネイティブ依存関係、ビルドの比較 | 静的な検査;[マネージドコードのガイド](docs/managed-code-analysis.md) |
|
||||
| Android APK | マニフェストの宣言、クラス、逆コンパイルされたメソッド、参照 | Linux/macOS/Windows x64 上のヘッドレス JADX と完全な JDK;[Android ガイド](docs/android-analysis.md) |
|
||||
| Android デバイス | デバイス、パッケージ、プロセス、ログ、ダンプ、スクリーンキャプチャ、APK/ファイル転送 | 呼び出し側が用意した adb(エミュレーターまたは実機);[ADB ガイド](docs/adb-device-analysis.md) |
|
||||
@@ -146,6 +147,8 @@ REA には Node.js 22.x(>=22.19)、24.x(>=24.11)、または 26+ と npm
|
||||
| Android リソース | デコードされたマニフェスト、文字列テーブル、ロケール、バージョン情報 | 呼び出し側が用意した Apktool ランチャー;[リソースガイド](docs/apktool-resource-analysis.md) |
|
||||
| Flutter アプリ | Dart スナップショットハッシュ、ABI ごとのライブラリダイジェスト、エンジンビルドの来歴 | 不要(静的 APK 解析);[Flutter ガイド](docs/flutter-analysis.md) |
|
||||
| ファームウェア | 領域、抽出結果、ネイティブ分析への引き継ぎ | Linux 上の Binwalk / Unblob;[ファームウェアガイド](docs/firmware-analysis.md) |
|
||||
| 記録済みパケット | 選択フィールド、繰り返し値、フィルター、デコード指定 | Linux: TShark 4.x + prlimit; [キャプチャガイド](docs/packet-captures.md) |
|
||||
| 記録済みロジック信号 | デコーダ注釈、サンプル範囲、診断情報 | Linux: sigrok-cli 0.7.2 + prlimit; [信号記録ガイド](docs/signal-recordings.md) |
|
||||
| パッケージとリソース | ファイル一覧、ダイジェスト、plist、Apple バンドルの構造、抽出されたリソース | [アーティファクトと JavaScript のガイド](docs/javascript-artifact-reconstruction.md)、[Apple アプリケーション](docs/apple-application-analysis.md) |
|
||||
| プロセスの動作 | ターミナル出力、対話、終了とファイルシステムの観測結果、実行の比較 | ネイティブ PTY を備えた Linux/macOS;[プロセスキャプチャ](docs/process-capture.md) |
|
||||
|
||||
@@ -261,7 +264,7 @@ npx rea-agents@latest setup
|
||||
|
||||
## スター履歴
|
||||
|
||||
🎉 **GitHub スター 110,000 件、ありがとうございます!**
|
||||
🎉 **GitHub スター 170,000 件、ありがとうございます!**
|
||||
|
||||
REA の利用、バグ報告、機能の要望、ビルドのテスト、修正への貢献に感謝します。
|
||||
|
||||
|
||||
+4
-1
@@ -139,6 +139,7 @@ REA에는 Node.js 22.x(>=22.19), 24.x(>=24.11) 또는 26+와 npm이 필요합니
|
||||
| JavaScript / Electron | 모듈, 임포트, 소스 맵, 라우트, IPC, 네이티브 애드온 관계 | Node.js와 npm; [애플리케이션 분석](https://rea.tools/ko/guides/javascript/) |
|
||||
| 웹사이트 | 페이지 구조, 스크립트, 네트워크 관찰 결과, 요청한 스크린샷 | Chrome 계열 브라우저; [브라우저 분석](https://rea.tools/ko/guides/browser/) |
|
||||
| 저장된 네트워크 캡처 | 요청, 응답, 접근 가능한 페이로드, 소스 위치 | HAR; mitmproxy 네이티브 캡처에는 Linux의 mitmdump; [캡처 가이드](docs/web-network-captures.md) |
|
||||
| 패킷 캡처 | 엔드포인트, DNS, TLS SNI, 프로토콜 분포, HTTP 요청 | 호출자 제공 tshark 및 capinfos; [pcap 가이드](docs/pcap-analysis.md) |
|
||||
| .NET 어셈블리 | 메타데이터, CIL 명령, 선언된 네이티브 의존성, 빌드 비교 | 정적 검사; [관리 코드 가이드](docs/managed-code-analysis.md) |
|
||||
| Android APK | 매니페스트 선언, 클래스, 디컴파일된 메서드, 참조 | Linux/macOS/Windows x64의 헤드리스 JADX와 전체 JDK; [Android 가이드](docs/android-analysis.md) |
|
||||
| Android 기기 | 기기, 패키지, 프로세스, 로그, 덤프, 화면 캡처, APK/파일 전송 | 호출자 제공 adb(에뮬레이터 또는 실제 기기); [ADB 가이드](docs/adb-device-analysis.md) |
|
||||
@@ -146,6 +147,8 @@ REA에는 Node.js 22.x(>=22.19), 24.x(>=24.11) 또는 26+와 npm이 필요합니
|
||||
| Android 리소스 | 디코딩된 매니페스트, 문자열 테이블, 로캘, 버전 정보 | 호출자 제공 Apktool 런처; [리소스 가이드](docs/apktool-resource-analysis.md) |
|
||||
| Flutter 앱 | Dart 스냅샷 해시, ABI별 라이브러리 해시, 엔진 빌드 출처 | 없음 (정적 APK 파싱); [Flutter 가이드](docs/flutter-analysis.md) |
|
||||
| 펌웨어 | 영역, 추출 결과, 네이티브 분석으로 전달할 정보 | Linux의 Binwalk / Unblob; [펌웨어 가이드](docs/firmware-analysis.md) |
|
||||
| 기록된 패킷 | 선택 필드, 반복 값, 필터 및 디코딩 규칙 | Linux: TShark 4.x + prlimit; [캡처 가이드](docs/packet-captures.md) |
|
||||
| 기록된 논리 신호 | 디코더 주석, 샘플 범위, 진단 정보 | Linux: sigrok-cli 0.7.2 + prlimit; [신호 기록 가이드](docs/signal-recordings.md) |
|
||||
| 패키지 및 리소스 | 파일 목록, 다이제스트, plist, Apple 번들 구조, 추출한 리소스 | [아티팩트 및 JavaScript 가이드](docs/javascript-artifact-reconstruction.md), [Apple 애플리케이션](docs/apple-application-analysis.md) |
|
||||
| 프로세스 동작 | 터미널 출력, 상호작용, 종료 및 파일 시스템 관찰 결과, 실행 비교 | 네이티브 PTY를 지원하는 Linux/macOS; [프로세스 캡처](docs/process-capture.md) |
|
||||
|
||||
@@ -261,7 +264,7 @@ npx rea-agents@latest setup
|
||||
|
||||
## 스타 기록
|
||||
|
||||
🎉 **GitHub 스타 110,000개, 감사합니다!**
|
||||
🎉 **GitHub 스타 170,000개, 감사합니다!**
|
||||
|
||||
REA를 사용하고, 버그를 보고하고, 기능을 요청하고, 빌드를 테스트하고, 수정에 기여해 주신 모든 분께 감사드립니다.
|
||||
|
||||
|
||||
+4
-1
@@ -139,6 +139,7 @@ REA wymaga Node.js 22.x (>=22.19), 24.x (>=24.11) lub 26+ oraz npm. Dodatkowe na
|
||||
| JavaScript / Electron | Moduły, importy, mapy źródeł, trasy, IPC i powiązania z natywnymi dodatkami | Node.js i npm; [analiza aplikacji](https://rea.tools/pl/guides/javascript/) |
|
||||
| Strony internetowe | Struktura strony, skrypty, obserwacje sieciowe i zamówione zrzuty ekranu | Przeglądarka z rodziny Chrome; [analiza w przeglądarce](https://rea.tools/pl/guides/browser/) |
|
||||
| Zapisane przechwycenia ruchu sieciowego | Żądania, odpowiedzi, dostępne dane przesyłane i lokalizacje źródłowe | HAR; mitmdump w systemie Linux dla natywnych przechwyceń mitmproxy; [przewodnik przechwytywania](docs/web-network-captures.md) |
|
||||
| Przechwytywanie pakietów | Punkty końcowe, DNS, TLS SNI, dystrybucja protokołów i żądania HTTP | Dostarczone przez wywołującego tshark i capinfos; [przewodnik pcap](docs/pcap-analysis.md) |
|
||||
| Zestawy .NET | Metadane, instrukcje CIL, zadeklarowane zależności natywne i porównania kompilacji | Analiza statyczna; [przewodnik kodu zarządzanego](docs/managed-code-analysis.md) |
|
||||
| Pakiety APK dla Androida | Deklaracje manifestu, klasy, zdekompilowane metody i odwołania | JADX bez interfejsu graficznego i pełny JDK w systemach Linux/macOS/Windows x64; [przewodnik Androida](docs/android-analysis.md) |
|
||||
| Urządzenia z Androidem | Urządzenia, pakiety, procesy, logi, zrzuty, zrzuty ekranu oraz transfery APK/plików | adb dostarczony przez wywołującego (emulator lub urządzenie); [przewodnik ADB](docs/adb-device-analysis.md) |
|
||||
@@ -146,6 +147,8 @@ REA wymaga Node.js 22.x (>=22.19), 24.x (>=24.11) lub 26+ oraz npm. Dodatkowe na
|
||||
| Zasoby Androida | Zdekodowany manifest, tablice ciągów, języki i informacje o wersji | Należący do wywołującego launcher Apktool; [przewodnik zasobów](docs/apktool-resource-analysis.md) |
|
||||
| Aplikacje Flutter | Hash migawki Dart, sumy bibliotek per ABI i pochodzenie kompilacji silnika | Brak (statyczna analiza APK); [przewodnik Flutter](docs/flutter-analysis.md) |
|
||||
| Oprogramowanie układowe | Regiony, wyniki wyodrębniania i przekazanie do analizy kodu natywnego | Binwalk / Unblob w systemie Linux; [przewodnik oprogramowania układowego](docs/firmware-analysis.md) |
|
||||
| Zapisane pakiety | Wybrane pola, powtarzane wartości, filtry i reguły dekodowania | Linux: TShark 4.x + prlimit; [przewodnik po przechwyconych pakietach](docs/packet-captures.md) |
|
||||
| Zapisane sygnały logiczne | Adnotacje dekodera, zakresy próbek i diagnostyka | Linux: sigrok-cli 0.7.2 + prlimit; [przewodnik po sygnałach](docs/signal-recordings.md) |
|
||||
| Pakiety i zasoby | Spisy plików, skróty, pliki plist, struktura pakietów Apple i wyodrębnione zasoby | [Przewodnik artefaktów i JavaScript](docs/javascript-artifact-reconstruction.md), [aplikacje Apple](docs/apple-application-analysis.md) |
|
||||
| Zachowanie procesów | Wyjście terminala, interakcje, obserwacje zakończenia i systemu plików oraz porównania uruchomień | Linux/macOS z natywnym PTY; [przechwytywanie procesów](docs/process-capture.md) |
|
||||
|
||||
@@ -259,7 +262,7 @@ Zgłaszaj podatności zgodnie z [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Historia gwiazdek
|
||||
|
||||
🎉 **110 000 gwiazdek na GitHubie — dziękujemy!**
|
||||
🎉 **170 000 gwiazdek na GitHubie — dziękujemy!**
|
||||
|
||||
Dziękujemy wszystkim, którzy używają REA, zgłaszają błędy, proponują nowe funkcje, testują kompilacje i przesyłają poprawki.
|
||||
|
||||
|
||||
+4
-1
@@ -139,6 +139,7 @@ O REA requer Node.js 22.x (>=22.19), 24.x (>=24.11) ou 26+, além do npm. As fer
|
||||
| JavaScript / Electron | Módulos, importações, source maps, rotas, IPC e relações com complementos nativos | Node.js e npm; [análise de aplicativos](https://rea.tools/pt-BR/guides/javascript/) |
|
||||
| Sites | Estrutura da página, scripts, observações de rede e capturas de tela solicitadas | Um navegador da família Chrome; [análise do navegador](https://rea.tools/pt-BR/guides/browser/) |
|
||||
| Capturas de rede salvas | Requisições, respostas, payloads acessíveis e localizações de origem | HAR; mitmdump no Linux para capturas no formato nativo do mitmproxy; [guia de capturas](docs/web-network-captures.md) |
|
||||
| Capturas de pacotes | Endpoints, DNS, TLS SNI, distribuição de protocolos e requisições HTTP | tshark e capinfos fornecidos pelo chamador; [guia pcap](docs/pcap-analysis.md) |
|
||||
| Assemblies .NET | Metadados, instruções CIL, dependências nativas declaradas e comparações de builds | Inspeção estática; [guia de código gerenciado](docs/managed-code-analysis.md) |
|
||||
| APKs Android | Declarações do manifesto, classes, métodos descompilados e referências | JADX sem interface gráfica e um JDK completo no Linux/macOS/Windows x64; [guia de Android](docs/android-analysis.md) |
|
||||
| Dispositivos Android | Dispositivos, pacotes, processos, logs, despejos, capturas de tela e transferências de APK/arquivos | adb fornecido pelo chamador (emulador ou aparelho); [guia de ADB](docs/adb-device-analysis.md) |
|
||||
@@ -146,6 +147,8 @@ O REA requer Node.js 22.x (>=22.19), 24.x (>=24.11) ou 26+, além do npm. As fer
|
||||
| Recursos do Android | Manifesto decodificado, tabelas de strings, idiomas e fatos de versão | Lançador do Apktool fornecido pelo chamador; [guia de recursos](docs/apktool-resource-analysis.md) |
|
||||
| Aplicativos Flutter | Hash de snapshot Dart, hashes de bibliotecas por ABI e procedência de build do motor | Nenhuma (análise estática de APK); [guia Flutter](docs/flutter-analysis.md) |
|
||||
| Firmware | Regiões, resultados de extração e encaminhamentos à análise nativa | Binwalk / Unblob no Linux; [guia de firmware](docs/firmware-analysis.md) |
|
||||
| Pacotes gravados | Campos selecionados, valores repetidos, filtros e regras de decodificação | Linux: TShark 4.x + prlimit; [guia de capturas](docs/packet-captures.md) |
|
||||
| Sinais lógicos gravados | Anotações do decodificador, intervalos de amostras e diagnósticos | Linux: sigrok-cli 0.7.2 + prlimit; [guia de sinais](docs/signal-recordings.md) |
|
||||
| Pacotes e recursos | Inventários de arquivos, digests, plists, estrutura de bundles Apple e recursos extraídos | [Guia de artefatos e JavaScript](docs/javascript-artifact-reconstruction.md), [aplicativos Apple](docs/apple-application-analysis.md) |
|
||||
| Comportamento de processos | Saída do terminal, interações, observações de encerramento e do sistema de arquivos, e comparações entre execuções | Linux/macOS com PTY nativo; [captura de processos](docs/process-capture.md) |
|
||||
|
||||
@@ -259,7 +262,7 @@ Relate vulnerabilidades conforme [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Histórico de estrelas
|
||||
|
||||
🎉 **110.000 estrelas no GitHub — muito obrigado!**
|
||||
🎉 **170.000 estrelas no GitHub — muito obrigado!**
|
||||
|
||||
Obrigado a todos que usam o REA, relatam bugs, sugerem funcionalidades, testam builds e contribuem com correções.
|
||||
|
||||
|
||||
+22
-19
@@ -130,24 +130,27 @@ npx rea-agents@latest setup
|
||||
|
||||
REA требует Node.js 22.x (>=22.19), 24.x (>=24.11) или 26+, а также npm. Дополнительные инструменты и поддерживаемые системы зависят от цели:
|
||||
|
||||
| Цель | Что возвращает REA | Требования и руководство |
|
||||
| --------------------------- | -------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Нативные бинарные файлы | Псевдокод, ассемблер, строки, символы, вызовы и ссылки | Hopper, Ghidra или IDA; [нативный анализ](https://rea.tools/ru/guides/native/) |
|
||||
| Структура ELF без запуска | Секции, сегменты, исходные символы/релокации и возможные механизмы защиты по данным статического анализа | pwntools, предоставленный вызывающей стороной, в Linux x64; [диагностика бинарных файлов](docs/binary-diagnostics.md) |
|
||||
| Байт-код EVM | Селекторы диспетчеризации, смещения байтов, предполагаемые аргументы и изменяемость состояния | Локальные необработанные байты или шестнадцатеричное представление в Linux x64; [руководство по EVM без запуска](docs/evm-bytecode.md) |
|
||||
| Записанные сбои Linux | Необработанные записи note, регистры/сигналы каждого записанного потока и необязательные кандидаты сопоставления с памятью | pwntools, предоставленный вызывающей стороной, в Linux x64; необязательные GDB/pwndbg; [записанные сбои](docs/recorded-crashes.md) |
|
||||
| JavaScript / Electron | Модули, импорты, карты исходного кода, маршруты, IPC и связи с нативными дополнениями | Node.js и npm; [анализ приложений](https://rea.tools/ru/guides/javascript/) |
|
||||
| Сайты | Структура страницы, скрипты, наблюдения за сетью и запрошенные снимки экрана | Браузер семейства Chrome; [анализ браузера](https://rea.tools/ru/guides/browser/) |
|
||||
| Сохранённые сетевые захваты | Запросы, ответы, доступные полезные данные и позиции в источнике | HAR; mitmdump в Linux для захватов в собственном формате mitmproxy; [руководство по сетевым захватам](docs/web-network-captures.md) |
|
||||
| Сборки .NET | Метаданные, инструкции CIL, объявленные нативные зависимости и сравнения сборок | Статическое исследование; [руководство по управляемому коду](docs/managed-code-analysis.md) |
|
||||
| Android APK | Объявления манифеста, классы, декомпилированные методы и ссылки | JADX без графического интерфейса и полный JDK в Linux/macOS/Windows x64; [руководство по Android](docs/android-analysis.md) |
|
||||
| Устройства Android | Устройства, пакеты, процессы, журналы, дампы, снимки экрана и передача APK/файлов | adb, предоставленный вызывающей стороной (эмулятор или устройство); [руководство по ADB](docs/adb-device-analysis.md) |
|
||||
| Проекты JEB | Единицы проекта от движка, дайджесты артефактов и псевдокод типов/методов | Запущенный вызывающей стороной клиент JEB с интерфейсом MCP, выбирается через REA_JEB_MCP_URL; [руководство по JEB](docs/jeb-analysis.md) |
|
||||
| Ресурсы Android | Декодированный манифест, таблицы строк, языки и сведения о версии | Запуск Apktool, предоставленный вызывающей стороной; [руководство по ресурсам](docs/apktool-resource-analysis.md) |
|
||||
| Приложения Flutter | Хеш снапшота Dart, дайджесты библиотек по ABI и происхождение сборки движка | Не требуется (статический разбор APK); [руководство по Flutter](docs/flutter-analysis.md) |
|
||||
| Прошивки | Области, результаты извлечения и передача в нативный анализ | Binwalk / Unblob в Linux; [руководство по прошивкам](docs/firmware-analysis.md) |
|
||||
| Пакеты и ресурсы | Списки файлов, хеши, plist, структура пакетов Apple и извлечённые ресурсы | [Руководство по артефактам и JavaScript](docs/javascript-artifact-reconstruction.md), [приложения Apple](docs/apple-application-analysis.md) |
|
||||
| Поведение процессов | Вывод терминала, взаимодействия, наблюдения за завершением и файловой системой, сравнения запусков | Linux/macOS с нативным PTY; [захват процессов](docs/process-capture.md) |
|
||||
| Цель | Что возвращает REA | Требования и руководство |
|
||||
| ----------------------------- | -------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Нативные бинарные файлы | Псевдокод, ассемблер, строки, символы, вызовы и ссылки | Hopper, Ghidra или IDA; [нативный анализ](https://rea.tools/ru/guides/native/) |
|
||||
| Структура ELF без запуска | Секции, сегменты, исходные символы/релокации и возможные механизмы защиты по данным статического анализа | pwntools, предоставленный вызывающей стороной, в Linux x64; [диагностика бинарных файлов](docs/binary-diagnostics.md) |
|
||||
| Байт-код EVM | Селекторы диспетчеризации, смещения байтов, предполагаемые аргументы и изменяемость состояния | Локальные необработанные байты или шестнадцатеричное представление в Linux x64; [руководство по EVM без запуска](docs/evm-bytecode.md) |
|
||||
| Записанные сбои Linux | Необработанные записи note, регистры/сигналы каждого записанного потока и необязательные кандидаты сопоставления с памятью | pwntools, предоставленный вызывающей стороной, в Linux x64; необязательные GDB/pwndbg; [записанные сбои](docs/recorded-crashes.md) |
|
||||
| JavaScript / Electron | Модули, импорты, карты исходного кода, маршруты, IPC и связи с нативными дополнениями | Node.js и npm; [анализ приложений](https://rea.tools/ru/guides/javascript/) |
|
||||
| Сайты | Структура страницы, скрипты, наблюдения за сетью и запрошенные снимки экрана | Браузер семейства Chrome; [анализ браузера](https://rea.tools/ru/guides/browser/) |
|
||||
| Сохранённые сетевые захваты | Запросы, ответы, доступные полезные данные и позиции в источнике | HAR; mitmdump в Linux для захватов в собственном формате mitmproxy; [руководство по сетевым захватам](docs/web-network-captures.md) |
|
||||
| Сборки .NET | Метаданные, инструкции CIL, объявленные нативные зависимости и сравнения сборок | Статическое исследование; [руководство по управляемому коду](docs/managed-code-analysis.md) |
|
||||
| Android APK | Объявления манифеста, классы, декомпилированные методы и ссылки | JADX без графического интерфейса и полный JDK в Linux/macOS/Windows x64; [руководство по Android](docs/android-analysis.md) |
|
||||
| Устройства Android | Устройства, пакеты, процессы, журналы, дампы, снимки экрана и передача APK/файлов | adb, предоставленный вызывающей стороной (эмулятор или устройство); [руководство по ADB](docs/adb-device-analysis.md) |
|
||||
| Проекты JEB | Единицы проекта от движка, дайджесты артефактов и псевдокод типов/методов | Запущенный вызывающей стороной клиент JEB с интерфейсом MCP, выбирается через REA_JEB_MCP_URL; [руководство по JEB](docs/jeb-analysis.md) |
|
||||
| Ресурсы Android | Декодированный манифест, таблицы строк, языки и сведения о версии | Запуск Apktool, предоставленный вызывающей стороной; [руководство по ресурсам](docs/apktool-resource-analysis.md) |
|
||||
| Приложения Flutter | Хеш снапшота Dart, дайджесты библиотек по ABI и происхождение сборки движка | Не требуется (статический разбор APK); [руководство по Flutter](docs/flutter-analysis.md) |
|
||||
| Прошивки | Области, результаты извлечения и передача в нативный анализ | Binwalk / Unblob в Linux; [руководство по прошивкам](docs/firmware-analysis.md) |
|
||||
| Записанные пакеты | Выбранные поля, повторяющиеся значения, фильтры и правила декодирования | Linux: TShark 4.x + prlimit; [руководство по захватам](docs/packet-captures.md) |
|
||||
| Захват пакетов | Конечные точки, DNS, TLS SNI, распределение протоколов и HTTP-запросы | Предоставленные вызывающей стороной tshark и capinfos; [руководство по pcap](docs/pcap-analysis.md) |
|
||||
| Записанные логические сигналы | Аннотации декодера, диапазоны отсчётов и диагностика | Linux: sigrok-cli 0.7.2 + prlimit; [руководство по сигналам](docs/signal-recordings.md) |
|
||||
| Пакеты и ресурсы | Списки файлов, хеши, plist, структура пакетов Apple и извлечённые ресурсы | [Руководство по артефактам и JavaScript](docs/javascript-artifact-reconstruction.md), [приложения Apple](docs/apple-application-analysis.md) |
|
||||
| Поведение процессов | Вывод терминала, взаимодействия, наблюдения за завершением и файловой системой, сравнения запусков | Linux/macOS с нативным PTY; [захват процессов](docs/process-capture.md) |
|
||||
|
||||
Статическое исследование JavaScript и .NET читает предоставленные файлы, не запуская приложение. Захват во время выполнения запускает выбранную цель или взаимодействует с ней с правами вашего пользователя; соответствующие руководства описывают последствия.
|
||||
|
||||
@@ -259,7 +262,7 @@ npx rea-agents@latest setup
|
||||
|
||||
## История звёзд
|
||||
|
||||
🎉 **110 000 звёзд на GitHub — спасибо!**
|
||||
🎉 **170 000 звёзд на GitHub — спасибо!**
|
||||
|
||||
Спасибо всем, кто использует REA, сообщает об ошибках, предлагает новые функции, тестирует сборки и помогает с исправлениями.
|
||||
|
||||
|
||||
+4
-1
@@ -139,6 +139,7 @@ REA ต้องใช้ Node.js 22.x (>=22.19), 24.x (>=24.11) หรือ 26
|
||||
| JavaScript / Electron | โมดูล การนำเข้า ซอร์สแมป เส้นทาง IPC และความสัมพันธ์กับส่วนเสริมเนทีฟ | Node.js และ npm; [การวิเคราะห์แอปพลิเคชัน](https://rea.tools/th/guides/javascript/) |
|
||||
| เว็บไซต์ | โครงสร้างหน้า สคริปต์ ข้อมูลเครือข่ายที่สังเกตได้ และภาพหน้าจอที่ร้องขอ | เบราว์เซอร์ในตระกูล Chrome; [การวิเคราะห์ผ่านเบราว์เซอร์](https://rea.tools/th/guides/browser/) |
|
||||
| ข้อมูลเครือข่ายที่บันทึกไว้ | คำขอ คำตอบ เพย์โหลดที่ปรากฏ และตำแหน่งต้นทาง | HAR; mitmdump บน Linux สำหรับข้อมูลที่บันทึกในรูปแบบเนทีฟของ mitmproxy; [คู่มือการบันทึกข้อมูล](docs/web-network-captures.md) |
|
||||
| การจับแพ็กเก็ต | จุดเชื่อมต่อ, DNS, TLS SNI, การกระจายโปรโตคอล และคำขอ HTTP | tshark และ capinfos ที่ผู้เรียกจัดเตรียม; [คู่มือ pcap](docs/pcap-analysis.md) |
|
||||
| แอสเซมบลี .NET | เมทาดาทา คำสั่ง CIL การพึ่งพาเนทีฟที่ประกาศไว้ และการเปรียบเทียบผลการบิลด์ | การตรวจสอบแบบสถิต; [คู่มือโค้ดแบบจัดการ](docs/managed-code-analysis.md) |
|
||||
| APK ของ Android | การประกาศใน manifest คลาส เมธอดที่ดีคอมไพล์ และการอ้างอิง | JADX แบบไม่มี GUI และ JDK แบบเต็มบน Linux/macOS/Windows x64; [คู่มือ Android](docs/android-analysis.md) |
|
||||
| อุปกรณ์ Android | อุปกรณ์ แพ็กเกจ โพรเซส บันทึก ดัมพ์ ภาพหน้าจอ และการถ่ายโอน APK/ไฟล์ | adb ที่ผู้เรียกจัดเตรียม (โปรแกรมจำลองหรืออุปกรณ์จริง); [คู่มือ ADB](docs/adb-device-analysis.md) |
|
||||
@@ -146,6 +147,8 @@ REA ต้องใช้ Node.js 22.x (>=22.19), 24.x (>=24.11) หรือ 26
|
||||
| ทรัพยากร Android | ไฟล์ manifest ที่ถอดรหัสแล้ว ตารางสตริง ภาษา และข้อมูลเวอร์ชัน | ตัวเรียก Apktool ที่ผู้เรียกจัดเตรียม; [คู่มือทรัพยากร](docs/apktool-resource-analysis.md) |
|
||||
| แอป Flutter | แฮชสแนปชอต Dart, แฮชไลบรารีราย ABI และที่มาของบิลด์เอนจิน | ไม่ต้อง (วิเคราะห์ APK แบบ static); [คู่มือ Flutter](docs/flutter-analysis.md) |
|
||||
| เฟิร์มแวร์ | พื้นที่ข้อมูล ผลการแตกไฟล์ และการส่งต่อไปวิเคราะห์เนทีฟ | Binwalk / Unblob บน Linux; [คู่มือเฟิร์มแวร์](docs/firmware-analysis.md) |
|
||||
| แพ็กเก็ตที่บันทึกไว้ | ฟิลด์ที่เลือก ค่าซ้ำ ตัวกรอง และกฎการถอดรหัส | Linux: TShark 4.x + prlimit; [คู่มือไฟล์แคปเจอร์](docs/packet-captures.md) |
|
||||
| สัญญาณลอจิกที่บันทึกไว้ | คำอธิบายจากตัวถอดรหัส ช่วงตัวอย่าง และข้อมูลวินิจฉัย | Linux: sigrok-cli 0.7.2 + prlimit; [คู่มือสัญญาณ](docs/signal-recordings.md) |
|
||||
| แพ็กเกจและทรัพยากร | รายการไฟล์ ค่าแฮช plist โครงสร้างบันเดิลของ Apple และทรัพยากรที่แตกออกมา | [คู่มืออาร์ติแฟกต์และ JavaScript](docs/javascript-artifact-reconstruction.md), [แอปพลิเคชัน Apple](docs/apple-application-analysis.md) |
|
||||
| พฤติกรรมของโปรเซส | เอาต์พุตเทอร์มินัล การโต้ตอบ ข้อมูลการสิ้นสุดและระบบไฟล์ที่สังเกตได้ และการเปรียบเทียบการรัน | Linux/macOS ที่มี PTY เนทีฟ; [การบันทึกโปรเซส](docs/process-capture.md) |
|
||||
|
||||
@@ -259,7 +262,7 @@ npx rea-agents@latest setup
|
||||
|
||||
## ประวัติดาว
|
||||
|
||||
🎉 **110,000 ดาวบน GitHub — ขอบคุณทุกคน!**
|
||||
🎉 **170,000 ดาวบน GitHub — ขอบคุณทุกคน!**
|
||||
|
||||
ขอบคุณทุกคนที่ใช้ REA รายงานบั๊ก เสนอฟีเจอร์ ทดสอบบิลด์ และช่วยแก้ไข
|
||||
|
||||
|
||||
+4
-1
@@ -139,6 +139,7 @@ REA; Node.js 22.x (>=22.19), 24.x (>=24.11) veya 26+ ve npm gerektirir. Ek araç
|
||||
| JavaScript / Electron | Modüller, içe aktarmalar, kaynak haritaları, rotalar, IPC ve yerel eklenti ilişkileri | Node.js ve npm; [uygulama analizi](https://rea.tools/tr/guides/javascript/) |
|
||||
| Web siteleri | Sayfa yapısı, betikler, ağ gözlemleri ve istenen ekran görüntüleri | Chrome ailesinden bir tarayıcı; [tarayıcı analizi](https://rea.tools/tr/guides/browser/) |
|
||||
| Kaydedilmiş ağ yakalamaları | İstekler, yanıtlar, erişilebilir yük içerikleri ve kaynak konumları | HAR; mitmproxy'nin kendi biçimindeki yakalamalar için Linux üzerinde mitmdump; [ağ yakalama rehberi](docs/web-network-captures.md) |
|
||||
| Paket yakalamaları | Uç noktalar, DNS, TLS SNI, protokol dağılımı ve HTTP istekleri | Çağıranın sağladığı tshark ve capinfos; [pcap rehberi](docs/pcap-analysis.md) |
|
||||
| .NET derlemeleri | Meta veriler, CIL talimatları, bildirilmiş yerel bağımlılıklar ve derleme karşılaştırmaları | Statik inceleme; [yönetilen kod rehberi](docs/managed-code-analysis.md) |
|
||||
| Android APK'leri | Manifest bildirimleri, sınıflar, tersine derlenmiş metotlar ve referanslar | Linux/macOS/Windows x64 üzerinde arayüzsüz JADX ve tam bir JDK; [Android rehberi](docs/android-analysis.md) |
|
||||
| Android cihazları | Cihazlar, paketler, süreçler, günlükler, dökümler, ekran görüntüleri ve APK/dosya aktarımları | Çağıranın sağladığı adb (emülatör veya cihaz); [ADB rehberi](docs/adb-device-analysis.md) |
|
||||
@@ -146,6 +147,8 @@ REA; Node.js 22.x (>=22.19), 24.x (>=24.11) veya 26+ ve npm gerektirir. Ek araç
|
||||
| Android kaynakları | Çözülmüş manifest, dize tabloları, diller ve sürüm bilgileri | Çağıranın sağladığı Apktool başlatıcısı; [kaynak rehberi](docs/apktool-resource-analysis.md) |
|
||||
| Flutter uygulamaları | Dart anlık görüntü karması, ABI başına kitaplık özetleri ve motor derleme kökeni | Yok (statik APK çözümleme); [Flutter rehberi](docs/flutter-analysis.md) |
|
||||
| Donanım yazılımı | Bölgeler, çıkarma sonuçları ve yerel kod analizine devirler | Linux üzerinde Binwalk / Unblob; [donanım yazılımı rehberi](docs/firmware-analysis.md) |
|
||||
| Kaydedilmiş paketler | Seçilen alanlar, yinelenen değerler, filtreler ve çözümleme kuralları | Linux: TShark 4.x + prlimit; [yakalama kılavuzu](docs/packet-captures.md) |
|
||||
| Kaydedilmiş mantık sinyalleri | Kod çözücü açıklamaları, örnek aralıkları ve tanılama | Linux: sigrok-cli 0.7.2 + prlimit; [sinyal kılavuzu](docs/signal-recordings.md) |
|
||||
| Paketler ve kaynaklar | Dosya envanterleri, özet değerleri, plist'ler, Apple paket yapısı ve çıkarılmış kaynaklar | [Artefakt ve JavaScript rehberi](docs/javascript-artifact-reconstruction.md), [Apple uygulamaları](docs/apple-application-analysis.md) |
|
||||
| Süreç davranışı | Terminal çıktısı, etkileşimler, sonlanma ve dosya sistemi gözlemleri, çalıştırma karşılaştırmaları | Yerel PTY ile Linux/macOS; [süreç yakalama](docs/process-capture.md) |
|
||||
|
||||
@@ -259,7 +262,7 @@ Güvenlik açıklarını [SECURITY.md](SECURITY.md) üzerinden bildirin.
|
||||
|
||||
## Yıldız geçmişi
|
||||
|
||||
🎉 **GitHub'da 110.000 yıldız — teşekkürler!**
|
||||
🎉 **GitHub'da 170.000 yıldız — teşekkürler!**
|
||||
|
||||
REA'yı kullanan, hata bildiren, özellik talep eden, derlemeleri test eden ve düzeltmelere katkıda bulunan herkese teşekkürler.
|
||||
|
||||
|
||||
+4
-1
@@ -139,6 +139,7 @@ REA потребує Node.js 22.x (>=22.19), 24.x (>=24.11) або 26+, а та
|
||||
| JavaScript / Electron | Модулі, імпорти, карти вихідного коду, маршрути, IPC і зв'язки з нативними доповненнями | Node.js і npm; [аналіз застосунків](https://rea.tools/uk/guides/javascript/) |
|
||||
| Сайти | Структура сторінки, скрипти, мережеві спостереження й запитані знімки екрана | Браузер сімейства Chrome; [аналіз браузера](https://rea.tools/uk/guides/browser/) |
|
||||
| Збережені мережеві захоплення | Запити, відповіді, доступні корисні дані та місця в джерелі | HAR; mitmdump у Linux для захоплень у власному форматі mitmproxy; [посібник мережевих захоплень](docs/web-network-captures.md) |
|
||||
| Захоплення пакетів | Кінцеві точки, DNS, TLS SNI, розподіл протоколів та HTTP-запити | Надані викликаючою стороною tshark та capinfos; [підручник pcap](docs/pcap-analysis.md) |
|
||||
| Збірки .NET | Метадані, інструкції CIL, оголошені нативні залежності й порівняння збірок | Статична перевірка; [посібник керованого коду](docs/managed-code-analysis.md) |
|
||||
| Android APK | Оголошення маніфесту, класи, декомпільовані методи й посилання | JADX без графічного інтерфейсу й повний JDK у Linux/macOS/Windows x64; [посібник Android](docs/android-analysis.md) |
|
||||
| Пристрої Android | Пристрої, пакети, процеси, журнали, дампи, знімки екрана та передавання APK/файлів | adb, наданий викликаючою стороною (емулятор або пристрій); [підручник ADB](docs/adb-device-analysis.md) |
|
||||
@@ -146,6 +147,8 @@ REA потребує Node.js 22.x (>=22.19), 24.x (>=24.11) або 26+, а та
|
||||
| Ресурси Android | Декодований маніфест, таблиці рядків, мови та відомості про версію | Засіб запуску Apktool, наданий викликаючою стороною; [підручник ресурсів](docs/apktool-resource-analysis.md) |
|
||||
| Додатки Flutter | Хеш знімка Dart, дайджести бібліотек за ABI та походження збірки рушія | Не потрібні (статичний розбір APK); [підручник Flutter](docs/flutter-analysis.md) |
|
||||
| Прошивки | Області, результати видобування й передавання до нативного аналізу | Binwalk / Unblob у Linux; [посібник прошивок](docs/firmware-analysis.md) |
|
||||
| Записані пакети | Вибрані поля, повторювані значення, фільтри та правила декодування | Linux: TShark 4.x + prlimit; [посібник із записів пакетів](docs/packet-captures.md) |
|
||||
| Записані логічні сигнали | Анотації декодера, діапазони відліків і діагностика | Linux: sigrok-cli 0.7.2 + prlimit; [посібник із сигналів](docs/signal-recordings.md) |
|
||||
| Пакети й ресурси | Переліки файлів, хеші, plist, структура пакетів Apple та видобуті ресурси | [Посібник артефактів і JavaScript](docs/javascript-artifact-reconstruction.md), [застосунки Apple](docs/apple-application-analysis.md) |
|
||||
| Поведінка процесів | Вивід термінала, взаємодії, спостереження за завершенням і файловою системою та порівняння запусків | Linux/macOS із нативним PTY; [захоплення процесів](docs/process-capture.md) |
|
||||
|
||||
@@ -259,7 +262,7 @@ npx rea-agents@latest setup
|
||||
|
||||
## Історія зірок
|
||||
|
||||
🎉 **110 000 зірок на GitHub — дякуємо!**
|
||||
🎉 **170 000 зірок на GitHub — дякуємо!**
|
||||
|
||||
Дякуємо всім, хто використовує REA, повідомляє про помилки, пропонує нові функції, тестує збірки й допомагає з виправленнями.
|
||||
|
||||
|
||||
+4
-1
@@ -139,6 +139,7 @@ REA yêu cầu Node.js 22.x (>=22.19), 24.x (>=24.11) hoặc 26+, cùng với np
|
||||
| JavaScript / Electron | Mô-đun, các khai báo import, source map, tuyến, IPC và quan hệ với tiện ích mở rộng mã máy | Node.js và npm; [phân tích ứng dụng](https://rea.tools/vi/guides/javascript/) |
|
||||
| Trang web | Cấu trúc trang, script, quan sát mạng và ảnh chụp màn hình được yêu cầu | Trình duyệt thuộc họ Chrome; [phân tích trình duyệt](https://rea.tools/vi/guides/browser/) |
|
||||
| Bản ghi lưu lượng mạng đã lưu | Yêu cầu, phản hồi, payload có thể truy cập và vị trí nguồn | HAR; mitmdump trên Linux cho bản ghi định dạng gốc của mitmproxy; [hướng dẫn bản ghi mạng](docs/web-network-captures.md) |
|
||||
| Bắt gói tin | Điểm cuối, DNS, TLS SNI, phân bố giao thức và yêu cầu HTTP | tshark và capinfos do nơi gọi cung cấp; [hướng dẫn pcap](docs/pcap-analysis.md) |
|
||||
| Assembly .NET | Siêu dữ liệu, lệnh CIL, các phụ thuộc mã máy được khai báo và so sánh bản dựng | Kiểm tra tĩnh; [hướng dẫn mã được quản lý](docs/managed-code-analysis.md) |
|
||||
| APK Android | Khai báo manifest, lớp, phương thức được dịch ngược và tham chiếu | JADX không giao diện đồ họa và JDK đầy đủ trên Linux/macOS/Windows x64; [hướng dẫn Android](docs/android-analysis.md) |
|
||||
| Thiết bị Android | Thiết bị, gói, tiến trình, nhật ký, dump, ảnh chụp màn hình và truyền APK/tệp | adb do nơi gọi cung cấp (trình giả lập hoặc thiết bị); [hướng dẫn ADB](docs/adb-device-analysis.md) |
|
||||
@@ -146,6 +147,8 @@ REA yêu cầu Node.js 22.x (>=22.19), 24.x (>=24.11) hoặc 26+, cùng với np
|
||||
| Tài nguyên Android | Manifest đã giải mã, bảng chuỗi, ngôn ngữ và thông tin phiên bản | Trình khởi chạy Apktool do nơi gọi cung cấp; [hướng dẫn tài nguyên](docs/apktool-resource-analysis.md) |
|
||||
| Ứng dụng Flutter | Băm snapshot Dart, hash thư viện theo ABI và nguồn gốc build engine | Không cần (phân tích APK tĩnh); [hướng dẫn Flutter](docs/flutter-analysis.md) |
|
||||
| Phần sụn | Các vùng, kết quả trích xuất và dữ liệu chuyển sang phân tích mã máy | Binwalk / Unblob trên Linux; [hướng dẫn phần sụn](docs/firmware-analysis.md) |
|
||||
| Gói tin đã ghi | Trường được chọn, giá trị lặp, bộ lọc và quy tắc giải mã | Linux: TShark 4.x + prlimit; [hướng dẫn bản ghi gói tin](docs/packet-captures.md) |
|
||||
| Tín hiệu logic đã ghi | Chú thích bộ giải mã, phạm vi mẫu và chẩn đoán | Linux: sigrok-cli 0.7.2 + prlimit; [hướng dẫn tín hiệu](docs/signal-recordings.md) |
|
||||
| Gói và tài nguyên | Danh sách tệp, giá trị băm, plist, cấu trúc bundle Apple và tài nguyên được trích xuất | [Hướng dẫn hiện vật phần mềm và JavaScript](docs/javascript-artifact-reconstruction.md), [ứng dụng Apple](docs/apple-application-analysis.md) |
|
||||
| Hành vi tiến trình | Đầu ra terminal, tương tác, quan sát khi thoát và hệ thống tệp, cùng so sánh các lần chạy | Linux/macOS có PTY gốc; [ghi lại tiến trình](docs/process-capture.md) |
|
||||
|
||||
@@ -259,7 +262,7 @@ Báo cáo lỗ hổng theo [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Lịch sử sao
|
||||
|
||||
🎉 **110.000 sao trên GitHub — xin cảm ơn!**
|
||||
🎉 **170.000 sao trên GitHub — xin cảm ơn!**
|
||||
|
||||
Cảm ơn mọi người đã sử dụng REA, báo lỗi, đề xuất tính năng, kiểm thử bản dựng và đóng góp bản sửa lỗi.
|
||||
|
||||
|
||||
+4
-1
@@ -139,6 +139,7 @@ REA 需要 Node.js 22.x(>=22.19)、24.x(>=24.11)或 26+,以及 npm。
|
||||
| JavaScript / Electron | 模組、匯入、source map、路由、IPC 和原生附加元件關係 | Node.js 和 npm;[應用分析](https://rea.tools/zh-TW/guides/javascript/) |
|
||||
| 網站 | 頁面結構、腳本、網路觀察結果和按請求獲取的螢幕擷取畫面 | Chrome 系瀏覽器;[瀏覽器分析](https://rea.tools/zh-TW/guides/browser/) |
|
||||
| 已保存的網路擷取 | 請求、回應、可訪問的酬載和來源位置 | HAR;原生 mitmproxy 擷取需要 Linux 上的 mitmdump;[擷取指南](docs/web-network-captures.md) |
|
||||
| 封包擷取 | 端點、DNS、TLS SNI、通訊協定分佈和 HTTP 請求 | 呼叫端提供的 tshark 和 capinfos;[pcap 指南](docs/pcap-analysis.md) |
|
||||
| .NET 程式集 | 中繼資料、CIL 指令、聲明的原生依賴和建置對比 | 靜態檢查;[托管程式碼指南](docs/managed-code-analysis.md) |
|
||||
| Android APK | 清單聲明、類別、反編譯的方法和引用 | Linux/macOS/Windows x64 上的無界面 JADX 和完整 JDK;[Android 指南](docs/android-analysis.md) |
|
||||
| Android 裝置 | 裝置、套件、程序、日誌、傾印、螢幕擷取以及 APK/檔案傳輸 | 呼叫端提供的 adb(模擬器或實體裝置);[ADB 指南](docs/adb-device-analysis.md) |
|
||||
@@ -146,6 +147,8 @@ REA 需要 Node.js 22.x(>=22.19)、24.x(>=24.11)或 26+,以及 npm。
|
||||
| Android 資源 | 解碼後的清單、字串表、語系和版本資訊 | 呼叫端提供的 Apktool 啟動器;[資源指南](docs/apktool-resource-analysis.md) |
|
||||
| Flutter 應用 | Dart 快照雜湊、依 ABI 的程式庫雜湊及引擎建置來源 | 無(靜態 APK 解析);[Flutter 指南](docs/flutter-analysis.md) |
|
||||
| 韌體 | 區域、提取結果和轉交原生分析的內容 | Linux 上的 Binwalk / Unblob;[韌體指南](docs/firmware-analysis.md) |
|
||||
| 封包記錄 | 指定欄位、重複值、篩選條件與解碼規則 | Linux: TShark 4.x + prlimit; [封包分析指南](docs/packet-captures.md) |
|
||||
| 錄製的邏輯訊號 | 解碼註解、樣本範圍與診斷資訊 | Linux: sigrok-cli 0.7.2 + prlimit; [訊號記錄指南](docs/signal-recordings.md) |
|
||||
| 軟體包與資源 | 檔案清單、摘要、plist、Apple bundle 結構和提取的資源 | [製品與 JavaScript 指南](docs/javascript-artifact-reconstruction.md)、[Apple 應用](docs/apple-application-analysis.md) |
|
||||
| 處理程序行為 | 終端輸出、互動、退出和檔案系統觀察結果,以及執行對比 | 支援原生 PTY 的 Linux/macOS;[處理程序擷取](docs/process-capture.md) |
|
||||
|
||||
@@ -261,7 +264,7 @@ npx rea-agents@latest setup
|
||||
|
||||
## Star 歷史
|
||||
|
||||
🎉 **GitHub Star 達到 110,000 個,感謝大家!**
|
||||
🎉 **GitHub Star 達到 170,000 個,感謝大家!**
|
||||
|
||||
感謝每一位使用 REA、報告錯誤、提出功能需求、測試建置和貢獻修復的朋友。
|
||||
|
||||
|
||||
+4
-1
@@ -139,6 +139,7 @@ REA 需要 Node.js 22.x(>=22.19)、24.x(>=24.11)或 26+,以及 npm。
|
||||
| JavaScript / Electron | 模块、导入、source map、路由、IPC 和原生扩展关系 | Node.js 和 npm;[应用分析](https://rea.tools/zh-CN/guides/javascript/) |
|
||||
| 网站 | 页面结构、脚本、网络观察结果和按请求获取的截图 | Chrome 系浏览器;[浏览器分析](https://rea.tools/zh-CN/guides/browser/) |
|
||||
| 已保存的网络捕获 | 请求、响应、可访问的载荷和来源位置 | HAR;原生 mitmproxy 捕获需要 Linux 上的 mitmdump;[捕获指南](docs/web-network-captures.md) |
|
||||
| 数据包捕获 | 端点、DNS、TLS SNI、协议分布和 HTTP 请求 | 调用方提供的 tshark 和 capinfos;[pcap 指南](docs/pcap-analysis.md) |
|
||||
| .NET 程序集 | 元数据、CIL 指令、声明的原生依赖和构建对比 | 静态检查;[托管代码指南](docs/managed-code-analysis.md) |
|
||||
| Android APK | 清单声明、类、反编译的方法和引用 | Linux/macOS/Windows x64 上的无界面 JADX 和完整 JDK;[Android 指南](docs/android-analysis.md) |
|
||||
| Android 设备 | 设备、软件包、进程、日志、转储、屏幕截图以及 APK/文件传输 | 调用方提供的 adb(模拟器或实体设备);[ADB 指南](docs/adb-device-analysis.md) |
|
||||
@@ -146,6 +147,8 @@ REA 需要 Node.js 22.x(>=22.19)、24.x(>=24.11)或 26+,以及 npm。
|
||||
| Android 资源 | 解码后的清单、字符串表、语言区域和版本信息 | 调用方提供的 Apktool 启动器;[资源指南](docs/apktool-resource-analysis.md) |
|
||||
| Flutter 应用 | Dart 快照哈希、按 ABI 的库摘要及引擎构建溯源 | 无(静态 APK 解析);[Flutter 指南](docs/flutter-analysis.md) |
|
||||
| 固件 | 区域、提取结果和转交原生分析的内容 | Linux 上的 Binwalk / Unblob;[固件指南](docs/firmware-analysis.md) |
|
||||
| 抓包文件 | 指定字段、重复值、过滤条件与解码规则 | Linux: TShark 4.x + prlimit; [抓包分析指南](docs/packet-captures.md) |
|
||||
| 录制的逻辑信号 | 解码注释、样本范围和诊断信息 | Linux: sigrok-cli 0.7.2 + prlimit; [信号记录指南](docs/signal-recordings.md) |
|
||||
| 软件包与资源 | 文件清单、摘要、plist、Apple bundle 结构和提取的资源 | [制品与 JavaScript 指南](docs/javascript-artifact-reconstruction.md)、[Apple 应用](docs/apple-application-analysis.md) |
|
||||
| 进程行为 | 终端输出、交互、退出和文件系统观察结果,以及运行对比 | 支持原生 PTY 的 Linux/macOS;[进程捕获](docs/process-capture.md) |
|
||||
|
||||
@@ -261,7 +264,7 @@ npx rea-agents@latest setup
|
||||
|
||||
## Star 历史
|
||||
|
||||
🎉 **GitHub Star 达到 110,000 个,感谢大家!**
|
||||
🎉 **GitHub Star 达到 170,000 个,感谢大家!**
|
||||
|
||||
感谢每一位使用 REA、报告错误、提出功能需求、测试构建和贡献修复的朋友。
|
||||
|
||||
|
||||
+6
-3
@@ -24,9 +24,12 @@ provider-dependent behavior to be reproduced safely.
|
||||
|
||||
## Security boundary
|
||||
|
||||
REA authenticates each local provider bridge session with a random capability
|
||||
token and a current-user Unix socket. Tokens are transferred through private
|
||||
session descriptors rather than process arguments or environment variables.
|
||||
REA authenticates its Hopper and Ghidra bridges with per-session random
|
||||
capability tokens. POSIX uses current-user Unix sockets; Windows Ghidra uses
|
||||
authenticated loopback TCP with a private NTFS runtime whose DACL admits only
|
||||
the current user and SYSTEM. See [Windows Ghidra P0](docs/windows-ghidra-p0.md#private-runtime-transport-and-lifecycle).
|
||||
Tokens are transferred through private session descriptors rather than process
|
||||
arguments or environment variables.
|
||||
This is not a sandbox and does not protect against malicious processes already
|
||||
running as the same operating-system user. Opening an untrusted binary
|
||||
delegates parsing and analysis to the selected local provider with that user's
|
||||
|
||||
@@ -28,6 +28,13 @@ entries preserve the declared license and author metadata and include a clearly
|
||||
labeled standard license text. These are documented upstream notice gaps, not
|
||||
recovered copyright notices or a claim that attribution has been fully verified.
|
||||
|
||||
The optional `frida@17.23.2` npm archive and tagged Node binding source also
|
||||
omit a standalone license/copyright document. Its declared
|
||||
`LGPL-2.0 WITH WxWindows-exception-3.1` license is recorded with unchanged
|
||||
[Frida licensing text](licenses/frida-17.23.2-COPYING.txt) and the
|
||||
[GNU Library GPL v2](licenses/LGPL-2.0.txt). These are licensing supplements,
|
||||
not recovered Node binding or prebuild-component copyright notices.
|
||||
|
||||
## Open-source engine credits and licenses
|
||||
|
||||
We thank the maintainers and contributors of the projects below. Their
|
||||
|
||||
@@ -93,7 +93,7 @@ def project_record(state, secrets):
|
||||
safe_bytes, credential_url = transport_value(value, path, transport_url)
|
||||
content_base64 = base64.b64encode(value).decode("ascii")
|
||||
digest = hashlib.sha256(value).hexdigest()
|
||||
hidden = any(secret in value or secret in safe_bytes for secret in byte_secrets) or any(secret in content_base64 or secret in digest for secret in secrets)
|
||||
hidden = any(secret in value or secret in safe_bytes for secret in byte_secrets) or any(secret in content_base64 or secret in digest or secret in str(len(value)) for secret in secrets)
|
||||
safe_text = None
|
||||
try:
|
||||
safe_text = safe_bytes.decode("utf-8", errors="strict")
|
||||
|
||||
@@ -37,12 +37,19 @@ export default defineConfig({
|
||||
link: "/javascript-application-workflows",
|
||||
},
|
||||
{ text: "Browser observation", link: "/browser-observation" },
|
||||
{
|
||||
text: "Web network captures",
|
||||
link: "/web-network-captures",
|
||||
},
|
||||
{ text: "Electron observation", link: "/electron-observation" },
|
||||
{ text: "Process capture", link: "/process-capture" },
|
||||
{ text: ".NET assemblies", link: "/managed-code-analysis" },
|
||||
{ text: "Android applications", link: "/android-analysis" },
|
||||
{ text: "Apple applications", link: "/apple-application-analysis" },
|
||||
{ text: "Firmware", link: "/firmware-analysis" },
|
||||
{ text: "SQLite databases", link: "/sqlite-databases" },
|
||||
{ text: "Packet captures", link: "/packet-captures" },
|
||||
{ text: "Signal recordings", link: "/signal-recordings" },
|
||||
],
|
||||
},
|
||||
{
|
||||
|
||||
@@ -136,8 +136,8 @@ existing analysis families directly, for example
|
||||
## Resource and lifecycle limits
|
||||
|
||||
- Caller values reach adb only as positional arguments in fixed vectors;
|
||||
no tool accepts an arbitrary shell command line, and UI interaction
|
||||
(`input`, `am`, `monkey`) is deliberately not exposed.
|
||||
no tool accepts an arbitrary shell command line, `input`, or `monkey`.
|
||||
The app/activity lifecycle tools use `am start -W` and `am force-stop`.
|
||||
- Per-invocation budgets: client/devices 10-20 s, shell reads 20-60 s with
|
||||
64 KiB-8 MiB capture caps (logcat 4 MiB, screen 32 MiB), transfers and
|
||||
bugreports 600 s. Exceeded budgets report `resource_constraint`; stopped
|
||||
|
||||
@@ -9,6 +9,27 @@ This family is present on repository main and in npm 4.1.0. Check the
|
||||
[released package boundary](installation.md#released-package-and-main) and the
|
||||
connected server's tool list before selecting it.
|
||||
|
||||
## Optional package delivery
|
||||
|
||||
The MCP `inspect_android_package` request keeps complete output when `detail` is
|
||||
omitted or `"complete"`. Select `detail: "summary"` explicitly to receive all
|
||||
package, SDK and permission declarations, engine identity and class/resource
|
||||
counts without decoded manifest text or duplicate raw provider replies. A
|
||||
retaining session keeps the complete original Evidence and returns a distinct
|
||||
linked `inspect_analysis_view` summary. This changes delivery, not JADX analysis.
|
||||
|
||||
Use the returned `normalized_result.parent_evidence_id` with
|
||||
`inspect_analysis_view`, source kind `retained-evidence`, and
|
||||
`view: {kind: "android", facet: "manifest", offset: 0, limit: 4096}` to read the
|
||||
exact decoded manifest without reopening the APK. Offsets and limits count UTF-16
|
||||
code units; follow `coverage.next_offset`. The page preserves `source_status` and
|
||||
`reported_total_bytes`: exhausting retained text does not make a partial decoded
|
||||
manifest complete. Complete source Evidence remains exportable under the existing
|
||||
session lifetime. Servers without retention reject summary before starting JADX.
|
||||
One-shot CLI output remains complete. Older servers require the existing complete
|
||||
request after schema discovery. APK authenticity and runtime behavior remain
|
||||
unverified.
|
||||
|
||||
## Supply tools
|
||||
|
||||
Use a **full JDK 17 or newer** (including its compiler module) and
|
||||
|
||||
@@ -174,7 +174,7 @@ Resolution follows dyld:
|
||||
- `@rpath` tries each `LC_RPATH` of the loading image first, then those of the
|
||||
image that loaded it, up to the process root.
|
||||
- Dependencies load dependents-first in load-command order, and each image
|
||||
loads once per process. A later request whose install name matches an
|
||||
loads once per process. A later @rpath request whose install name matches an
|
||||
already loaded image reuses it.
|
||||
- Symlinks such as `Versions/Current` are followed inside the analyzed root.
|
||||
An escape through `..` or an absolute link is `escapes-target`.
|
||||
|
||||
@@ -6,7 +6,7 @@ graph TD
|
||||
end
|
||||
|
||||
subgraph "Production Composition"
|
||||
COMPOSITION["src/composition/<br/>fresh typed provider and session factories<br/>one-shot binding and independent optional loading<br/>static WASM descriptor: named contract, factory, retryable lifetime"]
|
||||
COMPOSITION["src/composition/<br/>fresh typed provider and session factories<br/>one-shot binding and independent optional loading<br/>static WASM/EVM descriptors: named contracts, factories, retryable lifetimes"]
|
||||
end
|
||||
|
||||
subgraph "Adapters"
|
||||
|
||||
@@ -8,6 +8,10 @@ exported script's native imports under an explicit URL/import-map context.
|
||||
|
||||
REA can attach to a user-owned Chrome-family browser through the Chrome DevTools Protocol (CDP) and produce bounded Evidence about an existing page. This is a passive reverse-engineering capability, not a general browser automation or remote-control surface.
|
||||
|
||||
For request/response bodies and traffic debugging, see
|
||||
[Fiddler Everywhere captures and MCP](web-network-captures.md#reading-fiddler-captures-saz).
|
||||
Inspect native SAZ or HAR 1.2 offline with REA, or connect Everywhere's separate MCP server.
|
||||
|
||||
## Shipped surfaces
|
||||
|
||||
- `list_browser_targets` / `rea list-browser-targets` discovers all eligible page targets at the endpoint by default, with an optional exact-origin filter.
|
||||
|
||||
@@ -312,3 +312,9 @@ artifacts carry their normalized before/after digests and capture states.
|
||||
Every changed or unknown artifact record is returned inline for each aligned
|
||||
step. Missing, truncated, or mismatched capture coverage remains unknown
|
||||
instead of being treated as equality.
|
||||
|
||||
`scenario.event_families` records the sorted, de-duplicated `capture.events`
|
||||
selection. Captures with different selections report event evidence as unknown
|
||||
with a recovery instruction, even when no selected family emitted an event.
|
||||
Captures written before this field existed omit it, so their event evidence is
|
||||
also unknown.
|
||||
|
||||
+11
-1
@@ -62,6 +62,14 @@ unpack .NET single-file hosts or decode IL2CPP metadata. NativeAOT method bodies
|
||||
require native analysis. See the [managed-code guide](managed-code-analysis.md)
|
||||
for supported deployment forms, selection semantics, and coverage limits.
|
||||
|
||||
## SQLite snapshots
|
||||
|
||||
For a local database snapshot, use `rea inspect-sqlite-database PATH --json`.
|
||||
Add `--table NAME --row-limit 20` to read a bounded sample from an ordinary
|
||||
table. This command does not require a native analysis provider. See
|
||||
[SQLite database snapshots](sqlite-databases.md) for database/WAL identity,
|
||||
source-file handling and row-value semantics.
|
||||
|
||||
## Native analysis
|
||||
|
||||
Inspect embedded Go build metadata without installing Go or configuring a
|
||||
@@ -207,7 +215,9 @@ rea import-reference-source /absolute/path/to/source
|
||||
```
|
||||
|
||||
The import records hashes and metadata for the supplied files, separately
|
||||
from observations of the current app. It honors the source tree's `.gitignore`
|
||||
from observations of the current app. For a directory inside a Git checkout,
|
||||
it records the HEAD of the nearest enclosing repository; whether the working
|
||||
tree matches that commit remains unknown. It honors the source tree's `.gitignore`
|
||||
and applies default exclusions for common generated and dependency paths such
|
||||
as `node_modules/`, `dist/`, and `*.log`. Pattern-based exclusions record the
|
||||
matched pattern and whether it came from project, default, caller, or sensitive
|
||||
|
||||
@@ -6,6 +6,12 @@ for the reviewed setup transaction and [client IDs](installation.md#supported-ag
|
||||
The versioned verification notes below describe their stated runs; live service
|
||||
acceptance is verified only where explicitly recorded.
|
||||
|
||||
## Fiddler Everywhere alongside REA
|
||||
|
||||
Connect Fiddler Everywhere's MCP server separately for HTTP traffic debugging,
|
||||
preserving your REA client entry. See the
|
||||
[setup instructions and public skills](web-network-captures.md#fiddler-everywhere-mcp-and-public-skills).
|
||||
|
||||
## GitHub Copilot CLI
|
||||
|
||||
For GitHub Copilot CLI 1.0.95, `gpt-4.1` model metadata can block ordinary
|
||||
@@ -95,6 +101,11 @@ Setup and doctor follow Qwen Code's tilde and working-directory-relative home
|
||||
overrides. Qwen Code also discovers personal skills in `~/.agents/skills`,
|
||||
where setup installs the bundled workflow.
|
||||
|
||||
## Antigravity
|
||||
|
||||
Antigravity 2.0 discovers personal skills in `~/.gemini/config/skills`.
|
||||
Setup installs its selected skill copy there.
|
||||
|
||||
## Gemini CLI
|
||||
|
||||
Gemini CLI uses `$GEMINI_CLI_HOME/.gemini/settings.json` when that home override
|
||||
|
||||
@@ -298,13 +298,42 @@ subsequent wire recovery and clean shutdown. Caller timeout does not interrupt
|
||||
Hopper's synchronous native operation. Socket boundary tests deterministically
|
||||
cover active and queued expiry, late replies and timer cleanup.
|
||||
|
||||
The Linux demo lane remains a separate `verify:hopper:linux` command. Its
|
||||
lifecycle checks attempt competing CLI launches for the active target and a
|
||||
The Linux demo lane remains a separate `verify:hopper:linux` command. Hopper
|
||||
6.4.2's ELF loader did not recognize the original 4,119-byte literal or three
|
||||
non-ASCII byte strings in the measured fixtures. The lane retains complete raw
|
||||
byte checks for those literals through source-owned symbols. Recognition of the
|
||||
original full-length literal is bound to its exact source address; its decoded
|
||||
object coverage remains explicitly unverified on Linux even when recognized.
|
||||
Absent non-ASCII recognition also reports decoded-object coverage as unverified.
|
||||
Recognized non-ASCII and control-text objects must still match their source
|
||||
bytes, encoding, filters and searches.
|
||||
A separate 548-byte ASCII literal requires native fragmentation, shortened
|
||||
provider display, complete decoded fragments and dossier provenance on Linux;
|
||||
the macOS lane retains the original full-length recognition requirement. These
|
||||
are observed fixture/build capabilities, not universal Hopper length limits.
|
||||
Its lifecycle checks attempt competing CLI launches for the active target and a
|
||||
different target, requiring rejection before either can change the owning
|
||||
MCP session's documents or procedures. Lease boundary tests also preserve
|
||||
unresponsive or malformed live endpoints while allowing confirmed stale
|
||||
sockets to recover.
|
||||
|
||||
`verify:setup:linux` exercises installation on a clean supported Linux runner as
|
||||
root. Use a disposable VM: this lane installs Hopper and its Linux runtime
|
||||
dependencies through the system package manager. Set `REA_VERIFY_SERVER_COMMAND`
|
||||
to the CLI entrypoint being verified. The command builds and checks source-owned
|
||||
ELF fixtures before testing the read-only Hopper setup plan, approved
|
||||
installation, scoped doctor, real CLI/MCP analysis, rejection of a modified
|
||||
launcher, idempotent setup without requesting a reinstall, and owned child
|
||||
cleanup. Before installation and after teardown, a clean-runner process scan
|
||||
also rejects surviving Hopper demo helpers or Xvfb groups, including orphans
|
||||
whose launcher exited. This scan observes processes without signaling them.
|
||||
Hopper detection compares executable file identity through `/proc`, covering
|
||||
alternate or configured launcher paths even after an invoked symlink is removed.
|
||||
The dedicated runner must have no unrelated Hopper/Xvfb sessions. It requires
|
||||
the repository's pinned development toolchain and an
|
||||
existing host C compiler, `file`, `nm`, and `strings`. Use `verify:hopper:linux`
|
||||
instead when Hopper is already installed.
|
||||
|
||||
`verify:hopper:fat` is a separate macOS lane requiring installed Hopper and the
|
||||
existing Xcode clang/lipo toolchain. It compiles arm64/x86-64 thin executables and
|
||||
one- and two-slice FAT32 containers, verifies exact/interior address mappings
|
||||
|
||||
+44
-1
@@ -35,7 +35,7 @@ license](https://github.com/morluto/rea/blob/main/third_party/evmole/README.md)
|
||||
and whitespace. Decoded-byte SHA-256 identifies actual selected EVM bytes.
|
||||
Neither digest is the Ethereum Keccak code hash.
|
||||
- Full bytecode, upstream function offsets and original producer representation
|
||||
are returned inline. Offsets are byte positions in decoded bytecode.
|
||||
are returned inline by default. Offsets are byte positions in decoded bytecode.
|
||||
- Upstream also returns terminal metadata during selector recovery. Its reported
|
||||
values remain in `raw_result`; integer precision and non-text-key coverage are
|
||||
unknown in the JS binding. Full bytecode retains the original encoded metadata.
|
||||
@@ -47,6 +47,49 @@ license](https://github.com/morluto/rea/blob/main/third_party/evmole/README.md)
|
||||
- Analysis is offline: no target runtime execution, implicit chain/RPC/explorer
|
||||
lookup, wallet action, transaction broadcast or network request.
|
||||
|
||||
## Selected interface delivery and retained views
|
||||
|
||||
Existing CLI calls and MCP calls with omitted or `complete` detail keep the
|
||||
complete inline result. On a retaining MCP server, select `detail: "summary"`
|
||||
when the question needs every inferred function and its identity/uncertainty,
|
||||
without bytecode text or duplicated raw stdout:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_evm_interface",
|
||||
"arguments": {
|
||||
"path": "/artifacts/runtime.hex",
|
||||
"encoding": "hex",
|
||||
"detail": "summary"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The result is distinct derived Evidence linked to its complete parent. Its
|
||||
`normalized_result.summary` preserves every producer-returned selector, decoded
|
||||
byte offset, dispatch, argument/mutability string including null, carrier/decoded
|
||||
identities, original limitations and explicit unknowns. Payload `evidence_kind`
|
||||
remains `inferred`. It preserves stderr/truncation and records stdout text length.
|
||||
Complete-within-view coverage covers returned candidates; discovery completeness
|
||||
remains unknown even for an empty or exhausted collection. Analysis still runs;
|
||||
delivery selection does not save interpreter work or change source parameters.
|
||||
|
||||
Use `normalized_result.parent_evidence_id` with `inspect_analysis_view` source
|
||||
kind `retained-evidence`. EVM view kind `evm` pages `functions` by original row
|
||||
order, or `bytecode_hex`, `stdout`, `stderr` by UTF-16 code units. Hex pages preserve
|
||||
the original lowercase text without a prefix; two characters represent one decoded
|
||||
byte. A text-page offset differs from a function's decoded-byte offset. Complete
|
||||
parent Evidence remains accessible through the normal ledger/export workflow;
|
||||
reading views does not rerun analysis or change its identity. Existing record
|
||||
retention and close semantics are unchanged.
|
||||
|
||||
Portable historical complete EVM Evidence can use source kind `inline` without
|
||||
a provider. A fixed server without Evidence retention rejects selected summary
|
||||
delivery before starting analysis and continues to support complete delivery.
|
||||
Discover the actual advertised schema before sending `detail` to an older server;
|
||||
use its complete operation when an equivalent workflow is required. The one-shot
|
||||
CLI keeps its existing complete result.
|
||||
|
||||
## Resource and lifecycle limits
|
||||
|
||||
Inputs are limited to 4 MiB, complete replies to 16 MiB and combined diagnostic
|
||||
|
||||
@@ -25,6 +25,26 @@ The equivalent MCP methods are `identify_flutter_build` (with `path`) and
|
||||
arm64-v8a, then x86_64, then x86, then armeabi-v7a, then the remaining
|
||||
ABIs alphabetically).
|
||||
|
||||
## Optional retained AOT delivery
|
||||
|
||||
MCP `inspect_dart_aot` keeps complete output when `detail` is omitted or
|
||||
`"complete"`. Explicit `detail: "summary"` returns the selected target/ABI/library
|
||||
identity, all snapshot headers, structural coverage, string-pool coverage and
|
||||
original lower-bound counts without the string lists. It retains complete source
|
||||
Evidence and returns a distinct linked `inspect_analysis_view` summary. Analysis
|
||||
and the caller's ABI choice are unchanged; one-shot CLI output stays complete.
|
||||
|
||||
Use `normalized_result.parent_evidence_id` as the retained source for
|
||||
`inspect_analysis_view`, with `view: {kind: "dart-aot", collection: "package_uris",
|
||||
offset: 0, limit: 64}`. Collections are `package_uris`, `dart_uris` and
|
||||
`class_like_tokens`. Pages preserve the original strings and order, selected ABI,
|
||||
library identity and source coverage. `returned_counts` describe producer-returned
|
||||
rows; counts ending in `_at_least` remain lower bounds. Exhausting a page sequence
|
||||
does not make a partial pool complete or turn byte-pattern tokens into typed
|
||||
classes/functions. Views do not reopen the APK. Original Evidence remains
|
||||
exportable under its existing session lifetime. Summary requires a retaining
|
||||
server; older servers keep the complete request after schema discovery.
|
||||
|
||||
## What the AOT inspection returns
|
||||
|
||||
`inspect_dart_aot` reads one ABI's `libapp.so` in full and reports:
|
||||
|
||||
+10
-10
@@ -22,16 +22,16 @@ header declarations, not proof that every instruction conforms to that ISA.
|
||||
Provider selection returns the specific unsupported constraint rather than
|
||||
guessing a compatible profile:
|
||||
|
||||
| Declaration | Outcome |
|
||||
| ----------------------------------------------- | --------------------------------------------------------------------------- |
|
||||
| Little-endian ELF64 `ET_EXEC`, double-float ABI | admitted, `RISCV:LE:64:default` + `gcc` |
|
||||
| Little-endian ELF32 `ET_EXEC`, double-float ABI | admitted, `RISCV:LE:32:default` + `gcc` |
|
||||
| Big-endian | refused; Ghidra 12.1.4 ships no big-endian RISC-V language |
|
||||
| `ET_REL`, `ET_DYN`, other `e_type` | refused with the declared value |
|
||||
| `EF_RISCV_RVE` | refused; the embedded 15-register ABI needs its own processor specification |
|
||||
| `EF_RISCV_TSO` | refused; the total-store-order memory model is unverified here |
|
||||
| Soft, single, or quad float ABI | refused; the pinned generic compiler specification models double-float |
|
||||
| Reserved or newer `e_flags` bits | refused with the offending bits named |
|
||||
| Declaration | Outcome |
|
||||
| ----------------------------------------------- | ---------------------------------------------------------------------------------------- |
|
||||
| Little-endian ELF64 `ET_EXEC`, double-float ABI | admitted, `RISCV:LE:64:default` + `gcc` |
|
||||
| Little-endian ELF32 `ET_EXEC`, double-float ABI | admitted, `RISCV:LE:32:default` + `gcc` |
|
||||
| Big-endian | refused; Ghidra 12.1.4 ships no big-endian RISC-V language |
|
||||
| `ET_REL`, `ET_DYN`, other `e_type` | refused with the declared value |
|
||||
| `EF_RISCV_RVE` | refused; the embedded 16-register profile (x0–x15) needs its own processor specification |
|
||||
| `EF_RISCV_TSO` | refused; the total-store-order memory model is unverified here |
|
||||
| Soft, single, or quad float ABI | refused; the pinned generic compiler specification models double-float |
|
||||
| Reserved or newer `e_flags` bits | refused with the offending bits named |
|
||||
|
||||
`e_type` is the only load-shape declaration this lane reads. A fixed-address
|
||||
`ET_EXEC` declaration does **not** establish static linkage: `PT_INTERP` and
|
||||
|
||||
+37
-5
@@ -1,6 +1,6 @@
|
||||
# IDA Pro provider
|
||||
# IDA provider
|
||||
|
||||
REA adapts an existing [mrexodia/ida-pro-mcp](https://github.com/mrexodia/ida-pro-mcp) server into its native analysis tools. Install and configure IDA, Hex-Rays, Python, and upstream MCP using that repository's instructions. REA does not install, upgrade, activate, or license them, and does not bundle another IDA server.
|
||||
REA adapts an existing IDA MCP server into its native analysis tools. Existing IDA Pro profiles use [mrexodia/ida-pro-mcp](https://github.com/mrexodia/ida-pro-mcp); an optional native bridge for IDA Pro and IDA Free is described below. Install and configure your selected engine and upstream MCP using their instructions. REA does not install, upgrade, activate, or license them, and does not bundle another IDA server.
|
||||
|
||||
Choose one lifecycle mode in an upstream MCP JSON registration. Set `REA_IDA_MCP_CONFIG` to that file, then select `ida` once through `REA_ANALYSIS_PROVIDER`, the CLI's `--provider ida`, or MCP's `open_binary.provider_id`. No provider startup occurs during discovery.
|
||||
|
||||
@@ -32,7 +32,33 @@ Open the **original input binary** in REA, not an `.idb` or `.i64` database. Its
|
||||
|
||||
REA's close operation releases its MCP connection or proxy. It never saves or closes the existing GUI database. External edits remain visible because results are live and are not replayed from snapshots. These checks do not lock the GUI or prove that the database is unpatched.
|
||||
|
||||
The legacy producer moves the active GUI as a side effect: assembly and raw function instructions jump to the function, and successful pseudocode opens or reuses a pseudocode view. A function dossier does both. `binary_session` reports these operations with `effects.may_show_ui: true` in attached mode; headless operations and the other attached operations report `false`.
|
||||
## Optional native IDA MCP bridge
|
||||
|
||||
IDA Pro and IDA Free can use the separate [ida-free-mcp native plugin](https://github.com/0xshlomil/ida-free-mcp) through an explicit attached registration. Install and start that plugin using its upstream instructions, open the original binary in the IDA GUI, and wait for analysis. The native MCP 1.1.2 release binds `0.0.0.0` without a listen-address setting. Restrict inbound access with the host firewall or use an upstream build that binds only to loopback before starting it. REA's loopback URL restriction does not change the plugin's listener. REA does not install IDA, build the plugin, activate a license, or launch the GUI.
|
||||
|
||||
```json
|
||||
{
|
||||
"mode": "attached",
|
||||
"protocol": "native",
|
||||
"url": "http://127.0.0.1:13337/mcp"
|
||||
}
|
||||
```
|
||||
|
||||
Set `REA_IDA_MCP_CONFIG` to this file and select `ida` using the same CLI/MCP commands above. Omitting `protocol` preserves existing IDA Pro registrations. Native mode requires `resources/read` for `ida://idb/metadata` and its recorded input SHA-256; it never substitutes a digest of an unrelated local file. The native resource's `path` describes the IDB in the upstream protocol, not the original input path. REA retains it and checks database identity and image base before and after requests.
|
||||
|
||||
This experimental profile provides function/string inventory and search, function lookup, assembly, GUI pseudocode, callees, address xrefs, instruction windows, and function dossiers. Direct callers remain unavailable: native code/data xref labels do not prove call edges. It requires neither IDAPython nor idalib. Native headless registrations are rejected.
|
||||
|
||||
Native string observations use upstream C-string rendering; other representations remain unknown. Literal searches request conservative candidates and apply REA's original case and Unicode matching rules. Regex searches retain the full scan because native byte-oriented regex semantics cannot replace REA's Unicode regex semantics. All candidate pages are examined, and upstream search errors remain failures.
|
||||
|
||||
Native callee resolution batches unique queries and checks each reply's query identity. Results remain live: a recorded input hash does not establish database revision, so cross-request analysis caches cannot prove freshness. For focused questions, use pseudocode, callees, or instruction-window tools; a function dossier requests additional facets and returns more Evidence.
|
||||
|
||||
IDA Free's cloud decompiler needs network access and may request consent. Follow its licensing and supported architecture limits; see [IDA Free](https://hex-rays.com/ida-free). Decompilation moves GUI focus and the upstream plugin closes existing pseudocode tabs before requesting fresh output. REA advertises UI/network effects and closes only its MCP connection, preserving the open database. Rendered pseudocode has upstream traversal limits and does not prove complete source recovery. Cloud/widget failures remain explicit failures; retry after the GUI finishes or dismiss its prompt.
|
||||
|
||||
Native MCP 1.1.2 can navigate in the previously focused pseudocode tab when switching functions. Use upstream 1.1.3 or a source build including [the navigation correction](https://github.com/0xshlomil/ida-free-mcp/commit/6adf2c615a2a03661059673e9c136ef28d0dc778), which focuses the disassembly view **before** jumping to the requested entry. This REA profile does not repair stale GUI output. Verification must include two different functions before claiming reliable function switching. The initial Linux verification used a locally corrected 1.1.2 plugin; upstream 1.1.3 has not been verified by that run.
|
||||
|
||||
Native tool observations retain actual tool names and complete JSON text. Upstream `structuredContent` can be a shortened preview even while text contains the full result. REA reads the full native representation and rejects missing/malformed text instead of treating previews as complete observations.
|
||||
|
||||
The legacy producer moves the active GUI as a side effect: assembly and raw function instructions jump to the function, and successful pseudocode opens or reuses a pseudocode view. A function dossier does both. Native GUI decompilation also changes the active view. `binary_session` reports these operations with `effects.may_show_ui: true`; headless operations and the other attached operations report `false`.
|
||||
|
||||
## Give REA a target for headless analysis
|
||||
|
||||
@@ -98,9 +124,9 @@ For guided registration, set `REA_IDA_MCP_CONFIG` before `rea setup --client <cl
|
||||
|
||||
## Coverage and evidence
|
||||
|
||||
Both profiles support procedure/string inventory and literal or regex search, procedure resolution, pseudocode, assembly, raw function instructions, resolved callees, incoming address references, and a function dossier. Inventories and modern pseudocode/assembly follow upstream pagination. Upstream limits on modern xrefs and callees are retained as explicit truncation limitations. In the modern headless profile, targets that are not proven x86-only use processor-neutral `callgraph` edges instead of the x86 instruction checks in upstream `callees` (mrexodia/ida-pro-mcp#549). REA requests depth zero: the pinned upstream emits the root's outgoing edges without scanning child functions, then REA batches function-name lookup for those addresses. The graph covers recognized function destinations only; imports and other non-function targets remain unknown. A server without `callgraph` returns an empty relationship list with an explicit coverage limitation. Graph and path workflows retain those limitations alongside observed edges. A failed or inconsistent graph reply remains an error. A transport-level truncated preview is rejected rather than reported as complete evidence; the error preserves upstream retrieval guidance.
|
||||
All profiles support procedure/string inventory and literal or regex search, procedure resolution, pseudocode, assembly, raw function instructions, resolved callees, incoming address references, and a function dossier. Inventories and modern pseudocode/assembly follow upstream pagination. Upstream limits on modern xrefs and callees are retained as explicit truncation limitations. In the modern headless profile, targets that are not proven x86-only use processor-neutral `callgraph` edges instead of the x86 instruction checks in upstream `callees` (mrexodia/ida-pro-mcp#549). REA requests depth zero: the pinned upstream emits the root's outgoing edges without scanning child functions, then REA batches function-name lookup for those addresses. The graph covers recognized function destinations only; imports and other non-function targets remain unknown. A server without `callgraph` returns an empty relationship list with an explicit coverage limitation. Graph and path workflows retain those limitations alongside observed edges. A failed or inconsistent graph reply remains an error. A transport-level truncated preview is rejected rather than reported as complete evidence; the error preserves upstream retrieval guidance.
|
||||
|
||||
The legacy profile additionally supports direct callers. Its producer may report call-site or interior addresses; REA resolves the reported function symbol to its canonical entry. The modern profile does not prove direct callers from its code/data xref labels, so that capability is unavailable and the dossier marks it unknown.
|
||||
The legacy profile additionally supports direct callers. Its producer may report call-site or interior addresses; REA resolves the reported function symbol to its canonical entry. The modern and native profiles do not prove direct callers from its code/data xref labels, so that capability is unavailable and the dossier marks it unknown.
|
||||
|
||||
Complete body ranges, typed reference edges, CFG, referenced data, source locals, comment kinds, and database revision are unavailable. Empty unsupported dossier facets mean **unknown**, not observed absence. `binary_overview`, whole-symbol inventory, mutation, debugging, arbitrary Python forwarding, and requested GUI navigation are outside this adapter's coverage. Check session availability before composing a workflow that needs those facets.
|
||||
|
||||
@@ -110,4 +136,10 @@ Complete body ranges, typed reference edges, CFG, referenced data, source locals
|
||||
|
||||
Run `npm run verify:ida -- --target /absolute/path/to/program --procedure main` with a configured upstream server. It exercises production CLI and MCP tools, validates their output contracts and Evidence, checks original-input preservation and session cleanup, and keeps local target data out of its console summary. Supply `--package-root` to verify an extracted or installed package. See [testing](testing.md) for the distinction between adapter tests and real-provider evidence.
|
||||
|
||||
Native verification requires `--second-procedure other_function --second-marker unique_second_body_text`; these options remain optional for the other profiles. The marker must occur in the second function's pseudocode and be absent from the first; the verifier also checks that returning to the first restores its text.
|
||||
|
||||
The verifier requires the resolved procedure in both inventory and procedure search, then compares address, assembly, pseudocode, instruction windows and relationship results with the CLI/MCP dossier. Supply `--expected-string known_literal` to check a known positive string inventory/search result and `--expected-xref 0xSOURCE` to check a known reference source address. Choose those oracles independently from the operation under test. Without them, string and xref correctness are listed in `unverified`; valid output shapes alone do not establish those behaviors. Callee parity does not independently establish semantic completeness. Native callee successes must report an explicit truncation marker; missing markers remain protocol failures.
|
||||
|
||||
The initial real workflows use stdio registrations for the legacy Windows GUI profile and a Windows x64 headless supervisor with IDA 9.3. The headless workflow has also passed on macOS arm64 with IDA Professional 9.1 (build 9.1.250226) against the same upstream commit `c133c3853faa111a9b00ee615c013b720d0c4acd`, using a source-built arm64 Mach-O fixture; Mach-O C symbols carry a leading underscore, so the procedure name is `_rea_fixture_add`. Streamable HTTP is tested against a loopback MCP fixture with the pinned client SDK; a real IDA HTTP-supervisor workflow remains unverified. Linux headless engine operation, other IDA releases, architectures, and modern attached GUI tools require their corresponding real workflows before support is claimed.
|
||||
|
||||
Native Windows verification uses IDA Pro 9.3 with the public two-function fixture and Windows `whoami.exe`, using upstream `c9d15d98e8a143fc74f6e14037c48f63889b6d6c` with a loopback-listener change. It covers production CLI/MCP parity, distinct-function switching, identity and connection cleanup. See the [correctness and performance measurements](https://github.com/morluto/rea/blob/main/experiments/analysis-efficiency/ida-native-windows.md). This follow-up did not run Windows IDA Free or native macOS.
|
||||
|
||||
+50
-14
@@ -57,6 +57,10 @@ Hopper installation or any file write. Declining or pressing Ctrl-C makes no
|
||||
changes. Agent configuration writes preserve unrelated entries and comments,
|
||||
create backups, use atomic replacement, and verify their result. Setup and
|
||||
uninstall retain an existing `.rea.backup` rather than replacing the first snapshot.
|
||||
If that snapshot differs from the current configuration, setup previews and
|
||||
preserves a verified `.rea.backup.<sha256>.recovery` copy of the current bytes.
|
||||
Both snapshots remain available; a failed rollback reports the current recovery
|
||||
copy's path and the restore failure.
|
||||
Setup also retains valid environment overrides in REA's existing server entry,
|
||||
such as `REA_MCP_INPUT_SCHEMA_PROFILE` and `REA_MCP_MAX_RESPONSE_BYTES`.
|
||||
Detected provider settings replace only matching environment keys.
|
||||
@@ -101,13 +105,11 @@ rea setup --yes --all-detected --install-hopper --json
|
||||
|
||||
### Skill destinations
|
||||
|
||||
The bundled skill is installed where each selected client discovers personal
|
||||
skills: Claude Code uses `~/.claude/skills` (or
|
||||
`$CLAUDE_CONFIG_DIR/skills` when configured). Gemini CLI uses `.agents/skills`
|
||||
under its selected home; Antigravity 2.0 uses `~/.gemini/config/skills`.
|
||||
Other integrations, including Qwen Code, use `~/.agents/skills`.
|
||||
A mixed selection plans each distinct destination. Selecting the skill without a client uses the shared
|
||||
directory. Setup leaves existing skill copies in other locations untouched.
|
||||
The bundled skill is installed in each selected client's personal skill directory.
|
||||
See [client configuration](client-configuration.md) for client-specific roots,
|
||||
profiles, and environment overrides. A mixed selection plans each distinct
|
||||
destination. Selecting the skill without a client uses `~/.agents/skills`.
|
||||
Setup leaves existing skill copies in other locations untouched.
|
||||
|
||||
`rea setup --skill-client claude_code --yes --json` refreshes and verifies the
|
||||
selected skill destination independently of MCP registration selection. The
|
||||
@@ -319,8 +321,8 @@ The Hopper launcher action used by REA creates a document from an executable;
|
||||
its supported command-line interface does not attach to an already-open
|
||||
document. While another live REA session owns the same target and loader
|
||||
profile, a second session reports the owning run ID instead of opening a
|
||||
duplicate document. Closing the owning REA session releases this guard; Hopper
|
||||
keeps its document open. A later REA session can therefore open another
|
||||
duplicate document. On macOS, closing the owning REA session releases this guard;
|
||||
Hopper keeps its document open. A later REA session can therefore open another
|
||||
document for the same target. REA does not currently identify, focus, or reuse
|
||||
that existing GUI document, and the supported launcher exposes no attach or
|
||||
reuse action for it.
|
||||
@@ -343,8 +345,14 @@ Hopper application per user across cooperating REA processes. A competing
|
||||
CLI or MCP session receives the owning session ID before Hopper is launched,
|
||||
for both the same target and a different target. Switch targets through that
|
||||
MCP session, or close it before starting another session. Closing the owner
|
||||
releases the reservation. This does not attach to or claim ownership of
|
||||
manually opened Hopper applications.
|
||||
releases the reservation and stops REA's owned demo application and private
|
||||
display; the headless Linux demo does not retain documents after MCP shutdown.
|
||||
Its helper watches a private lifetime pipe, so an owner crash or SIGKILL also
|
||||
stops those children, without relying on a reusable PID or a process-name scan.
|
||||
A later session recovers the dead owner's stale lease after child cleanup;
|
||||
private runtime files can remain when REA itself was killed. This does not attach to
|
||||
or claim ownership of manually opened Hopper applications. An application left
|
||||
running by an older REA version must still be closed before starting a new session.
|
||||
|
||||
Stale lease recovery is serialized. If an interrupted recovery leaves its
|
||||
reservation directory, the diagnostic identifies that exact path. Stop REA
|
||||
@@ -437,8 +445,9 @@ For cursor navigation, string decoding, symbol-name limits, and regex matching,
|
||||
see [Hopper analysis behavior](native-investigation.md#hopper-analysis-behavior).
|
||||
|
||||
Closing or switching a target closes its bound Hopper document, shuts down REA's
|
||||
bridge and removes its temporary socket directory while preserving the Hopper
|
||||
application and unrelated documents. A `cleanup_incomplete`
|
||||
bridge and removes its temporary socket directory. The macOS Hopper application
|
||||
and unrelated documents remain open; the owned Linux demo application stops.
|
||||
A `cleanup_incomplete`
|
||||
result identifies resources whose cleanup could not be verified.
|
||||
REA retains unresolved cleanup ownership and any confirmed shutdown phases.
|
||||
After addressing the reported failure, retry `close_binary` on the same
|
||||
@@ -605,12 +614,39 @@ rea doctor --json
|
||||
The copy step requires the destination to be absent and preserves an existing
|
||||
decompiler. This recipe is specific to macOS x64: the
|
||||
[Ghidra 12.1.4 Makefile](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/Ghidra/Features/Decompiler/src/decompile/cpp/Makefile)
|
||||
explicitly selects `-arch x86_64` on Darwin. For other architectures or a Gradle
|
||||
explicitly selects `-arch x86_64` on Darwin. For a Gradle
|
||||
build, follow the matching release's
|
||||
[native component instructions](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/GhidraDocs/GettingStarted.md#building-native-components).
|
||||
REA accepts the matching executable under either `os/<platform>/` or
|
||||
`build/os/<platform>/`; it does not build the component during setup or diagnosis.
|
||||
|
||||
### macOS arm64 native decompiler
|
||||
|
||||
On Apple Silicon the same source build works with explicit overrides; no
|
||||
Makefile edits and no Rosetta are needed. The Darwin block in the
|
||||
[Ghidra 12.1.4 Makefile](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/Ghidra/Features/Decompiler/src/decompile/cpp/Makefile)
|
||||
pins `-arch x86_64`, but `ARCH_TYPE` and `OSDIR` are ordinary make variables,
|
||||
and command-line assignments replace them for every compile and link rule:
|
||||
|
||||
```bash
|
||||
cd "$GHIDRA_INSTALL_DIR/Ghidra/Features/Decompiler/src/decompile/cpp"
|
||||
make ghidra_opt ARCH_TYPE="-arch arm64" OSDIR=mac_arm_64
|
||||
decompiler_dir="$GHIDRA_INSTALL_DIR/Ghidra/Features/Decompiler/os/mac_arm_64"
|
||||
mkdir -p "$decompiler_dir"
|
||||
test ! -e "$decompiler_dir/decompile" && test ! -L "$decompiler_dir/decompile" &&
|
||||
install -m 755 ghidra_opt "$decompiler_dir/decompile"
|
||||
rea doctor --json
|
||||
```
|
||||
|
||||
Verified end to end on macOS 15.6 arm64 with Apple clang 17 and the stock
|
||||
bison 2.3, against Ghidra 12.1.4: the build produces a Mach-O arm64
|
||||
`ghidra_opt`, the doctor check `ghidra-native_decompiler` turns healthy with
|
||||
"reports mach-o arm64, which matches darwin/arm64", and `rea decompile` on an
|
||||
arm64 Mach-O returns decompiled C through the headless provider. The JDK is
|
||||
not needed for the make step, only for the Ghidra provider. If your toolchain
|
||||
rejects the inherited `-mmacosx-version-min=10.6` flag, retry the make with
|
||||
`ADDITIONAL_FLAGS="-w"`.
|
||||
|
||||
### Ghidra import language and analysis seeds
|
||||
|
||||
Ghidra detects the language from the executable header. To set it (for example, Borland C++ PE images):
|
||||
|
||||
@@ -104,6 +104,15 @@ static inferences. Dynamic or ambiguous IPC is retained without a pairing edge.
|
||||
Runtime entities retain passive-CDP authority, while a successful static/runtime
|
||||
correspondence is a separately identified cross-layer inference.
|
||||
|
||||
Source module links retain their resolution mode. Static imports in `.ts` and
|
||||
`.tsx` are `compiler-dependent` when compiler options have not been retained.
|
||||
REA compares Node import and require resolution: a shared outcome remains
|
||||
available with a limitation; divergent outcomes remain unavailable with both
|
||||
conditional branches and their limitations. This models source relationships,
|
||||
not verified emitted code or execution. A declaration file found only through
|
||||
an emitted-extension fallback is retained as a candidate, not a runtime target;
|
||||
an explicitly selected path keeps its original identity.
|
||||
|
||||
## Evidence and authority
|
||||
|
||||
Every observation and edge carries all of the following:
|
||||
|
||||
@@ -23,6 +23,14 @@ artifact digest and structural graph ID. Semantic tracing requires the semantic
|
||||
relation graph and reports when it is unavailable rather than treating missing
|
||||
data as an empty graph.
|
||||
|
||||
Explicit TypeScript `import type` and `export type` declarations remain source
|
||||
facts and do not create runtime import or export relationships. Type export names
|
||||
are reported separately from value export names. Inline `type` specifiers can
|
||||
leave an empty import or re-export with side effects under
|
||||
[`verbatimModuleSyntax`](https://www.typescriptlang.org/tsconfig/verbatimModuleSyntax.html);
|
||||
REA retains those dependency candidates and reports the unavailable compiler
|
||||
options rather than assuming the dependency is erased.
|
||||
|
||||
The semantic graph wire format stores shared non-location provenance once in
|
||||
its required `evidence_contexts` table. Each node, relation, fingerprint, and
|
||||
unknown retains its exact `evidence.location` and names the owning context with
|
||||
@@ -156,9 +164,12 @@ to the comparison Evidence in a live session.
|
||||
`compare_source_to_bundle` compares one cryptographically committed
|
||||
`HistoricalSourceGraph` with authenticated application-graph Evidence. The
|
||||
stable scoring model reports every admitted signal and weight: exact source
|
||||
digest, source-map original path, exact current path, path suffix, basename, and
|
||||
language extension. Exact digest wins over location inference; weak basename
|
||||
or extension evidence never forces a mapping.
|
||||
digest, source-map original path, exact current path, path suffix, Unicode-
|
||||
equivalent path, basename, and language extension. Exact digest wins over
|
||||
location inference; weak basename, extension, and Unicode-equivalence evidence
|
||||
never forces a mapping. Unicode-equivalent spellings are candidate evidence
|
||||
only because distinct filesystem paths can preserve those spellings as separate
|
||||
files; multiple canonical-equivalent candidates remain unknown.
|
||||
|
||||
Each historical source file is classified as `unchanged`, `modified`,
|
||||
`removed`, `split`, `merged`, `duplicated`, or `unknown`. Multiple equally
|
||||
|
||||
@@ -1,28 +1,31 @@
|
||||
# JavaScript static/runtime reconciliation
|
||||
|
||||
REA can combine an existing static JavaScript Application Graph with one or
|
||||
more passive browser, Electron page, or Node/Electron V8 Inspector captures without collapsing their different
|
||||
authorities. The `reconcile_javascript_runtime` MCP tool and
|
||||
more browser, Electron, or Node/Electron V8 Inspector observations while
|
||||
preserving their different authorities. The `reconcile_javascript_runtime` MCP tool and
|
||||
`rea reconcile-javascript-runtime` CLI command accept Evidence records and
|
||||
return derived Evidence containing a combined JavaScript Application Graph
|
||||
and explicit match, ambiguity, mismatch, and unknown classifications.
|
||||
|
||||
The operation performs no filesystem, browser, or Electron I/O. It accepts only
|
||||
semantically verified Evidence produced by `analyze_javascript_application`,
|
||||
`inspect_web_page`, `inspect_electron_page`, or
|
||||
`observe_javascript_runtime`; matching operation names alone
|
||||
`inspect_web_page`, `inspect_electron_page`, `observe_javascript_runtime`, or
|
||||
`capture_electron_scenario`; matching operation names alone
|
||||
is insufficient. Provider identity, predicate type, subject digest, target
|
||||
parameters, and normalized-result schemas are checked before reconciliation.
|
||||
|
||||
`capture_electron_scenario` supplies target-only partial observations. It does
|
||||
not establish loaded-script coverage; see [Electron observation](electron-observation.md).
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Run `analyze_javascript_application` against exactly one application layer.
|
||||
Optional cache or assets directories can be analyzed independently and
|
||||
supplied as additional layers.
|
||||
2. Run `inspect_web_page`, `inspect_electron_page`, or the attach-only
|
||||
`observe_javascript_runtime`. Capturing script sources
|
||||
is optional, but a source capture provides the strongest byte
|
||||
identity.
|
||||
`observe_javascript_runtime`, or supply existing `capture_electron_scenario`
|
||||
Evidence with the partial scope above. Capturing script sources is optional,
|
||||
but a source capture provides the strongest byte identity.
|
||||
3. Pass the resulting Evidence records to `reconcile_javascript_runtime`.
|
||||
Browser URLs normally need an explicit URL-prefix mapping. Extracted
|
||||
Electron and V8 Inspector file locations are mapped automatically when runtime files remain
|
||||
@@ -104,7 +107,7 @@ nodes. REA evaluates compatible static candidates conservatively:
|
||||
|
||||
Every successful cross-layer `observed_as` edge has
|
||||
`authority: cross-layer-reconciliation` and `state: inferred`. The underlying
|
||||
artifact/AST observations and passive-CDP observations remain unchanged and
|
||||
artifact/AST observations and runtime observations remain unchanged and
|
||||
retain their own Evidence links.
|
||||
|
||||
## Loaded, resident, and not observed
|
||||
|
||||
@@ -40,6 +40,47 @@ inspection is needed; see [selected member inspection](#selected-member-inspecti
|
||||
Compare separate builds with [member comparison](#member-comparison), preserving
|
||||
their paths, digests, and selection scope.
|
||||
|
||||
### Metadata overview and retained rows
|
||||
|
||||
The existing CLI and omitted/complete MCP detail return complete metadata
|
||||
inventories. For identity and classification questions, select
|
||||
`inspect_managed_artifact` with
|
||||
`{"path":"/absolute/path/Assembly.dll","detail":"summary"}` on the first call.
|
||||
This keeps the complete source in the session and returns a distinct linked
|
||||
`inspect_analysis_view` overview with complete PE/CLI, classification and its
|
||||
evidence, metadata status, module/MVID, assembly, frameworks, returned-row counts,
|
||||
coverage/issues and limitations. No assembly is loaded or executed.
|
||||
|
||||
Use `normalized_result.parent_evidence_id` to page original rows:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "<parent_evidence_id>"
|
||||
},
|
||||
"view": {
|
||||
"kind": "managed",
|
||||
"collection": "attributes",
|
||||
"offset": 0,
|
||||
"limit": 16
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Collections are `references`, `resources` and `attributes`; rows retain their
|
||||
original values, nulls, order, metadata tokens and offsets. Pages retain source
|
||||
metadata status and coverage. Exhaustion covers the returned parent collection,
|
||||
not all possible metadata, and empty rows under partial/unavailable metadata do
|
||||
not establish absence. Tokens are build-local and require the recorded artifact
|
||||
SHA-256 and known module MVID for cross-observation matching.
|
||||
|
||||
Retained views do not repeat the parser or require the source file to remain
|
||||
available. Portable views use the authenticated original Evidence inline,
|
||||
including the existing `inspect-analysis-view` CLI. Session record lifetime and
|
||||
export/import semantics remain unchanged. A server without Evidence retention
|
||||
rejects selected summary before analysis; complete delivery remains available.
|
||||
|
||||
## Shipped scope
|
||||
|
||||
The canonical parser admits PE/CLI bytes and their metadata/CIL without loading
|
||||
@@ -110,7 +151,7 @@ assembly. Older member Evidence without `selection` remains readable.
|
||||
|
||||
Start by identifying the artifact with `inspect_managed_artifact`, then inspect
|
||||
its types and CIL with `inspect_managed_members`. Use [member comparison](#member-comparison)
|
||||
for another build, and [managed/native boundary rules](#managednative-boundary-rules)
|
||||
for another build, and [managed/native boundary rules](#managed-native-boundaries)
|
||||
when matching declared native calls to supplied native Evidence. Static inspection
|
||||
does not load or execute the assembly. Reconstructed structure and unresolved
|
||||
native/runtime questions retain their own authority and coverage.
|
||||
@@ -179,6 +220,8 @@ to native function addresses requires verified
|
||||
provider evidence; names or vtable similarity alone do not prove that a native
|
||||
function implements a specific managed method.
|
||||
|
||||
<div id="managed-native-boundaries"></div>
|
||||
|
||||
## Managed/native boundary rules
|
||||
|
||||
`verify_managed_native_boundaries` compares declared P/Invoke module and entry
|
||||
|
||||
+66
-2
@@ -21,6 +21,12 @@ directly to `compare_functions`, and `inspect_artifact` Evidence to
|
||||
`compare_artifacts`. Use `get_evidence_bundle` when the task needs broader
|
||||
retained session history or an explicit bundle for transfer.
|
||||
|
||||
Evidence and bundle imports reject unsupported fields in their envelopes,
|
||||
identities, manifests and other fixed metadata before authentication. Imports
|
||||
leave retained session state unchanged when validation fails. Keep application
|
||||
metadata in the supported JSON payloads (`parameters`, `raw_result`,
|
||||
`normalized_result`, or analysis-profile parameters), where keys are preserved.
|
||||
|
||||
Tool execution failures return `isError: true` and the complete canonical
|
||||
`{ "error": ... }` diagnostic as JSON in text `content`. Read that diagnostic
|
||||
for the error code, target details, partial observations, and remediation.
|
||||
@@ -78,6 +84,12 @@ unknown metadata. Missing omission counts remain null. Summary output carries
|
||||
every matching record; actual transport admission still applies. Omitted
|
||||
`detail` preserves complete canonical bundle delivery; filters require summary.
|
||||
|
||||
`list_unknowns` returns every matching current head by default. For a smaller
|
||||
inline response, supply `offset` and `limit`, then follow `next_offset` until it
|
||||
is null. `total` counts all matching heads; each page preserves complete records
|
||||
in stable unknown ID order. Filters apply before paging. Pages reflect current
|
||||
session state, so keep the unknown ledger unchanged while traversing it.
|
||||
|
||||
For a portable exported bundle, the CLI shares this projection:
|
||||
`rea inspect-evidence-bundle '{"path":"/tmp/evidence.json","detail":"summary","filters":{"operation":"analyze_function"}}' --format json`.
|
||||
The CLI validates and materializes that local bundle under its existing memory
|
||||
@@ -94,10 +106,49 @@ delivery for responses that fit that buffer and Node's single-string limit.
|
||||
It does not change the client's buffer, analysis coverage, or retained content.
|
||||
Ordinary responses keep their existing complete result contract.
|
||||
|
||||
### Untrusted target content
|
||||
|
||||
Strings, symbols, decompiled code and pseudocode, page text, script bodies,
|
||||
network captures, logs, and file listings come from the analysis target. A
|
||||
target can embed text written to look like instructions to the agent, for
|
||||
example a string asking it to run a command or download a "required" helper.
|
||||
REA treats that text as data, and its server instructions tell the agent to do
|
||||
the same: never follow directives found in tool results, and report any request
|
||||
to run, install, download, or send something as a prompt-injection attempt.
|
||||
|
||||
Results from tools that can return such content carry a stable marker in the
|
||||
result's `_meta`, whether the call succeeded or failed:
|
||||
|
||||
```json
|
||||
{ "provenance": { "untrusted_content": true, "source": "analysis_target" } }
|
||||
```
|
||||
|
||||
The marker is declared once per tool contract (`returnsTargetContent`) and is
|
||||
on by default: only `export_evidence_bundle` and `record_unknown`, whose
|
||||
results and errors contain only REA session state or caller-supplied values,
|
||||
omit it. Tools whose results or errors can include provider, launcher, or
|
||||
device output stay marked, including `binary_session` (provider failure
|
||||
reasons and availability diagnostics), `close_binary` (provider cleanup
|
||||
diagnostics with captured launcher output), `inspect_adb_client` and
|
||||
`inspect_apktool_client` (version output and stderr excerpts), and
|
||||
`update_unknown` (it echoes the stored question and domain, which an imported
|
||||
bundle can supply). A new tool is marked until it is audited into
|
||||
that exception list. Error results (`isError: true`) from a marked tool carry
|
||||
the marker too, because a failure can include target-derived text: an error
|
||||
message, captured output, or a `partial_observation` such as terminal data
|
||||
retained by a cancelled `capture_process_scenario`. That covers provider and
|
||||
admission failures, the response-budget error that replaces an oversized
|
||||
result, its oversized-error replacement, and errors the MCP SDK builds itself,
|
||||
such as input validation failures. The marker adds a fixed number of bytes
|
||||
that the response budget accounts for. It does not alter `content` or
|
||||
`structuredContent`. Hosts and hooks can use it to apply stricter handling,
|
||||
such as requiring approval for process-launching or device-mutating tools
|
||||
after target content was read.
|
||||
|
||||
## Retained application Evidence inputs
|
||||
|
||||
`analyze_javascript_application`, `inspect_binary_layout`, `inspect_artifact`, and
|
||||
`inspect_wasm_artifact` accept `detail`:
|
||||
`analyze_javascript_application`, `inspect_binary_layout`, `inspect_artifact`,
|
||||
`inspect_wasm_artifact`, and `inspect_macho` accept `detail`:
|
||||
`complete` (default) returns the complete analysis Evidence; `summary` records
|
||||
that complete Evidence in the session and returns the derived
|
||||
`inspect_analysis_view` summary Evidence instead. Its
|
||||
@@ -167,6 +218,19 @@ attempting oversized delivery, and returns the same summary with an exact
|
||||
`parent_evidence_id` for later export or facet reads. Delivery detail does not
|
||||
reach WABT or change the original source Evidence ID.
|
||||
|
||||
Completed `inspect_macho` Evidence supports a summary and
|
||||
`view: {kind: "macho", facet: "exports", offset: 0, limit: 64}`. Facets are
|
||||
`entrypoints`, `architectures`, `build_metadata`, `load_commands`,
|
||||
`dependencies`, `imports`, `exports`, `segments`, and `provenance`; pages return
|
||||
the original rows. The summary keeps the selected slice, header fields and UUID,
|
||||
and replaces each collection with its returned count, reported total and
|
||||
`exhaustive` flag. Each page repeats the paged collection's `source_total` and
|
||||
`source_exhaustive`; exhausting a non-exhaustive collection does not establish
|
||||
that unreturned symbols are absent. Large frameworks return megabytes of
|
||||
symbols (Electron Framework: 2.5 MB of imports and exports, 3.3 KB as a
|
||||
summary), so select `detail: "summary"` on the first call. Delivery detail
|
||||
never reaches the native tools or changes the source Evidence ID.
|
||||
|
||||
`trace_application_feature`, `trace_javascript_semantics`,
|
||||
`compare_application_versions`, `compare_source_to_bundle`, and
|
||||
`compare_javascript_export_shapes` accept complete inline application Evidence
|
||||
|
||||
@@ -0,0 +1,176 @@
|
||||
# Recorded packet capture inspection
|
||||
|
||||
`inspect_packet_capture` answers which selected protocol fields TShark observes
|
||||
in an existing capture. Network, USB or Bluetooth recordings can use the same
|
||||
field/filter interface when the installed TShark supports their encapsulation
|
||||
and dissectors. This tool does not acquire traffic, open an interface or control
|
||||
a device. HAR/SAZ/mitmproxy web-capture workflows remain separate.
|
||||
|
||||
Supply an existing TShark 4.x installation **on Linux**, with util-linux
|
||||
`prlimit` available at `/usr/bin/prlimit`. Override its absolute path with
|
||||
`REA_TSHARK_PRLIMIT_COMMAND` if required. Unsupported hosts, including macOS
|
||||
and Windows, are refused before inspection because this integration requires
|
||||
an enforced address-space limit:
|
||||
|
||||
```sh
|
||||
export REA_TSHARK_COMMAND=/absolute/path/to/tshark
|
||||
rea inspect-packet-capture /artifacts/device.pcapng \
|
||||
--fields ip.addr --fields udp.port --fields data.data \
|
||||
--display_filter 'udp' --format json
|
||||
```
|
||||
|
||||
The corresponding MCP call is:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_packet_capture",
|
||||
"arguments": {
|
||||
"path": "/artifacts/device.pcapng",
|
||||
"fields": ["ip.addr", "udp.port", "data.data"],
|
||||
"display_filter": "udp"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`frame.number` is always included. Repeated values are ordered string arrays,
|
||||
for example `"ip.addr": ["192.0.2.1", "192.0.2.2"]`. Missing fields remain
|
||||
absent; they are not converted to zero or empty arrays. Export-only index/score
|
||||
metadata is discarded. No payload bytes are automatically added; select a
|
||||
supported byte field such as `data.data` or `@data` when needed.
|
||||
|
||||
Selectors are limited before argv expansion: at most 1,024 fields and 1,024
|
||||
`decode_as` rules, 8,192 UTF-16 code units per selector, and 32 KiB total encoded
|
||||
selector arguments (UTF-8 text, terminators, option names and argv pointers).
|
||||
This leaves headroom below Linux's [execve argument/environment limits](https://man7.org/linux/man-pages/man2/execve.2.html);
|
||||
an unusually large inherited environment can still prevent launching.
|
||||
|
||||
Output is paged after filtering: `match_limit` defaults to 50 and `match_offset`
|
||||
to 0. The result retains original frame numbers and reports `output_window`
|
||||
with the matching count within the input read scope, returned count and
|
||||
`has_more`. Request later pages with the offset, or explicit `match_limit: null`
|
||||
(CLI `--all_matches`) for all remaining matches. Paging does not shorten
|
||||
input reading or change two-pass reassembly. Producer output must still fit its
|
||||
resource limit; use selected fields, a display filter or an input-packet limit
|
||||
when dissection output is too large. Each call checks the recording's current
|
||||
digest, so pages from different source digests must not be combined.
|
||||
|
||||
Additional selections:
|
||||
|
||||
- `display_filter`: Wireshark display-filter syntax; omitted means no filter.
|
||||
- `decode_as`: explicit dissector overrides, such as `["udp.port==5555,dns"]`.
|
||||
Repeat `--decode_as` in the CLI for multiple overrides.
|
||||
- `packet_limit`: optional maximum **input packets read before filtering**.
|
||||
A limit of 1 can therefore yield zero matching packets even when later frames
|
||||
match. Omitted means read to EOF within resource limits. The report records
|
||||
`to-eof` or `up-to-input-packet-limit`, never inferring total capture size from
|
||||
the returned packet count.
|
||||
- `two_pass`: defaults to true for fields that need later packets and reassembly
|
||||
dependencies. MCP may set false; CLI uses `--no-two_pass` for a single pass.
|
||||
|
||||
See the [TShark manual](https://www.wireshark.org/docs/man-pages/tshark.html)
|
||||
for field names, display filters, decode-as syntax and pass semantics. Invalid
|
||||
fields or filters return producer diagnostics. An empty result does not prove
|
||||
absence of traffic outside the selected recording, filter or read range.
|
||||
Dissector results are observations, not independently verified device behavior.
|
||||
|
||||
Evidence includes source path/digest/size, exact selections, typed packet fields,
|
||||
configured and resolved executable paths/digests/full version banner, arguments and stderr. Warnings remain
|
||||
visible even on exit zero. `tool.version_diagnostics` also retains warnings from
|
||||
the separate version probe. Post-run identity failures identify the affected
|
||||
private copy, executable or limiter; resource-signal errors retain the signal
|
||||
and effective ceilings. Two-pass mode cannot recover missing capture bytes.
|
||||
|
||||
Each call uses a private read-only capture copy and an empty personal
|
||||
configuration directory (`WIRESHARK_CONFIG_DIR`), with name resolution disabled.
|
||||
The executable, resource limiter and private input copy are rechecked after execution. Owned
|
||||
processes, file descriptors and private files are cleaned up; failed cleanup retains ownership and
|
||||
completed observations for retry. The one-shot CLI awaits a cleanup retry while
|
||||
preserving the original failure and completed Evidence. Completed observations
|
||||
remain in the MCP session after cancellation or cleanup failure; bounded
|
||||
post-run integrity checks finish before honoring a late abort. Source files
|
||||
are never passed to TShark.
|
||||
Installed libraries, global configuration and plugins remain trusted, are not
|
||||
fully fingerprinted, and can influence dissection. This is not an OS sandbox.
|
||||
|
||||
Input is limited to 32 MiB before reading, each command to 30 seconds, and
|
||||
combined stdout/stderr to 4 MiB during collection. Output overflow fails instead
|
||||
of returning truncated success. Before execution, `prlimit` enforces **1 GiB
|
||||
address space per process**, **64 MiB per file**, **30 CPU seconds** and no core
|
||||
dumps. Stricter inherited soft/hard limits are preserved; Evidence records the effective
|
||||
ceilings. Both limits are set before execution, so dissector allocations and
|
||||
compressed-capture expansion cannot exceed the per-process address-space cap.
|
||||
These are not aggregate process-memory or filesystem quotas. The selected
|
||||
limiter is trusted; its path, digest and arguments are retained in Evidence.
|
||||
No arbitrary process flags are accepted.
|
||||
|
||||
## Verification
|
||||
|
||||
```sh
|
||||
npm run build:cached
|
||||
REA_TSHARK_COMMAND=/absolute/path/to/tshark node scripts/verify-packet-capture.mjs
|
||||
```
|
||||
|
||||
The verifier generates its own Ethernet/IPv4/UDP/DNS and Bluetooth HCI PCAPs.
|
||||
It exercises real compiled CLI/MCP parity, advertised schemas, repeated/missing
|
||||
fields, filtering, decode-as, both pass modes, a packet limit before matching
|
||||
frames, Evidence persistence, failure diagnostics, source identity and cleanup.
|
||||
An optional positional REA entrypoint verifies an installed package.
|
||||
|
||||
The real-tool lane requires Linux and TShark 4.x. A separate kernel-limit
|
||||
regression checks the inherited soft/hard limits and verifies an oversized
|
||||
allocation is refused before it can consume memory. The Ubuntu workflow uses
|
||||
its distribution package; check its outcome before claiming that build verified.
|
||||
The earlier macOS check does not establish the current bounded integration,
|
||||
which deliberately refuses unsupported hosts. Other formats (including PCAPNG),
|
||||
USB, BLE ATT and actual physical capture are not established by these fixtures.
|
||||
|
||||
Each provider admits one inspection at a time, including acquisition and cleanup.
|
||||
An occupied provider returns a retryable capacity error before acquiring another
|
||||
file or worker. Closing the provider stops new admission and drains accepted work;
|
||||
failed cleanup remains owned and can be retried.
|
||||
|
||||
## Discover protocols and field names
|
||||
|
||||
`inspect_packet_fields` / `inspect-packet-fields` queries the installed TShark
|
||||
registration glossary before selecting fields for capture inspection:
|
||||
|
||||
```sh
|
||||
rea inspect-packet-fields --query usb
|
||||
rea inspect-packet-fields --kind fields --protocol usb --query idVendor
|
||||
rea inspect-packet-fields --kind fields --protocol udp --all
|
||||
```
|
||||
|
||||
`kind` defaults to `protocols`; `fields` selects field definitions. `protocol`
|
||||
selects an exact protocol abbreviation (the protocol itself or its child fields).
|
||||
`query` is a case-insensitive literal substring in name, abbreviation or field
|
||||
description. `offset` skips matching records; `limit` defaults to 50. MCP
|
||||
`limit: null` or CLI `--all` requests all remaining matches within the resource
|
||||
budget. `--all` and `--limit` conflict. The window reports examined records of
|
||||
the selected kind before filters, matching records, returned records and
|
||||
`has_more`; it never counts capture packets.
|
||||
|
||||
Returned records preserve field type, parent protocol, display base, bitmask,
|
||||
description and original glossary line number. Empty producer strings remain
|
||||
empty; protocol records have null field-only properties. Repeated abbreviations
|
||||
remain separate registrations. A listed field is an installation capability, not
|
||||
proof that the field appears in a selected recording.
|
||||
|
||||
The adapter uses the documented `tshark -G fields` grammar under its existing
|
||||
private empty profile and Linux process limits. TShark 4.6.9's full glossary was
|
||||
measured at 22,303,197 bytes; discovery admits at most **32 MiB** combined command
|
||||
output (capture inspection remains at **4 MiB**). Parsing walks lines once and
|
||||
constructs objects only for the selected page. Returned strings are copied so
|
||||
retained pages do not retain the complete glossary string. Before constructing a selected
|
||||
record it charges six times the source UTF-8 bytes plus 512 bytes against an
|
||||
**8 MiB** representation budget, covering JSON escaping and record bookkeeping.
|
||||
If that budget would be exceeded, narrow the query or page size; results are not
|
||||
silently truncated. This is a conservative admission budget, not measured heap
|
||||
usage. Parent pipe buffers and producer per-process limits remain distinct.
|
||||
|
||||
Tool/limiter identity, version, stderr and selected arguments are retained in
|
||||
Evidence. No capture, interface or device is opened. Discovery shares capture
|
||||
analysis's admission slot and cleanup owner. Each call rereads the glossary;
|
||||
installed plugins, libraries and global configuration remain trusted and are not
|
||||
fully fingerprinted. There is no cache or snapshot-consistent pagination claim.
|
||||
The real Linux lane checks public CLI/MCP queries against the direct glossary,
|
||||
including USB vendor-ID typing, negative queries and paging.
|
||||
@@ -0,0 +1,118 @@
|
||||
# Packet capture summaries
|
||||
|
||||
REA summarizes already saved pcap/pcapng files through caller-selected Wireshark
|
||||
4.x `tshark` and `capinfos`. Use `inspect_packet_capture` for selected fields,
|
||||
filters and decode-as rules; use this family for address, DNS, SNI, protocol-stack
|
||||
and URI inventories. It does not acquire traffic or fetch recorded targets.
|
||||
|
||||
## Supply tools
|
||||
|
||||
```sh
|
||||
export REA_TSHARK_COMMAND=/absolute/path/tshark
|
||||
export REA_CAPINFOS_COMMAND=/absolute/path/capinfos
|
||||
# Linux only; default /usr/bin/prlimit
|
||||
export REA_TSHARK_PRLIMIT_COMMAND=/absolute/path/prlimit
|
||||
rea inspect-pcap-client
|
||||
rea inspect-pcap-capture /captures/session.pcapng --json
|
||||
```
|
||||
|
||||
Without explicit tool commands, this family resolves them through PATH and
|
||||
records that selection source. Client inspection reads no capture file. Capture
|
||||
inspection requires Linux with util-linux prlimit or Windows with owned Job
|
||||
Objects. macOS capture inspection returns a capability gap because the new
|
||||
native memory boundary is unverified; its version probes remain available.
|
||||
Tools and installed libraries/plugins are caller-trusted. An empty private
|
||||
personal Wireshark configuration is used; this is not an OS sandbox.
|
||||
|
||||
## Inspect only the needed facts
|
||||
|
||||
MCP `inspect_pcap_capture` accepts `path` and optional `detail: "summary"`.
|
||||
Complete delivery remains the default. Summary commits and retains the same
|
||||
complete Evidence, returning identity, capture metadata, observed counts,
|
||||
retained counts, source coverage and diagnostic lengths without large lists.
|
||||
Then use its `normalized_result.parent_evidence_id` with
|
||||
`inspect_analysis_view` source kind `retained-evidence`:
|
||||
|
||||
```json
|
||||
{"kind":"pcap","facet":"endpoints","offset":0,"limit":20}
|
||||
{"kind":"pcap","facet":"dns-queries","offset":0,"limit":20}
|
||||
{"kind":"pcap","facet":"dns-addresses","query":"example.com","offset":0,"limit":20}
|
||||
{"kind":"pcap","facet":"tls-sni","offset":0,"limit":20}
|
||||
{"kind":"pcap","facet":"diagnostics","stream":"tshark","offset":0,"limit":4096}
|
||||
```
|
||||
|
||||
Other row facets are `protocols`, `http-uris` and `unattributed-addresses`.
|
||||
Defaults are offset 0 and limit 64. DNS question pages give address counts;
|
||||
select one exact retained question to page its addresses. Diagnostic windows
|
||||
count UTF-16 code units; other windows count rows. Missing/ambiguous selectors
|
||||
fail. Unknown legacy diagnostics remain unknown, not an observed empty stream.
|
||||
Portable CLI views use the existing `inspect-analysis-view` workflow with inline
|
||||
complete Evidence. Views never reread the source or launch Wireshark and work
|
||||
after the original file is deleted. Page exhaustion covers the retained facet;
|
||||
it does not establish network completeness or recover missing traffic.
|
||||
|
||||
## Interpret provenance and coverage
|
||||
|
||||
`target.sha256` binds observations to an admitted stable capture snapshot. Both
|
||||
tools read that copy, which is hashed again after analysis. Capture metadata and
|
||||
literal field values are producer observations; address/protocol counts and DNS
|
||||
associations are derivations. Capture Evidence has historical-reference authority.
|
||||
|
||||
Addresses count once per packet, including loopback and nested IP occurrences.
|
||||
DNS addresses are associated only with one question in the same response packet;
|
||||
additional records may be included, so this does not prove final resolution.
|
||||
Ambiguous addresses remain in a separate inventory. Protocol distribution counts
|
||||
exact `frame.protocols` stack strings. TLS SNI is recorded handshake metadata,
|
||||
not process identity or proof of the destination's authenticity.
|
||||
|
||||
All examined inventories are retained within the resource budget; no fixed
|
||||
200/100/50-item truncation is applied. HTTP transport credentials are excluded
|
||||
without rewriting other URI bytes; ambiguous credential spans cause omission.
|
||||
Counts preserve distinct observed source URIs before omission or coalescing.
|
||||
`coverage: "partial"` records unresolved DNS attribution or URI item loss;
|
||||
component credential exclusion is also counted explicitly. Diagnostics retain
|
||||
both tool stderr streams. Recorded fields do not establish decrypted payloads,
|
||||
causality, process identity, absence of unrecorded traffic or current deployments.
|
||||
|
||||
## Resources and lifecycle
|
||||
|
||||
One inspection is admitted at a time per provider. Concurrent calls return a
|
||||
capacity error; close drains active work before releasing resources. Captures
|
||||
are streamed into a read-only owned snapshot with a 2 GiB disk budget. Native
|
||||
capture commands enforce Linux address-space ceilings of at most 1 GiB,
|
||||
300 CPU seconds, 64 MiB per file and no core dumps, preserving stricter inherited
|
||||
limits. Windows uses a 1 GiB committed-memory limit for each owned tool tree.
|
||||
These are not aggregate machine or filesystem quotas.
|
||||
|
||||
TShark output is bounded at 8 MiB; its known five-level selected-field shape and
|
||||
128 MiB graph-expansion estimate are checked before allocation. Retained facts
|
||||
have a separate 16 MiB estimate. Capinfos output is limited to 64 KiB. Command
|
||||
wall deadlines are 300 s for dissection, 60 s for metadata and 20 s for probes.
|
||||
Native ownership preparation has its own bounded lifecycle.
|
||||
|
||||
Malformed/oversized output returns no partial success. Unvalidated partial JSON
|
||||
is not returned as facts. Cancellation reaches probes, acquisition, commands and
|
||||
projection. Cleanup failure returns a typed error, retains completed Evidence
|
||||
when available, and retains owned resources for retry on close.
|
||||
|
||||
## Verification scope
|
||||
|
||||
Current follow-up verification uses Linux/Node 24.18.0 and Wireshark 4.0.17.
|
||||
Private real recorded corpora are checked against independent raw-frame/IP
|
||||
parsing and Python aggregation, public CLI/stdio MCP parity, SDK schemas, source
|
||||
hashes and retained views. Synthetic native tools separately test malformed
|
||||
output, cancellation, source/snapshot mutation, capacity and cleanup; they do
|
||||
not establish real Wireshark or other-platform behavior.
|
||||
|
||||
The original proposal reported Linux 4.6.6, macOS/Windows 4.6.9 verification on
|
||||
its earlier implementation. Those reports do not verify the new final memory
|
||||
boundary. Actual Windows/Bun Wireshark execution is unverified here; macOS
|
||||
capture support needs a separate enforceable-bound decision.
|
||||
|
||||
```sh
|
||||
npm run verify:pcap -- --pcap /absolute/saved-capture.pcapng
|
||||
```
|
||||
|
||||
This verifier exercises built CLI/stdio MCP, schemas, artifact identity and
|
||||
cleanup. Without `--pcap`, capture inspection is reported as unverified. Private
|
||||
captures, keys and provider runtime files are not committed.
|
||||
@@ -416,3 +416,52 @@ while its token became unavailable after this change; npm changes its title
|
||||
as well. A newly started unreadable process can prevent verified cleanup even
|
||||
after the selected command exits. REA preserves this uncertainty and leaves
|
||||
that process untouched.
|
||||
|
||||
Reading those tokens requires compiling a small Swift helper with `xcrun swiftc`.
|
||||
By default that compilation reuses Swift's shared module cache, while the
|
||||
compiler's intermediate objects stay inside the per-run temporary root that REA
|
||||
removes afterwards. Set `REA_DARWIN_MODULE_CACHE_PATH` to a writable directory
|
||||
when a shared cache is unavailable, for example under an enforced write
|
||||
boundary. REA creates the directory, passes it to the compiler as
|
||||
`-module-cache-path`, and exports it as `CLANG_MODULE_CACHE_PATH` so the
|
||||
Objective-C header modules the helper builds land there too. REA never removes
|
||||
that directory: it outlives each run, so the first compilation into a new
|
||||
directory pays a cold-cache build and later runs reuse it. An absent or empty
|
||||
value keeps the shared default.
|
||||
|
||||
`xcrun` also maintains a separate tool-lookup cache. Its `--no-cache` option
|
||||
refreshes entries and does not guarantee a write-free lookup. To avoid that
|
||||
lookup while compiling REA's helper, set `REA_DARWIN_SWIFTC_PATH` to the absolute
|
||||
path of an installed `swiftc` and `SDKROOT` to the absolute path of a compatible
|
||||
macOS SDK. REA invokes that compiler directly with `-sdk` and the selected SDK;
|
||||
an unset or empty compiler setting keeps the normal `xcrun swiftc` path. A
|
||||
missing or relative SDK/compiler path fails with an actionable prerequisite
|
||||
error before compilation. An incompatible installation reports the compiler's
|
||||
bounded diagnostics; REA does not repair or replace the toolchain.
|
||||
|
||||
For example, select paths from your existing Xcode or Command Line Tools
|
||||
installation **before** entering an enforced write boundary:
|
||||
|
||||
```sh
|
||||
export REA_DARWIN_SWIFTC_PATH="$(xcrun --find swiftc)"
|
||||
export SDKROOT="$(xcrun --show-sdk-path)"
|
||||
export REA_DARWIN_MODULE_CACHE_PATH="$PWD/rea-cache/modules"
|
||||
export TMPDIR="$PWD/rea-cache/tmp"
|
||||
mkdir -p "$TMPDIR"
|
||||
rea analyze-javascript-application ./fixture --artifact-format directory
|
||||
```
|
||||
|
||||
Keep the selected compiler and SDK paired. Permit REA's evidence/output paths
|
||||
and its temporary directory as well as the caller-owned module cache. REA does
|
||||
not create a sandbox or claim that these settings contain every write made by
|
||||
arbitrary toolchains or analysis providers. They bypass REA's `xcrun` lookup
|
||||
and select the helper's Swift/Clang cache without disabling ownership checks.
|
||||
The caller-owned cache survives cancellation and cleanup; per-run compiler
|
||||
intermediates and the compiled helper remain REA-owned temporary files.
|
||||
|
||||
The boundary must also permit process-table inspection. On macOS, `/bin/ps`
|
||||
is setuid and a blanket sandbox may reject its launch with `spawn EPERM`.
|
||||
REA then preserves ownership uncertainty and can report `cleanup_incomplete`;
|
||||
it does not disable verification to continue. A sandbox that explicitly allows
|
||||
this read-only process inspector is a separate prerequisite from the compiler
|
||||
and cache settings.
|
||||
|
||||
+1
-1
@@ -61,7 +61,7 @@ Independent raw verification streams the complete private snapshot digest and
|
||||
extent with bounded memory and supports cancellation during the read.
|
||||
|
||||
Hopper and IDA refuse raw-binary targets. Windows Ghidra P0 continues to admit
|
||||
PE applications only.
|
||||
native PE applications and DLLs only; raw-binary targets are unsupported.
|
||||
|
||||
## Limitations
|
||||
|
||||
|
||||
@@ -218,6 +218,15 @@ release baseline and can propose an already-published version again.
|
||||
After publication, open a PR from the release branch back to main. Preserve
|
||||
main's later implementation changes and regenerate ignored build outputs from
|
||||
the combined contracts with the released package version.
|
||||
Before merging, check whether GitHub permits merge commits:
|
||||
|
||||
```bash
|
||||
gh api repos/morluto/rea --jq .allow_merge_commit
|
||||
```
|
||||
|
||||
If disabled, a maintainer must arrange an approved ancestry-preserving merge
|
||||
path before synchronization can finish. Squash and rebase merges do not put the
|
||||
released tag in main's ancestry. Do not substitute them for this step.
|
||||
Review and test this synchronization PR, then use a merge commit so the release
|
||||
tag remains in main's ancestry. Keep the released tag unchanged.
|
||||
|
||||
|
||||
@@ -0,0 +1,142 @@
|
||||
# Recorded signal decoding
|
||||
|
||||
`decode_signal_recording` answers what a selected protocol decoder observes in
|
||||
an existing recording. It uses caller-supplied **sigrok-cli 0.7.2** with
|
||||
libsigrokdecode on **Linux**. util-linux `prlimit` is required (default
|
||||
`/usr/bin/prlimit`, overridable with absolute `REA_SIGROK_PRLIMIT_COMMAND`).
|
||||
The limiter sets soft and hard address-space limits before the decoder starts.
|
||||
macOS and Windows are refused because this adapter cannot enforce that limit on
|
||||
those hosts. It does not acquire samples or communicate with hardware.
|
||||
|
||||
```sh
|
||||
export REA_SIGROK_COMMAND=/absolute/path/to/sigrok-cli
|
||||
rea decode-signal-recording /artifacts/uart.vcd \
|
||||
--decoders uart:rx=RX:baudrate=9600 --input_format vcd \
|
||||
--annotations uart=rx-data --format json
|
||||
```
|
||||
|
||||
The corresponding MCP request is:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "decode_signal_recording",
|
||||
"arguments": {
|
||||
"path": "/artifacts/uart.vcd",
|
||||
"decoders": "uart:rx=RX:baudrate=9600",
|
||||
"input_format": "vcd",
|
||||
"annotations": "uart=rx-data"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`decoders` uses sigrok's decoder stack and option/channel syntax. Channel names
|
||||
must match the recording; REA does not guess wiring, baud rate or protocol.
|
||||
Omitting `input_format` requests sigrok's autodetection. Omitting `annotations`
|
||||
retains every annotation class. Both omitted selections are recorded as null.
|
||||
See the [upstream manual](https://sigrok.org/wiki/Sigrok-cli) for formats,
|
||||
decoder stacking and annotation selectors. No arbitrary command flags are accepted.
|
||||
|
||||
Output defaults to 50 literal LF-separated lines. Use `line_offset` and
|
||||
`line_limit` for later pages, or MCP `line_limit: null` / CLI `--all_lines`
|
||||
for all remaining output. `output_window` reports `total_lines`, `returned_lines`
|
||||
and `has_more`; these count display lines, **not annotations or samples**.
|
||||
Line endings, blank lines and text within the selected window stay unchanged.
|
||||
REA does not parse display text into protocol facts, as explained by
|
||||
[upstream's annotation output guidance](https://sigrok.org/wiki/Protocol_decoder_output).
|
||||
Paging does not shorten decoding and producer output must still fit its limit.
|
||||
Use annotation-class selection to reduce producer output. Calls independently
|
||||
check the source digest; do not combine pages from different digests.
|
||||
|
||||
Each selector is limited to 8,192 UTF-16 code units, and total encoded selector
|
||||
arguments (UTF-8, option prefixes, terminators and 64-bit argv pointers) to
|
||||
32 KiB before launching. This reserves headroom below Linux execve's minimum
|
||||
argument/environment budget; a large inherited environment may still exhaust it.
|
||||
|
||||
The returned Evidence includes the input path/digest, exact selections, tool
|
||||
configured and resolved paths/digests, full version banner (including library versions), annotation text
|
||||
with sample-number ranges, and stderr. For example, `1104-1938 uart-1: 52`
|
||||
is producer display text over a sample range, not a timestamp or an independently
|
||||
verified protocol fact. Text is retained verbatim; it is not parsed as a lossless
|
||||
binary payload. Empty output does not establish absence of traffic. Warnings
|
||||
remain visible even when sigrok exits zero, including `tool.version_diagnostics`
|
||||
from the separate version probe. Post-run identity failures name the tool, limiter
|
||||
or private copy involved, rather than blaming the admitted source. Resource-signal
|
||||
failures retain the signal and effective limits without inferring its exact cause.
|
||||
|
||||
REA decodes a private, read-only copy of the recording, verifies that copy and
|
||||
the selected executable and resource limiter after decoding, and removes the workspace after owned
|
||||
process cleanup. Failed descriptor, process or workspace cleanup retains ownership for retry, and the one-shot
|
||||
CLI awaits a retry without replacing the original failure or completed Evidence.
|
||||
Completed observations remain in the MCP session after cancellation or cleanup
|
||||
failure; bounded post-run integrity checks finish before honoring a late abort. The source file is
|
||||
never passed to sigrok. Caller-supplied tools and their libraries/decoder modules
|
||||
are trusted, and hashes identify bytes rather than publisher authenticity.
|
||||
|
||||
The input copy is bounded at 32 MiB before reading, each command has a 30-second
|
||||
deadline, and combined stdout/stderr is bounded at 4 MiB during collection.
|
||||
Oversized output fails rather than becoming truncated success. These limits do
|
||||
not form an OS sandbox. Before execution, `prlimit` enforces **1 GiB address
|
||||
space per process**, **64 MiB per file**, **30 CPU seconds** and no core dumps.
|
||||
Stricter inherited soft/hard limits are preserved; Evidence records the effective
|
||||
ceilings. Both limits are set before execution, so decoder allocations and compressed
|
||||
recording expansion cannot exceed the per-process address-space cap. These are
|
||||
not aggregate process-memory or filesystem quotas. The selected limiter is
|
||||
trusted and its path, digest and arguments are retained in Evidence.
|
||||
|
||||
## Verification
|
||||
|
||||
```sh
|
||||
npm run build:cached
|
||||
REA_SIGROK_COMMAND=/absolute/path/to/sigrok-cli node scripts/verify-signal-recording.mjs
|
||||
```
|
||||
|
||||
The verifier synthesizes a 9600-baud 8N1 UART VCD carrying `REA`, then checks
|
||||
actual CLI/MCP parity, advertised schemas, stored Evidence, source hashes,
|
||||
annotation filtering/autodetection, invalid decoder/input failures and cleanup.
|
||||
An optional positional REA entrypoint verifies an installed package.
|
||||
The real-tool lane requires Linux, sigrok-cli 0.7.2 and libsigrokdecode. A separate
|
||||
kernel-limit regression checks the inherited soft/hard limits and verifies an
|
||||
oversized allocation is refused before it can consume memory. Other formats,
|
||||
decoders and physical acquisition are not established by the UART fixture.
|
||||
The workflow result must be checked before claiming a specific Linux build
|
||||
verified. The earlier macOS tool check does not establish the current bounded
|
||||
integration, which deliberately refuses unsupported hosts.
|
||||
|
||||
One decode is admitted at a time, including acquisition and cleanup. Excess
|
||||
requests receive a retryable capacity error before acquiring another file or
|
||||
worker. Shutdown closes admission and drains accepted work; failed cleanup
|
||||
remains owned and retryable.
|
||||
|
||||
## Discover decoders before choosing settings
|
||||
|
||||
`inspect_signal_decoders` / `inspect-signal-decoders` queries the **installed**
|
||||
sigrok catalog without opening a recording or scanning devices:
|
||||
|
||||
```sh
|
||||
rea inspect-signal-decoders --query uart
|
||||
rea inspect-signal-decoders --decoder uart --query baud
|
||||
rea inspect-signal-decoders --decoder i2c --offset 0 --limit 50
|
||||
```
|
||||
|
||||
Omit `decoder` to search decoder IDs and names. Supply one exact decoder ID to
|
||||
read its literal help, including required/optional channels, options, annotation
|
||||
classes and documentation. `query` is a case-insensitive literal substring;
|
||||
matching help retains original one-based line numbers. Empty query selects all.
|
||||
`offset` skips matches; `limit` defaults to 50. MCP `limit: null` or CLI `--all`
|
||||
returns all remaining matches; CLI `--all` and `--limit` conflict.
|
||||
The window reports examined, matching and returned records and `has_more`.
|
||||
A filtered empty page is not evidence that the installation has no decoders.
|
||||
|
||||
Discovery refuses more than 65,535 line separators before constructing record
|
||||
objects (65,536-line representation budget), in addition to the 4 MiB output cap.
|
||||
Exceeding that budget is a resource failure with captured output. Returned text
|
||||
is copied so a retained page does not keep the complete producer output alive.
|
||||
|
||||
Discovery shares decoding's owned process, admission slot, resource limits,
|
||||
cleanup retries and Evidence delivery. It records the executable and limiter
|
||||
identities, version banner and warnings. Failed version checks and malformed
|
||||
producer output preserve diagnostics. Help is producer text, not an independently
|
||||
validated parameter schema. Each request probes again; it does not cache or claim
|
||||
stable pages across installation changes. Libraries/decoder modules remain
|
||||
trusted and are not fully fingerprinted. Catalog output includes a SHA-256 of
|
||||
its executable, not a digest of the complete installation.
|
||||
@@ -0,0 +1,144 @@
|
||||
# SQLite database snapshots
|
||||
|
||||
`inspect_sqlite_database` / `inspect-sqlite-database` inspects one explicitly
|
||||
selected local SQLite database snapshot without an active native analysis
|
||||
target. Use it to understand application storage, configuration or retained
|
||||
database evidence. No separate SQLite installation is required.
|
||||
|
||||
This operation requires Node.js 24.x >=24.15 or 26+ with the native
|
||||
`DatabaseSync.limits` API. The worker verifies that capability before querying
|
||||
the copied database. Other REA features retain their published Node.js 22.x,
|
||||
24.x or 26+ requirements.
|
||||
|
||||
`rea capabilities` reports `inspect_sqlite_database` availability and the
|
||||
runtime requirement without opening a database. Unsupported runtimes show
|
||||
`available: false`, `availability_code: "unsupported_version"`, and an upgrade
|
||||
note while the inventory command still succeeds.
|
||||
|
||||
```sh
|
||||
rea capabilities --json
|
||||
rea inspect-sqlite-database ./application.sqlite --json
|
||||
rea inspect-sqlite-database ./application.sqlite --table settings --column key --column value --row-limit 20 --json
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_sqlite_database",
|
||||
"arguments": {
|
||||
"path": "/artifacts/application.sqlite",
|
||||
"table": "settings",
|
||||
"columns": ["key", "value"],
|
||||
"row_limit": 20
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
MCP paths must be absolute filesystem paths on the current host. CLI paths
|
||||
resolve against the operator's working directory. Select a quiescent snapshot:
|
||||
stop the writer or obtain a consistent database/WAL pair using the application's
|
||||
backup workflow before inspection. A file copy of a live pair does not establish
|
||||
an atomic SQLite snapshot, even if no change is detected during inspection.
|
||||
|
||||
## Returned evidence
|
||||
|
||||
Without a table selection, the default `schema_scope` is `inventory`: compact
|
||||
table metadata and index, view and trigger names, without SQL declarations or
|
||||
column details. Selecting a table defaults to `selected_table`, which expands
|
||||
that table and its indexes and triggers. Explicit `schema_scope: "database"`
|
||||
(or CLI `--schema-scope database`) expands all stored declarations and supported
|
||||
columns. `schema.completeness` refers to the reported scope; omitted column
|
||||
details are marked `not_requested`. SQL declarations retain the original text. Virtual and
|
||||
shadow tables are identified as such; their supported metadata coverage is
|
||||
reported. If a virtual table cannot be resolved, possible backing tables have
|
||||
`kind: "unknown"` and an explicit limitation; their rows are refused. Unrelated
|
||||
ordinary tables remain selectable. Schema inventory does not execute views,
|
||||
triggers or arbitrary SQL.
|
||||
|
||||
Rows are optional. `table` selects one exact ordinary-table name. `columns`
|
||||
(CLI repeated `--column`) selects exact visible column names in the requested
|
||||
order; omission selects all visible columns, including generated columns.
|
||||
Unselected cells are not evaluated or projected. Column selection requires a
|
||||
table and at least one name; empty-string identifiers are valid. `row_limit`
|
||||
requires `table`, defaults to 20 and accepts positive safe integers. Samples
|
||||
remain subject to the byte and worker resource limits below, regardless of row count. The result reports
|
||||
the selected limit, returned count and whether more rows were observed. Row
|
||||
order is unspecified; a truncated sample does not prove anything about omitted
|
||||
records. View, virtual-table and shadow-table row reads are unsupported.
|
||||
|
||||
SQLite integers are decimal strings so every signed 64-bit value survives JSON
|
||||
serialization. Real values retain their numbers; `"Infinity"`, `"-Infinity"`
|
||||
and `"-0"` explicitly preserve infinities and negative zero through JSON.
|
||||
Text returns the decoded string together with its bytes as hex and the database
|
||||
encoding, preserving malformed text that the SQLite binding may replace for
|
||||
display. Blobs are hex, and SQL NULL is a distinct tagged value.
|
||||
|
||||
Database and existing sibling `-wal` files have separate original SHA-256 and
|
||||
byte counts. The Evidence subject identifies the original database file;
|
||||
the WAL identity is also necessary to interpret pending committed data. The
|
||||
SQLite engine version is reported with the observed result. Neither digest is
|
||||
a claim that the database is healthy, complete or trusted.
|
||||
|
||||
## Source files and failures
|
||||
|
||||
REA copies the selected database and existing sibling WAL to an owned private
|
||||
temporary directory before SQLite opens them. Source files are not opened by
|
||||
SQLite, so its locks or shared-memory sidecars cannot modify the original
|
||||
snapshot. Temporary copies and the owned worker are cleaned up on success,
|
||||
failure and cancellation. Extension loading and schema trust are disabled.
|
||||
Empty rollback journals and fully zeroed journal headers left by successful
|
||||
`TRUNCATE` and `PERSIST` commits are accepted after stable identity checks.
|
||||
Journal headers are checked both before and after acquisition; unchanged file
|
||||
metadata alone does not establish that a journal remains inactive.
|
||||
Other retained rollback journals are refused because recovery is outside this
|
||||
profile. Journal files are never modified or copied into the worker snapshot.
|
||||
|
||||
A provider admits at most two inspections through acquisition, worker execution
|
||||
and cleanup. Additional requests receive a typed capacity error before reading
|
||||
files or creating workers; retry after an active inspection finishes. This
|
||||
bounds simultaneous input buffering and private snapshot/worker counts.
|
||||
|
||||
The database and WAL together are limited to 256 MiB. SQLite values/records,
|
||||
SQL statements and the complete worker JSON reply each have a 16 MiB limit.
|
||||
The owned worker has a 256 MiB JavaScript heap limit and a 30-second inspection
|
||||
deadline. SQLite native allocations are separate from the JavaScript heap;
|
||||
overall native heap and total resident memory are not capped. Row limits do not
|
||||
override these byte and resource limits. Resource-limit failures do not return
|
||||
an incomplete schema as successful complete evidence.
|
||||
|
||||
MCP also applies its configured response budget to the complete Evidence
|
||||
envelope. If delivery exceeds that budget, it reports a transport constraint
|
||||
with guidance for exporting retained Evidence, requesting a matching larger
|
||||
client/server response budget or using complete CLI JSON output.
|
||||
|
||||
A cancellation received after the worker has completed does not erase the
|
||||
completed observation. If subsequent private-root cleanup fails, the error
|
||||
retains the completed Evidence and the resource owner for cleanup retry. MCP
|
||||
records that Evidence before delivering the error, which carries its retained
|
||||
evidence ID for later retrieval or export.
|
||||
|
||||
Changed input identities, special files, malformed databases, unsupported
|
||||
formats and resource failures are reported as errors with the selected path and
|
||||
meaningful reason. Encrypted databases, database repair, arbitrary SQL,
|
||||
cross-database queries and live capture are outside this inspection's coverage.
|
||||
|
||||
## Compatibility verification
|
||||
|
||||
Run `npm run test:sqlite` after changing the SQLite adapter or its shared
|
||||
process, filesystem, CLI or MCP boundaries. It prepares the compiled runtime
|
||||
and runs SQLite service, snapshot, inspection, cleanup and public CLI/MCP
|
||||
tests. Directory-based selection includes new SQLite service and boundary
|
||||
tests automatically. `npm run test:sqlite:run` runs the same cases when the
|
||||
compiled runtime and generated test artifacts are already current.
|
||||
|
||||
The reusable SQLite CI workflow consumes the shared build artifact and tests
|
||||
the latest Node 22.x unavailable-runtime diagnostics, the minimum supported
|
||||
24.15.0 release, and the latest 24.x and 26.x releases. The repository-pinned
|
||||
runtime also runs the full suite. CI's required gate waits for every selected
|
||||
SQLite matrix job; a failed, cancelled, skipped or missing selected lane blocks
|
||||
the gate. Workflow changes themselves select the build and compatibility lane.
|
||||
|
||||
PR checks run against GitHub's merge with the current base branch. A later
|
||||
change to `main` can introduce failures outside SQLite; inspect the failing
|
||||
job and recheck the updated merge before treating an earlier green run as
|
||||
current verification. Compatibility checks detect regressions, but cannot
|
||||
guarantee that an arbitrary future upstream change remains compatible.
|
||||
+26
-3
@@ -68,6 +68,16 @@ correctness. Select relevant boundary files and real-provider lanes explicitly.
|
||||
Source projects also contain large capacity regressions; `test:fast` promises
|
||||
no compiled-runtime prerequisite, not a fixed time budget.
|
||||
|
||||
`npm run lint` prints errors and a warning summary; `npm run jscpd` prints
|
||||
duplication statistics. Complete reports remain in `.cache/static-checks/` and
|
||||
CI uploads them as `static-check-reports`, including after a failed static gate.
|
||||
Use `npm run lint:details` or `npm run jscpd:details` for all console diagnostics.
|
||||
Both duplicate commands scan production and test code with the same 3% limit;
|
||||
the summary commands preserve checker failures and restore reports from cached
|
||||
static tasks. Both lint caches include the website, documentation configuration,
|
||||
and experiment code scanned by the checker; a regression compares the actual
|
||||
lint inventory with Turbo's resolved inputs.
|
||||
|
||||
Routine iterations and rebases need focused regressions and relevant checks.
|
||||
Before handing off a PR, run `npm run check` and generated-document checks when
|
||||
applicable; CI owns the full suite and coverage. Use the full local gate for
|
||||
@@ -617,11 +627,12 @@ or upgrade Java, Ghidra, .NET or native toolchains. See
|
||||
`npm run verify:ida -- --target /absolute/path/to/program --procedure main`
|
||||
uses the existing `REA_IDA_MCP_CONFIG` registration. It installs no engine,
|
||||
Python package, or compiler. The target must already be open in the GUI for
|
||||
the legacy attached profile; the database-supervisor headless profile opens
|
||||
the legacy or native attached profile; the database-supervisor headless profile opens
|
||||
a digest-verified private copy. A caller-supplied fixture keeps prerequisites
|
||||
limited to the selected engine and host. `tests/conformance/ida/inventory.c`
|
||||
provides an optional small native fixture source with an exported
|
||||
`rea_fixture_add` function.
|
||||
provides an optional small native fixture source with exported
|
||||
`rea_fixture_add` and `rea_fixture_subtract` functions. For their switching check,
|
||||
use `--second-procedure rea_fixture_subtract --second-marker='- 13'`.
|
||||
|
||||
The lane invokes the production CLI dispatcher and connects the pinned MCP
|
||||
client SDK to the production REA server. It verifies function Evidence and
|
||||
@@ -649,6 +660,10 @@ failed intermittently on that host. Linux headless, modern attached GUI tools,
|
||||
other engine versions and architectures remain unverified; see the
|
||||
[provider guide](ida-provider.md).
|
||||
|
||||
Native IDA MCP verification uses `"protocol": "native"` with an already-running native plugin and requires `--second-procedure other_function --second-marker unique_second_body_text` to exercise GUI function switching and a return to the first function. These options remain optional for other profiles. The literal marker must appear in the second function's pseudocode and be absent from the first. Native cloud/widget failures are reported without hidden retries.
|
||||
|
||||
The initial Linux x64 IDA Free 9.4 run used native MCP 1.1.2 with a local navigation correction. Windows x64 verification uses **IDA Pro 9.3**, the two-function public fixture, and the shipped Windows `whoami.exe` application. The native plugin was built from upstream `c9d15d98e8a143fc74f6e14037c48f63889b6d6c`, including the merged navigation fix, with a local loopback-listener change. Production CLI/MCP parity, switching, identity and cleanup passed. Correctness and performance measurements are recorded in the [Windows native verification notes](https://github.com/morluto/rea/blob/main/experiments/analysis-efficiency/ida-native-windows.md). Stock 1.1.2 switching, native macOS, and Windows IDA Free remain unverified. See the [native bridge prerequisites](ida-provider.md#optional-native-ida-mcp-bridge).
|
||||
|
||||
## DOS Ghidra analysis
|
||||
|
||||
`npm run verify:ghidra:dos` requires the supported Ghidra and JDK installation
|
||||
@@ -834,6 +849,14 @@ establish passing Android coverage; inspect its execution receipt.
|
||||
The `termux-browser-diagnostics` artifact retains the Termux command log, exit
|
||||
status, and Android logcat on success or failure. The emulator is disposable.
|
||||
|
||||
## Packet capture analysis
|
||||
|
||||
`npm run verify:pcap -- --pcap PATH` exercises caller-supplied tshark and
|
||||
capinfos against a saved capture. Linux verification uses Wireshark 4.6.6
|
||||
and captures containing DNS, TLS and HTTP traffic. Native Windows and macOS
|
||||
engine behavior remains unverified; process fixtures do not establish it.
|
||||
See [Packet capture analysis](pcap-analysis.md) for contracts and budgets.
|
||||
|
||||
## Apple native metadata and UI
|
||||
|
||||
`npm run verify:apple-dispatch` compiles the Objective-C fixture (classes,
|
||||
|
||||
@@ -25,6 +25,36 @@ and original zero-based producer ordinals; duplicate or out-of-range selections
|
||||
are input errors. Every record is validated before selection, so selecting one
|
||||
entry does not hide a malformed entry elsewhere.
|
||||
|
||||
For agent discovery over MCP, request `detail: "summary"`. REA retains the
|
||||
complete selected Evidence in the session and returns identity and counts.
|
||||
Use `normalized_result.parent_evidence_id` with `inspect_analysis_view`:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "<parent_evidence_id>"
|
||||
},
|
||||
"view": { "kind": "capture", "detail": "preview", "offset": 0, "limit": 20 }
|
||||
}
|
||||
```
|
||||
|
||||
Preview pages select recorded HTTP headline fields, event names/times when
|
||||
present, original locations and byte descriptors; they omit headers, XML and
|
||||
payload content. Choose `detail: "complete"` in that view for exact record
|
||||
pages, including all sidecars. Offsets follow the parent's selected record
|
||||
order; producer ordinals remain unchanged. `total_records` counts the original
|
||||
capture while `retained_records` and page coverage count records in the parent.
|
||||
An exhausted page does not establish absence from unselected records.
|
||||
|
||||
Views do not reopen the file or rerun its decoder. They remain historical
|
||||
Evidence if the source changes or disappears; inspect the capture again to
|
||||
observe new bytes. The inspection tool's default `detail: "complete"` preserves
|
||||
existing inline delivery. For CLI use, save complete inspection JSON and pass
|
||||
it as inline Evidence to `inspect-analysis-view`; the same summary and capture
|
||||
page contracts apply. Large complete records can still exceed MCP's response
|
||||
budget; export the retained Evidence when a single record cannot fit.
|
||||
|
||||
## Formats and upstream profiles
|
||||
|
||||
HAR uses unchanged `har-schema@2.0.0` draft-06 schemas, `ajv@8.20.0`,
|
||||
@@ -57,11 +87,80 @@ tested upstream source is tag `v12.2.3`, commit
|
||||
an explicit unsupported-profile diagnostic. Non-Linux native decoding is not
|
||||
supported by this adapter.
|
||||
|
||||
## Reading Fiddler captures (`.saz`)
|
||||
|
||||
Fiddler Everywhere saves traffic in **Session Archive Zip (`.saz`)** files.
|
||||
Inspect native SAZ directly, without installing Fiddler or converting to HAR:
|
||||
|
||||
```sh
|
||||
rea inspect-web-network-capture /absolute/session.saz saz --json
|
||||
rea inspect-web-network-capture /absolute/session.saz saz --record 0 --json
|
||||
```
|
||||
|
||||
MCP uses the same `capture_path`, `format: "saz"`, and optional
|
||||
`record_ordinals` fields as the examples above. REA pairs `raw/<id>_c.txt`
|
||||
requests, `_s.txt` responses, and `_m.xml` metadata by filename stem. Archive
|
||||
member names identify the source; zero-based ordinals follow each session's
|
||||
first member in ZIP directory order, not its numeric ID or capture time.
|
||||
|
||||
HTTP fields and ordered duplicate headers are retained, with stored body bytes
|
||||
in binary sidecars. Bodies are not dechunked or decompressed. Metadata is an
|
||||
XML node tree; optional `_w.txt` and `_g.txt` files retain opaque WebSocket and
|
||||
gRPC message bytes. Missing members remain null, distinct from empty bodies.
|
||||
See Telerik's [SAZ format reference](https://www.telerik.com/fiddler/fiddler-everywhere/documentation/knowledge-base/fiddler-archives).
|
||||
|
||||
The existing ZIP reader verifies read members; expanded archive contents are
|
||||
limited to 32 MiB in addition to the input/output budgets below. Encrypted
|
||||
archives and unrecognized raw session members are unsupported. SAZ requires an
|
||||
enforced decoder memory bound: Node's fixed heap, or the Windows Job Object
|
||||
commit limit. Bun on hosts without that Job Object limit returns capability
|
||||
unavailable before starting the SAZ decoder. Non-session
|
||||
members are inventoried without decoding their content. Original files remain
|
||||
unchanged. Linux verification includes independently checked public recorded
|
||||
Mattermost captures from [FOREST](https://github.com/dfrc-korea/FOREST/tree/405c9154658a128ea203a6c10040cec55111ed03/api-results/dataset)
|
||||
and the Fiddler author's [encoding corpus](https://github.com/ericlaw1979/Clearinet/blob/b6faf78cde1e72042980364f180ceb34b77d1434/Test/data/Encodings.saz),
|
||||
plus synthetic malformed/limit fixtures. Their exporting product versions are
|
||||
not authenticated. An interactive export from a current Fiddler Everywhere
|
||||
installation and its live MCP server remain unverified. Support here is the
|
||||
bounded documented raw HTTP/XML SAZ layout, not every product/version profile.
|
||||
|
||||
For visual inspection, use Everywhere's **Import Snapshot**, then **Inspectors**
|
||||
and **Overview**. HAR remains an alternative: select sessions, choose
|
||||
**Export > HTTPArchive**, and confirm `log.version` is `"1.2"`. Preserve the
|
||||
original SAZ when exporting. See the [import/export instructions](https://www.telerik.com/fiddler/fiddler-everywhere/documentation/collaboration/exporting-and-importing).
|
||||
|
||||
## Fiddler Everywhere MCP and public skills
|
||||
|
||||
For agent-driven traffic debugging, connect Everywhere's MCP server separately
|
||||
from REA. Run a current Everywhere release with MCP access (Trial, Pro, or
|
||||
higher), internet access, and a compatible MCP client. Use
|
||||
**Settings > MCP Server > Coding agent**, or **Manual setup** with the generated
|
||||
API key. The default endpoint is `http://localhost:8868/mcp`. See Telerik's
|
||||
[MCP setup guide](https://www.telerik.com/fiddler/fiddler-everywhere/documentation/agent-tools/fiddler-mcp-server)
|
||||
and [agent-skills prerequisites](https://www.telerik.com/fiddler/fiddler-everywhere/documentation/agent-tools/agent-skills).
|
||||
|
||||
Telerik's [public skills](https://github.com/telerik/fiddler-agent-tools) cover
|
||||
installation (`fiddler-download-setup`), connection setup, and traffic analysis:
|
||||
|
||||
```sh
|
||||
npx skills add telerik/fiddler-agent-tools --skill fiddler-mcp-setup
|
||||
npx skills add telerik/fiddler-agent-tools --skill fiddler-traffic-debugging
|
||||
```
|
||||
|
||||
After setup, reload the agent client and use the connected server's advertised
|
||||
tool names and schemas. Check status, list sessions, then inspect relevant
|
||||
session IDs. Set `sessionsSource` to `LiveTraffic` for HTTP(S) traffic or
|
||||
`AgentCalls` for enriched LLM calls; active filters affect the session list.
|
||||
For an existing run, skip the skill's pre-capture steps that replace process
|
||||
filters and clear sessions. The documented MCP session sources do not provide
|
||||
a saved-snapshot selector; use the SAZ workflow above for saved files. REA does
|
||||
not register or wrap the Fiddler MCP server.
|
||||
|
||||
## Reading the evidence
|
||||
|
||||
- `artifact` identifies the caller-selected file and stable snapshot digest.
|
||||
- `records` preserve producer order/ordinals and original JSON pointers or byte
|
||||
ranges. No live browser transaction or scenario IDs are invented.
|
||||
- `records` preserve producer order/ordinals and original JSON pointers, byte
|
||||
ranges, or SAZ member names. No live browser transaction or scenario IDs are invented.
|
||||
- `reported` retains the producer's fields. HAR container metadata is returned
|
||||
separately, with `log.entries` replaced by `null` and `records_pointer`
|
||||
identifying the record collection. This is an inspection representation,
|
||||
@@ -156,7 +255,7 @@ response fields, including HAR's `status: 0` and an empty `mimeType`, remain
|
||||
unknown, not invented statuses or MIME types.
|
||||
|
||||
The view makes no handler-identity, schema or causality claim. It performs no
|
||||
network activity. Native mitmproxy captures return a typed unsupported error.
|
||||
network activity. Native mitmproxy, SAZ and qlog captures return a typed unsupported error.
|
||||
Parents inspected with `sensitive_values`, or whose Evidence does not record
|
||||
`sensitive_value_count`, return a typed unsupported error whose reason is the
|
||||
canonical token `sensitive_values_unretained`. The view would write path
|
||||
@@ -192,14 +291,15 @@ Complete evidence has a 32 MiB input, 96 MiB decoder reply and 64-level nesting
|
||||
budget. Exceeding a budget returns no partial success.
|
||||
Oversized selected captures return an input error identifying `capture_path`
|
||||
and the byte budget; owned reply/output failures remain output or provider errors.
|
||||
HAR runs in an owned
|
||||
192 MiB Node old-generation heap with one V8 worker. Native decoding applies
|
||||
HAR, SAZ and qlog run in an owned
|
||||
192 MiB Node old-generation heap with one V8 worker. Mitmproxy decoding applies
|
||||
Linux limits of 768 MiB address space, 30 seconds CPU and 96 MiB file output;
|
||||
both commands have a 30-second wall deadline and independently supervised
|
||||
process cleanup. Owned decoder diagnostics retain at most 1 MiB across stdout
|
||||
and stderr; the collector drops overflow as it arrives, reports the limit
|
||||
failure and verifies cleanup. Heap limits alone do not establish aggregate RSS limits.
|
||||
When REA runs on Bun, the HAR decoder runs on Bun, which ignores these V8 heap
|
||||
SAZ and qlog refuse decoding when neither a Node heap limit nor a Windows Job Object bound can be enforced. Existing HAR availability is preserved.
|
||||
When REA runs on Bun, the HAR/SAZ/qlog decoder runs on Bun, which ignores these V8 heap
|
||||
flags; its limitations and resource failures then omit the heap sizes. On
|
||||
Windows, the decoder's owned Job Object instead limits its committed memory to
|
||||
704 MiB, the 192 MiB budget plus a 512 MiB reserve for Bun itself. The
|
||||
@@ -207,7 +307,7 @@ limitations state that limit, and reaching it returns `resource_constraint`
|
||||
with `process_memory_limit_bytes`, whatever exit status Bun reports.
|
||||
|
||||
Observed decoder memory exhaustion returns `resource_constraint`, with the
|
||||
selected capture, applicable limits and retained process diagnostics. HAR's
|
||||
selected capture, applicable limits and retained process diagnostics. HAR/SAZ/qlog's
|
||||
old-generation heap is fixed; inherited `NODE_OPTIONS` cannot raise it.
|
||||
Record selection happens after complete decoding and does not reduce that
|
||||
workload. Use a smaller capture exported by its producer or another decoder
|
||||
@@ -234,3 +334,108 @@ Command cleanup failures retain the original status and both diagnostic streams;
|
||||
the decoder's ownership preparation receives the same cancellation signal.
|
||||
|
||||
MCP advertises the same strict named input contract used by the application. Historical inspection delegates argument validation to that shared application boundary so accepted explicit-sensitive declarations also cover invalid argument names and other correction details. The SDK receives raw arguments through its Standard Schema interface; malformed inputs still fail before decoder or filesystem effects and return the normal structured REA error. Protocol-envelope validation remains owned by MCP.
|
||||
|
||||
## Retained aioquic qlog
|
||||
|
||||
Select `format: "qlog"` through `inspect_web_network_capture`, or:
|
||||
|
||||
```bash
|
||||
rea inspect-web-network-capture /absolute/origin.qlog qlog --json
|
||||
rea inspect-web-network-capture /absolute/origin.qlog qlog --record 95 --record 0 --json
|
||||
```
|
||||
|
||||
The initial profile is **aioquic 1.5.0, JSON qlog 0.3**: one JSON object with
|
||||
`qlog_format: "JSON"`, `qlog_version: "0.3"`, `traces` containing object events
|
||||
with `name`, finite numeric `time` and object `data`. JSON-SEQ, JSONL, columnar
|
||||
array events and other versions are unsupported. Duplicate JSON members,
|
||||
malformed UTF-8, invalid interpreted QUIC integers and excessive nesting fail
|
||||
without partial success, including in events outside the selected ordinals.
|
||||
Well-formed event subsets and missing frame metadata remain inspectable.
|
||||
Unknown extensions are retained; this is not validation of every qlog schema.
|
||||
|
||||
Choosing the format selects the interpretation profile; the file does not
|
||||
necessarily report or authenticate an aioquic version. The profile's clock
|
||||
uses Unix-epoch milliseconds, verified in aioquic 1.5.0's `QuicLoggerTrace`
|
||||
`log_event`/`encode_time`. The `qlog.clock.basis` and Evidence analysis profile
|
||||
state that assumption explicitly. Numeric lexemes survive independently of
|
||||
safe JSON-number projections; decimal spelling does not establish physical
|
||||
clock precision or monotonicity.
|
||||
|
||||
Every event has a global ordinal from trace/event source traversal and its
|
||||
original `/traces/T/events/E` pointer. This order is not a global time sort.
|
||||
Trace index and the selected artifact identify a connection observation;
|
||||
reported ODCID is preserved without claiming global uniqueness. Streams are
|
||||
scoped by trace, stream ID and sent/received direction. Control/QPACK streams
|
||||
are retained, so counting streams does not count HTTP requests.
|
||||
|
||||
`qlog` is a typed facet required exactly for this format. Its scope is
|
||||
`all-recorded-events-before-selection`: selection changes returned event
|
||||
records while facts cite the original ordinals across the entire recorded
|
||||
file. The container retains trace metadata with each events array replaced
|
||||
by null; there is no single events pointer shared by all traces.
|
||||
|
||||
| Fact | Basis and limits |
|
||||
| ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Original event and loss declaration | Observed endpoint report; `recovery:packet_lost` does not prove actual network loss. Missing packet number/type remains unknown. |
|
||||
| STREAM ranges and byte totals | Derived per trace and direction from reported offsets and lengths using exact integer arithmetic. `unique_bytes` unions intervals; `frame_length_sum` includes overlaps; `repeated_bytes` is their difference. Overlap alone does not prove retransmission or equal contents. |
|
||||
| FIN final sizes | Reported offset plus length where FIN is true; multiple conflicting sizes remain visible. A final size does not establish complete coverage or a received body. |
|
||||
| Incomplete fact records | Ordinals with omitted/redacted interpretable frame metadata; usable fields in other records survive. An empty list does not establish complete logging or capture. |
|
||||
| HTTP bodies and causality | `body_bytes: "not-reported"`; lengths and headers are metadata, with no reconstructed body, hash, timing causality or live process attribution. |
|
||||
|
||||
QUIC integer fields interpreted here accept exact nonnegative JSON integer
|
||||
values through `2^62 - 1`, including equivalent exponent notation. Offsets plus
|
||||
lengths must fit that range. Derived sums are decimal strings and are not
|
||||
rounded to JavaScript numbers. A missing offset is unknown, never defaulted to
|
||||
zero. Ranges can start above zero or contain gaps. Packet headers, HTTP events,
|
||||
raw lengths and unknown extensions remain producer evidence rather than being
|
||||
normalized into invented HAR requests.
|
||||
|
||||
The adapter excludes authorization/proxy authorization/cookie/set-cookie
|
||||
values in known HTTP frame events, reset tokens in transport parameters and
|
||||
NEW_CONNECTION_ID frames, and NEW_TOKEN frame tokens. Known URL header userinfo
|
||||
and explicit sensitive values follow the shared exclusions. Ordinary extension
|
||||
fields and token-looking query text remain retained. Explicit declarations also
|
||||
cover normalized numeric projections, derived totals and trace pointers; facts
|
||||
that cannot be safely shown are omitted with incomplete-record coordinates.
|
||||
No keylog, PCAP, live URL or recorded target is opened by the qlog decoder.
|
||||
|
||||
To repeat the real-provider lane with **local caller-supplied evidence** from
|
||||
the educational aioquic 1.5.0 experiment (one origin trace, known 32,803-byte
|
||||
stream coverage and one 1,169-byte repeated range in the loss case):
|
||||
|
||||
```bash
|
||||
npm run verify:web:qlog -- /absolute/control/origin.qlog /absolute/loss/origin.qlog /absolute/new-report.json
|
||||
```
|
||||
|
||||
This lane checks built CLI/stdio MCP parity, SDK-converted JSON Schemas,
|
||||
artifact digests, original event count, selection scope, retained parent lookup,
|
||||
credential exclusions and the known control/loss facts. The report is created
|
||||
exclusively and contains no keys or captured credential values. It is a
|
||||
fixture-specific verification, not a general loss detector. CI uses synthetic
|
||||
format inputs for malformed cases and limits; those tests do not establish
|
||||
producer provenance. PCAP parsing and correlation remain outside this stage.
|
||||
|
||||
### Focused qlog views
|
||||
|
||||
Use MCP `inspect_web_network_capture` with `format: "qlog", detail: "summary"`
|
||||
for identity, profile, scope and retained-fact counts. Then pass its
|
||||
`normalized_result.parent_evidence_id` to `inspect_analysis_view` as a
|
||||
`retained-evidence` source.
|
||||
|
||||
```json
|
||||
{"kind":"qlog","facet":"traces","offset":0,"limit":10}
|
||||
{"kind":"qlog","facet":"streams","trace_index":0,"offset":0,"limit":10}
|
||||
{"kind":"qlog","facet":"contributions","trace_index":0,"stream_id":"0","direction":"sent","offset":0,"limit":20}
|
||||
```
|
||||
|
||||
Trace pages omit nested lists and give their counts. Stream pages preserve
|
||||
exact byte totals but omit contributions, ranges and final-size lists. For
|
||||
those lists select `contributions`, `ranges` or `final-sizes` with all three
|
||||
stream selectors. `loss-declarations` and `incomplete-records` need only
|
||||
`trace_index`. Offsets count rows in that retained facet; defaults are 0 and 64. Missing or ambiguous selectors fail rather than choosing a connection.
|
||||
|
||||
These views page already interpreted facts without decoding again or opening
|
||||
the source, and work after its deletion. Facts still cover all recorded events
|
||||
before parent record selection. Their exhaustion does not prove logging or
|
||||
network completeness. Event preview/complete pages remain available through
|
||||
kind `capture`, with original numeric lexemes and producer ordinals preserved.
|
||||
|
||||
@@ -153,8 +153,12 @@ the private runtime, isolating Ghidra preferences, logs, and caches.
|
||||
Process creation assigns a suspended child atomically to an owned Job Object.
|
||||
Membership and kill-on-owner-close policy are verified before resume.
|
||||
Breakaway is disabled. Only the child's default file owner is normalized;
|
||||
the caller token stays unchanged. Caller CPU affinity and priority are
|
||||
preserved. Close, cancellation, timeout, startup failure, and provider failure
|
||||
the caller token stays unchanged. Caller priority is preserved, and caller CPU
|
||||
affinity whenever the caller and the child share a processor group. A child that
|
||||
Windows places in another group keeps that group's default when the caller asked
|
||||
for no bound of its own; a caller pinned to fewer processors than its group
|
||||
holds is refused there rather than widened.
|
||||
Close, cancellation, timeout, startup failure, and provider failure
|
||||
terminate the job and verify all members have settled before runtime cleanup.
|
||||
Cleanup removes admitted objects by handle and unlinks reparse entries without
|
||||
following their targets. Abrupt owner death kills the job but can leave
|
||||
|
||||
@@ -7,10 +7,57 @@ JSON measurements belong outside the checkout. This document keeps the measured
|
||||
findings, verification scope and revision references; the scripts and `corpus.json`
|
||||
keep the reproduction inputs. Generated `results/` is ignored.
|
||||
|
||||
Follow-up: [latest-main capability audit](latest-main-audit.md) records the
|
||||
Flutter, SQLite, TShark and sigrok priorities and the first shipped Dart AOT
|
||||
baseline, preserving existing controls before introducing another selector.
|
||||
|
||||
Follow-up: [Dart AOT summary and retained string pools](flutter-aot-summary.md)
|
||||
records real LocalSend delivery pairs and old Evidence continuation, preserving
|
||||
ABI selection, snapshot identity, original lower bounds and partial coverage.
|
||||
|
||||
Follow-up: [SQLite existing-control measurements](sqlite-existing-controls.md)
|
||||
tests QGIS's shipped GeoPackage through the existing column selection, preserving
|
||||
full selected-table schema and typed cells before considering another API.
|
||||
|
||||
Follow-up: [native PE/DOS entry views and real model verification](native-entry-views.md)
|
||||
records the post-merge native continuation, including a small-function case where
|
||||
four selected replies cost more bytes than the complete dossier.
|
||||
|
||||
Follow-up: [EVM descriptor and real-template controls](evm-descriptor.md) extends
|
||||
the offline corpus with digest-pinned Uniswap runtime templates and an independent
|
||||
published ABI selector oracle, preserving unknown deployment/immutable metadata.
|
||||
|
||||
Follow-up: [EVM interface summary delivery](evm-interface-summary.md) records
|
||||
the opt-in retained view, old-client/Evidence migration checks and repeated SDK
|
||||
and real-model pairs, including mixed token/time results and strict-answer limits.
|
||||
|
||||
Follow-up: [task-scoped preset lifecycle verification](preset-lifecycle.md) checks
|
||||
real Ghidra profile/snapshot boundaries, startup cancellation and retained cleanup
|
||||
retry while preserving defaults and distinguishing verification from performance.
|
||||
|
||||
Follow-up: [EVM descriptor lifecycle acceptance](evm-lifecycle.md) verifies actual
|
||||
worker replies, late SDK cancellation, permission-induced cleanup failures,
|
||||
healthy WASM peers and old CLI parity with explicit runtime and platform scope.
|
||||
|
||||
Follow-up: [managed artifact delivery and retained views](managed-artifact-summary.md)
|
||||
records two shipped assembly checks, previous-runtime portable Evidence, and four
|
||||
actual Codex trials. Selected replies shrink substantially; observed model input
|
||||
tokens increase and latency is mixed, so model-token savings are not claimed.
|
||||
|
||||
Follow-up: [model tool context and per-response usage](model-context.md) separates
|
||||
raw MCP replies from observed client execution/discovery and usage. Optional
|
||||
trial capture retains tool/usage items while excluding account and ambient state;
|
||||
existing ephemeral trials keep their default behavior.
|
||||
|
||||
Follow-up: [Android package delivery and retained manifest](android-package-summary.md)
|
||||
records real NewPipe SDK pairs, unchanged old requests/Evidence, exact retained
|
||||
text, and actual Codex outcomes including client errors and a failed discovery.
|
||||
Reply bytes fall; model-token/time gains are not established.
|
||||
|
||||
Follow-up: [resource counters at payload completion](resource-counters.md) adds
|
||||
optional owned counter observations while preserving unknown whole-scope final
|
||||
counters, default receipt shape and the unchanged CPU/memory safeguards.
|
||||
|
||||
The 12 detailed checkpoint receipts were archived byte-for-byte before removal
|
||||
from the final PR tree. The archive `review-receipts-00d51e0fc-20261012` is in the
|
||||
private experiment lab outside the checkout; its `archive-index.json` records
|
||||
@@ -40,8 +87,9 @@ acceptance conditions for every optimization.
|
||||
experiment control; it does not itself establish reduced CPU consumption.
|
||||
|
||||
Migrate domain by domain with focused real-artifact checks and relevant shared
|
||||
contract checks. The domain matrix is a growing roadmap; native PE and DOS, for
|
||||
example, remain unverified in this study. A passed lane does not imply all
|
||||
contract checks. The native follow-up verifies selected PE and DOS MZ entry facts;
|
||||
DOS COM and complete native application behavior remain unverified. The domain
|
||||
matrix is a growing roadmap. A passed lane does not imply all
|
||||
advertised platforms/providers have passed. Newly discovered REA bugs remain
|
||||
in the separate local handoff document for later team repair.
|
||||
|
||||
|
||||
@@ -11,6 +11,8 @@ import subprocess
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
from codex_context import capture_tool_context
|
||||
|
||||
|
||||
def file_digest(path):
|
||||
digest = hashlib.sha256()
|
||||
@@ -48,7 +50,20 @@ def main():
|
||||
parser.add_argument('--providers', type=Path)
|
||||
parser.add_argument('--enabled-tools', type=Path)
|
||||
parser.add_argument('--authority', choices=['static', 'browser'], default='static')
|
||||
parser.add_argument('--capture-tool-context', action='store_true',
|
||||
help='Retain owned tool/usage items; excludes account/ambient state')
|
||||
parser.add_argument('--guide-enabled-tools', action='store_true',
|
||||
help='Opt-in exact-name discovery guidance for selected REA tools')
|
||||
args = parser.parse_args()
|
||||
if args.guide_enabled_tools and args.enabled_tools is None:
|
||||
parser.error('--guide-enabled-tools requires --enabled-tools')
|
||||
enabled_tools = None
|
||||
if args.enabled_tools:
|
||||
enabled_tools = json.loads(args.enabled_tools.read_text())
|
||||
if not isinstance(enabled_tools, list) or not all(isinstance(name, str) and name for name in enabled_tools) or len(enabled_tools) != len(set(enabled_tools)):
|
||||
raise ValueError('enabled-tools must be a unique list of exact tool names')
|
||||
if args.guide_enabled_tools and not enabled_tools:
|
||||
parser.error('--guide-enabled-tools requires a nonempty selected tool list')
|
||||
def terminate(signum, _frame):
|
||||
# The outer resource scope terminates descendants together. Ensure the
|
||||
# runner's copied credentials are also removed on a resource abort.
|
||||
@@ -130,11 +145,14 @@ def main():
|
||||
'possible and select the requested normalized_result fields before returning '
|
||||
'intermediate output to model context. Keep real Evidence IDs and limitations.'
|
||||
)
|
||||
enabled_tools = None
|
||||
if args.enabled_tools:
|
||||
enabled_tools = json.loads(args.enabled_tools.read_text())
|
||||
if not isinstance(enabled_tools, list) or not all(isinstance(name, str) and name for name in enabled_tools) or len(enabled_tools) != len(set(enabled_tools)):
|
||||
raise ValueError('enabled-tools must be a unique list of exact tool names')
|
||||
if args.guide_enabled_tools:
|
||||
instruction += (
|
||||
' Configured REA tool function names for this trial: '
|
||||
+ json.dumps(['mcp__rea__' + name for name in enabled_tools])
|
||||
+ '. For discovery select only these exact names from ALL_TOOLS and '
|
||||
'return only their metadata. Preserve the original analyst question, '
|
||||
'dependency order and source Evidence.'
|
||||
)
|
||||
quote = json.dumps
|
||||
config = '\n'.join([
|
||||
'approval_policy = "never"', 'sandbox_mode = "read-only"',
|
||||
@@ -160,6 +178,7 @@ def main():
|
||||
'inspect_android_package', 'search_android_classes',
|
||||
'inspect_android_class', 'inspect_android_method',
|
||||
'inspect_firmware_regions', 'inspect_managed_artifact',
|
||||
'inspect_sqlite_database',
|
||||
'inspect_managed_members', 'inspect_managed_native_boundaries',
|
||||
'inspect_artifact', 'inspect_wasm_artifact',
|
||||
'inspect_evm_interface', 'get_evidence_bundle', 'export_evidence_bundle',
|
||||
@@ -183,6 +202,8 @@ def main():
|
||||
(args.output / 'prompt.txt').write_text(prompt)
|
||||
(args.output / 'trial-settings.json').write_text(json.dumps({
|
||||
'mode': args.mode, 'model': args.model, 'reasoning': args.reasoning,
|
||||
'capture_tool_context': args.capture_tool_context,
|
||||
'guide_enabled_tools': args.guide_enabled_tools,
|
||||
'driver_commit': subprocess.check_output(
|
||||
['git', 'rev-parse', 'HEAD'], cwd=driver_repository, text=True).strip(),
|
||||
'runtime_revision': args.revision,
|
||||
@@ -196,7 +217,7 @@ def main():
|
||||
'package_lock_sha256': file_digest(repository / 'package-lock.json'),
|
||||
'experiment_sources_sha256': {
|
||||
name: hashlib.sha256((experiment / name).read_bytes()).hexdigest()
|
||||
for name in ['agent-trial.py', 'provider-environment.json']
|
||||
for name in ['agent-trial.py', 'codex_context.py', 'provider-environment.json']
|
||||
},
|
||||
'developer_instructions': instruction,
|
||||
'developer_instructions_sha256': hashlib.sha256(instruction.encode()).hexdigest(),
|
||||
@@ -212,7 +233,8 @@ def main():
|
||||
'multi_agent': False, 'code_mode': args.mode == 'client-code',
|
||||
'code_mode_only': args.mode == 'client-code'},
|
||||
}, indent=2) + '\n')
|
||||
invocation = [args.codex, '--no-daemon', 'exec', '--strict-config', '--ephemeral',
|
||||
invocation = [args.codex, '--no-daemon', 'exec', '--strict-config',
|
||||
*([] if args.capture_tool_context else ['--ephemeral']),
|
||||
'--skip-git-repo-check', '--ignore-rules', '--json',
|
||||
'--color', 'never', '-C', str(workspace),
|
||||
'--output-last-message', str(args.output.resolve() / 'answer.txt'), '-']
|
||||
@@ -234,26 +256,28 @@ def main():
|
||||
usage = []
|
||||
tools = []
|
||||
event_types = {}
|
||||
for line in (args.output / 'events.jsonl').open():
|
||||
if not line.strip():
|
||||
continue
|
||||
event = json.loads(line)
|
||||
kind = event.get('type', 'unknown')
|
||||
event_types[kind] = event_types.get(kind, 0) + 1
|
||||
if 'usage' in event:
|
||||
usage.append(event['usage'])
|
||||
item = event.get('item', {})
|
||||
if item.get('type') == 'mcp_tool_call' and kind == 'item.completed':
|
||||
tools.append({
|
||||
**{k: item.get(k) for k in ['server', 'tool', 'arguments', 'status', 'error']},
|
||||
'codex_event_result_json_utf8_bytes': len(json.dumps(
|
||||
item.get('result'), separators=(',', ':'), ensure_ascii=False
|
||||
).encode()),
|
||||
})
|
||||
with (args.output / 'events.jsonl').open() as event_file:
|
||||
for line in event_file:
|
||||
if not line.strip():
|
||||
continue
|
||||
event = json.loads(line)
|
||||
kind = event.get('type', 'unknown')
|
||||
event_types[kind] = event_types.get(kind, 0) + 1
|
||||
if 'usage' in event:
|
||||
usage.append(event['usage'])
|
||||
item = event.get('item', {})
|
||||
if item.get('type') == 'mcp_tool_call' and kind == 'item.completed':
|
||||
tools.append({
|
||||
**{k: item.get(k) for k in ['server', 'tool', 'arguments', 'status', 'error']},
|
||||
'codex_event_result_json_utf8_bytes': len(json.dumps(
|
||||
item.get('result'), separators=(',', ':'), ensure_ascii=False
|
||||
).encode()),
|
||||
})
|
||||
final_target_digest = file_digest(Path(args.artifact_path)) if args.artifact_path else None
|
||||
summary = {
|
||||
'mode': args.mode, 'requested_model': args.model, 'reasoning': args.reasoning,
|
||||
'runtime_revision': args.revision,
|
||||
'capture_tool_context_requested': args.capture_tool_context,
|
||||
'selected_artifact_sha256_before': target_digest,
|
||||
'selected_artifact_sha256_after': final_target_digest,
|
||||
'selected_artifact_bytes_unchanged': target_digest == final_target_digest if target_digest else None,
|
||||
@@ -268,9 +292,13 @@ def main():
|
||||
(args.output / 'summary.json').write_text(json.dumps(summary, indent=2) + '\n')
|
||||
print(json.dumps(summary))
|
||||
finally:
|
||||
# Always remove copied credentials, config, caches and profile state.
|
||||
shutil.rmtree(profile)
|
||||
(args.output / 'cleanup.json').write_text(json.dumps({'profile_removed': not profile.exists()}) + '\n')
|
||||
try:
|
||||
if args.capture_tool_context:
|
||||
capture_tool_context(trial_codex_home, args.output)
|
||||
finally:
|
||||
# Capture failure must never prevent copied-credential cleanup.
|
||||
shutil.rmtree(profile)
|
||||
(args.output / 'cleanup.json').write_text(json.dumps({'profile_removed': not profile.exists()}) + '\n')
|
||||
raise SystemExit(124 if timeout else exit_code)
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,166 @@
|
||||
# Android package summary and exact retained manifest
|
||||
|
||||
Continues [#2016](https://github.com/morluto/rea/issues/2016) and
|
||||
[#2438](https://github.com/morluto/rea/pull/2438). The recurring NewPipe question
|
||||
asks for APK/package/version identity, all case-sensitive MainActivity matches
|
||||
and all reported onCreate entries, including explicit null DEX descriptors.
|
||||
Complete package replies delivered decoded manifest text and duplicate upstream
|
||||
calls unrelated to those selected facts.
|
||||
|
||||
## Compatibility and ownership
|
||||
|
||||
MCP `inspect_android_package` accepts explicit `detail: "summary"`; omitted and
|
||||
`"complete"` requests preserve the complete output and provider request. The
|
||||
one-shot CLI stays complete. Summary selection happens in the application
|
||||
workflow, independently of the engine protocol, using the existing retained
|
||||
analysis-view machinery. No third-party plugin API or persistent analysis cache
|
||||
is introduced. Servers lacking retention reject summary before provider work.
|
||||
|
||||
The complete original Evidence stays under its original ID and session lifetime.
|
||||
A distinct linked derived view returns all package/SDK/permission declarations,
|
||||
engine identity, class/resource counts and signature-verification status.
|
||||
Manifest metadata retains the producer's partial/complete status, nullable
|
||||
reported byte total, and observed UTF-16 length. Text and duplicate raw calls
|
||||
remain in the original record, rather than being rewritten under its ID.
|
||||
|
||||
Use `inspect_analysis_view` with the parent's ID and
|
||||
`view: {kind: "android", facet: "manifest", offset: 0, limit: 4096}` to read exact
|
||||
retained decoded text. Follow `coverage.next_offset`; surrogate boundaries use
|
||||
the existing text-page implementation. Exhaustion describes the retained text,
|
||||
not global completeness when the producer says partial. APK signatures and
|
||||
runtime behavior remain unverified. Engine artifact SHA-256 identifies the JAR;
|
||||
views require the authenticated APK subject and never substitute the engine
|
||||
hash for a missing target identity. Older servers require schema discovery and
|
||||
the original complete request; automatic fallback is not introduced.
|
||||
|
||||
## Frozen runtime and real SDK pairs
|
||||
|
||||
Source runtime: `28c84e78cf2fbfde4aceed345b773c59eb889f68`; later test-only schema
|
||||
typing corrections do not relabel these receipts. Compiled dist/bridge file-table
|
||||
SHA-256: `5332dda13981f043e146b3601ebec1f380d9f698280ca53d75786b638a912e11`.
|
||||
The frozen old runtime is `8bc1601ca51e417ca472d90ea0367e9418a1a0f2`, digest
|
||||
`9e7380f537301bf0cfe2926f5619011af848af7c220750e75189498e65d825e1`.
|
||||
|
||||
Real NewPipe 0.29.1 bytes, acquisition and digest are pinned in `corpus.json`.
|
||||
Actual configured JADX headless MCP 0.7.1/Java runs use the same selected
|
||||
provider settings and search-path digests, one worker and 768 MiB Java heap.
|
||||
Node is 24.18.0; actual SDK client/server are 2.3.1. Processes and engines are
|
||||
fresh per trial; host page caches are not flushed. SDK schema mode is selected
|
||||
in both treatments. Heavy runs are serial, with one CPU, unchanged 3 GiB early
|
||||
stop/4 GiB hard memory, no swap, and 192 tasks.
|
||||
|
||||
All four candidate workflows use three successful public MCP calls. Only the
|
||||
first package delivery changes. Both pairs pass settings and selected-fact
|
||||
comparison, including all declarations, manifest status/length, engine identity,
|
||||
original limitations, matching class rows and onCreate entries. Original replies
|
||||
and IDs remain separate. A fresh old-runtime complete workflow also passes
|
||||
comparison against the candidate complete request; both complete package replies
|
||||
are exactly 119,527 compact client-result JSON bytes. This is scoped compatibility
|
||||
evidence, not a guarantee for every historical release/client/host.
|
||||
|
||||
| Trial | Package reply bytes | SDK workflow seconds | Payload CPU seconds | Kernel peak bytes |
|
||||
| -------------- | ------------------: | -------------------: | ------------------: | ----------------: |
|
||||
| sdk-complete-1 | 119,527 | 24.059 | 25.259 | 1,225,076,736 |
|
||||
| sdk-summary-1 | 8,659 | 23.694 | 24.894 | 1,226,870,784 |
|
||||
| sdk-summary-2 | 8,659 | 24.407 | 25.611 | 1,228,296,192 |
|
||||
| sdk-complete-2 | 119,527 | 24.509 | 25.740 | 1,229,844,480 |
|
||||
|
||||
Summary replies are 8,659 bytes, about 92.8% smaller; all three replies together
|
||||
fall from 149,577 to 38,709 bytes. This is a delivery result, not a model-token
|
||||
or engine-work saving. The SDK timing differences are small. CPU and memory do
|
||||
not establish a reduction. No provider setting or analysis scope changes.
|
||||
Transport roundtrip includes server/client work and is not an engine-only timer.
|
||||
|
||||
The complete catalog is 2,506,213 compact JSON bytes versus 2,497,635 on the old
|
||||
runtime: about 0.34% growth, including intervening main changes. Discovery cost
|
||||
remains visible alongside selected-delivery savings.
|
||||
|
||||
## Authentic old Evidence continuation
|
||||
|
||||
A real package record from old runtime 8bc160 imports into the candidate and
|
||||
supports retained summary, exact manifest pages, inline portable summary and
|
||||
export through actual public SDK tools. The new session explicitly selects an
|
||||
absent JADX JAR; this workflow performs no package analysis. Eleven successful
|
||||
calls preserve the original record, original ID, all declarations and all 25,222
|
||||
manifest UTF-16 units. Manifest UTF-8 SHA-256:
|
||||
`fee192fe2a6d7892c22cba115184c74c7e2c42ce464e8bb37be6289f2c16881d`.
|
||||
The exported original record is deeply equal to the imported record.
|
||||
|
||||
## Actual Codex task outcomes
|
||||
|
||||
Four serial trials run in complete/summary/summary/complete order on the same
|
||||
frozen candidate. Requested configuration: Codex CLI 0.161.0, `gpt-5.6-terra` high,
|
||||
three exact selected tool names and the same optional context/discovery capture
|
||||
settings. The requested alias does not certify the server-resolved model. The
|
||||
original analyst question is unchanged; equal treatment instructions explicitly
|
||||
choose package detail and explain the summary carrier. Temporary configuration
|
||||
points to the frozen modified REA MCP; no global configuration is edited.
|
||||
|
||||
| Trial | Input | Cached | Uncached | Output | Seconds | MCP calls | Usage records | Facts |
|
||||
| ---------------- | ------: | -----: | -------: | -----: | ------: | --------: | ------------: | ----- |
|
||||
| model-complete-1 | 39,267 | 23,040 | 16,227 | 1,690 | 94.053 | 3 | 3 | Pass |
|
||||
| model-summary-1 | 128,417 | 93,440 | 34,977 | 3,633 | 150.401 | 8 | 9 | Pass |
|
||||
| model-summary-2 | 54,437 | 37,120 | 17,317 | 2,185 | 123.677 | 3 | 4 | Pass |
|
||||
| model-complete-2 | 21,332 | 9,984 | 11,348 | 667 | 20.774 | 0 | 2 | Fail |
|
||||
|
||||
Three answers pass the original selected-fact gate against the actual SDK
|
||||
producer. Each trial preserves APK bytes and removes its isolated profile.
|
||||
The first summary trial makes five successful calls and three rejected requests:
|
||||
an invented `case_sensitive` argument, empty class-search input and empty
|
||||
class-inspection input. It then repeats successful package/search work. These
|
||||
errors and correction turns remain part of measured task cost; strict existing
|
||||
schemas correctly reject them. The second summary trial uses the intended three
|
||||
successful calls, but still does not establish improved total/uncached input or
|
||||
latency against the factually acceptable first complete control.
|
||||
|
||||
The final complete control's first client executor filters `ALL_TOOLS` for the
|
||||
same three exact names and receives an empty array. It makes zero MCP calls,
|
||||
reports all requested facts unknown and fails the factual gate. Identical name
|
||||
filters worked in the other trials. Stderr contains no explanatory error; the
|
||||
client visibility failure remains unexplained and is not attributed to REA or
|
||||
summary delivery. Its short duration is not comparable completed work. The
|
||||
original failed receipt is retained rather than replaced by a successful trial.
|
||||
|
||||
These measurements demonstrate no real-model token or speed gain. Smaller raw
|
||||
MCP envelopes do not establish smaller model context; observed usage also
|
||||
includes discovery, execution context, corrections and extra responses. Usage
|
||||
records count captured usage items rather than asserting exact remote request
|
||||
contents or equating MCP calls with model rounds. The complete default and
|
||||
opt-in choice remain unchanged; this is useful delivery/retention capability,
|
||||
not a general performance recommendation.
|
||||
|
||||
All model runs have verified payload-completion witnesses with only the observer
|
||||
remaining and zero memory/task limit events at that phase. Kernel peaks are
|
||||
602,472,448–1,444,945,920 bytes; the failed no-analysis control is included in that
|
||||
range and is not evidence of lower analysis memory. Whole-scope final counters
|
||||
remain unknown and the existing full resource grader stays false. No broader
|
||||
CPU, memory, OS/provider or model claim is made.
|
||||
|
||||
## Resource interpretation
|
||||
|
||||
Every SDK run and portable continuation has a verified owned
|
||||
payload-exited-observer-alive witness, with only the observer remaining and no
|
||||
memory/task limit events at that phase. Kernel peak and CPU include that opt-in
|
||||
observer. Whole-scope terminal counters remain unknown; the existing complete
|
||||
physical resource grader is not changed or called successful. No OOM stress or
|
||||
resource-limit increase is used.
|
||||
|
||||
## Verification and scope
|
||||
|
||||
Twenty-seven relevant converted-schema, SDK, domain and existing Android
|
||||
failure/cancellation/cleanup checks pass. A three-case focused rerun passes after
|
||||
a test-only schema typing correction. The 1 MiB manifest protocol fixture verifies
|
||||
original default/full output, reconstruction after source removal, unchanged
|
||||
exported Evidence and one engine launch. Other cases preserve nulls and partial
|
||||
status, Unicode boundaries and tagged malformed/missing-identity failures, and
|
||||
reject unavailable retention or cancelled requests before provider work. These
|
||||
fixtures verify production boundaries; actual JADX claims use the real NewPipe
|
||||
runs above. Full supported single-thread compiler and unchanged lint pass under
|
||||
the same memory safeguards. No other Android OS/device/provider lane is inferred.
|
||||
|
||||
Raw SDK/model replies, private profiles and resource receipts stay outside Git.
|
||||
Reproduce SDK pairs using the existing `measure.mjs` with `--artifact newpipe.apk`,
|
||||
`--schemas selected`, identical explicit provider configuration and frozen runtime,
|
||||
and alternating `--delivery complete` / `--delivery summary`. Preserve failed
|
||||
attempts and run `compare.mjs` on both pair orders. Model findings require their
|
||||
own identical analyst facts and explicit delivery controls.
|
||||
@@ -0,0 +1,117 @@
|
||||
"""Retain owned Codex tool/usage observations without account or ambient state."""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
|
||||
|
||||
def token_counts(value):
|
||||
if not isinstance(value, dict):
|
||||
raise ValueError('Expected a token-count object')
|
||||
result = {}
|
||||
for key in ['input_tokens', 'cached_input_tokens', 'cache_write_input_tokens',
|
||||
'output_tokens', 'reasoning_output_tokens', 'total_tokens']:
|
||||
if key in value:
|
||||
count = value[key]
|
||||
if not isinstance(count, int) or isinstance(count, bool) or count < 0:
|
||||
raise ValueError('Expected nonnegative integer token counts')
|
||||
result[key] = count
|
||||
return result
|
||||
|
||||
|
||||
def selected_event(event):
|
||||
"""Select observed items; these are not the exact submitted model request."""
|
||||
if not isinstance(event, dict) or not isinstance(event.get('payload'), dict):
|
||||
raise ValueError('Expected an event object with an object payload')
|
||||
if not isinstance(event.get('type'), str):
|
||||
raise ValueError('Expected an event type string')
|
||||
if event.get('timestamp') is not None and not isinstance(event['timestamp'], str):
|
||||
raise ValueError('Expected a timestamp string or omission')
|
||||
payload = event['payload']
|
||||
kind = event.get('type')
|
||||
if kind == 'response_item' and payload.get('type') in [
|
||||
'custom_tool_call', 'custom_tool_call_output',
|
||||
'function_call', 'function_call_output',
|
||||
]:
|
||||
keys = ['type', 'call_id', 'name', 'input', 'arguments', 'output']
|
||||
elif kind == 'token_usage_record':
|
||||
keys = ['response_id', 'usage', 'turn_token_usage', 'thread_token_usage']
|
||||
payload = {key: token_counts(payload[key]) if key != 'response_id'
|
||||
else payload[key] for key in keys if key in payload}
|
||||
elif kind == 'event_msg' and payload.get('type') == 'token_count':
|
||||
keys = ['type', 'info']
|
||||
info = payload.get('info')
|
||||
if info is not None:
|
||||
if not isinstance(info, dict):
|
||||
raise ValueError('Expected token-count info')
|
||||
selected_info = {key: token_counts(info[key]) for key in
|
||||
['total_token_usage', 'last_token_usage'] if key in info}
|
||||
window = info.get('model_context_window')
|
||||
if 'model_context_window' in info:
|
||||
if window is not None and (not isinstance(window, int) or isinstance(window, bool) or window <= 0):
|
||||
raise ValueError('Expected a positive model context window')
|
||||
selected_info['model_context_window'] = window
|
||||
payload = {'type': 'token_count', 'info': selected_info}
|
||||
else:
|
||||
payload = {'type': 'token_count', 'info': None}
|
||||
else:
|
||||
return None
|
||||
return {
|
||||
'timestamp': event.get('timestamp'),
|
||||
'type': kind,
|
||||
'payload': {key: payload[key] for key in keys if key in payload},
|
||||
}
|
||||
|
||||
|
||||
def capture_tool_context(codex_home, output):
|
||||
"""Capture only an explicitly selected trial's rollouts; never its auth/config."""
|
||||
sessions = codex_home / 'sessions'
|
||||
blocked = sessions.is_symlink()
|
||||
sources = [] if blocked else sorted(sessions.rglob('*.jsonl'))
|
||||
target = output / 'codex-tool-context.jsonl'
|
||||
source_receipts = []
|
||||
errors = [{'source': 'sessions', 'reason': 'not_an_owned_directory'}] if blocked else []
|
||||
retained = 0
|
||||
with os.fdopen(os.open(target, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600),
|
||||
'w', encoding='utf-8') as result:
|
||||
for source in sources:
|
||||
name = source.relative_to(codex_home).as_posix()
|
||||
if source.is_symlink() or not source.is_file():
|
||||
errors.append({'source': name, 'reason': 'not_an_owned_regular_file'})
|
||||
continue
|
||||
digest = hashlib.sha256()
|
||||
counts = {}
|
||||
with source.open('rb') as lines:
|
||||
for ordinal, line in enumerate(lines, 1):
|
||||
digest.update(line)
|
||||
try:
|
||||
event = json.loads(line)
|
||||
selected = selected_event(event)
|
||||
except ValueError as error:
|
||||
errors.append({'source': name, 'line': ordinal,
|
||||
'reason': type(error).__name__})
|
||||
continue
|
||||
kind = event.get('type')
|
||||
counts[str(kind)] = counts.get(str(kind), 0) + 1
|
||||
if selected is not None:
|
||||
result.write(json.dumps(selected, ensure_ascii=False) + '\n')
|
||||
retained += 1
|
||||
source_receipts.append({'source': name, 'sha256': digest.hexdigest(),
|
||||
'event_counts': counts})
|
||||
retained_digest = hashlib.sha256()
|
||||
with target.open('rb') as captured:
|
||||
for chunk in iter(lambda: captured.read(1024 * 1024), b''):
|
||||
retained_digest.update(chunk)
|
||||
receipt = {
|
||||
'status': 'partial' if errors else 'captured' if source_receipts else 'unavailable',
|
||||
'retained_file': target.name,
|
||||
'retained_sha256': retained_digest.hexdigest(),
|
||||
'retained_events': retained, 'sources': source_receipts, 'errors': errors,
|
||||
'scope': 'Owned client tool-call/reply and per-response usage items, not '
|
||||
'the exact submitted model request. Session/account metadata, '
|
||||
'world state, user/developer messages, encrypted reasoning and '
|
||||
'internal passthrough are omitted. Authentication, configuration '
|
||||
'and ambient environment are never copied by this capture.',
|
||||
}
|
||||
(output / 'context-receipt.json').write_text(json.dumps(receipt, indent=2) + '\n')
|
||||
return receipt
|
||||
@@ -24,8 +24,10 @@ const sameSettings = [
|
||||
"artifact",
|
||||
"artifact_sha256",
|
||||
"artifact_sha256_after",
|
||||
"retained_verification",
|
||||
"native_question",
|
||||
"javascript_question",
|
||||
"managed_question",
|
||||
"provider_settings_sha256",
|
||||
"provider_search_path_sha256",
|
||||
"provider_environment_policy_sha256",
|
||||
|
||||
@@ -179,6 +179,90 @@
|
||||
"transport_integrity": "sha512-RJFoomET6FajjG511fmQxeBQfU6M24a0aFZPqpid+ttIxanWf1VGytBG0UmsGjt07qmIPJS8U31D+aecuCucsQ==",
|
||||
"derivation": "selected regular tar member package/wasm/tree-sitter-css.wasm; no target execution"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "uniswap-v3.factory.hex",
|
||||
"file": "UniswapV3Factory.runtime.hex",
|
||||
"bytes": 49073,
|
||||
"sha256": "fb152cc9b597ef4e54e9d72e81dd72c202050aae5e98c63df139864274504276",
|
||||
"decoded_bytes": 24535,
|
||||
"decoded_sha256": "95c34fe1e6b26c79ae42800041a3e2bdb6a978d19297cd74001ac6b316a48d8d",
|
||||
"provenance": {
|
||||
"project": "Uniswap v3-core UniswapV3Factory",
|
||||
"version": "1.0.1",
|
||||
"url": "https://registry.npmjs.org/@uniswap/v3-core/-/v3-core-1.0.1.tgz",
|
||||
"transport_bytes": 86371,
|
||||
"transport_sha256": "141b001e2a376fff867cb24e0713b2834095cb1250c13aae263d9a646446d329",
|
||||
"transport_integrity": "sha512-7pVk4hEm00j9tc71Y9+ssYpO6ytkeI0y7WE9P6UcmNzhxPePwyAxImuhVsTqWK9YFvzgtvzJHi64pBl4jUzKMQ==",
|
||||
"archive_member": "package/artifacts/contracts/UniswapV3Factory.sol/UniswapV3Factory.json",
|
||||
"compiler_artifact_bytes": 104076,
|
||||
"compiler_artifact_sha256": "599479f60ebb056804aff7b2d05bdd0830ddbb1fdfaa0b6c62c02294ca7188b0",
|
||||
"derivation": "Parse the exact published Hardhat deployedBytecode field and preserve its original hex carrier with one trailing LF; preserve template placeholders",
|
||||
"authority": "Published compiler runtime template, not a live deployed instance; original placeholders/immutables preserved; no execution or chain lookup",
|
||||
"published_abi_function_count": 7,
|
||||
"deployed_link_references": {},
|
||||
"immutable_references": {
|
||||
"recorded": false,
|
||||
"value": null
|
||||
},
|
||||
"immutable_scope": "Published Hardhat artifact does not record an immutable-reference map; absence is unknown, not proof of no immutables. Runtime template bytes remain exact."
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "uniswap-v3.pool.hex",
|
||||
"file": "UniswapV3Pool.runtime.hex",
|
||||
"bytes": 44287,
|
||||
"sha256": "84d90218eb125074383287877cdc564eca3765241e3bf6f6afdd64a922fbed2b",
|
||||
"decoded_bytes": 22142,
|
||||
"decoded_sha256": "3ef71385c2a22b945bead7904f24b25db877de9db51d1401f2f9daea07ea5c0c",
|
||||
"provenance": {
|
||||
"project": "Uniswap v3-core UniswapV3Pool",
|
||||
"version": "1.0.1",
|
||||
"url": "https://registry.npmjs.org/@uniswap/v3-core/-/v3-core-1.0.1.tgz",
|
||||
"transport_bytes": 86371,
|
||||
"transport_sha256": "141b001e2a376fff867cb24e0713b2834095cb1250c13aae263d9a646446d329",
|
||||
"transport_integrity": "sha512-7pVk4hEm00j9tc71Y9+ssYpO6ytkeI0y7WE9P6UcmNzhxPePwyAxImuhVsTqWK9YFvzgtvzJHi64pBl4jUzKMQ==",
|
||||
"archive_member": "package/artifacts/contracts/UniswapV3Pool.sol/UniswapV3Pool.json",
|
||||
"compiler_artifact_bytes": 111557,
|
||||
"compiler_artifact_sha256": "204fb2fa7ad1a639b1df5601f035f9924b064b721b97adf60c7d5d4712643e34",
|
||||
"derivation": "Parse the exact published Hardhat deployedBytecode field and preserve its original hex carrier with one trailing LF; preserve template placeholders",
|
||||
"authority": "Published compiler runtime template, not a live deployed instance; original placeholders/immutables preserved; no execution or chain lookup",
|
||||
"published_abi_function_count": 26,
|
||||
"deployed_link_references": {},
|
||||
"immutable_references": {
|
||||
"recorded": false,
|
||||
"value": null
|
||||
},
|
||||
"immutable_scope": "Published Hardhat artifact does not record an immutable-reference map; absence is unknown, not proof of no immutables. Runtime template bytes remain exact."
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "localsend-arm64.apk",
|
||||
"file": "LocalSend-1.18.2-android-arm64v8.apk",
|
||||
"bytes": 46558706,
|
||||
"sha256": "82ec3568fba2aa5295b9aae8b76f701d7a4703d86b9f8bad749472038fbaeab3",
|
||||
"provenance": {
|
||||
"project": "LocalSend",
|
||||
"version": "1.18.2",
|
||||
"upstreamFile": "LocalSend-1.18.2-android-arm64v8.apk",
|
||||
"url": "https://github.com/localsend/localsend/releases/download/v1.18.2/LocalSend-1.18.2-android-arm64v8.apk",
|
||||
"transport_sha256": "82ec3568fba2aa5295b9aae8b76f701d7a4703d86b9f8bad749472038fbaeab3",
|
||||
"transport_bytes": 46558706,
|
||||
"derivation": "Official arm64 release APK, compared with GitHub release asset digest; byte-only analysis without execution"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "qgis-world-map.gpkg",
|
||||
"file": "qgis-world-map.gpkg",
|
||||
"bytes": 12914688,
|
||||
"sha256": "01d293159d1b77bd9212b5252587853f61aa9e25b2387135f34afaba8d057190",
|
||||
"provenance": {
|
||||
"project": "QGIS bundled world map",
|
||||
"source_commit": "e1223b8e9e71fc50ee995ed556f711c54ccc101b",
|
||||
"git_blob_sha1": "c9ff80228dff0de93c9acdef16a97545ba360244",
|
||||
"url": "https://raw.githubusercontent.com/qgis/QGIS/e1223b8e9e71fc50ee995ed556f711c54ccc101b/resources/data/world_map.gpkg",
|
||||
"derivation": "unmodified official resource file; streaming Git blob and SHA-256 verification"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -0,0 +1,218 @@
|
||||
# EVM descriptor and real-template checkpoint
|
||||
|
||||
Continues [#2016](https://github.com/morluto/rea/issues/2016) after the native
|
||||
checkpoint merged. The internal descriptor binds the existing exact EVM
|
||||
contract, CLI name and typed service lifetime. Existing complete inputs, output,
|
||||
provider selection and cleanup remain the baseline. Wiring alone does not establish
|
||||
token or latency savings.
|
||||
|
||||
The descriptor contains only the named contract, existing CLI name and existing
|
||||
factory. The legacy `createEvmInterfaceService` export remains usable. The MCP
|
||||
host selects one complete service instance, preserves injection identity, passes
|
||||
that same instance to registration and retains it for cleanup/retry. No bound
|
||||
service wrapper, generic registry, automatic dependency acquisition or new pack
|
||||
selection is introduced.
|
||||
|
||||
Five actual in-memory SDK boundary scenarios verify converted schemas, duplicate
|
||||
registration rejection before use, malformed inputs, explicit unavailable or
|
||||
incompatible-output failures with a healthy WASM peer, cleanup retry, and request
|
||||
cancellation followed by recovery. Their injected ports are recording seams, not
|
||||
real EVMole/WABT claims. Nine service and 26 provider-boundary cases also pass;
|
||||
two unsupported-host scenarios are skipped on this Linux host. The existing seven
|
||||
WASM MCP scenarios pass. Full `npm run check` passes; its approximately 3 GiB
|
||||
sampled peak and `memory.high` events are excluded from performance claims.
|
||||
Candidate real-artifact/catalog and model observations are recorded below.
|
||||
|
||||
## Public inputs and independent checks
|
||||
|
||||
[corpus.json](corpus.json) retains the original WETH historical example and adds
|
||||
Uniswap v3-core 1.0.1 Factory/Pool compiler runtime templates from the official npm
|
||||
package. Archive integrity, exact regular member, compiler JSON, original carrier
|
||||
and decoded-byte digests are pinned. The carrier is the original `deployedBytecode`
|
||||
field plus one trailing LF. No placeholder is rewritten and no contract is executed.
|
||||
|
||||
These are compiler templates, not live deployed instances. Their published Hardhat
|
||||
artifacts omit immutable-reference maps; that metadata remains explicitly unknown.
|
||||
The recorded empty link-reference maps do not establish absence of immutables.
|
||||
WETH remains a published historical RPC example, with unknown live authenticity.
|
||||
|
||||
Actual control CLI observations infer 7 Factory and 26 Pool functions. The
|
||||
[independent verifier](verify-evm-template.mjs) hashes each canonical published ABI
|
||||
signature with caller-supplied Ethereum Keccak and checks the complete ABI-dispatch
|
||||
selector multiset against the original REA response. Both sets match. This does not
|
||||
establish inferred argument/mutability equivalence, deployment authenticity or
|
||||
runtime semantics; all producer unknowns remain visible.
|
||||
|
||||
## Latest-main control
|
||||
|
||||
Production control: `1770f38cfc6cbe87491d0de8b8730c305c5cf8ea`, Node 24.18.0,
|
||||
npm 11.16.0, matching locked dependencies and unchanged EVMole 0.9.3.
|
||||
The frozen `dist/`/`bridge/` file-table SHA-256 is
|
||||
`7fd6c2c37b4e166df158ec29d48d31f3712b9b087d9807ee019ee997d06ea144`.
|
||||
Measurement driver SHA-256 is
|
||||
`f767c0c0232be1e11918e70e3197a6919680d16688076bef28d4320662f712f7`.
|
||||
Each public SDK run validates the selected named advertised schema and verifies
|
||||
original carrier and decoded identities, inference/unknown scope and unchanged
|
||||
input bytes. All three runs return the same complete 2,420,709-byte catalog.
|
||||
|
||||
| Question | Inferred functions | SDK tool-result JSON bytes | First workflow seconds |
|
||||
| ------------------------ | -----------------: | -------------------------: | ---------------------: |
|
||||
| WETH historical carrier | 11 | 26,760 | 11.432 |
|
||||
| Uniswap Factory template | 7 | 110,424 | 3.628 |
|
||||
| Uniswap Pool template | 26 | 110,998 | 11.211 |
|
||||
|
||||
These are initial observations, not paired performance conclusions or model-token
|
||||
counts. Each workflow includes connection, complete catalog discovery and one
|
||||
interface query. Full bytecode remains inline. Candidate/control repetitions must
|
||||
use the same source driver, inputs, provider settings and Node/worker limits.
|
||||
|
||||
Preparation against earlier frozen main `b1bf4a70d44191348ca20a91a52fb663da4daed0`
|
||||
also passed all 59 cases of the existing real CLI/MCP EVM verifier with privately
|
||||
supplied solc 0.8.30 and strace 6.1. Those cases cover compiler configurations,
|
||||
raw/hex identity, malformed/unsupported carriers, missing limiter and resource
|
||||
constraints. Its measured scope took 233.1 seconds, sampled peak 1,192,730,624
|
||||
bytes and last-observed CPU 181.018 seconds. This is a control, not candidate proof.
|
||||
|
||||
All heavy runs are serial under one CPU, 4 GiB hard memory/3 GiB early-stop, no swap
|
||||
and bounded tasks. Final cgroup counters disappear at scope teardown and remain
|
||||
unverified; last-observed zero limit events are not substituted for final counters.
|
||||
Raw replies, tools, compiler artifacts and detailed receipts remain outside Git.
|
||||
|
||||
## Reproduction
|
||||
|
||||
Build and freeze each pinned runtime using the repository toolchain, then run
|
||||
`measure.mjs --artifact uniswap-v3.factory.hex` or `uniswap-v3.pool.hex` with the
|
||||
exact derived carrier, `--schemas selected`, explicit runtime/revision and a fresh
|
||||
private output directory. Existing `weth.hex` answer fields and invocation remain
|
||||
unchanged. The same common workflow verifies all three offline carriers.
|
||||
|
||||
Run `verify-evm-template.mjs` with `--artifact` selecting the corpus ID,
|
||||
`--artifact-json` selecting its original published compiler JSON, `--evidence`
|
||||
selecting the recorded CLI Evidence or SDK structured reply, `--keccak-module`
|
||||
selecting the absolute `js-sha3/src/sha3.js` module from the private solc toolchain,
|
||||
and `--output` selecting a new private receipt. It acquires no dependencies and
|
||||
does not overwrite an existing receipt.
|
||||
|
||||
The [template model question](questions/evm-uniswap.txt) requires the same complete
|
||||
inferred interface facts and explicit unknowns. An ABI-count match is not used as
|
||||
a model grade.
|
||||
|
||||
## Candidate compatibility and real-provider verification
|
||||
|
||||
Candidate: `152984da771450048ee9d60db8a410994a125878`, frozen runtime file-table
|
||||
SHA-256 `725563640c27eb50ddb6f977d2dfbe1bdaddf7e05c02897318bceaa36fa602d7`.
|
||||
Two repetitions for each EVM input produce six passing paired fact/settings
|
||||
comparisons with the same driver and explicit limits. Each report preserves both
|
||||
original Evidence reference maps. The complete actual SDK catalog is byte-identical
|
||||
on every pair, including order, names, descriptions, schemas and annotations;
|
||||
catalog SHA-256 is
|
||||
`8664e42fd5225645cfd910f440f731c67ac198d3d53aa85b5c485374c74cf603`.
|
||||
Result byte counts remain unchanged. Candidate Factory/Pool selectors also match
|
||||
the independent published ABI oracle.
|
||||
|
||||
| Question | Control seconds, repetitions 1 / 2 | Candidate seconds, repetitions 1 / 2 |
|
||||
| -------- | ---------------------------------: | -----------------------------------: |
|
||||
| Factory | 3.628 / 3.650 | 3.708 / 3.605 |
|
||||
| Pool | 11.211 / 3.591 | 3.749 / 3.581 |
|
||||
| WETH | 11.432 / 3.694 | 3.572 / 3.677 |
|
||||
|
||||
First controls preceded wiring; each second pair ran candidate then control.
|
||||
Every measurement starts a new MCP process; host cache/page state is not fully
|
||||
controlled. Slower initial controls do not establish a descriptor speedup.
|
||||
Second-pair latency and last-observed CPU are similar. Sampled EVM SDK family peaks
|
||||
range from 510,255,104 to 527,622,144 bytes. No scope abort occurred; final counters
|
||||
remain unknown, so these measurements are descriptive.
|
||||
|
||||
The candidate passes all 59 existing real CLI/MCP verifier cases with solc 0.8.30
|
||||
and unchanged EVMole 0.9.3. Actual syscall traces report target execution and network
|
||||
sockets absent for those scenarios, and the verifier reports no remaining
|
||||
descendants. The scope took 177.456 seconds, sampled peak 1,237,020,672 bytes and
|
||||
last-observed CPU 144.913 seconds. Different totals from the earlier control are
|
||||
not attributed to unchanged provider algorithms.
|
||||
|
||||
Post-descriptor public MCP regression uses the original pinned shipped corpus:
|
||||
|
||||
| Domain / selected question | Actual provider / scope | Calls | Seconds |
|
||||
| ------------------------------------------- | -------------------------------------------------- | ----: | ------: |
|
||||
| SQLite WASM counts / exports | WABT 1.0.42; selected summary and retained read | 2 | 5.516 |
|
||||
| ShareX managed metadata / native boundaries | Static PE/CLI, no assembly execution | 2 | 3.710 |
|
||||
| Obsidian manifest / inventory | Complete inventory retained, no AST claim | 4 | 5.376 |
|
||||
| NewPipe package / MainActivity methods | Real headless JADX 0.7.1 | 3 | 21.669 |
|
||||
| OpenWrt regions | Real firmware signature scan, no device execution | 1 | 3.581 |
|
||||
| jq native entry | Ghidra 12.1.4, default analysis, ephemeral Program | 4 | 270.986 |
|
||||
|
||||
All six workflows pass named SDK output validation and original digest checks;
|
||||
input bytes remain unchanged. Maximum sampled family peak is 1,284,648,960 bytes.
|
||||
They establish those available Linux workflows, not paired performance or
|
||||
Windows/macOS, IDA/Hopper, Android-device, DOS COM, browser or JavaScript AST
|
||||
behavior. Browser/AST were not repeated for this narrow migration.
|
||||
|
||||
## Real model complete-output calibration
|
||||
|
||||
Six serial Codex CLI 0.161.0 trials request `gpt-5.6-terra`, high reasoning, with
|
||||
isolated temporary profiles pointing to the frozen candidate. Resolved server-side
|
||||
model identity is not inferred. Only the EVM inspection tool is enabled, matching
|
||||
the earlier WETH calibration; this is not full-catalog tool selection verification.
|
||||
Every trial makes one successful call and no failed call.
|
||||
|
||||
| Trial | Total input | Uncached input | Output | Seconds |
|
||||
| --------- | ----------: | -------------: | -----: | ------: |
|
||||
| Factory 1 | 66,158 | 36,974 | 1,329 | 43.483 |
|
||||
| Pool 1 | 65,600 | 36,416 | 2,307 | 69.546 |
|
||||
| WETH 1 | 54,700 | 25,516 | 1,144 | 56.372 |
|
||||
| Factory 2 | 65,964 | 36,780 | 1,453 | 26.412 |
|
||||
| Pool 2 | 65,753 | 36,569 | 1,939 | 74.706 |
|
||||
| WETH 2 | 54,502 | 25,318 | 1,208 | 35.137 |
|
||||
|
||||
Every trial reports 29,184 cached input tokens. Selected fact/reference/lifecycle
|
||||
grades all pass: original/decoded identities, every inferred selector, offset,
|
||||
dispatch, argument/mutability string including null, explicit unknowns, no runtime
|
||||
execution, observed Evidence references, unchanged bytes and removed profiles.
|
||||
Raw transcripts and grades remain private. Sampled peaks are
|
||||
594,853,888–776,335,360 bytes; last-observed CPU is 6.181–7.034 seconds. Strict
|
||||
overall grades remain unverified because final resource counters are unknown;
|
||||
they are not relabeled as wholly passing resource-qualified runs.
|
||||
|
||||
This calibrates the current complete path after compatible wiring, not a paired
|
||||
model performance experiment. The next token experiment should answer the same
|
||||
interface question through an explicitly selected precise view, retain full
|
||||
bytecode/original Evidence under the existing lifetime, and preserve complete
|
||||
defaults. Discover support before sending new options to an older server. Include
|
||||
discovery, follow-up, failures and retries in the complete task cost.
|
||||
|
||||
## Static contract admission verification
|
||||
|
||||
The follow-up from main `c462a4ebd3a10e59253bcf38353a0914b176738b` tests the
|
||||
pilot's actual static boundary, without adding a dynamic module registry or
|
||||
changing production code. Complete-project TypeScript checking now verifies
|
||||
that the inferred descriptor type rejects a WASM operation contract, a WASM
|
||||
CLI command, a WASM service factory, an unrelated input schema and an unrelated
|
||||
output schema. These are negative assignability assertions: their guarantee
|
||||
comes from the compiler, not from executing an incompatible descriptor.
|
||||
The existing named contract and CLI identity are also checked at runtime;
|
||||
separate factory calls produce distinct service/provider owners. Invalid target
|
||||
input remains an application error without launching a worker.
|
||||
|
||||
The actual public in-memory MCP SDK test additionally spies on the provider:
|
||||
duplicate EVM registration throws the named `already registered` error before
|
||||
any provider call. Converted input/output JSON Schemas validate before the
|
||||
first valid operation. Invalid client arguments are rejected before execution;
|
||||
incompatible provider reply data is validated **after** execution and is not
|
||||
presented as a pre-use contract guarantee. Existing healthy-peer, cancellation,
|
||||
retained summary/default and cleanup-retry scenarios remain covered.
|
||||
|
||||
Full project typecheck passes with the installed compiler's supported single
|
||||
thread scheduling; it includes all source/test files. The two exact focused
|
||||
files pass 12 runtime cases. The resource scopes sample 2,057,961,472 bytes for
|
||||
typecheck and 2,426,290,176 bytes for required build plus focused tests, below the
|
||||
unchanged 3 GiB early-stop/4 GiB hard ceiling. Final transient-cgroup counters
|
||||
remain unknown. No new real-provider or performance improvement is claimed by
|
||||
these compile/SDK checks; the earlier pinned real EVMole/WABT/lifecycle evidence
|
||||
retains its original scope and source identity.
|
||||
|
||||
This proves the one statically composed descriptor's named/schema/factory
|
||||
compatibility and duplicate-registration guard. It does not implement runtime
|
||||
admission of arbitrary JavaScript descriptors, third-party plugin ABI/version
|
||||
negotiation or a general plugin SDK. Those remain deferred by the original plan.
|
||||
Existing callers, full defaults, provider inputs, owned lifetimes and output
|
||||
identities are unchanged.
|
||||
@@ -0,0 +1,208 @@
|
||||
# EVM interface delivery and migration checkpoint
|
||||
|
||||
Continues [#2016](https://github.com/morluto/rea/issues/2016) after the static
|
||||
descriptor merged in [#2261](https://github.com/morluto/rea/pull/2261). Opt-in
|
||||
`inspect_evm_interface` MCP `detail: "summary"` returns every inferred interface
|
||||
candidate without bytecode hex or duplicate stdout. Complete Evidence remains
|
||||
retained under its original identity. Delivery bytes fall; the measured model
|
||||
cohort does **not** establish a stable token or task-time improvement.
|
||||
|
||||
## Frozen scope
|
||||
|
||||
Control production is `ad08510a113d7a9b8d92fb1e385757880e469869`; candidate
|
||||
production is `32c7548b9ae5c8cda348653d177b1747127c74ba`. Their compiled file-table
|
||||
SHA-256 values are respectively
|
||||
`f68e85030cb05082520ea9f0d7650298ae316ce7bba59a8942e0cc1318ae9850` and
|
||||
`978b3a014e3b2a1e1c6620364dcd793fa7cb40f19892d44cfb0312739a8c14a3`.
|
||||
The SDK driver SHA-256 is
|
||||
`a95d8d8e9f72c0b2b885c61907ed13054ebf7d1642f6f13b0126163840b1221a`.
|
||||
Driver commit `03bff83e70e09a917341c1fe3ebffbfdc67785c5` fixes only reading the
|
||||
public `get_evidence_bundle.result.records` envelope; production is unchanged.
|
||||
Both runtimes use Node 24.18.0, matching locked dependencies, SDK client/server
|
||||
2.3.1 and unchanged bundled EVMole 0.9.3.
|
||||
|
||||
Inputs are the digest-pinned WETH historical hex carrier and Uniswap v3-core
|
||||
1.0.1 Factory/Pool compiler runtime templates in [corpus.json](corpus.json).
|
||||
Original carrier and decoded-byte identities remain distinct. Templates are
|
||||
not authenticated deployed instances; constructor-dependent values and absent
|
||||
immutable-reference metadata stay unknown. No chain, target execution or
|
||||
signature lookup is introduced.
|
||||
|
||||
## Public SDK pairs
|
||||
|
||||
Two pairs per artifact run serially: complete control then selected candidate,
|
||||
followed by candidate then control. Each starts a fresh MCP process, discovers
|
||||
the catalog and validates the actual named advertised schema. Host caches are
|
||||
not independently controlled. All six fact/settings comparisons pass. Both
|
||||
answer projections omit unrequested hex; original complete replies remain intact.
|
||||
|
||||
| Input | Complete / summary result bytes | Reduction | Control seconds, 1 / 2 | Candidate seconds, 1 / 2 |
|
||||
| ------- | ------------------------------: | --------: | ---------------------: | -----------------------: |
|
||||
| Factory | 110,424 / 14,442 | 86.92% | 3.670 / 3.766 | 3.664 / 3.867 |
|
||||
| Pool | 110,998 / 19,922 | 82.05% | 4.110 / 3.740 | 3.977 / 3.870 |
|
||||
| WETH | 26,760 / 15,378 | 42.53% | 3.671 / 3.710 | 3.930 / 3.673 |
|
||||
|
||||
These measurements support smaller delivery, not faster interpretation. Analysis
|
||||
still runs and the full parent still incurs producer/retention costs. Catalog
|
||||
count and order stay at 184 tools. The complete catalog grows from 2,422,229 to
|
||||
2,429,938 compact JSON bytes (+7,709); all name/description/input definitions
|
||||
grow from 512,655 to 514,074 bytes. The EVM input definition grows from 1,443 to
|
||||
2,029 bytes. Only `inspect_analysis_view` and `inspect_evm_interface` definitions
|
||||
change. These discovery costs are visible and are not converted into token counts.
|
||||
|
||||
Nine separate measurements verify new-server complete defaults, old-server
|
||||
summary fallback and retained reads for all three inputs. The four-call retained
|
||||
proofs retrieve the complete canonical parent plus pages and cost 160,110,
|
||||
177,106 and 79,160 bytes for Factory, Pool and WETH respectively. They are
|
||||
verification workflows, not examples of one-call delivery savings.
|
||||
|
||||
## Real Codex pairs and answer qualification
|
||||
|
||||
Twelve serial Codex CLI 0.161.0 trials use the existing identical questions for
|
||||
each pair, the same artifact path, one enabled EVM tool, temporary MCP profiles
|
||||
and requested `gpt-5.6-terra` high. The requested alias is not proof of the
|
||||
resolved model identity. Trials include schema discovery, reasoning, one actual
|
||||
successful EVM call and final answer. No call fails or retries. Profiles are
|
||||
removed and input bytes remain unchanged. This fixed one-tool profile does not
|
||||
test selection from the full catalog.
|
||||
|
||||
| Input / repetition | Uncached input, control → candidate | Seconds, control → candidate | Candidate delivery |
|
||||
| ------------------ | ----------------------------------: | ---------------------------: | ------------------ |
|
||||
| Factory / 1 | 22,901 → 25,668 | 52.485 → 48.768 | summary |
|
||||
| Factory / 2 | 37,181 → 15,270 | 53.404 → 45.345 | summary |
|
||||
| Pool / 1 | 36,652 → 26,657 | 57.764 → 66.642 | summary |
|
||||
| Pool / 2 | 28,305 → 36,259 | 62.068 → 71.756 | complete |
|
||||
| WETH / 1 | 21,128 → 25,187 | 44.869 → 46.809 | summary |
|
||||
| WETH / 2 | 17,417 → 17,529 | 30.364 → 39.943 | summary |
|
||||
|
||||
Keep all trials, including the candidate that explicitly chose complete delivery.
|
||||
Five of six candidates select summary, but uncached input improves in only two
|
||||
pairs and time improves in only two. Aggregate control → candidate observations
|
||||
are 362,496 → 357,770 total input; 198,912 → 211,200 cached input;
|
||||
163,584 → 146,570 uncached input; 8,953 → 10,397 output; and
|
||||
300.954 → 319.263 seconds. Cache state and model/service variation are not
|
||||
separated; the aggregate does not establish a repeatable saving for an individual
|
||||
question or a qualified performance recommendation.
|
||||
|
||||
Every answer passes exact core-field checks: both identities/byte counts, every
|
||||
function selector/offset/dispatch/argument/mutability including null, inference,
|
||||
unknown completeness/kind/hardfork/authenticity, and absent runtime execution.
|
||||
All reported IDs occur in actual responses. However, the existing grader requires
|
||||
the final limitation list to equal the producer list verbatim and in order.
|
||||
Only three trials pass that additional check; nine fail. The questions request
|
||||
preserving limitations but do not explicitly request a verbatim ordered list.
|
||||
Four failures contain all original strings with extra correct scope statements
|
||||
or different ordering. The other five paraphrase or omit source caveats, including
|
||||
worker or upstream-metadata limitations. Do not silently waive those omissions
|
||||
or turn these into twelve fully acceptable answers. The strict reports and
|
||||
original transcripts remain intact; core-field success is a narrower result.
|
||||
|
||||
Model scope sampled peaks range from 562.5 to 710.5 MiB, with last-observed CPU
|
||||
5.945–6.878 seconds. CPU includes the local process family, not remote model
|
||||
computation. All heavy jobs use one CPU, a 4 GiB hard limit, 3 GiB early-stop,
|
||||
no swap and 192 tasks. No scope abort occurs in this cohort. Final cgroup
|
||||
counters are unavailable after teardown: final OOM/task/limit-event assertions
|
||||
remain **unverified**. Last-observed counters are not substituted for final ones.
|
||||
Consequently neither strict answer acceptance nor resource-qualified performance
|
||||
success is claimed for the whole cohort.
|
||||
|
||||
## Existing-user compatibility
|
||||
|
||||
Omitted and explicit-complete MCP requests, the existing CLI and provider inputs
|
||||
preserve their full behavior. `detail` is removed before the unchanged provider
|
||||
input; original full Evidence parameters, raw result and identity are retained.
|
||||
Summary receives its own identity linked to that parent. All candidate facts,
|
||||
original report limitations, stderr/truncation and explicit unknowns stay inline.
|
||||
Complete-within-view candidate coverage is not global discovery completeness.
|
||||
Function pages retain original order; text pages use UTF-16 code units, with
|
||||
hex characters distinguished from decoded-byte function offsets.
|
||||
|
||||
Actual new default replies for all three artifacts validate against the frozen
|
||||
old advertised input/output schemas, and structured/text payloads agree. Older
|
||||
servers reject neither request nor unsupported fields because the experimental
|
||||
client discovers support before selecting summary and otherwise calls complete.
|
||||
This is an explicit client fallback, not universal server forward compatibility.
|
||||
|
||||
Three authentic old-runtime records and old-created canonical bundles pass new
|
||||
MCP import, retained summary, CLI inline summary and export. Original records,
|
||||
IDs and bundle source bytes stay deeply equal. The reproducible verifier below
|
||||
also checks function/hex pages with the analysis limiter explicitly unavailable,
|
||||
so retained reads cannot silently start successful producer analysis. This
|
||||
proves the pinned old/new Linux workflows, not every historical client release.
|
||||
|
||||
The initial retained experiment failed because its driver read the bundle's
|
||||
wrong envelope. Correcting that consumer produced passing retained proofs; the
|
||||
failed receipt stays archived and is not recorded as a REA defect. Separately,
|
||||
one static-check attempt hit the unchanged 3 GiB early-stop. The bounded retry
|
||||
passes with Node heap 512 MiB/native checker memory controls; its roughly 3 GiB
|
||||
peak and soft-limit events are excluded from efficiency comparisons.
|
||||
|
||||
## Reproduction and checks
|
||||
|
||||
Use [the shared instructions](README.md), explicit frozen revisions/runtimes and
|
||||
fresh private output directories. Run `measure.mjs` for `weth.hex`,
|
||||
`uniswap-v3.factory.hex` and `uniswap-v3.pool.hex`, with `--schemas selected` and
|
||||
paired `--delivery complete` / `summary`; run `compare.mjs` on equivalent pairs.
|
||||
Add `--verify-retained` only for the separate four-call proof. Use the unchanged
|
||||
[WETH](questions/evm.txt) or [template](questions/evm-uniswap.txt) question and the
|
||||
same one-tool profile for `agent-trial.py`, preserving all usage and grader results.
|
||||
|
||||
Export complete Evidence with the old runtime's public ledger workflow, then run:
|
||||
|
||||
```sh
|
||||
node experiments/analysis-efficiency/verify-evm-retention.mjs \
|
||||
--runtime-root ABS_FROZEN_NEW_RUNTIME --revision NEW_SHA \
|
||||
--bundle ABS_OLD_BUNDLE_JSON --output NEW_PRIVATE_ABS_DIR
|
||||
```
|
||||
|
||||
The script acquires no dependencies and preserves the old bundle. Revision is
|
||||
caller-supplied; retain the frozen file-table receipt alongside it. Raw JSON,
|
||||
transcripts, profiles, bytecode, compiler inputs and generated artifacts stay
|
||||
outside Git. This checkpoint adds source verification and curated observations.
|
||||
|
||||
The production change passes 64 focused EVM/WASM/generic-view/domain/service
|
||||
checks and full static/docs gates. Recording provider seams establish exact
|
||||
schemas, old defaults, inference/null preservation, historical identity failures,
|
||||
peer availability, cancellation and cleanup retry; they are not real-provider
|
||||
claims. Real-provider and shared-domain checks are recorded separately.
|
||||
|
||||
The same frozen candidate also passes six actual public MCP artifact workflows:
|
||||
SQLite WASM with WABT summary, ShareX managed metadata, Obsidian inventory,
|
||||
NewPipe with JADX, an OpenWrt region inspection, and jq entry analysis with
|
||||
Ghidra. These are shared-contract regressions; they do not establish AST analysis
|
||||
of all Obsidian code, full firmware reconstruction, browser coverage, or new token
|
||||
savings in those domains. Native Windows PE and DOS coverage remains at the
|
||||
[earlier native checkpoint](native-entry-views.md), not a new Ghidra/Windows claim.
|
||||
The source retention verifier passes for all three authentic old EVM bundles,
|
||||
including exact summaries, function/hex pages, CLI inline views and unchanged
|
||||
exported originals with the limiter explicitly unavailable.
|
||||
|
||||
The frozen candidate passes all 59 existing real EVM CLI/MCP verifier cases with
|
||||
caller-supplied solc 0.8.30 and strace 6.1, including malformed/unsupported
|
||||
carriers, unavailable limiter and capacity failures. Its scope takes 96.076
|
||||
seconds with 1,345,052,672 sampled peak bytes and 95.130 last-observed CPU seconds.
|
||||
This is regression evidence, not an interpreter speedup comparison; final
|
||||
resource counters remain unverified. Fresh fast static and generated-document
|
||||
checks also pass after adding the migration verifier and this report.
|
||||
|
||||
## Latest-main integration
|
||||
|
||||
The branch rebases without conflict onto main
|
||||
`0043d9b134c0a353e8bf85c0bde4492275f1f720`. Production integration commit
|
||||
`3766749316c345ccb594b1b748747b1af9582b34` preserves the same EVM projection,
|
||||
provider and delivery semantics; intervening main adds unrelated capabilities.
|
||||
The above performance cohort stays pinned to its original frozen runtimes and
|
||||
is not relabeled as a measurement of the larger integrated catalog.
|
||||
|
||||
At `cda794fb9d4a5ca2bd661853a899d579fb8a47b2`, all 30 affected EVM/WASM/generic-view
|
||||
MCP tests pass again, as do full `npm run check` and `npm run docs:check`. A fresh
|
||||
real Factory SDK selected-summary/retained-parent/page workflow and the original
|
||||
Pool bundle's SDK/CLI migration verifier also pass against the rebuilt integrated
|
||||
runtime. The latter still has an explicitly unavailable analysis limiter.
|
||||
|
||||
The initial integrated static-check scope stops at 3,221,446,656 sampled bytes,
|
||||
just above the unchanged 3 GiB early-stop. Running the same typecheck through
|
||||
its single Turbo task with less launcher overhead passes at 3,219,353,600 sampled
|
||||
bytes; the subsequent full gate uses that valid cached typecheck and passes.
|
||||
No memory threshold or check requirement is relaxed. Preserve both receipts;
|
||||
these near-limit validation jobs are excluded from efficiency conclusions.
|
||||
@@ -0,0 +1,159 @@
|
||||
# EVM descriptor lifecycle acceptance
|
||||
|
||||
Continues [#2016](https://github.com/morluto/rea/issues/2016) after
|
||||
[#2349](https://github.com/morluto/rea/pull/2349) merged. The
|
||||
[public verifier](verify-evm-lifecycle.mjs) exercises actual compiled CLI and
|
||||
stdio MCP workflows with unchanged EVMole 0.9.3/WASM. This increment adds
|
||||
verification only; production contracts, factories, algorithms and defaults
|
||||
remain unchanged. It does not introduce a plugin SDK or database cache.
|
||||
|
||||
## Runtime and public artifacts
|
||||
|
||||
The verified runtime is explicitly frozen from
|
||||
`67f817e3653efad35f7044e68fbd6f163b86864b`, built with Node 24.18.0 and the
|
||||
matching locked dependencies. Its `dist/` and `bridge/` file-table SHA-256 is
|
||||
`cf03167870ac9def929cfaab536d6bd791089031660d1b35bff20672ac07cbbf`.
|
||||
The table is checked again after each workflow; source and target bytes must
|
||||
remain unchanged before a success receipt is published.
|
||||
|
||||
Inputs are the digest-pinned [corpus](corpus.json): published historical WETH
|
||||
and official Uniswap v3-core 1.0.1 Factory/Pool compiler runtime templates.
|
||||
They are not independently authenticated live deployments. Their deployment,
|
||||
hardfork, immutable-reference and inferred-interface limitations remain as
|
||||
recorded in [the descriptor checkpoint](evm-descriptor.md). The healthy peer
|
||||
is the shipped SQLite WASM 3.53.4-build1 module, inspected by real WABT 1.0.42.
|
||||
No contract, native target, chain/RPC query or transaction is executed.
|
||||
|
||||
## Actual cancellation and retained cleanup
|
||||
|
||||
The verifier supplies a private test launcher through the existing explicit
|
||||
`REA_EVM_PRLIMIT_COMMAND` setting. It passes the original arguments to real
|
||||
`/usr/bin/prlimit`, runs the unchanged Node/EVMole worker, then briefly gates
|
||||
launcher completion after that worker exits and writes its complete reply.
|
||||
This controls a real lifecycle boundary without replacing the analysis engine.
|
||||
It adds process overhead and is not a performance comparison.
|
||||
|
||||
Before cancellation or permission changes, production inspection primitives
|
||||
verify the live launcher's run token, parent/group relationship and native
|
||||
start identity. The complete worker reply must match the original selected
|
||||
bytecode and producer facts. Filesystem changes apply only to the verified,
|
||||
non-root user's temporary runtime directory.
|
||||
|
||||
Three scenarios are verified:
|
||||
|
||||
- **Late cancellation:** the real worker has completed, while the public SDK
|
||||
request remains pending. SDK cancellation reports `REQUEST_TIMEOUT` locally;
|
||||
token sweeps and directory removal independently confirm provider cleanup.
|
||||
A subsequent complete request returns identical analysis facts.
|
||||
- **Successful analysis followed by cleanup failure:** removing write permission
|
||||
produces actual `EACCES`. MCP reports `cleanup_incomplete` and retains the
|
||||
original resource path. A retry before permission repair fails on that same
|
||||
resource. Real SQLite WASM analysis still succeeds. Restoring permission lets
|
||||
the provider retire the old resource and perform a fresh complete EVM query.
|
||||
- **Format failure followed by cleanup failure:** a deliberately malformed
|
||||
odd-nibble carrier reaches the unchanged worker. The cleanup error preserves
|
||||
its original `invalid_input` category and exact producer diagnostic rather
|
||||
than overwriting the failure. The same repair/retry and healthy-peer checks
|
||||
succeed.
|
||||
|
||||
The existing CLI `inspect-evm-interface PATH hex --json` produces the same
|
||||
complete normalized facts, parameters and subject as MCP. Fresh Evidence IDs
|
||||
are not equated. Final verifier lineage is checked after client/transport close;
|
||||
an empty descendant list is a point-in-time observation, not proof that the
|
||||
workflow never launched children. Cleanup callbacks always run and preserve
|
||||
verification and cleanup failures together.
|
||||
|
||||
This proves cancellation after actual analysis, not interruption inside
|
||||
EVMole's synchronous WASM computation. Server-shutdown cleanup retry still has
|
||||
the separately labeled injected-owner SDK tests from the descriptor checkpoint.
|
||||
These Linux x64 observations do not establish Windows/macOS provider behavior.
|
||||
Earlier real CLI/MCP malformed/unavailable/capacity checks remain linked in the
|
||||
descriptor checkpoint; they are not silently rerun or attributed to this script.
|
||||
|
||||
## Measurements and compatibility scope
|
||||
|
||||
Initial three-carrier verifier source SHA-256 is
|
||||
`2fec069b53324ef82070d323b26dd613b6da0caa7f18100adbeb39b045cf98dc`.
|
||||
All three carriers pass all three scenarios and CLI parity with this same source.
|
||||
|
||||
| Carrier | Seconds | Sampled peak family bytes | Last-observed CPU seconds |
|
||||
| ------- | ------: | ------------------------: | ------------------------: |
|
||||
| WETH | 25.595 | 1,116,884,992 | 23.926 |
|
||||
| Factory | 23.979 | 1,132,994,560 | 23.419 |
|
||||
| Pool | 24.589 | 1,141,415,936 | 24.043 |
|
||||
|
||||
Runs are serial under the existing one-CPU quota, 4 GiB hard memory, unchanged
|
||||
3 GiB early-stop, no swap and 192-task ceiling. Resource receipts, complete
|
||||
producer replies, derived malformed input, launcher and runtime copies stay in
|
||||
the private lab outside Git. Final cgroup counters disappear at scope teardown
|
||||
and remain unknown; sampled zero events do not prove final no-OOM/no-limit
|
||||
results. This is compatibility and recovery evidence, not a model-token,
|
||||
whole-task latency or CPU saving claim.
|
||||
|
||||
Latest main `81cca2e0e3ebd98c21f41d7ad63d6a00e4cbc3f9` could not build during
|
||||
setup: its shared partial-error handler passes three arguments to an interface
|
||||
requiring four. Independently completed main CI at
|
||||
`fdd8f826023c568937d237c2f97d855de283a9b8` reports the same TypeScript failure
|
||||
([run 38309578208](https://github.com/morluto/rea/actions/runs/38309578208)).
|
||||
The unrelated regression is recorded in the maintainer-requested private bug
|
||||
handoff; this change does not repair it. The maintainer independently corrected
|
||||
the producing caller in main `8fc2043eb8bcedbf34dce064f977e455438635d6`.
|
||||
The frozen-runtime observations above remain pinned to their original revision;
|
||||
synchronizing the branch does not relabel them as latest-main verification.
|
||||
|
||||
After rebasing, a fresh build and all 10 focused EVM MCP boundary tests pass.
|
||||
The same unchanged source verifier also passes all three real Factory scenarios,
|
||||
CLI parity, healthy SQLite WASM checks and final cleanup on the newly built
|
||||
runtime `b3eadf147221a2fa997ac26667a183ca5bccb1f4` (parent is corrected main
|
||||
`8fc2043eb8bcedbf34dce064f977e455438635d6`). Its `dist/`/`bridge/` file-table
|
||||
SHA-256 is `7a4cec0fa4f7e929356b50a6b13472f80bd6f526221e427125370cca3ef98d00`.
|
||||
This related rerun takes 24.285 seconds, samples 1,146,167,296 peak family bytes
|
||||
and records 23.662 last-observed CPU seconds. The older failed build and earlier
|
||||
cohort receipts are preserved; wider domain/model matrices are not repeated.
|
||||
|
||||
Static composition continues to bind one named canonical contract and one
|
||||
typed factory. Existing SDK checks reject duplicate registration before use
|
||||
and reject malformed inputs before invoking the provider. Producer contract
|
||||
violations are detected when the producer returns a result; they cannot be
|
||||
validated in advance without that result. A dynamic third-party module ABI
|
||||
preflight is not implemented or claimed. The corresponding broader acceptance
|
||||
item remains open rather than creating the deferred generic plugin registry.
|
||||
|
||||
## Final launcher review
|
||||
|
||||
A missing gate marker while gating is disabled cannot establish absence of a
|
||||
new launch. The fixture now appends a launch record before starting the actual
|
||||
worker; both permission-failure retries must leave that log byte-identical.
|
||||
This independently verifies that retained cleanup failure stops a new EVM
|
||||
launch, while preserving the original resource and healthy WASM peer.
|
||||
|
||||
Final verifier source SHA-256 is
|
||||
`fc9a50e2e54e4c8eede4b8a19283e11ddaa1faadea52eebc390b55b5a1902668`.
|
||||
All three related real Factory scenarios, CLI parity and final cleanup pass
|
||||
again on the corrected-main runtime: 23.579 seconds, 1,163,685,888 peak sampled
|
||||
family bytes and 23.110 last-observed CPU seconds. The earlier three-carrier
|
||||
receipts remain pinned to their earlier source; wider artifacts/models are
|
||||
not repeated for this review fix.
|
||||
|
||||
## Reproduction
|
||||
|
||||
Build and freeze the explicit source revision as documented in the measurement
|
||||
README. Supply an object of explicit string-valued provider settings, including
|
||||
the caller-installed WABT paths. As a non-root Linux x64 user, run under the
|
||||
existing resource wrapper:
|
||||
|
||||
```bash
|
||||
python3 experiments/analysis-efficiency/resource-run.py -- \
|
||||
node experiments/analysis-efficiency/verify-evm-lifecycle.mjs \
|
||||
--runtime-root /absolute/frozen-runtime \
|
||||
--revision 67f817e3653efad35f7044e68fbd6f163b86864b \
|
||||
--path /absolute/UniswapV3Factory.runtime.hex \
|
||||
--peer-path /absolute/sqlite3.wasm \
|
||||
--providers /absolute/explicit-providers.json \
|
||||
--output /absolute/new-private-receipt-directory
|
||||
```
|
||||
|
||||
The verifier checks the carrier and SQLite module against the committed corpus,
|
||||
creates an owner-only receipt directory without overwriting existing receipts,
|
||||
and acquires no dependencies. Preserve failed attempts separately. A shorter
|
||||
reply or a successful cleanup verifier does not establish an agent-cost gain.
|
||||
@@ -0,0 +1,160 @@
|
||||
# Dart AOT summary and retained string pools
|
||||
|
||||
Continues [#2016](https://github.com/morluto/rea/issues/2016) and
|
||||
[#2455](https://github.com/morluto/rea/pull/2455). The selected analyst question
|
||||
asks for shipped LocalSend target/ABI/library identity, every nullable snapshot
|
||||
observation, string-pool lower bounds and returned counts, and original coverage
|
||||
and limitations. It does not require individual observed strings.
|
||||
|
||||
## Compatibility and ownership
|
||||
|
||||
MCP `inspect_dart_aot` accepts explicit `detail: "summary"`. Omitted and complete
|
||||
requests preserve the complete result and original provider inputs. The CLI
|
||||
remains complete. ABI omission and explicit ABI selection retain their original
|
||||
meaning; the provider never receives the delivery field. Missing retention,
|
||||
invalid detail and pre-cancellation fail before provider work.
|
||||
|
||||
The existing application workflow retains the complete original Evidence, then
|
||||
returns a distinct derived summary linked to its parent ID. It preserves target
|
||||
and libapp digests, every snapshot section and nullable fact, original pool and
|
||||
overall coverage, lower-bound counts and limitations. `returned_counts` reports
|
||||
the lengths of the producer-returned arrays; it does not upgrade lower bounds
|
||||
to a complete snapshot inventory. The analysis and original source retention
|
||||
are unchanged. No new engine, persistent cache or plugin SDK is introduced.
|
||||
|
||||
Read exact retained strings using `inspect_analysis_view`, the parent ID, and
|
||||
`view: {kind: "dart-aot", collection: "package_uris", offset: 0, limit: 64}`.
|
||||
Other collections are `dart_uris` and `class_like_tokens`. Follow the returned
|
||||
next offset. Pages preserve source order, ABI/library identity and partial pool
|
||||
coverage without opening the APK again. Page exhaustion describes retained
|
||||
strings, not the entire snapshot. Strings remain byte-pattern observations,
|
||||
rather than recovered typed class/function semantics. Missing historical
|
||||
subjects remain explicitly unknown; conflicting recorded target digests fail.
|
||||
|
||||
## Frozen runtime and real SDK pairs
|
||||
|
||||
The official LocalSend 1.18.2 arm64 APK acquisition, size and SHA-256 are pinned
|
||||
in `corpus.json`. No APK installation, target execution or version-dataset
|
||||
lookup occurred. Source runtime: `811034a7283c9751a3ecbff33e8b2f28bec55793`;
|
||||
compiled dist/bridge file-table SHA-256:
|
||||
`dabbb63536c5862be839ecdc0f3eaad05b29c2a3816f57041528d30f754311be`.
|
||||
The previous runtime is `28c84e78cf2fbfde4aceed345b773c59eb889f68`, digest
|
||||
`5332dda13981f043e146b3601ebec1f380d9f698280ca53d75786b638a912e11`.
|
||||
Its production sources were verified byte-equal to main f254 at acquisition.
|
||||
Subsequent test-only changes and main's SAZ addition do not relabel these pins.
|
||||
|
||||
Node is 24.18.0 and the SDK client/server are 2.3.1. Each workflow starts a fresh
|
||||
server and validates the selected live advertised schema. Artifact bytes,
|
||||
provider environment/search-path fingerprints, driver and locked dependencies
|
||||
stay equal. Host page caches are not flushed. Heavy tasks run serially under
|
||||
the unchanged CPU, memory, task and host guards.
|
||||
|
||||
| Trial | AOT reply bytes | SDK workflow seconds | Payload CPU seconds | Kernel peak bytes |
|
||||
| -------------- | --------------: | -------------------: | ------------------: | ----------------: |
|
||||
| sdk-complete-1 | 151,341 | 5.139 | 6.819 | 691,441,664 |
|
||||
| sdk-summary-1 | 12,037 | 5.219 | 6.608 | 639,430,656 |
|
||||
| sdk-summary-2 | 12,037 | 5.064 | 6.405 | 640,868,352 |
|
||||
| sdk-complete-2 | 151,341 | 5.128 | 6.468 | 633,614,336 |
|
||||
|
||||
Both pairs use one successful public tool call and pass settings/fact comparison,
|
||||
including all snapshot fields, coverage, original limitations and all counts.
|
||||
Reply bytes fall 92.05%. Whole SDK time stays approximately 5.1 seconds; these
|
||||
trials do not establish engine speed, CPU or memory savings. Payload CPU includes
|
||||
driver/catalog work beyond the SDK workflow timer. Catalog JSON grows from
|
||||
2,506,213 bytes on the previous runtime to 2,514,896 (about 0.35%).
|
||||
|
||||
A fresh old-runtime complete request passes comparison against candidate
|
||||
complete, including identical 151,341-byte replies. This is scoped compatibility
|
||||
evidence, not a guarantee for every historical release or third-party client.
|
||||
|
||||
## Real old Evidence continuation
|
||||
|
||||
Fifteen successful public MCP calls import genuine old-runtime AOT Evidence,
|
||||
project retained summary, reconstruct all 500 package URIs, 216 Dart URIs and
|
||||
2,000 class-like tokens, compare inline portable summary, and export the original
|
||||
record. The APK is absent during all these operations. Original string order,
|
||||
lower bounds, partial coverage, nullable snapshot facts, subject and parent ID
|
||||
remain intact; the original exported record is deeply equal to its source.
|
||||
|
||||
The first private verification attempt used a nonexistent receipt filename and
|
||||
failed before import or artifact relocation. That driver error is retained;
|
||||
the corrected attempt selects the fresh old-runtime record. It is not a REA
|
||||
defect or a successful migration trial.
|
||||
|
||||
## Actual Codex task outcomes
|
||||
|
||||
Four serial trials use the same frozen candidate in complete/summary/summary/
|
||||
complete order. Requested configuration is Codex CLI 0.161.0, `gpt-5.6-terra`
|
||||
high, one exact enabled tool (`inspect_dart_aot`), and identical optional owned
|
||||
context capture/discovery guidance. This requested alias does not certify the
|
||||
server-resolved model. Temporary configuration points to the modified frozen
|
||||
REA MCP; all copied credential profiles are removed and target bytes remain
|
||||
unchanged. The source question is fixed; equal treatment instructions choose
|
||||
detail and explain the summary carrier and parent reference.
|
||||
|
||||
| Trial | Input | Cached | Uncached | Output | Seconds | MCP attempts | Failed attempts | Facts |
|
||||
| ---------------- | -----: | -----: | -------: | -----: | ------: | -----------: | --------------: | ----- |
|
||||
| model-complete-1 | 74,641 | 50,176 | 24,465 | 2,765 | 116.898 | 3 | 1 | pass |
|
||||
| model-summary-1 | 38,253 | 22,016 | 16,237 | 1,449 | 88.904 | 1 | 0 | pass |
|
||||
| model-summary-2 | 38,363 | 22,016 | 16,347 | 1,515 | 55.190 | 1 | 0 | pass |
|
||||
| model-complete-2 | 66,585 | 38,144 | 28,441 | 2,217 | 79.180 | 2 | 0 | pass |
|
||||
|
||||
Both pairs pass checks for every selected producer fact, exact nullable
|
||||
observations, target digest, observed response Evidence IDs, original limitations,
|
||||
artifact preservation and profile cleanup. The grader compares the reported
|
||||
facts with producer observations; it is not an independent Dart deserializer.
|
||||
Total input falls 48.75% / 42.38%, uncached input 33.63% / 42.52%, reported output
|
||||
47.59% / 31.66%, and whole-task time 23.95% / 30.30%. Usage is the final aggregate
|
||||
reported by the actual CLI, one usage record per trial; cached tokens are not
|
||||
subtracted from total input. Reported output includes reasoning output (1,262,
|
||||
321, 383 and 709 tokens respectively).
|
||||
|
||||
The first complete trial initially supplied `artifact_path` instead of the
|
||||
advertised `path`, corrected it, then repeated a successful complete call.
|
||||
The second complete trial made two successful calls. Both summary trials made
|
||||
one successful call. All attempts, discovery and correction costs remain
|
||||
included. The paired whole-task improvement therefore includes actual model
|
||||
decisions and the convenience of explicit returned counts; it does not isolate
|
||||
serialization as the sole cause or establish engine acceleration. Two pairs on
|
||||
one app/question/model configuration support this scope, not a universal gain.
|
||||
|
||||
Payload CPU seconds / kernel peak bytes are respectively 12.556 / 876,937,216;
|
||||
9.213 / 785,272,832; 9.108 / 793,939,968; and 13.164 / 803,209,216. Model payload
|
||||
observations verify owned cleanup and zero phase limit events, with no guard
|
||||
aborts. Whole-scope final counters remain unknown, so every strict all-selected
|
||||
resource grade remains false even though the separate factual grade passes.
|
||||
|
||||
## Resource and verification scope
|
||||
|
||||
SDK and migration payload-exit observations verify owned cleanup and zero
|
||||
observed memory/pids limit events, with peaks below 692 MB. Missing whole-scope
|
||||
terminal counters remain unknown; the strict whole-resource grader stays false.
|
||||
One-CPU quotas are experiment controls rather than CPU-efficiency claims.
|
||||
|
||||
Production-provider fixture tests cover explicit ABI, old complete defaults,
|
||||
actual SDK-converted schemas, exact pages after APK removal, portable records,
|
||||
nullable/partial facts, malformed inputs, target digest conflicts and early
|
||||
refusals. CLI/MCP acceptance covers original report equality, duplicate-member
|
||||
refusals and unsupported snapshot kinds. Fixture successes are distinguished
|
||||
from the real LocalSend SDK/migration workflows. Other Flutter apps, architectures,
|
||||
SDK layouts, typed class recovery and runtime behavior remain unverified here.
|
||||
Raw replies, binaries, model traces and temporary credentials stay outside Git.
|
||||
|
||||
## Integration with the newer main
|
||||
|
||||
The rebased verification runtime is `63984ce0898bd7997e6d9b138e6d96d0a24f2b09`
|
||||
on main `a2d8b94dd50dab8d966239dd63ab84718aa8b4b4`, with compiled file-table
|
||||
SHA-256 `19c8ccdd3e99bb46a5c54f4edd8c6b2d2eb4f3a4870b3c129061d38145382b68`.
|
||||
Targeted Flutter/SAZ/shared-view integration passes 35 tests; one opposite
|
||||
runtime-capacity branch is skipped because this host enforces decoder memory.
|
||||
Documentation checks and two README conformance tests pass.
|
||||
|
||||
Fresh real complete/summary SDK checks on this runtime match the earlier selected
|
||||
facts and settings, with unchanged 151,341 / 12,037-byte AOT replies. Fifteen
|
||||
old-Evidence migration calls again preserve every original string and record
|
||||
with the source APK absent. The real one-shot CLI complete result also matches
|
||||
the previous-runtime normalized report, subject, parameters and original ID.
|
||||
An initial CLI-only controller check lacked an owned resource receipt; it is
|
||||
retained and excluded from resource/performance claims, then repeated in the
|
||||
bounded owned scope. These checks establish integration, rather than relabeling
|
||||
the earlier paired model/SDK measurements as the rebased runtime.
|
||||
@@ -0,0 +1,120 @@
|
||||
# Native IDA MCP verification on Windows
|
||||
|
||||
Verification date: 2026-10-13. These runs used **IDA Pro 9.3** (executable
|
||||
product version `9.3.26.0213`), not Windows IDA Free. Personal paths, raw
|
||||
observations, and license registration information are excluded.
|
||||
|
||||
The native plugin was built from upstream
|
||||
[`c9d15d98e8a143fc74f6e14037c48f63889b6d6c`](https://github.com/0xshlomil/ida-free-mcp/commit/c9d15d98e8a143fc74f6e14037c48f63889b6d6c)
|
||||
with MinGW GCC 13 and its pinned public SDK. The only local source change
|
||||
bound the listener to `127.0.0.1`; the upstream function-navigation fix was
|
||||
already present. This does not verify the stock 1.1.2 release listener or
|
||||
its older navigation order.
|
||||
|
||||
## Correctness and authority
|
||||
|
||||
The public [IDA fixture](../../tests/conformance/ida/inventory.c) contains
|
||||
two exported functions with distinct bodies. Production CLI/MCP verification
|
||||
checks matching function Evidence, assembly, a unique second-function marker,
|
||||
return to the first function, original-input identity, invalid requests, and
|
||||
connection cleanup. Native verification requires the second procedure and its
|
||||
unique marker; missing, incomplete and empty arguments are rejected before
|
||||
connecting. Dossier relationship coverage remains unknown when direct callers
|
||||
are unavailable, while standalone callee coverage retains its own observed state.
|
||||
It preserves the attached GUI database.
|
||||
|
||||
The later verification follow-up rejects native callee successes without an
|
||||
explicit truncation marker, checks positive procedure inventory/search and
|
||||
compares normalized function results. String and xref correctness require
|
||||
independent positive oracles; runs without those oracles label them unverified.
|
||||
Earlier reports established contract validity for these operations and did not
|
||||
by themselves establish string/xref semantic correctness.
|
||||
|
||||
The follow-up passed eight production checks with a known CRT literal confirmed
|
||||
in the unchanged fixture bytes and a reference source address independently
|
||||
identified from `main` assembly matching the C source. A loopback response-only
|
||||
proxy in front of the real IDA Pro provider reproduced the earlier verifier
|
||||
passing empty function inventories and xrefs. The corrected verifier rejected
|
||||
those replies with positive oracles, empty string results and missing native
|
||||
callee truncation markers. The comparison used verifier source `d1aaeee40`
|
||||
with the same current adapter runtime; it is a verifier comparison, not an
|
||||
engine performance comparison. Only response payloads were modified; the original
|
||||
input digest stayed unchanged, the target was never executed, and the attached
|
||||
GUI database was preserved.
|
||||
|
||||
The real application was a digest-verified, unchanged copy of Windows
|
||||
`whoami.exe`: 94,208 bytes, 243 recognized functions, and 314 upstream
|
||||
C-string observations. The selected 1,741-byte function had 20 resolved
|
||||
callees. Before and after, the task returned identical pseudocode, callee
|
||||
addresses, and its one `whoami` string match. The executable was never run.
|
||||
Other string representations remain unknown in the native profile.
|
||||
|
||||
A Windows C++ producer golden checked 24 literal/case combinations and
|
||||
408 candidate comparisons, including metacharacters, Kelvin sign, dotted I,
|
||||
sigma, supplementary characters, and null bytes. Adapter tests additionally
|
||||
cover malformed UTF-16 fallback, candidate pagination, explicit search errors,
|
||||
and reordered, missing, duplicate, unrelated, or failed lookup replies.
|
||||
These checks do not establish IDA recognition of every Unicode string format.
|
||||
|
||||
## Equivalent completed tasks
|
||||
|
||||
Both builds used main `59be86671`. The baseline was `fb34ccbec`, before
|
||||
the native performance edits; the comparison build added those edits.
|
||||
Schemas were identical. Each run opened the same analyzed binary, read the
|
||||
specified function's pseudocode and callees, searched literal `whoami`, and
|
||||
closed the REA session. The task then repeated on the same MCP connection.
|
||||
|
||||
There were two runs per build on a shared host. Node 24.18.0 and IDA were
|
||||
restricted to one logical CPU with below-normal priority; each Node process
|
||||
had a 768 MiB heap ceiling. The attached database was already analyzed.
|
||||
Node compile caching was disabled for the primary comparison.
|
||||
|
||||
| Measurement | Before | After |
|
||||
| --------------------------------------------------- | -----------: | -----------: |
|
||||
| Task response text | 49,139 bytes | 32,546 bytes |
|
||||
| Task response token estimate | 16,618 | 11,509 |
|
||||
| Callee lookup calls, including selector | 17 | 2 |
|
||||
| String-search pages | 4 | 1 |
|
||||
| String-search Evidence | 19,022 bytes | 3,344 bytes |
|
||||
| Mean task time, warm MCP connection | 675 ms | 317 ms |
|
||||
| Mean task time, fresh MCP process and catalog | 19.72 s | 20.86 s |
|
||||
| Mean Node family CPU, fresh and warm tasks together | 17.72 s | 18.62 s |
|
||||
| Largest Node family peak-RSS upper bound | 776.8 MiB | 780.0 MiB |
|
||||
| Mean attached IDA CPU during both tasks | 226.6 ms | 171.9 ms |
|
||||
|
||||
The warm task was about 53% faster and its response estimate about 31%
|
||||
smaller. Fresh-start, overall CPU, and memory improvements were not
|
||||
established. Startup and full catalog conversion dominate the fresh task.
|
||||
Two samples do not establish a general performance guarantee.
|
||||
|
||||
Token counts use local `cl100k_base` tokenization of complete serialized
|
||||
MCP response text and request arguments; they are estimates, not model usage.
|
||||
Requests added 99 estimated tokens per task. The unchanged full schema
|
||||
catalog contained 2,422,229 bytes and 613,965 estimated tokens. Including it
|
||||
once gives first-task estimates of 630,682 before and 625,573 after. A host
|
||||
may select schemas lazily; actual host injection, prompts, reasoning tokens,
|
||||
and model-side costs were not measured. No retries were needed.
|
||||
|
||||
Node CPU counters were captured at shutdown. The memory figure sums the
|
||||
two processes' individual OS peak RSS values: an upper bound, not a
|
||||
simultaneous family peak. Attached IDA is outside that process family; its
|
||||
unchanged lifetime peak working set was 376.9 MiB, including initial analysis.
|
||||
Measurement overhead and whole-machine resource use were not measured.
|
||||
|
||||
## Reuse and cleanup
|
||||
|
||||
Callee queries are deduplicated and batched within an operation. Literal
|
||||
search fetches a conservative candidate set, including non-ASCII text when
|
||||
needed, and preserves REA's final Unicode predicate. It examines all candidate
|
||||
pages and retains full source observations. Regex searches keep the full scan.
|
||||
|
||||
Cross-request analysis results remain live because an input digest does not
|
||||
establish database revision or freshness. An exploratory caller-selected
|
||||
[Node compile cache](https://nodejs.org/download/release/v24.16.0/docs/api/module.html#module-compile-cache)
|
||||
did not demonstrate startup improvement: empty-cache and cache-hit MCP starts
|
||||
were 7.35 s and 7.56 s. It occupied 6.7 MiB and was limited to this experiment;
|
||||
no new default cache or installation behavior was introduced.
|
||||
|
||||
Temporary source, SDK, build, portable runtime, cache, and raw measurement
|
||||
outputs are test-owned cleanup targets. Attached GUI databases are retained;
|
||||
REA never saves or closes them.
|
||||
@@ -0,0 +1,75 @@
|
||||
# Latest-main efficiency continuation
|
||||
|
||||
Continues [#2016](https://github.com/morluto/rea/issues/2016). The 2026-10-13
|
||||
fetch resolved main to `f254f5ac2b804b353a711264bb178f43fdf07867`, the same
|
||||
revision as the preceding Android conclusion. Newer capabilities below were
|
||||
added during that earlier cycle; they are additional measurement scopes, not
|
||||
new commits after that conclusion.
|
||||
|
||||
| Capability | Existing controls | Next evidence needed |
|
||||
| ----------------------------- | ---------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- |
|
||||
| Flutter / Dart AOT (#1582) | Explicit ABI, bounded string observations, partial coverage | Real shipped snapshot/count question; opt-in delivery only if unrelated strings dominate |
|
||||
| SQLite (#2271) | Compact inventory, selected table/columns, row limit | Real application database and model workflow before adding another selector |
|
||||
| TShark (#2318) | Fields, filter, decode-as, pass, default 50 matching rows, paging, explicit null for all | Real public capture and repeated task cost; paging repeats dissection |
|
||||
| sigrok (#2339) | Decoder/channels/annotation selection, default 50 display lines, paging | Real public capture; display-line count is not annotation count and paging repeats decoding |
|
||||
| Retained unknown head results | Existing retained paging | Reuse the existing view rather than duplicate its implementation |
|
||||
|
||||
Attached IDA, Frida and other live runtime workflows involve separate authority
|
||||
and lifecycle questions and are outside this static parsing continuation.
|
||||
Plugin SDKs and cross-session engine caches remain deferred until an observed
|
||||
need and benefit justify them.
|
||||
|
||||
## First selected scope: Dart AOT
|
||||
|
||||
The official LocalSend 1.18.2 arm64 APK is pinned in `corpus.json`. The initial
|
||||
real public SDK baseline inspected actual bytes without executing/installing
|
||||
the APK. Its complete default AOT reply was 151,341 compact client-result JSON
|
||||
bytes (70,487 normalized bytes), with 500 package URIs, 216 Dart URIs and 2,000
|
||||
class-like tokens. String-pool and overall coverage were partial. A second
|
||||
explicit arm64 request returned the same normalized facts.
|
||||
|
||||
These are exploratory first/warm calls, not an optimization pair. The frozen
|
||||
runtime source is `28c84e78cf2fbfde4aceed345b773c59eb889f68`, with production
|
||||
source bytes verified equal to main f254 at baseline acquisition. Its compiled
|
||||
file-table digest is
|
||||
`5332dda13981f043e146b3601ebec1f380d9f698280ca53d75786b638a912e11`;
|
||||
the receipt is not relabeled as a newer build.
|
||||
|
||||
The candidate adds explicit MCP `detail: "summary"` and retained pool pages.
|
||||
Omitted/complete requests and the CLI preserve the old complete path. Analysis,
|
||||
ABI selection and original Evidence stay intact. Summary retains target/libapp
|
||||
identity, exact nullable snapshot observations, lower bounds, returned counts,
|
||||
coverage and limitations. Pages preserve original string order and source
|
||||
coverage without rescanning. No engine cache, provider protocol or third-party
|
||||
plugin API is introduced.
|
||||
|
||||
Follow-up measurements must compare identical selected facts, portable old
|
||||
Evidence and actual model tasks. Reply bytes do not establish token or latency
|
||||
improvements. Whole-scope final counters remain unknown if their cgroup has
|
||||
disappeared, even when the narrower payload-exit observation is verified.
|
||||
Raw records, binaries and model traces stay in the private lab outside Git.
|
||||
Heavy work remains serial under the unchanged one-CPU, 3 GiB early-stop / 4 GiB
|
||||
hard memory, no-swap and host-availability guards.
|
||||
|
||||
## Main advanced during the experiment
|
||||
|
||||
Main subsequently advanced to `a2d8b94dd50dab8d966239dd63ab84718aa8b4b4`,
|
||||
adding native Fiddler SAZ parsing. Its web-network capture contract already
|
||||
offers opt-in summary and retained capture pages, archive-member coordinates
|
||||
and exact exposed encoded bytes. It decodes all records before ordinal
|
||||
selection and rejects malformed/oversized inputs without partial success.
|
||||
Measure real public SAZ artifacts using these controls before adding another
|
||||
API or changing decoding or credential-redaction semantics. SAZ performance
|
||||
remains unverified in this Flutter continuation.
|
||||
|
||||
The Flutter cohorts keep their original frozen runtime; subsequent integration
|
||||
onto this newer main does not relabel earlier measurement receipts.
|
||||
|
||||
Main advanced again to `bb154d106` during CI, adding retained aioquic 1.5.0
|
||||
qlog 0.3 JSON parsing and qlog views. The existing capture summary/page path
|
||||
already applies. Qlog's trace-scoped STREAM interval unions and endpoint loss
|
||||
declarations have separate semantics: they do not recover bodies, prove network
|
||||
loss or establish PCAP correlation. Measure genuine supported recordings and
|
||||
existing retained views before adding delivery or caching. Other qlog layouts
|
||||
and qlog performance remain unverified in the Flutter experiment. Shared-view
|
||||
integration is checked after rebasing; earlier runtime receipts remain pinned.
|
||||
@@ -0,0 +1,234 @@
|
||||
# Managed artifact delivery and retained views
|
||||
|
||||
Continues [#2016](https://github.com/morluto/rea/issues/2016) after
|
||||
[#2404](https://github.com/morluto/rea/pull/2404) merged. This checkpoint pins
|
||||
the complete managed workflow before implementing another optional view.
|
||||
The baseline below precedes implementation; the follow-up adds explicit summary
|
||||
delivery and portable collection pages while preserving complete defaults.
|
||||
|
||||
## Frozen control and analyst question
|
||||
|
||||
Control source is merged main `ae8051302b5834322fa00cfa31c968ae74ab6ed9`,
|
||||
Node 24.18.0, matching locked dependencies and actual compiled stdio MCP.
|
||||
The frozen `dist/`/`bridge/` file-table SHA-256 is
|
||||
`7a4cec0fa4f7e929356b50a6b13472f80bd6f526221e427125370cca3ef98d00`.
|
||||
Measurement source SHA-256 is
|
||||
`a95d8d8e9f72c0b2b885c61907ed13054ebf7d1642f6f13b0126163840b1221a`.
|
||||
|
||||
Use the digest-pinned shipped ShareX DLL from [the corpus](corpus.json),
|
||||
2,162,688 bytes, SHA-256
|
||||
`191a191be964e434daf0f9d76d1e0d8ddce90d7e248cba2a6b5ac9577f288d53`.
|
||||
The [existing model question](questions/managed.txt) requests exact identity,
|
||||
complete managed classification and its evidence, frameworks and every declared
|
||||
native ModuleRef/PInvoke boundary with coverage and limitations. It explicitly
|
||||
excludes unrelated resource/custom-attribute inventories. Analysis reads PE/CLI
|
||||
metadata without loading or executing the assembly.
|
||||
|
||||
The current complete result has exactly the same normalized facts as the earlier
|
||||
real managed checkpoint linked from [the EVM descriptor study](evm-descriptor.md).
|
||||
This comparison does not equate fresh Evidence IDs, provider versions or whole
|
||||
catalogs. Original target bytes remain unchanged; SDK-converted named schemas
|
||||
and actual replies validate in the public measurement workflow.
|
||||
|
||||
## Observed data costs
|
||||
|
||||
| Public operation | SDK result JSON UTF-8 bytes | Tool milliseconds |
|
||||
| ----------------------------------- | --------------------------: | ----------------: |
|
||||
| `inspect_managed_artifact` | 1,270,839 | 450.306 |
|
||||
| `inspect_managed_native_boundaries` | 27,913 | 142.663 |
|
||||
|
||||
The complete workflow, including connection/discovery and both calls, takes
|
||||
3.721 seconds. The full discovered catalog is 2,482,204 compact JSON bytes;
|
||||
its input definitions are 527,569 bytes. These costs include the current main
|
||||
changes and are not silently substituted for an older catalog baseline.
|
||||
|
||||
The artifact inspection's normalized payload alone is 607,605 compact JSON UTF-8
|
||||
bytes, measured with actual Node `JSON.stringify`:
|
||||
|
||||
| Returned collection | Rows | Compact JSON UTF-8 bytes |
|
||||
| ------------------- | ----: | -----------------------: |
|
||||
| References | 38 | 12,153 |
|
||||
| Resources | 36 | 10,357 |
|
||||
| Custom attributes | 1,945 | 582,586 |
|
||||
|
||||
Custom attributes dominate the payload. The remaining identity, PE/CLI,
|
||||
classification, metadata, framework and coverage facts are comparatively small.
|
||||
`raw_result` is null. SDK result size includes both structured and text
|
||||
representations; it is not the exact context a model receives or a token count.
|
||||
|
||||
The serial owned scope takes 4.852 seconds including the driver, samples
|
||||
520,110,080 peak family bytes and records 4.699 last-observed CPU seconds.
|
||||
One CPU, 4 GiB hard memory, unchanged 3 GiB early-stop, no swap and 192 tasks
|
||||
remain in effect. No guard abort occurs. Final cgroup counters disappear and
|
||||
remain unknown; sampled zero events are not final no-OOM proof.
|
||||
|
||||
## Bounded candidate and acceptance
|
||||
|
||||
Evaluate an explicit overview on the first artifact inspection, retaining the
|
||||
unchanged complete source and returning a distinct linked view. Keep complete
|
||||
identity/classification, metadata status, nullable module/MVID, assembly,
|
||||
frameworks, original coverage/issues/limitations and returned-collection counts
|
||||
inline. Typed pages can expose original references/resources/attributes with
|
||||
their nulls, token coordinates, offsets, order and values. Returned-array
|
||||
coverage must remain distinct from whole-metadata completeness.
|
||||
|
||||
Reuse the existing managed schemas, retained Evidence application workflow and
|
||||
named registration hooks. Keep omitted/complete requests, other managed tools,
|
||||
selected target/provider behavior and original export/import identities usable.
|
||||
Do not strengthen historical admission to require a non-null subject or
|
||||
reinterpret foreign recorded paths through this host's filesystem. Delivery
|
||||
selection must not be passed to the PE/CLI parser as an analysis option.
|
||||
|
||||
At the baseline checkpoint the candidate was not implemented or measured. Compare the
|
||||
same completed analyst question, actual discovery/results, full default
|
||||
compatibility and retained paging without producer reruns. Follow with other
|
||||
shipped managed inputs, partial/unavailable metadata and real Codex trials with
|
||||
strict factual/reference checks. Preserve failed or truncated transcripts.
|
||||
Count total/cached/uncached input, output, calls/retries, whole-task time, CPU
|
||||
and family memory before recommending a token or latency improvement. A smaller
|
||||
reply alone is insufficient.
|
||||
|
||||
## Reproduce the control
|
||||
|
||||
Build and freeze the explicit source revision before editing production code.
|
||||
Supply the original shipped DLL and an explicit string-valued provider dictionary
|
||||
in a fresh private receipt directory:
|
||||
|
||||
```bash
|
||||
python3 experiments/analysis-efficiency/resource-run.py -- \
|
||||
node experiments/analysis-efficiency/measure.mjs \
|
||||
--runtime-root /absolute/control-ae805 \
|
||||
--revision ae8051302b5834322fa00cfa31c968ae74ab6ed9 \
|
||||
--artifact sharex.dll --path /absolute/ShareX.dll \
|
||||
--providers /absolute/explicit-providers.json --schemas selected \
|
||||
--output /absolute/new-control-receipts
|
||||
```
|
||||
|
||||
Raw replies, runtime copies, fingerprints and detailed resource receipts stay
|
||||
outside Git. No new provider, plugin SDK, database cache or target execution is
|
||||
introduced by this baseline.
|
||||
|
||||
## Implemented candidate and real SDK observations
|
||||
|
||||
Candidate `8bc1601ca51e417ca472d90ea0367e9418a1a0f2` adds
|
||||
`inspect_managed_artifact({path, detail: "summary"})` and managed collection pages
|
||||
through the existing `inspect_analysis_view` contract. The complete parser result
|
||||
remains retained. Omitted detail and explicit `complete` keep complete output;
|
||||
other managed tools do not acquire this selector. CLI inspection stays complete.
|
||||
The frozen candidate runtime file-table SHA-256 is
|
||||
`9e7380f537301bf0cfe2926f5619011af848af7c220750e75189498e65d825e1`.
|
||||
The revised measurement driver SHA-256 is
|
||||
`8c22f59e931fdc347dea1204ea0faf91330578db684357261ec6f03bd0efa85e`.
|
||||
|
||||
Use `--managed-question identity` on **both** sides, with `--delivery complete`
|
||||
for the control and `--delivery summary` for the candidate. The original default
|
||||
`inventory` recipe remains unchanged. Both identity recipes make the same two
|
||||
public calls, validate their advertised SDK schemas, and compare complete
|
||||
identity/classification, frameworks, coverage and native-boundary facts.
|
||||
|
||||
| Artifact | Complete metadata bytes | Summary metadata bytes | Complete two-call bytes | Summary two-call bytes |
|
||||
| --------------------- | ----------------------: | ---------------------: | ----------------------: | ---------------------: |
|
||||
| ShareX.dll | 1,270,839 | 10,549 | 1,298,752 | 38,462 |
|
||||
| ShareX.HelpersLib.dll | 1,296,839 | 10,937 | 1,586,970 | 301,068 |
|
||||
|
||||
The helper assembly is a second shipped member of the same pinned ShareX release,
|
||||
1,076,224 bytes, SHA-256
|
||||
`e91bd1ec904e2bb8a1add4bb6187443b878bf33d24fe9e27f0b799bfee9de33d`.
|
||||
It is not an independent application or a new runtime/provider lane. Its native
|
||||
boundary report remains comparatively large and unchanged. Metadata reply bytes
|
||||
fall by about 99.17% and 99.16%; two-call totals fall by about 97.04% and 81.03%.
|
||||
The first committed-candidate observations take 3.699 and 3.838 seconds, versus
|
||||
3.809 and 3.925 seconds for controls. These timings include discovery and do not
|
||||
establish a general latency improvement. The full candidate catalog grows from
|
||||
2,482,204 to 2,497,635 bytes; input definitions grow from 527,569 to 528,862.
|
||||
|
||||
An earlier dirty prototype produced the same runtime file-table hash and a
|
||||
separate provenance receipt. Its observations remain separate; a clean baseline
|
||||
commit is never presented as the source of uncommitted production changes.
|
||||
|
||||
## Actual Codex findings: token benefit is not established
|
||||
|
||||
Four sequential trials use actual Codex CLI 0.161.0, requested model
|
||||
`gpt-5.6-terra`, high reasoning, the unchanged question above and the same two
|
||||
enabled tools. The requested alias does not prove a resolved server model ID.
|
||||
Order is control, candidate, candidate, control. Both candidate trials independently
|
||||
select summary; all trials preserve dependency order, make two successful calls,
|
||||
leave the artifact unchanged and remove temporary profiles.
|
||||
|
||||
| Trial | Input tokens | Cached input | Uncached input | Output tokens | Wall seconds |
|
||||
| ----------- | -----------: | -----------: | -------------: | ------------: | -----------: |
|
||||
| Control 1 | 54,821 | 29,184 | 25,637 | 1,782 | 63.207 |
|
||||
| Candidate 1 | 77,594 | 50,432 | 27,162 | 1,583 | 46.685 |
|
||||
| Candidate 2 | 82,426 | 49,408 | 33,018 | 2,120 | 67.379 |
|
||||
| Control 2 | 55,296 | 29,184 | 26,112 | 2,361 | 62.159 |
|
||||
|
||||
Uncached input is reported input minus cached input. **Candidate input and
|
||||
uncached input are higher in both comparisons.** Latency is mixed. This feature
|
||||
demonstrates smaller selected delivery, not measured model-token savings or a
|
||||
stable whole-task speedup. Event result bytes are also not a measurement of the
|
||||
exact context presented to the model. No hidden truncation/cache mechanism is
|
||||
inferred from these aggregates. Keep further model-context investigation separate
|
||||
from this additive view implementation.
|
||||
|
||||
The question prescribes a `coverage` field without prescribing its nested labels.
|
||||
The grader now accepts only explicit equivalent artifact/native scope names;
|
||||
conflicting duplicate scopes still fail. Original failure receipts remain intact.
|
||||
Control 1 and both candidates pass the selected factual checks after that
|
||||
correction. Control 2 omits `classification.container` and does not report
|
||||
separate artifact/native coverage in the expected representation; it fails these
|
||||
strict selected checks. The prompt did not specify every nested key, so this is
|
||||
also an answer-representation limitation. It is not counted as a fully acceptable
|
||||
control answer or repaired by assuming ambiguous flat coverage. Original
|
||||
limitations were additionally reviewed against producer replies.
|
||||
|
||||
All four reports still fail the **complete** selected gate because final physical
|
||||
resource counters are unavailable. The factual subset is reported separately;
|
||||
it does not turn unknown final resource counters into a pass. Trial guards remain
|
||||
unchanged, no guard abort is observed, and sampled peaks stay below the 3 GiB
|
||||
early-stop threshold. A sampled zero OOM count is not final no-OOM proof.
|
||||
|
||||
## Previous-runtime Evidence compatibility
|
||||
|
||||
Complete actual producer Evidence from control `ae8051302b5834322fa00cfa31c968ae74ab6ed9`
|
||||
for both shipped assemblies is passed, unchanged, through the candidate's public
|
||||
SDK inline view. Each artifact makes 14 view calls: a summary, all original
|
||||
reference/resource/attribute pages at a caller-selected 256 rows, and exhausted
|
||||
pages for each collection. Summary facts, every row and order, source coverage,
|
||||
parent ID, links and target digest match; the original Evidence stays unchanged.
|
||||
ShareX has 38/36/1,945 reported rows and HelpersLib has 49/17/1,988. No producer
|
||||
operation or target mutation is requested by this portable workflow.
|
||||
|
||||
The private verification driver SHA-256 is
|
||||
`7bb1f24b488beda012b1e78132de29b868fe1ea91a5e1499d7f271801fa40380`;
|
||||
the receipt scope takes 5.860 seconds, samples 642,244,608 peak family bytes and
|
||||
5.619 last-observed CPU seconds, with no guard abort. Final counters remain unknown.
|
||||
These are actual previous-runtime records, unlike the separate synthetic
|
||||
null-subject compatibility test. They do not prove every historical release.
|
||||
|
||||
Focused SDK/domain checks cover first-call summary, complete defaults, selected
|
||||
target reuse, paging after the source disappears, malformed metadata, historical
|
||||
null subjects, digest mismatch, portable projection and unavailable retention
|
||||
before file access. The exact candidate CI passes all checks after only the
|
||||
browser cleanup and Apple `hdiutil` jobs are retried on the same source revision;
|
||||
their original failures remain recorded in the private team handoff.
|
||||
|
||||
## Committed-runtime repetition and complete defaults
|
||||
|
||||
A second SDK pair reverses the order (candidate before control) for both
|
||||
assemblies. Result byte totals repeat exactly. ShareX workflow times are
|
||||
3.644 versus 3.687 seconds; HelpersLib times are 3.714 versus 3.859 seconds.
|
||||
Both pairs preserve selected settings/facts and have no tool errors. Two pairs
|
||||
per assembly, sharing one application and host, remain a narrow descriptive
|
||||
sample rather than a general performance claim.
|
||||
|
||||
Actual omitted-detail calls on the candidate additionally reproduce **all**
|
||||
normalized metadata inventories and native-boundary facts from both controls,
|
||||
not just the identity recipe's selected subset. Their canonical complete
|
||||
Evidence IDs also match. This verifies the recorded old request and complete
|
||||
result path alongside the explicit new delivery option.
|
||||
|
||||
The changed shared projector also handles an actual shipped SQLite WASM run
|
||||
through the candidate's SDK summary workflow, with real configured WABT:
|
||||
two calls, 18,687 result JSON bytes, 5.614 seconds and no tool failure. This
|
||||
checks that neighboring dispatch path; it does not establish a new WASM
|
||||
performance improvement or additional operating-system/provider coverage.
|
||||
@@ -3,7 +3,7 @@ import { createReadStream } from "node:fs";
|
||||
import { mkdir, readFile, readdir, stat, writeFile } from "node:fs/promises";
|
||||
import { dirname, isAbsolute, resolve } from "node:path";
|
||||
import { fileURLToPath, pathToFileURL } from "node:url";
|
||||
import { parseArgs } from "node:util";
|
||||
import { isDeepStrictEqual, parseArgs } from "node:util";
|
||||
import {
|
||||
NATIVE_ARTIFACTS,
|
||||
measureNativeEntry,
|
||||
@@ -23,9 +23,12 @@ const { values } = parseArgs({
|
||||
providers: { type: "string" },
|
||||
output: { type: "string" },
|
||||
delivery: { type: "string", default: "complete" },
|
||||
"verify-retained": { type: "boolean", default: false },
|
||||
"native-question": { type: "string", default: "entry" },
|
||||
"javascript-question": { type: "string", default: "inventory" },
|
||||
"managed-question": { type: "string", default: "inventory" },
|
||||
schemas: { type: "string", default: "default" },
|
||||
"sqlite-selection": { type: "string", default: "all" },
|
||||
},
|
||||
});
|
||||
if (
|
||||
@@ -41,6 +44,21 @@ if (
|
||||
) ||
|
||||
!["inventory", "selected"].includes(values["javascript-question"]) ||
|
||||
!["default", "selected"].includes(values.schemas) ||
|
||||
!["all", "columns"].includes(values["sqlite-selection"]) ||
|
||||
(values.artifact !== "qgis-world-map.gpkg" &&
|
||||
values["sqlite-selection"] !== "all") ||
|
||||
(values.artifact === "qgis-world-map.gpkg" &&
|
||||
values.delivery !== "complete") ||
|
||||
!["inventory", "identity"].includes(values["managed-question"]) ||
|
||||
(!["sharex.dll", "sharex.helpers.dll"].includes(values.artifact) &&
|
||||
values["managed-question"] !== "inventory") ||
|
||||
(["sharex.dll", "sharex.helpers.dll"].includes(values.artifact) &&
|
||||
values.delivery === "summary" &&
|
||||
values["managed-question"] !== "identity") ||
|
||||
(values["verify-retained"] &&
|
||||
!["weth.hex", "uniswap-v3.factory.hex", "uniswap-v3.pool.hex"].includes(
|
||||
values.artifact,
|
||||
)) ||
|
||||
(!NATIVE_ARTIFACTS.includes(values.artifact) &&
|
||||
values["native-question"] !== "entry") ||
|
||||
(values.artifact !== "jq" &&
|
||||
@@ -238,15 +256,55 @@ const targetSha = (value) => value.subject?.digest.sha256;
|
||||
const inventoryPaths = ["package.json", "main.js", "preload.js"];
|
||||
const unique = (values) => [...new Set(values)].sort();
|
||||
const inspectManagedTarget = async () => {
|
||||
const info = await call("inspect_managed_artifact", { path: values.path });
|
||||
const arguments_ = {
|
||||
path: values.path,
|
||||
...(values.delivery === "summary" ? { detail: "summary" } : {}),
|
||||
};
|
||||
const definition = catalogDefinitions.get("inspect_managed_artifact");
|
||||
if (definition === undefined)
|
||||
throw new Error("Managed inspection is not advertised");
|
||||
const accepted = new Ajv2020({
|
||||
strict: false,
|
||||
validateFormats: false,
|
||||
}).compile(definition.inputSchema)(arguments_);
|
||||
if (!accepted)
|
||||
throw new Error(
|
||||
"Selected managed delivery is not advertised by this runtime",
|
||||
);
|
||||
const info = await call("inspect_managed_artifact", arguments_);
|
||||
const boundaries = await call("inspect_managed_native_boundaries", {
|
||||
path: values.path,
|
||||
});
|
||||
const received = normalized(info);
|
||||
const report = received.kind === "summary" ? received.summary : received;
|
||||
let artifact = report;
|
||||
if (values["managed-question"] === "identity") {
|
||||
const {
|
||||
references,
|
||||
resources,
|
||||
attributes,
|
||||
reported_counts,
|
||||
identity_scope: _scope,
|
||||
...identity
|
||||
} = report;
|
||||
artifact = {
|
||||
...identity,
|
||||
reported_counts: reported_counts ?? {
|
||||
references: references.length,
|
||||
resources: resources.length,
|
||||
attributes: attributes.length,
|
||||
},
|
||||
};
|
||||
}
|
||||
return {
|
||||
artifact_sha256: targetSha(info),
|
||||
artifact: normalized(info),
|
||||
artifact,
|
||||
boundaries: normalized(boundaries),
|
||||
evidence_ids: [info.evidence_id, boundaries.evidence_id],
|
||||
evidence_ids: [
|
||||
info.evidence_id,
|
||||
...(received.kind === "summary" ? [received.parent_evidence_id] : []),
|
||||
boundaries.evidence_id,
|
||||
],
|
||||
};
|
||||
};
|
||||
|
||||
@@ -350,7 +408,170 @@ const inspectWasmTarget = async () => {
|
||||
};
|
||||
};
|
||||
|
||||
/** Identical interface facts for complete delivery, selected summary or old-server fallback. */
|
||||
const inspectEvmTarget = async () => {
|
||||
const definition = catalogDefinitions.get("inspect_evm_interface");
|
||||
if (!definition) throw new Error("Missing advertised EVM definition");
|
||||
const input = { path: values.path, encoding: "hex" };
|
||||
const supportsSummary = new Ajv2020({
|
||||
strict: false,
|
||||
validateFormats: false,
|
||||
}).compile(definition.inputSchema)({ ...input, detail: "summary" });
|
||||
const info = await call("inspect_evm_interface", {
|
||||
...input,
|
||||
...(values.delivery === "summary" && supportsSummary
|
||||
? { detail: "summary" }
|
||||
: {}),
|
||||
});
|
||||
const result = normalized(info);
|
||||
const projected = result.kind === "summary";
|
||||
const n = projected ? result.summary : result;
|
||||
const references = [info.evidence_id];
|
||||
if (projected) {
|
||||
if (
|
||||
result.parent_operation !== "inspect_evm_interface" ||
|
||||
result.parent_evidence_id === info.evidence_id ||
|
||||
!info.evidence_links.includes(result.parent_evidence_id) ||
|
||||
"hex" in n.bytecode ||
|
||||
"stdout" in n.diagnostics
|
||||
)
|
||||
throw new Error(
|
||||
"EVM summary changed source provenance or precise delivery",
|
||||
);
|
||||
references.push(result.parent_evidence_id);
|
||||
}
|
||||
if (
|
||||
n.evidence_kind !== "inferred" ||
|
||||
n.artifact.path !== values.path ||
|
||||
n.artifact.sha256 !== inputSha ||
|
||||
n.artifact.bytes !== artifact.bytes ||
|
||||
n.artifact.encoding !== "hex" ||
|
||||
n.discovery_completeness !== "unknown" ||
|
||||
n.runtime_execution !== "not-performed" ||
|
||||
["kind", "hardfork", "deployment_authenticity"].some(
|
||||
(key) => n.bytecode[key] !== "unknown",
|
||||
) ||
|
||||
(artifact.decoded_sha256 !== undefined &&
|
||||
n.bytecode.sha256 !== artifact.decoded_sha256) ||
|
||||
(artifact.decoded_bytes !== undefined &&
|
||||
n.bytecode.bytes !== artifact.decoded_bytes)
|
||||
)
|
||||
throw new Error("EVM interface identity or uncertainty changed");
|
||||
if (values["verify-retained"]) {
|
||||
if (!projected)
|
||||
throw new Error(
|
||||
"Retained verification requires supported selected summary",
|
||||
);
|
||||
const bundle = (await call("get_evidence_bundle", {})).result;
|
||||
const parent = bundle.records.find(
|
||||
(record) => record.evidence_id === result.parent_evidence_id,
|
||||
);
|
||||
if (
|
||||
!parent ||
|
||||
parent.operation !== "inspect_evm_interface" ||
|
||||
parent.subject?.digest.sha256 !== inputSha ||
|
||||
parent.parameters.path !== values.path ||
|
||||
parent.parameters.encoding !== "hex" ||
|
||||
"detail" in parent.parameters
|
||||
)
|
||||
throw new Error(
|
||||
"Complete original EVM source was not retained unchanged",
|
||||
);
|
||||
const original = normalized(parent);
|
||||
const { hex: _hex, ...identity } = original.bytecode;
|
||||
const { stdout, ...diagnostics } = original.diagnostics;
|
||||
const expected = {
|
||||
...original,
|
||||
bytecode: identity,
|
||||
diagnostics: { ...diagnostics, stdout_code_units: stdout.length },
|
||||
};
|
||||
if (!isDeepStrictEqual(expected, n))
|
||||
throw new Error("EVM summary changed selected original facts");
|
||||
for (const [facet, item] of [
|
||||
["functions", original.functions.slice(0, 64)],
|
||||
[
|
||||
"bytecode_hex",
|
||||
{ text: original.bytecode.hex.slice(0, 64), unit: "utf16-code-units" },
|
||||
],
|
||||
]) {
|
||||
const page = await call("inspect_analysis_view", {
|
||||
source: { kind: "retained-evidence", evidence_id: parent.evidence_id },
|
||||
view: { kind: "evm", facet, offset: 0, limit: 64 },
|
||||
});
|
||||
const v = normalized(page);
|
||||
if (
|
||||
!isDeepStrictEqual(v.item, item) ||
|
||||
v.parent_evidence_id !== parent.evidence_id ||
|
||||
v.artifact.sha256 !== inputSha
|
||||
)
|
||||
throw new Error("EVM retained read changed source facts or identity");
|
||||
if (references.includes(page.evidence_id))
|
||||
throw new Error("EVM retained read reused an Evidence identity");
|
||||
references.push(page.evidence_id);
|
||||
}
|
||||
}
|
||||
const { hex: _hex, ...bytecode } = n.bytecode;
|
||||
return {
|
||||
artifact_sha256: targetSha(info),
|
||||
artifact: n.artifact,
|
||||
bytecode,
|
||||
functions: n.functions,
|
||||
discovery_completeness: n.discovery_completeness,
|
||||
runtime_execution: n.runtime_execution,
|
||||
limitations: n.limitations,
|
||||
evidence_ids: references,
|
||||
};
|
||||
};
|
||||
|
||||
const workflows = {
|
||||
"qgis-world-map.gpkg": async () => {
|
||||
const columns = ["fid", "NAME", "ISO_A3"];
|
||||
const input = {
|
||||
path: values.path,
|
||||
table: "countries",
|
||||
row_limit: 1,
|
||||
...(values["sqlite-selection"] === "columns" ? { columns } : {}),
|
||||
};
|
||||
const definition = catalogDefinitions.get("inspect_sqlite_database");
|
||||
if (
|
||||
definition === undefined ||
|
||||
!new Ajv2020({ strict: false, validateFormats: false }).validate(
|
||||
definition.inputSchema,
|
||||
input,
|
||||
)
|
||||
)
|
||||
throw new Error(
|
||||
"Selected SQLite request is not advertised by this runtime",
|
||||
);
|
||||
const inspected = await call("inspect_sqlite_database", input);
|
||||
const report = normalized(inspected);
|
||||
if (report.rows === null || report.rows.table !== "countries")
|
||||
throw new Error("Runtime returned no selected country sample");
|
||||
const indexes = columns.map((column) => {
|
||||
const index = report.rows.columns.indexOf(column);
|
||||
if (index < 0)
|
||||
throw new Error(`Missing requested SQLite column ${column}`);
|
||||
return index;
|
||||
});
|
||||
return {
|
||||
artifact_sha256: targetSha(inspected),
|
||||
artifact: report.artifact,
|
||||
wal: report.wal,
|
||||
engine: report.engine,
|
||||
database_encoding: report.database_encoding,
|
||||
schema: report.schema,
|
||||
rows: {
|
||||
...report.rows,
|
||||
columns,
|
||||
values: report.rows.values.map((row) =>
|
||||
indexes.map((index) => row[index]),
|
||||
),
|
||||
},
|
||||
producer_limitations: report.limitations,
|
||||
limitations: inspected.limitations,
|
||||
evidence_ids: [inspected.evidence_id],
|
||||
};
|
||||
},
|
||||
"obsidian.asar": async () => {
|
||||
if (values["javascript-question"] === "selected") {
|
||||
const inspected = await call("analyze_javascript_application", {
|
||||
@@ -499,7 +720,40 @@ const workflows = {
|
||||
}
|
||||
},
|
||||
"newpipe.apk": async () => {
|
||||
const pkg = await call("inspect_android_package", { path: values.path });
|
||||
const packageInput = {
|
||||
path: values.path,
|
||||
...(values.delivery === "summary" ? { detail: "summary" } : {}),
|
||||
};
|
||||
const definition = catalogDefinitions.get("inspect_android_package");
|
||||
if (
|
||||
definition === undefined ||
|
||||
!new Ajv2020({ strict: false, validateFormats: false }).validate(
|
||||
definition.inputSchema,
|
||||
packageInput,
|
||||
)
|
||||
)
|
||||
throw new Error(
|
||||
"Selected Android delivery is not advertised by this runtime",
|
||||
);
|
||||
const pkg = await call("inspect_android_package", packageInput);
|
||||
const result = normalized(pkg);
|
||||
const report = values.delivery === "summary" ? result.summary : result;
|
||||
if (
|
||||
values.delivery === "summary" &&
|
||||
(result.kind !== "summary" ||
|
||||
result.parent_operation !== "inspect_android_package")
|
||||
)
|
||||
throw new Error("Runtime returned no retained Android package summary");
|
||||
const { manifest, ...declarations } = report;
|
||||
const manifestObservation =
|
||||
values.delivery === "summary"
|
||||
? manifest
|
||||
: {
|
||||
status: manifest.status,
|
||||
reported_total_bytes: manifest.reported_total_bytes,
|
||||
text_code_units: manifest.text.length,
|
||||
text_included: false,
|
||||
};
|
||||
const found = await call("search_android_classes", {
|
||||
path: values.path,
|
||||
query: "MainActivity",
|
||||
@@ -510,9 +764,12 @@ const workflows = {
|
||||
});
|
||||
return {
|
||||
artifact_sha256: targetSha(pkg),
|
||||
package_name: normalized(pkg).package_name,
|
||||
version_name: normalized(pkg).version_name,
|
||||
version_code: normalized(pkg).version_code,
|
||||
package_name: report.package_name,
|
||||
version_name: report.version_name,
|
||||
version_code: report.version_code,
|
||||
package_declarations: declarations,
|
||||
manifest_observation: manifestObservation,
|
||||
package_limitations: pkg.limitations,
|
||||
matches: normalized(found).matches,
|
||||
coverage: normalized(found).coverage,
|
||||
onCreate: normalized(cls).methods.filter(
|
||||
@@ -523,6 +780,54 @@ const workflows = {
|
||||
},
|
||||
"sharex.dll": inspectManagedTarget,
|
||||
"sharex.helpers.dll": inspectManagedTarget,
|
||||
"localsend-arm64.apk": async () => {
|
||||
const input = {
|
||||
path: values.path,
|
||||
...(values.delivery === "summary" ? { detail: "summary" } : {}),
|
||||
};
|
||||
const definition = catalogDefinitions.get("inspect_dart_aot");
|
||||
if (
|
||||
definition === undefined ||
|
||||
!new Ajv2020({ strict: false, validateFormats: false }).validate(
|
||||
definition.inputSchema,
|
||||
input,
|
||||
)
|
||||
)
|
||||
throw new Error(
|
||||
"Selected Dart AOT delivery is not advertised by this runtime",
|
||||
);
|
||||
const inspected = await call("inspect_dart_aot", input);
|
||||
const result = normalized(inspected);
|
||||
if (
|
||||
values.delivery === "summary" &&
|
||||
(result.kind !== "summary" ||
|
||||
result.parent_operation !== "inspect_dart_aot")
|
||||
)
|
||||
throw new Error("Runtime returned no retained Dart AOT summary");
|
||||
const report = values.delivery === "summary" ? result.summary : result;
|
||||
const { package_uris, dart_uris, class_like_tokens, ...pool } =
|
||||
report.string_pool;
|
||||
return {
|
||||
artifact_sha256: targetSha(inspected),
|
||||
target: report.target,
|
||||
abi: report.abi,
|
||||
libapp: report.libapp,
|
||||
string_pool:
|
||||
values.delivery === "summary"
|
||||
? pool
|
||||
: {
|
||||
...pool,
|
||||
returned_counts: {
|
||||
package_uris: package_uris.length,
|
||||
dart_uris: dart_uris.length,
|
||||
class_like_tokens: class_like_tokens.length,
|
||||
},
|
||||
},
|
||||
coverage: report.coverage,
|
||||
limitations: inspected.limitations,
|
||||
evidence_ids: [inspected.evidence_id],
|
||||
};
|
||||
},
|
||||
"openwrt.bin": async () => {
|
||||
const info = await call("inspect_firmware_regions", { path: values.path });
|
||||
return {
|
||||
@@ -534,23 +839,9 @@ const workflows = {
|
||||
"sqlite.wasm": inspectWasmTarget,
|
||||
"vscode.tree-sitter.wasm": inspectWasmTarget,
|
||||
"vscode.tree-sitter-css.wasm": inspectWasmTarget,
|
||||
"weth.hex": async () => {
|
||||
const info = await call("inspect_evm_interface", {
|
||||
path: values.path,
|
||||
encoding: "hex",
|
||||
});
|
||||
const n = normalized(info);
|
||||
return {
|
||||
artifact_sha256: targetSha(info),
|
||||
artifact: n.artifact,
|
||||
bytecode: n.bytecode,
|
||||
functions: n.functions,
|
||||
discovery_completeness: n.discovery_completeness,
|
||||
runtime_execution: n.runtime_execution,
|
||||
limitations: n.limitations,
|
||||
evidence_ids: [info.evidence_id],
|
||||
};
|
||||
},
|
||||
"weth.hex": inspectEvmTarget,
|
||||
"uniswap-v3.factory.hex": inspectEvmTarget,
|
||||
"uniswap-v3.pool.hex": inspectEvmTarget,
|
||||
...Object.fromEntries(
|
||||
NATIVE_ARTIFACTS.map((artifact) => [
|
||||
artifact,
|
||||
@@ -654,11 +945,21 @@ const report = {
|
||||
artifact_sha256_after: finalInputSha,
|
||||
delivery: values.delivery,
|
||||
sdk_schema_mode: values.schemas,
|
||||
retained_verification: values["verify-retained"],
|
||||
native_question: NATIVE_ARTIFACTS.includes(values.artifact)
|
||||
? values["native-question"]
|
||||
: null,
|
||||
javascript_question:
|
||||
values.artifact === "obsidian.asar" ? values["javascript-question"] : null,
|
||||
managed_question: ["sharex.dll", "sharex.helpers.dll"].includes(
|
||||
values.artifact,
|
||||
)
|
||||
? values["managed-question"]
|
||||
: null,
|
||||
sqlite_selection:
|
||||
values.artifact === "qgis-world-map.gpkg"
|
||||
? values["sqlite-selection"]
|
||||
: null,
|
||||
provider_settings_sha256: sha(JSON.stringify(selectedProviders)),
|
||||
provider_search_path_sha256: sha(
|
||||
selectedProviders.PATH ?? process.env.PATH ?? "",
|
||||
|
||||
@@ -0,0 +1,209 @@
|
||||
# Observe the client boundary before claiming token savings
|
||||
|
||||
Continues [#2016](https://github.com/morluto/rea/issues/2016) after the managed
|
||||
summary [PR #2413](https://github.com/morluto/rea/pull/2413) merged as
|
||||
`82cfcee19f18b0eeae9a330ba04e0411cdf85225`. That opt-in view has verified smaller
|
||||
delivery and complete-default migration, while its original real-model cohort
|
||||
does not establish token savings. This follow-up measures why raw reply bytes
|
||||
and aggregate model usage cannot be treated as interchangeable observations.
|
||||
|
||||
## Concrete diagnostic observation
|
||||
|
||||
An individual real Codex CLI 0.161.0 control uses frozen runtime
|
||||
`ae8051302b5834322fa00cfa31c968ae74ab6ed9`, the unchanged ShareX identity question,
|
||||
requested `gpt-5.6-terra` high and the same two enabled REA tools. A private
|
||||
diagnostic copy removes the original runner's explicit `--ephemeral` flag.
|
||||
Authentication/profile cleanup still runs; only selected tool/usage items are
|
||||
retained after filtering the owned rollout. Its source rollout SHA-256 is
|
||||
`d4df41d6187880d177f00b7533d85ad032c9c4ec502a39e7d5a566180e10461b` and the retained
|
||||
filtered-item SHA-256 is
|
||||
`0d5b4fb67c5674d1ba14ac32ae37e9a72b0368f4d9f5f9f81eb4d13193cda260`.
|
||||
The original whole-copy is removed; account metadata, world state, messages and
|
||||
internal passthrough are not retained in the filtered receipt.
|
||||
|
||||
Despite the requested `direct` configuration label, the observed client uses
|
||||
three JavaScript `exec` calls. First it filters `ALL_TOOLS` with a broad
|
||||
`/REA|reverse|artifact|managed|assembly|metadata/i` expression and prints matching
|
||||
metadata. The client reports truncation of a 164,005-token original output.
|
||||
This is the client's output measurement, not billed input or a complete inventory.
|
||||
Next it invokes complete managed inspection but selects requested normalized
|
||||
identity/classification fields before printing. Finally it invokes native-boundary
|
||||
inspection and prints selected fields. The actual original complete MCP replies
|
||||
remain available in the separate public-event transcript.
|
||||
|
||||
The persisted custom-tool output values are 41,704 / 2,154 / 13,489 UTF-8 bytes.
|
||||
These are the serialized recorded output **values**, not raw wire bytes or exact
|
||||
submitted model requests. The trace reports four per-response usage observations:
|
||||
|
||||
| Response | Input | Cached input | Output |
|
||||
| -------- | -----: | -----------: | -----: |
|
||||
| 1 | 10,400 | 0 | 132 |
|
||||
| 2 | 19,992 | 9,984 | 456 |
|
||||
| 3 | 21,242 | 19,200 | 251 |
|
||||
| 4 | 24,799 | 20,224 | 1,535 |
|
||||
|
||||
Their totals match public turn usage: 76,433 input, 49,408 cached, 27,025 uncached
|
||||
and 2,374 output, with 61.436 seconds observed wall time. Two MCP calls alone do
|
||||
not describe the client's discovery/execution rounds. This proves the observed
|
||||
control can project a complete result before printing it; it does not establish
|
||||
that every earlier control/candidate did the same, or attribute every token
|
||||
difference to discovery. The earlier `direct` label specifies requested client
|
||||
configuration, not a verified absence of an executor.
|
||||
|
||||
Initial transcript-capture attempts retained empty source lists because the
|
||||
original runner intentionally passed `--ephemeral`. A first private flag-removal
|
||||
edit missed its newline and failed; that candidate remains an unsuccessful
|
||||
capture attempt. The later corrected control is an individual diagnostic,
|
||||
not a comparable pair with that candidate. Other exploratory aggregates remain
|
||||
private and are not silently folded into the original four-trial cohort.
|
||||
|
||||
## Reproducible optional capture
|
||||
|
||||
`agent-trial.py --capture-tool-context` omits `--ephemeral` only for that explicitly
|
||||
selected temporary trial. Its default is unchanged. The profile still disappears
|
||||
in `finally`, including when capture fails. The capture reads only its owned
|
||||
`sessions/*.jsonl` records, rejects symlink sources, and retains:
|
||||
|
||||
- Custom/function tool calls and replies, preserving call IDs and exact values.
|
||||
- Per-response token counts and cumulative counts, with original response IDs.
|
||||
- Client token-count events, retaining explicit unknown `info: null`.
|
||||
|
||||
It excludes session/account metadata, world state, user/developer messages,
|
||||
encrypted reasoning, internal passthrough and unknown nested usage fields. It
|
||||
does not copy authentication, configuration or the ambient environment. Receipts
|
||||
record source/retained digests, event counts, malformed-line failures and missing
|
||||
rollouts as unavailable. Capture is an observation of persisted client items,
|
||||
not proof of the exact request accepted by a remote model service.
|
||||
|
||||
Run the same frozen runtime, question, tool selection and capture mode on both
|
||||
sides. Preserve source/settings hashes and separate requested versus observed
|
||||
behavior. Use the existing resource wrapper with unchanged one CPU, 4 GiB hard
|
||||
memory, 3 GiB early-stop, no swap and task/host safeguards. Final physical counters
|
||||
remain unknown when transient cgroups disappear; this instrumentation does not
|
||||
repair that measurement limitation.
|
||||
|
||||
The focused Python tests run through the existing CI `test:python` gate.
|
||||
They cover account/ambient exclusion while preserving caller data, malformed and
|
||||
missing records, foreign symlinks, and capture failure without copied-credential
|
||||
leakage. The cleanup test simulates a CLI and establishes only cleanup semantics;
|
||||
it is not real-model/provider evidence.
|
||||
|
||||
## An explicit discovery-guidance experiment
|
||||
|
||||
`--guide-enabled-tools` is another opt-in experiment flag. It requires a nonempty
|
||||
`--enabled-tools` list and adds the exact corresponding `mcp__rea__...` function
|
||||
names to the temporary developer instructions, requesting only exact-name
|
||||
metadata discovery. It preserves the analyst question, operation order, tool
|
||||
selection and provider inputs. The default instruction is unchanged. Missing or
|
||||
invalid selected tools fail before creating an output/profile. The test gate
|
||||
also verifies the new prerequisite without acquiring a profile.
|
||||
|
||||
Compare guided and unguided runs on the **same** frozen REA runtime with matching
|
||||
capture mode and requested facts. A smaller discovery response or additional
|
||||
hint is not itself a task/token success: grade answers, count all response usage,
|
||||
preserve failed/truncated runs and repeat both orders before recommending this
|
||||
guidance. This is a caller-side experiment, not a new REA tool or provider preset.
|
||||
|
||||
## First committed capture pair
|
||||
|
||||
Checkpoint `9ebdcc458c399b9bdecaaca2ef47422888231181` runs the unchanged question
|
||||
against frozen control `ae8051302b5834322fa00cfa31c968ae74ab6ed9` and candidate
|
||||
`8bc1601ca51e417ca472d90ea0367e9418a1a0f2`, both with capture enabled and no tool
|
||||
guidance. Both captures succeed with 10 retained events, unchanged source bytes,
|
||||
removed profiles, two successful MCP calls and passing selected factual checks.
|
||||
Full grader gates remain false because final physical resource counters are
|
||||
unavailable; factual subsets do not override that limitation.
|
||||
|
||||
| Side | Input | Cached | Uncached | Output | Wall seconds |
|
||||
| ----------------- | -----: | -----: | -------: | -----: | -----------: |
|
||||
| Complete control | 52,656 | 27,136 | 25,520 | 2,015 | 57.736 |
|
||||
| Summary candidate | 56,655 | 29,184 | 27,471 | 1,534 | 55.055 |
|
||||
|
||||
Both clients make two executor calls and three response-usage observations.
|
||||
First they print broad discovered metadata, with truncation. Then each executor
|
||||
sequentially performs both MCP operations, preserving selected-target dependency
|
||||
order. The control prints a 2,086-byte output value; the candidate prints 22,770
|
||||
bytes. Their discovery output values are 41,723 and 41,675 bytes. These are
|
||||
recorded serialized values and remain distinct from raw MCP envelopes, billed
|
||||
tokens and exact submitted requests. Both requested modes are `direct`; the
|
||||
actual executor use is separately observed.
|
||||
|
||||
The candidate's smaller raw metadata reply does not result in a smaller printed
|
||||
client value in this pair. Candidate input/uncached tokens remain higher. The
|
||||
observed projection and round counts are useful constraints for the next
|
||||
experiment; this one pair does not attribute all usage differences to either
|
||||
formatting or discovery, or establish a reliable speedup.
|
||||
|
||||
## Same-runtime guided discovery, repeated in both orders
|
||||
|
||||
Checkpoint `e20bd0e2a905f06bc8c56fe04b0e61f3da88636a` runs four actual Codex trials
|
||||
in unguided/guided/guided/unguided order. All use the **same** frozen candidate
|
||||
runtime, artifact, original question, requested model/reasoning, two selected
|
||||
tools, provider settings and capture mode. Only the explicit discovery-guidance
|
||||
treatment changes. Every trial selects artifact summary and makes the same two
|
||||
successful MCP calls in dependency order; source bytes stay unchanged, captures
|
||||
succeed and profiles are removed.
|
||||
|
||||
| Trial | Input | Cached | Uncached | Output | Seconds | Observed responses |
|
||||
| ---------- | ------: | -----: | -------: | -----: | ------: | -----------------: |
|
||||
| Unguided 1 | 77,251 | 50,432 | 26,819 | 1,931 | 62.066 | 4 |
|
||||
| Guided 1 | 39,175 | 23,040 | 16,135 | 1,763 | 55.658 | 3 |
|
||||
| Guided 2 | 58,336 | 38,144 | 20,192 | 1,626 | 50.685 | 4 |
|
||||
| Unguided 2 | 103,301 | 72,704 | 30,597 | 1,983 | 60.336 | 5 |
|
||||
|
||||
Both guided discovery output values repeat at 12,336 bytes without truncation,
|
||||
versus 41,709 bytes with truncation on both unguided trials. The recorded
|
||||
executor/output usage explains observable discovery and round differences; it
|
||||
does not certify exact remote request contents or attribute every token to the
|
||||
hint. CPU samples are 6.303–6.554 seconds and peak family memory 684,519,424–
|
||||
712,028,160 bytes, with no guard abort. Final resource counters remain unknown.
|
||||
These samples do not establish a memory or CPU improvement.
|
||||
|
||||
Unguided 1 omits `classification.container` and fails the selected factual gate;
|
||||
it is not a fully acceptable comparison answer. Guided 1 and Unguided 2 pass
|
||||
that gate. Guided 2 initially fails exact coverage-object equality because it
|
||||
adds `module_ref_count: 0` and `pinvoke_import_count: 0` to the explicitly named
|
||||
native scope. Both numbers exactly match the producer's returned arrays. The
|
||||
grader now explicitly verifies only these two known additional counts, including
|
||||
integer type, while still requiring all original coverage facts. Unknown extra
|
||||
claims, conflicting aliases, wrong counts, boolean/string counts and omitted
|
||||
issues continue to fail. Source coverage remains unchanged; returned-row counts
|
||||
do not establish global absence. Original failure receipts remain preserved;
|
||||
the regrade separately records its new source hash. Guided 2 then passes selected
|
||||
facts, but every **full** gate still fails unknown final physical resource checks.
|
||||
|
||||
The second comparison has two factually acceptable answers: observed guided input
|
||||
falls about 43.5%, uncached input about 34.0%, and wall time about 16.0%. The first
|
||||
comparison has an incomplete unguided answer and is reported without counting it
|
||||
as equivalent completed work. Two trials per treatment on one question/application
|
||||
remain a preliminary signal, not a general default recommendation. The hint stays
|
||||
opt-in; wider questions/domains, count/coverage representation and client round
|
||||
behavior need further verification before promoting it into general guidance.
|
||||
|
||||
## Real Android follow-up: the primary outcomes are mixed
|
||||
|
||||
Checkpoint `22834c7d73056e2480b426848f58c31af0c307bc` checks the pinned real
|
||||
NewPipe 0.29.1 APK using actual configured JADX 0.7.1/Java, the original Android
|
||||
question and the same frozen candidate runtime. First a fresh SDK workflow
|
||||
validates three live advertised operations and produces the comparison facts:
|
||||
package/version identity, every case-sensitive MainActivity match and all
|
||||
reported onCreate entries, preserving signatures, overload indices and null DEX
|
||||
descriptors. It takes 26.961 seconds; the owned scope samples 1,234,735,104 bytes.
|
||||
|
||||
Both subsequent actual Codex trials use the same three selected tool names,
|
||||
capture mode, model/reasoning, question and provider settings. Only discovery
|
||||
guidance changes. Each makes three successful MCP calls and five observed model
|
||||
responses, passes selected facts against the fresh actual producer, preserves
|
||||
source bytes and removes its temporary profile. No APK execution is requested.
|
||||
|
||||
| Trial | Input | Cached | Uncached | Output | Wall seconds | Sampled peak bytes |
|
||||
| -------- | -----: | -----: | -------: | -----: | -----------: | -----------------: |
|
||||
| Unguided | 99,913 | 80,640 | 19,273 | 1,527 | 66.797 | 1,405,829,120 |
|
||||
| Guided | 89,220 | 62,464 | 26,756 | 2,142 | 76.936 | 1,345,576,960 |
|
||||
|
||||
Total input falls about 10.7%, but uncached input increases and wall time is
|
||||
longer. One pair does not demonstrate improved Android performance or memory.
|
||||
No guard abort occurs; final counters remain unknown and full resource gates
|
||||
remain false. This second domain is a concrete counterexample to generalizing
|
||||
the favorable managed comparison. Keep both flags optional and continue
|
||||
task-specific experiments before changing installed/default agent guidance.
|
||||
@@ -0,0 +1,208 @@
|
||||
# Task-scoped preset lifecycle verification
|
||||
|
||||
Continues [#2016](https://github.com/morluto/rea/issues/2016) after
|
||||
[#2308](https://github.com/morluto/rea/pull/2308) merged. This increment verifies
|
||||
existing caller-selected Ghidra settings and immutable JSON snapshots through
|
||||
actual public CLI/MCP workflows. Production defaults, provider selection and
|
||||
public contracts remain unchanged; an engine database cache is still deferred.
|
||||
|
||||
Runtime baseline is main `f76163c2467db623f9de0bda757c9d8610ca9565`, built with
|
||||
Node 24.18.0 and matching locked dependencies. Real Ghidra 12.1.4 uses the
|
||||
existing caller-supplied JDK 21 and 768 MiB Java heap under one CPU. No target
|
||||
execution, annotation or attached GUI database operation is requested.
|
||||
|
||||
## Small shipped input and profile boundaries
|
||||
|
||||
The first input is the installed `/usr/bin/true`, 35,664 bytes, SHA-256
|
||||
`c79bf44242829108e323378531f4ac839513ca1fba45efd6583643526e1e9fd2`.
|
||||
The local package database reports Debian `coreutils` `9.1-1` `amd64`; these are
|
||||
installed-artifact observations, not independent archive authenticity proof.
|
||||
The target itself is never invoked, including for version discovery. REA's
|
||||
actual `list_procedures` chooses an internal, non-thunk procedure at `0x102000`.
|
||||
|
||||
The default profile reports `analyzer_preset: "ghidra-default"`. The selected
|
||||
alternative reports `ghidra-default+disabled:Decompiler Switch Analysis` and
|
||||
the exact disabled name in its analysis options. Their profile digests differ.
|
||||
The selected initializer's complete normalized facts agree, while the profile
|
||||
and provenance remain distinct. This does not establish equivalence for other
|
||||
procedures, call graphs or applications. The earlier jq experiment found a
|
||||
callee body difference; the existing default remains the recommendation.
|
||||
|
||||
The public [snapshot verifier](verify-preset-snapshot.mjs) checks:
|
||||
|
||||
- The default `analyze_function` actually produces a stored query entry and an
|
||||
authenticated original Evidence record in the exported snapshot.
|
||||
- The same profile returns identical task facts, subject, parameters and profile
|
||||
through a fresh MCP connection. Its limitations explicitly mark snapshot reuse;
|
||||
its distinct response identity does not masquerade as the original record.
|
||||
- A disabled profile refuses the default snapshot before headless Ghidra starts,
|
||||
then successfully analyzes the same caller-selected target with its own profile.
|
||||
- Restoring the default refuses the disabled snapshot and accepts the original
|
||||
default snapshot. Original snapshot/source bytes stay unchanged.
|
||||
- The existing CLI `function --provider ghidra --snapshot ... --json` returns
|
||||
the original cached Evidence identity and exact task facts without modifying
|
||||
the source snapshot.
|
||||
|
||||
Actual `strace` records the known headless launch command once for fresh default
|
||||
and once for fresh disabled analysis, and zero times for either rejection or
|
||||
accepted reuse. The audit covers that known command, not an arbitrary execution
|
||||
or network-absence guarantee. Source bytes and snapshot bytes remain unchanged.
|
||||
|
||||
## Real cancellation and cleanup retry
|
||||
|
||||
The [lifecycle verifier](verify-preset-lifecycle.mjs) observes actual pending
|
||||
startup before the bridge endpoint exists. It verifies the JVM's run token,
|
||||
launcher relationship and native process start identity with production process
|
||||
inspection primitives, then cancels through the SDK's normal request signal.
|
||||
Read-only token sweeps confirm cleanup, including processes outside the old
|
||||
parent chain; owned runtime/socket directories are removed. The selected target
|
||||
remains open and a fresh provider query succeeds.
|
||||
|
||||
REA preserves the logical run ID when reimporting that selected target. A new
|
||||
native process start identity proves the new generation; requiring a new logical
|
||||
ID would test a different contract. No PID/path pair alone establishes ownership.
|
||||
|
||||
After recovery, the verifier removes write permission only from the verified,
|
||||
user-owned temporary Ghidra project directory. The actual public `close_binary`
|
||||
reports `cleanup_incomplete` with the retained resource and `EACCES` reason.
|
||||
Permissions are restored, a second close succeeds, and token sweeps/root removal
|
||||
confirm cleanup. The original executable is unchanged. This is a controlled real
|
||||
filesystem failure, not a recording provider simulation. The error does not
|
||||
authorize automatic retry before fixing its cause.
|
||||
|
||||
These observations pass for both the default and disabled-switch profiles on the
|
||||
small input. The source verifier preserves verification and restoration/close
|
||||
failures together, always attempts cleanup, validates the manifest at its read
|
||||
boundary and publishes success only after its cleanup callbacks complete.
|
||||
|
||||
## Failed harness assumptions and measurement limits
|
||||
|
||||
Keep the initial probes and their corrections outside Git:
|
||||
|
||||
- A primitive-only `procedure_info` snapshot contained no replay entries. Equal
|
||||
content-addressed output after another analysis was not proof of cache reuse.
|
||||
The corrected probe requires a stored `analyze_function` entry, a cache-reuse
|
||||
marker and actual headless launch traces.
|
||||
- After cancellation, activity was `not_observed`, rather than `idle`, because
|
||||
no provider activity snapshot remained. Preserve that unknown; successful fresh
|
||||
work proves recovery without relabeling absent metadata.
|
||||
- Reimport preserved the logical run ID. Native start identity, not that logical
|
||||
ID, establishes the new process generation.
|
||||
|
||||
These were verification assumptions, not newly demonstrated REA defects. No
|
||||
unrelated production repair is made. Existing defects remain in the requested
|
||||
private `/tmp` record.
|
||||
|
||||
The initial complete snapshot/profile/CLI source run takes 91.838 seconds and
|
||||
samples 1,180,246,016 peak family bytes. The real default/disabled cancellation
|
||||
and permission-retry prototype takes 66.977 seconds and samples 1,040,846,848
|
||||
bytes. Their last-observed CPU totals are 90.486 and 65.542 seconds respectively.
|
||||
Both use one CPU, 4 GiB hard memory, 3 GiB early-stop, no swap and 192 tasks.
|
||||
No scope abort occurs, but final counters disappear at teardown and remain
|
||||
unverified. Sampled values are not final OOM/limit-event proof.
|
||||
|
||||
These are lifecycle verification costs. Discovery, profile inspection, rejection,
|
||||
cleanup and syscall tracing make their call sequences different. They are not
|
||||
paired completed-question performance trials, and contain no model token counts.
|
||||
Explicit snapshot reuse is existing REA behavior, not a new speedup introduced by
|
||||
these verifiers. Broader native inputs and failure scenarios remain follow-ups.
|
||||
|
||||
## Shipped jq continuation
|
||||
|
||||
The same source snapshot verifier additionally passes on the pinned public jq
|
||||
binary, SHA-256
|
||||
`b1c22172dd303f3be49e935aa56aa48a8b7a46e0bc838b4997d3bb451495870f`.
|
||||
Procedure `0x403a50` comes from the earlier actual REA entry observation, rather
|
||||
than a newly guessed file offset. Default and disabled profiles each perform a
|
||||
fresh headless analysis. Both profile mismatch directions reject before a
|
||||
headless launch, both valid default reuses return exact task facts with zero
|
||||
headless launches, and CLI reuse retains the original Evidence ID and snapshot
|
||||
bytes. The selected entry agrees across profiles; the earlier differing callee
|
||||
still prevents a general equivalence or fast-preset recommendation.
|
||||
|
||||
Default/disabled fresh `analyze_function` calls take 294.525/298.621 seconds in
|
||||
this verification; default snapshot calls take 0.222 and 0.168 seconds. The CLI
|
||||
reuse takes 2.899 seconds. These are calls within different verification sequences,
|
||||
not repeated completed-question performance cohorts. They demonstrate the existing
|
||||
explicit snapshot mechanism without an engine database cache; they do not measure
|
||||
model tokens or introduce a new optimization.
|
||||
|
||||
The complete jq verification takes 624.537 seconds, samples 1,218,355,200 peak
|
||||
family bytes and observes 567.198 CPU seconds before teardown. The final resource
|
||||
counters remain unknown. The source verifier SHA-256 is
|
||||
`42060686f9633da06bb5f257fa5c70f588e66569cf0befebe6a1a13f43bf0e88`;
|
||||
the initial lifecycle verifier hash is
|
||||
`b1680f4f506ef9328d1beabb4b1fbe97c911b3f06986857f0a59a4b97a3b61d3`.
|
||||
Both hashes also match completed small-input source runs, including the final
|
||||
cleanup/error-preservation implementation. Runtime production remains the pinned
|
||||
main baseline; the follow-up changes only experiment sources and documentation.
|
||||
|
||||
## Invalid analyzer failure and healthy peers
|
||||
|
||||
Real MCP and CLI requests with `REA_GHIDRA_DISABLED_ANALYZERS` set to
|
||||
`REA Nonexistent Analyzer 2016` both report `execution_failure`. Their diagnostics
|
||||
preserve the exact unavailable analyzer name, selected provider, operation,
|
||||
target identity and Ghidra pre-script failure. MCP still answers a bundled EVM
|
||||
peer request and closes the selected native target; original target bytes stay
|
||||
unchanged and the owned Ghidra runtime directory is removed.
|
||||
|
||||
The first healthy-peer probe assumed ELF layout support was built in. This main
|
||||
requires an explicitly supplied pwntools interpreter for that format and correctly
|
||||
reports `provider_unavailable`; none was configured. The corrected probe uses
|
||||
the already verified bundled EVM peer. No installer or provider fallback is added.
|
||||
|
||||
The first CLI observer also looked inside `error` as if it had MCP's shape. The
|
||||
actual CLI envelope has an error string and sibling `code`, `message`, `details`
|
||||
and other fields. Regrading its saved complete reply proves the named diagnostics
|
||||
are preserved. The original incorrect assertion/report remains archived; no
|
||||
provider rerun or production repair is needed for that correction.
|
||||
|
||||
The invalid-name behavior is reproducible with the existing CLI:
|
||||
|
||||
```sh
|
||||
REA_GHIDRA_DISABLED_ANALYZERS='REA Nonexistent Analyzer 2016' \
|
||||
node ABS_RUNTIME/scripts/rea.mjs function ABS_TRUE_BINARY 0x102000 \
|
||||
--provider ghidra --json
|
||||
```
|
||||
|
||||
Use the address actually observed by REA for a different target. The deliberately
|
||||
invalid name is expected to fail; fixing the setting is required before retrying.
|
||||
|
||||
## Reproduction
|
||||
|
||||
Build the pinned runtime and supply an explicit string-valued provider settings
|
||||
JSON, real Ghidra/JDK, a caller-supplied Linux strace executable and a fresh private
|
||||
output directory. The snapshot verifier streams target hashes and stores actual
|
||||
SDK/CLI replies, profile commitments and exec traces outside the checkout:
|
||||
|
||||
```sh
|
||||
node experiments/analysis-efficiency/verify-preset-snapshot.mjs \
|
||||
--runtime-root ABS_RUNTIME --revision SHA --path ABS_NATIVE_TARGET \
|
||||
--providers ABS_SETTINGS_JSON --strace ABS_STRACE --output NEW_ABS_DIR
|
||||
```
|
||||
|
||||
Select `--procedure` from actual REA evidence when the question already identifies
|
||||
one; otherwise the verifier uses the actual procedure list. `--timeout-ms` is an
|
||||
explicit observation window; `--disabled-analyzers` uses the existing setting.
|
||||
|
||||
Run cancellation/permission retry as a non-root Linux user:
|
||||
|
||||
```sh
|
||||
node experiments/analysis-efficiency/verify-preset-lifecycle.mjs \
|
||||
--runtime-root ABS_RUNTIME --revision SHA --path ABS_NATIVE_TARGET \
|
||||
--providers ABS_SETTINGS_JSON --output NEW_ABS_DIR
|
||||
```
|
||||
|
||||
Runtime revision is caller-supplied; keep the built/frozen runtime receipt alongside
|
||||
the verifier source fingerprint. Target bytes, provider projects, raw JSON, traces
|
||||
and profiles are not committed. The scripts acquire no dependencies or change
|
||||
the user's installed configuration.
|
||||
|
||||
## Final cleanup review
|
||||
|
||||
The final review adds an independent filesystem check that the recovered
|
||||
bridge socket directory is absent after permission restoration and close retry,
|
||||
and preserves its ownership manifest in the receipt. Both default and disabled
|
||||
profiles pass this related real workflow again. Its source SHA-256 is
|
||||
`f5d29efdb056032072b99ac1e6fe1d6d2099bbe84c324c75032bd0178bb747c5`. The previous observations and source hashes stay archived;
|
||||
the jq/model/provider matrices were not repeated for this observer change.
|
||||
@@ -0,0 +1,113 @@
|
||||
# Process trace validation
|
||||
|
||||
A 6,000-event directed cycle previously overflowed the JavaScript stack inside
|
||||
the schema refinement. Cycle detection now uses iterative topological traversal.
|
||||
Cyclic inputs retain independent declaration diagnostics without expanding all
|
||||
event pairs. Unordered declarations retain group membership, with one diagnostic
|
||||
per overlapping or conflicting group, instead of allocating a pair set and
|
||||
repeating the same error for every pair.
|
||||
|
||||
A unique topological order proves that every event pair is ordered, allowing
|
||||
rank comparisons without retaining transitive closure. Other acyclic graphs
|
||||
retain only one origin's forward and backward reachability sets at a time.
|
||||
Complete pair validation preserves the missing count and first declaration-order
|
||||
missing pair. General partial orders still require quadratic pair examination
|
||||
and repeated graph traversal; they trade CPU time for bounded retained state.
|
||||
|
||||
## Initial cycle fix comparison
|
||||
|
||||
Flameox ran three randomized paired blocks (seed 917), each limited to 20 seconds
|
||||
and 512 MiB. Node 24.18.0 used a 384-MiB old-space limit on macOS arm64. Each
|
||||
process validated the same 2,500-event acyclic chain four times, then checked the
|
||||
known 6,000-event cycle failure with per-event not-before constraints and one
|
||||
unordered group containing every event. All six executions passed the embedded answer
|
||||
digest and expected rejection checks.
|
||||
|
||||
The baseline source is from `bb540826d`; the candidate is the initial cycle fix
|
||||
in `10935b7b1`, which still memoized reachability sets for every queried origin.
|
||||
Both modules had TypeScript annotations removed with Node's identical built-in
|
||||
transform and used the same copied runtime dependencies. This isolates schema
|
||||
validation rather than comparing complete packaged CLI/MCP processes.
|
||||
|
||||
| Median measurement | Baseline | Candidate |
|
||||
| -------------------------------------------- | -------: | --------: |
|
||||
| First validation elapsed milliseconds | 335.80 | 177.25 |
|
||||
| Warm validation elapsed milliseconds | 313.36 | 165.23 |
|
||||
| First validation CPU milliseconds | 366.85 | 218.39 |
|
||||
| Warm validation CPU milliseconds | 335.32 | 194.08 |
|
||||
| Whole-process elapsed milliseconds | 1,344.13 | 787.00 |
|
||||
| Process CPU milliseconds before final report | 1,457.67 | 941.05 |
|
||||
| Peak process-family MiB | 328.36 | 395.52 |
|
||||
|
||||
The input is identical: 233,095 JSON bytes, estimated at 77,010 tokens by tokenx
|
||||
1.3.0. The valid parsed result digest is
|
||||
`28bd99ce83f0da8e45600ef20f114cb52bb26b5f5e6bfc1601ca3e712eaf3147`.
|
||||
Actual agent tokens, catalog injection, transport, model reasoning and provider
|
||||
costs were not measured. This is a component measurement, not an end-to-end
|
||||
investigation speed claim.
|
||||
|
||||
First validation excludes module import and fixture preparation; warm runs reuse
|
||||
the process but create fresh per-validation graph state. Whole-process elapsed
|
||||
includes startup, imports, all four validations, the cyclic-input check, token
|
||||
estimation and reporting. CPU includes startup through metrics publication;
|
||||
peak RSS covers the complete single-process workload, with no children. Host
|
||||
caches were not flushed. Peak memory increased in this cohort. Its retained
|
||||
valid-order closure still required quadratic memory, prompting the follow-up.
|
||||
Three pairs describe this fixture only.
|
||||
|
||||
Preserved Flameox evidence:
|
||||
`d6211639446e8b6ec2e82636913eaf471def96673e28f03e9c7f98bec07f8e13`;
|
||||
analysis `e663b6bd8f5693975542d5b5e2b18437b3c167a0be939bb7386eb68549936231`.
|
||||
An earlier cohort lost modules during another checkout build; its three failed
|
||||
executions and timing results are excluded. The successful cohort uses an
|
||||
isolated runtime copy. Source-module SHA-256 values are
|
||||
`a7729faa2183b0e493b9ac4fc55f6feaee5b2e5294be13d1b29b57a44b899788`
|
||||
(baseline) and `725409ce84902125941251037a8a214ccc3c1eec577e484c164c361ca580d693`
|
||||
(candidate).
|
||||
|
||||
## Bounded retention follow-up
|
||||
|
||||
Flameox compared `10935b7b1` with the rank/streaming implementation in three
|
||||
randomized paired blocks (seed 624), under the same host, Node, heap, memory and
|
||||
20-second execution bounds. Each execution validated the chain above four times,
|
||||
then a 1,200-event partial order with 600 unordered two-event layers and all four
|
||||
edges between consecutive layers four times, then the same cyclic failure.
|
||||
All six executions passed both parsed-answer digests and the cycle diagnostic.
|
||||
Runtime dependencies were copied and isolated from concurrent checkout builds.
|
||||
|
||||
| Median measurement | Memoized closure | Rank/streaming |
|
||||
| ---------------------------------------- | ---------------: | -------------: |
|
||||
| Chain first elapsed milliseconds | 166.11 | 11.37 |
|
||||
| Chain warm elapsed milliseconds | 154.25 | 5.32 |
|
||||
| Chain first CPU milliseconds | 210.13 | 24.38 |
|
||||
| Chain warm CPU milliseconds | 184.76 | 16.84 |
|
||||
| Partial order first elapsed milliseconds | 49.44 | 116.54 |
|
||||
| Partial order warm elapsed milliseconds | 50.67 | 110.29 |
|
||||
| Partial order first CPU milliseconds | 72.47 | 137.69 |
|
||||
| Partial order warm CPU milliseconds | 70.65 | 121.77 |
|
||||
| Capture process elapsed milliseconds | 901.24 | 557.30 |
|
||||
| Process CPU milliseconds before report | 1,101.12 | 665.03 |
|
||||
| Peak process-family MiB | 391.13 | 127.72 |
|
||||
|
||||
First samples exclude module import and fixture preparation. Partial-order first
|
||||
samples follow four chain validations, so they are first use of that fixture
|
||||
within an already running process. Warm medians pool the three subsequent samples
|
||||
from each of three processes. CPU and peak RSS cover the combined workload;
|
||||
capture elapsed additionally includes Flameox capture startup and overhead.
|
||||
Host caches were not flushed. The partial-order fixture is slower after this
|
||||
change; these results establish a retention tradeoff, not universal acceleration.
|
||||
|
||||
The partial-order input is 165,590 JSON bytes, estimated at 52,783 tokens by tokenx
|
||||
1.3.0; its parsed-answer SHA-256 is
|
||||
`41d06e593813e497221c36793bf00d5fece7055081895b5f6c123ba6ad9a8227`.
|
||||
Chain bytes, token estimate and answer digest remain unchanged. Actual agent
|
||||
tokens, transport, catalog injection, model reasoning and provider costs remain
|
||||
unmeasured. Both cohorts are component measurements, not complete investigations.
|
||||
|
||||
Preserved evidence:
|
||||
`e29086af99afd8e67cbbd92e43d960ea40f73b4d318ea0cbca60f493575462b8`;
|
||||
analysis `086133516f2acf031032d357a3ddb1ac611ce4f12556a24065a2becd567c5f7b`.
|
||||
The baseline module SHA-256 is
|
||||
`725409ce84902125941251037a8a214ccc3c1eec577e484c164c361ca580d693`;
|
||||
the candidate module SHA-256 is
|
||||
`2109f1096406e9ecac8bbf4f7ae8abb5113404790a9382487285d2831fcddf26`.
|
||||
@@ -0,0 +1 @@
|
||||
Statically inspect the published Uniswap v3-core 1.0.1 compiler runtime template at {{artifact_path}} using only REA inspect_evm_interface with encoding hex. Report original carrier SHA-256/byte count, decoded bytecode SHA-256/byte count, every inferred function selector with bytecode_offset, dispatch, inferred_arguments and inferred_state_mutability exactly including null. Preserve evidence_kind, discovery_completeness, bytecode kind/hardfork/deployment_authenticity, runtime_execution, real Evidence IDs and limitations. The unchanged template came from the official published compiler artifact; constructor-dependent values and immutable-reference metadata are not established. Do not infer a live deployed instance, original source signatures or runtime behavior. No RPC, transactions, target execution, shell/file tools or other agents. Final JSON fields: artifact_sha256, artifact_bytes, bytecode_sha256, bytecode_bytes, functions, evidence_kind, discovery_completeness, bytecode_kind, hardfork, deployment_authenticity, runtime_execution, evidence_ids, limitations.
|
||||
@@ -0,0 +1 @@
|
||||
Statically inspect the shipped Flutter APK at {{artifact_path}} using only the registered REA inspect_dart_aot tool. Use its default ABI selection. Report target identity, chosen ABI, every libapp ELF/snapshot field exactly as reported including null, original string-pool coverage, all three counts ending in _at_least, printable_run_count, and returned_counts for package_uris, dart_uris and class_like_tokens. Compute returned_counts from the returned arrays if needed. Preserve original overall coverage and producer limitations exactly, including order. Counts ending in _at_least are lower bounds and class-like tokens are byte-pattern observations, not recovered typed classes. Do not return individual strings or infer Dart SDK versions, class semantics, runtime behavior or complete inventory. Final answer must be JSON with artifact_sha256, target, abi, libapp, string_pool, coverage, limitations, evidence_ids. Use actual response Evidence IDs. Do not execute the APK, use shell or host file tools, access the network, or call other agents. Stop when these facts are supported.
|
||||
@@ -0,0 +1 @@
|
||||
Statically inspect the supplied QGIS world-map GeoPackage at {{artifact_path}} using only the registered REA inspect_sqlite_database tool. Select the ordinary countries table with row_limit 1 and its default selected_table schema scope. Report the complete returned schema exactly as reported, including SQL, every declared column, indexes, views, triggers, completeness and explicit nulls. For the one returned sample, report only fid, NAME and ISO_A3 in that order, preserving the original tagged SQLite cells exactly: decimal integer strings, text values with bytes_hex and encoding, and SQL null. Preserve row_limit, returned_rows, truncated and order; row order is unspecified and this is not a complete country inventory. Preserve original artifact identity, wal including null, engine, database_encoding, both normalized-result producer limitations and outer Evidence limitations exactly including order. Do not return geometry blobs or unrequested sample cells, interpret boundaries/political labels as current facts, or infer missing values. Final JSON fields: artifact_sha256, artifact, wal, engine, database_encoding, schema, rows, producer_limitations, limitations, evidence_ids. Include real response Evidence IDs. Do not use shell or host file tools, network, other agents or execute QGIS. Stop once these facts are supported.
|
||||
@@ -0,0 +1,103 @@
|
||||
# Resource counters at payload completion
|
||||
|
||||
Continues [#2016](https://github.com/morluto/rea/issues/2016) from merged main
|
||||
`4f4f6cd7b5d7a400b90772a138d7b9f942c2a556`. A transient cgroup can disappear
|
||||
before the existing supervisor reads terminal counters. Earlier reports keep
|
||||
those counters unknown; sampled zero events are not final no-OOM proof.
|
||||
|
||||
An optional `resource-run.py --observe-payload-exit -- COMMAND...` adds one small
|
||||
observer inside the same owned scope. It waits for the direct payload, records
|
||||
kernel `memory.peak`, CPU and memory/task events, and reports remaining processes
|
||||
before acknowledging completion to the outer supervisor. The observer remains
|
||||
alive until receipt acknowledgement, bounded by the existing ten-second cleanup
|
||||
interval. Payload stdio and process exit status remain observable.
|
||||
|
||||
The receipt phase is **payload-exited-observer-alive**. The supervisor validates
|
||||
the owned-unit token, numeric fields, cgroup identity, live observer membership,
|
||||
payload exit status and absence of additional processes before marking that
|
||||
observation verified. Wrong-owner, malformed, missing or inconsistent receipts
|
||||
stay unknown; lingering actors stay explicit and cannot certify cleanup.
|
||||
Unexpected metadata is omitted rather than copying an inherited environment.
|
||||
|
||||
This is not a final whole-scope counter: the observer still has to exit. Existing
|
||||
`resource_events_final` and `resource_counters_verified` retain their original
|
||||
meaning and unknown state. The grader's existing full resource gate is unchanged.
|
||||
Earlier runs and source/runtime hashes are not relabeled. Observer CPU/memory and
|
||||
acknowledgement time are included in this new measurement mode; comparable runs
|
||||
must use the same mode. Default commands do not start an observer or add fields.
|
||||
|
||||
One CPU, the unchanged 3 GiB early-stop/4 GiB hard ceiling, no swap, task limits
|
||||
and host-available-memory safeguards remain in force. No resource limit is raised.
|
||||
The opt-in mode requires at least two task slots for observer plus payload;
|
||||
smaller caller-selected ceilings fail before process acquisition. Default mode
|
||||
continues to accept a one-task ceiling.
|
||||
Cleanup uses the uniquely owned systemd scope, without signaling a process merely
|
||||
because a recorded PID exists. Failed scope cleanup retains the named owner.
|
||||
|
||||
## Focused verification
|
||||
|
||||
Five semantic Python cases cover acquisition-free capacity rejection, nonzero exit/kernel facts, missing or malformed
|
||||
receipts, wrong ownership, integer types, contradictory cleanup, lingering actors,
|
||||
foreign symlinks and exclusion of unrelated metadata. They join the existing
|
||||
Python gate; simulated receipts do not establish actual cgroup behavior.
|
||||
|
||||
Actual Linux cgroup/systemd probes additionally verify:
|
||||
|
||||
- Default mode keeps its receipt shape and success status.
|
||||
- Success and exit-code 3 preserve the payload outcome and verify a sole observer.
|
||||
- A bounded 16 MiB Node allocation yields a kernel peak, including work that the
|
||||
100 ms sampler can miss.
|
||||
- A payload leaving a child reports both actors and refuses verification; the
|
||||
supervisor retires the owned scope.
|
||||
- A caller-selected task ceiling of 4 triggers the existing task-limit early stop
|
||||
with exit 125 and an unverified observation.
|
||||
- Interrupting a confirmed-live owned controller preserves failure and retires
|
||||
its scope; missing completion counters are not replaced with zeros.
|
||||
|
||||
These are metering/lifecycle probes, not a new model/provider performance claim.
|
||||
Actual REA workflows and repeated observer-overhead measurements remain necessary
|
||||
before relying on the new mode for domain comparisons. Raw receipts remain in
|
||||
the private experiment lab outside Git.
|
||||
|
||||
## Actual REA and model observations
|
||||
|
||||
Checkpoint `d1f2cfe76df28e6effe973123f2e19313f6d0d4b` repeats real SDK workflows
|
||||
against the unchanged frozen candidate `8bc1601ca51e417ca472d90ea0367e9418a1a0f2`
|
||||
(runtime file-table SHA-256
|
||||
`9e7380f537301bf0cfe2926f5619011af848af7c220750e75189498e65d825e1`).
|
||||
The resource supervisor changes; the compiled REA/provider algorithms do not.
|
||||
Managed pairs use default/observer/observer/default order with the same question,
|
||||
settings, two public calls and identical selected facts. Raw receipts remain
|
||||
private and new measurements do not replace historical resource unknowns.
|
||||
|
||||
| Workflow | SDK milliseconds | Scope seconds | Sampled peak bytes | Kernel peak at payload exit | Payload CPU microseconds |
|
||||
| -------------------- | ---------------: | ------------: | -----------------: | --------------------------: | -----------------------: |
|
||||
| Managed default 1 | 5,168.485 | 11.814 | 525,873,152 | unknown | unknown |
|
||||
| Managed observer 1 | 4,156.708 | 5.358 | 527,777,792 | 535,724,032 | 5,195,457 |
|
||||
| Managed observer 2 | 3,810.362 | 5.056 | 538,742,784 | 539,529,216 | 4,818,497 |
|
||||
| Managed default 2 | 4,263.993 | 5.360 | 526,032,896 | unknown | unknown |
|
||||
| SQLite WASM observer | 6,733.216 | 8.087 | 777,199,616 | 777,199,616 | 7,830,130 |
|
||||
| NewPipe APK observer | 23,162.195 | 24.600 | 1,286,586,368 | 1,287,548,928 | 23,965,627 |
|
||||
|
||||
The WASM and APK runs use actual configured WABT and JADX/Java. Every observer
|
||||
receipt verifies a sole remaining observer, successful payload exit and the
|
||||
expected owned group. No guard abort occurs. The initial default scope timing is
|
||||
an outlier relative to its SDK workflow and later scopes; its cause is not
|
||||
established. Two managed pairs are insufficient to characterize observer overhead
|
||||
or establish a speed/memory improvement. Kernel peak and CPU include the observer.
|
||||
|
||||
One actual Codex CLI 0.161.0 trial uses the original managed question, exact-name
|
||||
guidance, context capture and requested `gpt-5.6-terra` high. It passes selected
|
||||
facts, preserves source bytes, removes its profile and makes three successful MCP
|
||||
calls (including an additional artifact inspection). Reported usage is 61,173
|
||||
input, 48,128 cached, 13,045 uncached and 2,678 output tokens; wall time is 125.798
|
||||
seconds. The payload-exit witness records 812,896,256 kernel peak bytes and
|
||||
8,566,029 CPU microseconds with only the observer alive and zero recorded
|
||||
memory/task limit events at that phase. This confirms the completed model/MCP
|
||||
payload boundary, not a token/time benefit or final whole-scope no-OOM proof.
|
||||
Its full existing grader remains false for unknown terminal physical counters.
|
||||
|
||||
The subsequent review adds supervisor/helper source SHA-256 to opt-in receipts
|
||||
before process acquisition, without adding those fields to default receipts.
|
||||
This distinguishes measurement implementations independently of frozen runtime
|
||||
identity and keeps the observer's phase explicit.
|
||||
@@ -1,14 +1,18 @@
|
||||
"""Run one owned experiment in a bounded, transient Linux systemd scope."""
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
import uuid
|
||||
|
||||
from resource_payload import read_observation
|
||||
|
||||
|
||||
def available_bytes():
|
||||
for line in Path('/proc/meminfo').read_text().splitlines():
|
||||
@@ -64,10 +68,14 @@ def stop_owned_scope(unit, process, group):
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument('--task-ceiling', type=int, default=192)
|
||||
parser.add_argument('--observe-payload-exit', action='store_true',
|
||||
help='Observe payload-exit counters before the cgroup observer exits')
|
||||
parser.add_argument('command', nargs=argparse.REMAINDER)
|
||||
options = parser.parse_args()
|
||||
if options.task_ceiling <= 0:
|
||||
parser.error('--task-ceiling must be positive')
|
||||
if options.observe_payload_exit and options.task_ceiling < 2:
|
||||
parser.error('--observe-payload-exit requires capacity for the observer and payload (at least 2 tasks)')
|
||||
args = options.command
|
||||
if args and args[0] == '--':
|
||||
args = args[1:]
|
||||
@@ -78,22 +86,41 @@ def main():
|
||||
raise SystemExit('Refusing experiment: less than 8 GiB available')
|
||||
unit = 'rea-efficiency-' + uuid.uuid4().hex + '.scope'
|
||||
cpu = max(os.sched_getaffinity(0))
|
||||
receipt_root = tempfile.TemporaryDirectory(prefix='rea-payload-counters-') if options.observe_payload_exit else None
|
||||
receipt_path = Path(receipt_root.name) / 'observation.json' if receipt_root else None
|
||||
release_path = Path(receipt_root.name) / 'release' if receipt_root else None
|
||||
payload = args if receipt_root is None else [
|
||||
sys.executable, str(Path(__file__).with_name('resource_payload.py')),
|
||||
str(receipt_path), str(release_path), unit, *args,
|
||||
]
|
||||
command = [
|
||||
'systemd-run', '--user', '--scope', '--unit=' + unit,
|
||||
'-p', 'MemoryMax=4G', '-p', 'MemoryHigh=3G', '-p', 'MemorySwapMax=0',
|
||||
'-p', 'CPUQuota=100%', '-p', 'TasksMax=' + str(options.task_ceiling),
|
||||
'taskset', '-c', str(cpu), *args,
|
||||
'taskset', '-c', str(cpu), *payload,
|
||||
]
|
||||
process = subprocess.Popen(command)
|
||||
observer_sources = {
|
||||
name: hashlib.sha256(Path(__file__).with_name(name).read_bytes()).hexdigest()
|
||||
for name in ['resource-run.py', 'resource_payload.py']
|
||||
} if options.observe_payload_exit else None
|
||||
try:
|
||||
process = subprocess.Popen(command)
|
||||
except BaseException:
|
||||
if receipt_root is not None:
|
||||
receipt_root.cleanup()
|
||||
raise
|
||||
peak = 0
|
||||
peak_tasks = 0
|
||||
events = {}
|
||||
cpu_stats = {}
|
||||
abort = None
|
||||
final_events = None
|
||||
payload_observation = None
|
||||
payload_group = None
|
||||
code = None
|
||||
group = None
|
||||
started = time.monotonic()
|
||||
try:
|
||||
group = None
|
||||
while process.poll() is None:
|
||||
if group is None:
|
||||
reply = subprocess.run(
|
||||
@@ -104,6 +131,18 @@ def main():
|
||||
path = reply.stdout.strip()
|
||||
if path:
|
||||
group = Path('/sys/fs/cgroup') / path.lstrip('/')
|
||||
if receipt_path is not None and payload_observation is None and group is not None:
|
||||
candidate = read_observation(receipt_path, unit)
|
||||
if candidate is not None:
|
||||
payload_observation = candidate
|
||||
if candidate['control_group'] == str(group.relative_to('/sys/fs/cgroup')):
|
||||
try:
|
||||
actors = [int(x) for x in (group / 'cgroup.procs').read_text().splitlines()]
|
||||
if candidate['observer_pid'] in actors:
|
||||
payload_group = group
|
||||
except (OSError, ValueError):
|
||||
pass
|
||||
release_path.touch(exist_ok=False)
|
||||
current = group / 'memory.current' if group else None
|
||||
if current and current.exists():
|
||||
try:
|
||||
@@ -132,6 +171,8 @@ def main():
|
||||
# Read while the owned group still exists, before wait/cleanup can remove it.
|
||||
final_events = final_resource_events(group)
|
||||
code = process.wait()
|
||||
if receipt_path is not None and payload_observation is None:
|
||||
payload_observation = read_observation(receipt_path, unit)
|
||||
except BaseException:
|
||||
stop_owned_scope(unit, process, group)
|
||||
raise
|
||||
@@ -150,7 +191,22 @@ def main():
|
||||
'resource_events_final': final_events,
|
||||
'resource_counters_verified': final_events is not None,
|
||||
'cpu_stat_last_observed': cpu_stats,
|
||||
**({
|
||||
'payload_resource_observation': payload_observation,
|
||||
'payload_resource_observation_verified': (
|
||||
payload_observation is not None
|
||||
and payload_observation['payload_cleanup_verified']
|
||||
and payload_group is not None and abort is None
|
||||
and code is not None
|
||||
and (payload_observation['payload_exit_code'] if payload_observation['payload_exit_code'] >= 0
|
||||
else 128 - payload_observation['payload_exit_code']) == code
|
||||
),
|
||||
'payload_observation_scope': 'Counters at payload exit while observer remains alive; not final whole-scope counters.',
|
||||
'payload_observer_sources_sha256': observer_sources,
|
||||
} if options.observe_payload_exit else {}),
|
||||
}), file=sys.stderr)
|
||||
if receipt_root is not None:
|
||||
receipt_root.cleanup()
|
||||
raise SystemExit(125 if abort else code)
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,114 @@
|
||||
"""Observe a finished payload while its owned cgroup observer is still alive."""
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
|
||||
def pairs(path):
|
||||
values = {}
|
||||
for line in path.read_text().splitlines():
|
||||
name, value = line.split()
|
||||
number = int(value)
|
||||
if number < 0 or name in values:
|
||||
raise ValueError('Invalid kernel counter')
|
||||
values[name] = number
|
||||
return values
|
||||
|
||||
|
||||
def snapshot(group, token, code):
|
||||
actors = [int(line) for line in (group / 'cgroup.procs').read_text().splitlines()]
|
||||
observer = os.getpid()
|
||||
return {
|
||||
'phase': 'payload-exited-observer-alive', 'ownership_token': token,
|
||||
'payload_exit_code': code, 'observer_pid': observer,
|
||||
'remaining_processes': actors, 'payload_cleanup_verified': actors == [observer],
|
||||
'resource_events': {name: pairs(group / name)
|
||||
for name in ['memory.events', 'pids.events']},
|
||||
'cpu_stat': pairs(group / 'cpu.stat'),
|
||||
'memory_peak_bytes': int((group / 'memory.peak').read_text()),
|
||||
'control_group': str(group.relative_to('/sys/fs/cgroup')),
|
||||
}
|
||||
|
||||
|
||||
def read_observation(path, token):
|
||||
"""Validate the observer receipt; missing/malformed witnesses stay unknown."""
|
||||
try:
|
||||
if path.is_symlink():
|
||||
return None
|
||||
value = json.loads(path.read_text())
|
||||
if not isinstance(value, dict) or value.get('ownership_token') != token:
|
||||
return None
|
||||
if value.get('phase') != 'payload-exited-observer-alive':
|
||||
return None
|
||||
if not isinstance(value.get('control_group'), str) or not value['control_group']:
|
||||
return None
|
||||
for name in ['payload_exit_code', 'observer_pid', 'memory_peak_bytes']:
|
||||
if type(value.get(name)) is not int:
|
||||
return None
|
||||
if value['observer_pid'] <= 0 or value['memory_peak_bytes'] < 0:
|
||||
return None
|
||||
actors = value.get('remaining_processes')
|
||||
if not isinstance(actors, list) or any(type(x) is not int or x <= 0 for x in actors):
|
||||
return None
|
||||
if type(value.get('payload_cleanup_verified')) is not bool:
|
||||
return None
|
||||
if value['payload_cleanup_verified'] != (actors == [value['observer_pid']]):
|
||||
return None
|
||||
required = {'memory.events': ['high', 'max', 'oom', 'oom_kill'],
|
||||
'pids.events': ['max']}
|
||||
events = value.get('resource_events')
|
||||
if not isinstance(events, dict):
|
||||
return None
|
||||
for name, keys in required.items():
|
||||
record = events.get(name)
|
||||
if not isinstance(record, dict) or any(key not in record for key in keys):
|
||||
return None
|
||||
if any(type(x) is not int or x < 0 for x in record.values()):
|
||||
return None
|
||||
cpu = value.get('cpu_stat')
|
||||
if not isinstance(cpu, dict) or 'usage_usec' not in cpu:
|
||||
return None
|
||||
if any(type(x) is not int or x < 0 for x in cpu.values()):
|
||||
return None
|
||||
names = ['phase', 'ownership_token', 'payload_exit_code', 'observer_pid',
|
||||
'remaining_processes', 'payload_cleanup_verified',
|
||||
'cpu_stat', 'memory_peak_bytes', 'control_group']
|
||||
return {**{name: value[name] for name in names},
|
||||
'resource_events': {name: events[name] for name in required}}
|
||||
except (OSError, ValueError, TypeError):
|
||||
return None
|
||||
|
||||
|
||||
def main():
|
||||
output, release, token = Path(sys.argv[1]), Path(sys.argv[2]), sys.argv[3]
|
||||
command = sys.argv[4:]
|
||||
if not command:
|
||||
raise SystemExit('Missing explicit payload command')
|
||||
child = subprocess.Popen(command)
|
||||
code = child.wait()
|
||||
paths = [line[3:] for line in Path('/proc/self/cgroup').read_text().splitlines()
|
||||
if line.startswith('0::')]
|
||||
try:
|
||||
if len(paths) != 1:
|
||||
raise ValueError('Expected one unified cgroup')
|
||||
group = Path('/sys/fs/cgroup') / paths[0].lstrip('/')
|
||||
observation = snapshot(group, token, code)
|
||||
with os.fdopen(os.open(output, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600),
|
||||
'w') as file:
|
||||
json.dump(observation, file)
|
||||
# Keep the owner alive until its supervisor acknowledges the receipt.
|
||||
# Bound a broken supervisor by the existing ten-second cleanup interval.
|
||||
deadline = time.monotonic() + 10
|
||||
while not release.exists() and time.monotonic() < deadline:
|
||||
time.sleep(0.02)
|
||||
except (OSError, ValueError) as error:
|
||||
print('Payload counter observation unavailable: ' + type(error).__name__, file=sys.stderr)
|
||||
raise SystemExit(code if code >= 0 else 128 - code)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -39,9 +39,37 @@ def decode(result):
|
||||
return None
|
||||
|
||||
|
||||
def named_scope(value, names):
|
||||
"""Accept explicit equivalent scope labels; contradictory duplicates stay a failure."""
|
||||
if not isinstance(value, dict):
|
||||
return None
|
||||
selected = {name: value[name] for name in names if name in value}
|
||||
values = list(selected.values())
|
||||
if not values:
|
||||
return None
|
||||
if all(entry == values[0] for entry in values):
|
||||
return values[0]
|
||||
return {'conflicting_reported_scopes': selected}
|
||||
|
||||
|
||||
def scope_with_verified_counts(value, expected_scope, returned_counts):
|
||||
"""Validate explicit extra row counts; never erase unknown or conflicting facts."""
|
||||
if not isinstance(value, dict):
|
||||
return value, {}
|
||||
scope = dict(value)
|
||||
checks = {}
|
||||
for name, expected in returned_counts.items():
|
||||
if name in scope:
|
||||
reported = scope[name]
|
||||
checks[name] = type(reported) is int and reported == expected
|
||||
if name not in expected_scope:
|
||||
del scope[name]
|
||||
return scope, checks
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument('--domain', choices=['wasm', 'evm', 'android', 'managed', 'managed-nonempty', 'firmware', 'inventory', 'javascript', 'native', 'browser'], required=True)
|
||||
parser.add_argument('--domain', choices=['wasm', 'evm', 'android', 'flutter-aot', 'sqlite', 'managed', 'managed-nonempty', 'firmware', 'inventory', 'javascript', 'native', 'browser'], required=True)
|
||||
parser.add_argument('--trial', type=Path, required=True)
|
||||
parser.add_argument('--reference', type=Path)
|
||||
parser.add_argument('--resource-log', type=Path, required=True)
|
||||
@@ -98,8 +126,21 @@ def main():
|
||||
check('wabt_' + key, answer.get('wabt_' + key), reference['tool_profile'][key])
|
||||
elif domain == 'evm':
|
||||
n = producer_results['inspect_evm_interface'][-1]['normalized_result']
|
||||
if n.get('kind') == 'summary':
|
||||
check('evm_summary.parent_operation', n.get('parent_operation'), 'inspect_evm_interface')
|
||||
checks['evm_summary.parent_reference_observed'] = n.get('parent_evidence_id') in observed_ids
|
||||
n = n['summary']
|
||||
for key, expected in {'artifact_bytes': n['artifact']['bytes'], 'bytecode_sha256': n['bytecode']['sha256'], 'bytecode_bytes': n['bytecode']['bytes'], 'functions': n['functions'], 'evidence_kind': n['evidence_kind'], 'discovery_completeness': n['discovery_completeness'], 'runtime_execution': n['runtime_execution'], 'bytecode_kind': n['bytecode']['kind'], 'hardfork': n['bytecode']['hardfork'], 'deployment_authenticity': n['bytecode']['deployment_authenticity']}.items():
|
||||
check(key, answer.get(key), expected)
|
||||
check('evm_original_limitations', answer.get('limitations'), n['limitations'])
|
||||
elif domain == 'sqlite':
|
||||
for key in ['artifact', 'wal', 'engine', 'database_encoding', 'schema', 'rows', 'producer_limitations', 'limitations']:
|
||||
# Missing is not SQL/JSON null; booleans are not numeric counters.
|
||||
check(key, (key in answer, json.dumps(answer.get(key), sort_keys=True)),
|
||||
(True, json.dumps(reference[key], sort_keys=True)))
|
||||
elif domain == 'flutter-aot':
|
||||
for key in ['target', 'abi', 'libapp', 'string_pool', 'coverage', 'limitations']:
|
||||
check(key, answer.get(key), reference[key])
|
||||
elif domain == 'android':
|
||||
for key in ['package_name', 'version_name', 'version_code', 'matches']:
|
||||
check(key, answer.get(key), reference[key])
|
||||
@@ -115,8 +156,14 @@ def main():
|
||||
check(key, answer.get(key), n[key])
|
||||
for key in ['module_refs', 'pinvoke_imports']:
|
||||
check(key, answer.get('native_boundaries', {}).get(key), reference['boundaries'][key])
|
||||
check('artifact_coverage', answer.get('coverage', {}).get('artifact'), n['coverage'])
|
||||
check('native_boundaries_coverage', answer.get('coverage', {}).get('native_boundaries'), reference['boundaries']['coverage'])
|
||||
check('artifact_coverage', named_scope(answer.get('coverage'), ['artifact', 'managed_artifact', 'managed_metadata']), n['coverage'])
|
||||
boundary_scope, counts = scope_with_verified_counts(
|
||||
named_scope(answer.get('coverage'), ['native_boundaries', 'native_boundary_declarations']),
|
||||
reference['boundaries']['coverage'],
|
||||
{'module_ref_count': len(reference['boundaries']['module_refs']),
|
||||
'pinvoke_import_count': len(reference['boundaries']['pinvoke_imports'])})
|
||||
check('native_boundaries_coverage', boundary_scope, reference['boundaries']['coverage'])
|
||||
checks.update({'native_coverage_' + name: valid for name, valid in counts.items()})
|
||||
elif domain == 'managed-nonempty':
|
||||
n = reference['boundaries']
|
||||
check('module_mvid', answer.get('module_mvid'), n['module']['mvid'])
|
||||
@@ -275,6 +322,7 @@ def main():
|
||||
status = str(event['status'])
|
||||
statuses[status] = statuses.get(status, 0) + 1
|
||||
check('network_response_statuses', network['observed_response_statuses'], statuses)
|
||||
factual_checks = dict(checks)
|
||||
scopes = [json.loads(line) for line in args.resource_log.read_text().splitlines() if line.startswith('{')]
|
||||
scope = next(value for value in reversed(scopes) if 'resource_scope' in value)
|
||||
checks['no_resource_abort'] = scope['abort'] is None
|
||||
@@ -293,6 +341,7 @@ def main():
|
||||
report = {
|
||||
'review_source_sha256': hashlib.sha256(Path(__file__).read_bytes()).hexdigest(),
|
||||
'domain': domain, 'checks': checks, 'all_selected_checks_pass': all(checks.values()),
|
||||
'factual_checks': factual_checks, 'all_factual_checks_pass': all(factual_checks.values()),
|
||||
'differences': differences, 'tool_errors': tool_errors, 'summary': summary,
|
||||
'fingerprints': {key: value for key, value in settings.items() if 'sha256' in key or key in ['runtime_revision', 'enabled_tools', 'authority']},
|
||||
'answer_file_sha256': hashlib.sha256((args.trial / 'answer.txt').read_bytes()).hexdigest(),
|
||||
|
||||
@@ -0,0 +1,181 @@
|
||||
# SQLite existing-control measurements
|
||||
|
||||
Continues [#2016](https://github.com/morluto/rea/issues/2016) after the Flutter
|
||||
lane. The measured main runtime is pinned to
|
||||
`172407787f1842dedbcd27b5385177595f786ce4`. Its only post-Flutter
|
||||
commit repairs Windows transient runtime admission (#2467); this Linux round
|
||||
does not establish Windows performance. No production/API change is proposed
|
||||
merely because another summary or cache could be added.
|
||||
|
||||
Before PR integration, main advanced to
|
||||
`bb540826d65316c164d2de8ba4ef10f81a8ad073`. Its five intervening commits cover
|
||||
MCP archive contracts/documentation, an advertised-schema inventory test,
|
||||
pwntools cleanup identity, concurrent Mach-O slice inspection, and transient
|
||||
DMG verification retries. None changes the SQLite selector. Rebase and current
|
||||
main verification do not relabel the frozen measurements below.
|
||||
|
||||
The committed post-format driver passes a real SDK smoke against the frozen
|
||||
runtime. After rebase, a fresh build, real all/selected-column SDK pair and docs
|
||||
check pass on the updated main. Selected facts match each other and the frozen
|
||||
runtime; artifact bytes remain unchanged. These are integration checks, not
|
||||
additional model performance pairs.
|
||||
|
||||
## Real application data and question
|
||||
|
||||
Use QGIS's bundled `world_map.gpkg`, unmodified from official immutable source
|
||||
commit `e1223b8e9e71fc50ee995ed556f711c54ccc101b`. `corpus.json` pins its URL,
|
||||
12,914,688-byte size, Git blob `c9ff80228dff0de93c9acdef16a97545ba360244`, and
|
||||
SHA-256 `01d293159d1b77bd9212b5252587853f61aa9e25b2387135f34afaba8d057190`.
|
||||
Streaming acquisition matches the upstream blob. The upstream resource install
|
||||
list includes `data/*`; [its README](https://github.com/qgis/QGIS/blob/e1223b8e9e71fc50ee995ed556f711c54ccc101b/resources/data/README_world_map.md)
|
||||
records data provenance. This is shipped application data, not a generated REA
|
||||
fixture. QGIS is neither installed nor executed.
|
||||
|
||||
The source question asks for the complete returned `countries` table schema
|
||||
and one sample's `fid`, `NAME`, `ISO_A3` tagged cells, in that order. It preserves
|
||||
database/WAL identity, engine/encoding, exact original SQL and columns, integer
|
||||
strings, text bytes, nulls, row limit/count, unspecified order, truncation and
|
||||
both producer/outer limitations. It does not require geometry blobs or a full
|
||||
country inventory. Neither political labels nor the sampled ordering are
|
||||
inferred to be current or globally exhaustive.
|
||||
|
||||
SQLite already defaults to compact schema inventory without a table, detailed
|
||||
selected-table schema with a table, and exposes exact `columns`. Selecting
|
||||
columns avoids constructing unrelated cells without changing the requested
|
||||
table schema. Thus compare existing all-visible columns against exact columns
|
||||
on the same main runtime, rather than changing an old user's default or adding
|
||||
a new delivery mechanism. Inventory scope cannot substitute for the detailed
|
||||
schema requested by this question.
|
||||
|
||||
## Public SDK observations
|
||||
|
||||
Frozen runtime: main `172407787f1842dedbcd27b5385177595f786ce4`, compiled
|
||||
dist/bridge file-table SHA-256
|
||||
`72a4ac6c6db491c455f3e4a798f8f52ed5850c08831e7cdc9f25c1525a5a1d11`.
|
||||
Node 24.18.0, SDK client/server 2.3.1, observed SQLite 3.53.1, WAL null. Driver,
|
||||
provider settings/search-path fingerprints, artifact and locked dependencies
|
||||
remain equal. Each workflow starts a fresh server and validates the selected
|
||||
actual advertised schema. Host page caches are not flushed.
|
||||
|
||||
The initial default inventory returns 26 table entries, including explicit
|
||||
virtual/shadow classifications, no rows and `not_requested` column details:
|
||||
27,185 compact client-result JSON bytes. Selecting `countries` reveals eight
|
||||
columns, including a geometry blob. Its one-row full reply is 485,277 bytes.
|
||||
The selected request retains every requested declared-schema field and exact
|
||||
tagged sample values while omitting unrelated sample cells.
|
||||
|
||||
| Trial | Reply bytes | SDK workflow seconds | Payload CPU seconds | Kernel peak bytes |
|
||||
| ------------- | ----------: | -------------------: | ------------------: | ----------------: |
|
||||
| sdk-all-1 | 485,277 | 11.820 | 12.755 | 562,884,608 |
|
||||
| sdk-columns-1 | 18,509 | 14.017 | 13.715 | 557,510,656 |
|
||||
| sdk-columns-2 | 18,509 | 15.826 | 14.162 | 548,343,808 |
|
||||
| sdk-all-2 | 485,277 | 5.679 | 7.310 | 561,704,960 |
|
||||
|
||||
Both pairs make one successful call, pass settings/selected-fact comparison,
|
||||
retain complete original schema and all requested typed cells, and preserve
|
||||
source bytes. Reply bytes fall about 96.2%. These observations **do not show SDK
|
||||
speed or CPU improvement**: selected workflows are slower in both pairs and
|
||||
timing varies substantially. No cause is inferred from these data. Payload CPU
|
||||
includes runner/catalog work outside the SDK workflow timer. Memory changes
|
||||
are small and do not establish a general reduction. Raw replies and the exact
|
||||
pre-format driver source are archived privately.
|
||||
|
||||
Reproduce with `measure.mjs --artifact qgis-world-map.gpkg` and
|
||||
`--sqlite-selection all|columns`, using the same `--runtime-root`, `--revision`,
|
||||
artifact path and `--schemas selected`. `--delivery` stays `complete`; this is
|
||||
an existing request selection, not a new summary API. Original caller arguments
|
||||
and schemas remain in the private receipts. The source question is
|
||||
`questions/sqlite-qgis.txt` and the selected factual grader is
|
||||
`review-model.py --domain sqlite`.
|
||||
|
||||
## Decision and scope
|
||||
|
||||
The initial four actual Codex attempts never executed SQLite: the client rejected
|
||||
the tool because its private snapshot writes require approval and the isolated
|
||||
driver lacked this exact tool's approval override. Every factual grade fails;
|
||||
these are failed client attempts, not completed-task performance comparisons.
|
||||
All profiles are removed. The existing experiment-only per-tool approval list
|
||||
now includes `inspect_sqlite_database`; each corrected trial exposes only that
|
||||
exact enabled tool. REA annotations, target authority and global Codex configuration
|
||||
are unchanged. The [official configuration reference](https://learn.chatgpt.com/docs/config-file/config-reference)
|
||||
documents the per-tool override. Corrected cohorts use fresh output directories
|
||||
and retain the original failures separately.
|
||||
|
||||
Actual repeated Codex comparisons determine the task token/time benefit before
|
||||
crediting smaller replies. An existing selector that answers the question is
|
||||
evidence against adding another production mechanism. Keep negative outcomes
|
||||
and unavailable coverage, including failure/retry costs and profile cleanup.
|
||||
|
||||
Four corrected serial trials use Codex CLI 0.161.0, requested `gpt-5.6-terra`
|
||||
high, one enabled tool, identical owned context/discovery capture, and the same
|
||||
frozen main runtime. The requested alias is not a verified server-resolved model.
|
||||
Equal treatment instructions select the existing all-visible or exact-column
|
||||
request; the source question and requested schema/sample facts stay fixed.
|
||||
|
||||
| Corrected trial | Input | Cached | Uncached | Output | Seconds | MCP calls | Facts |
|
||||
| ------------------------ | -----: | -----: | -------: | -----: | ------: | --------: | ----- |
|
||||
| approved-model-all-1 | 88,131 | 52,480 | 35,651 | 3,369 | 84.007 | 2 | pass |
|
||||
| approved-model-columns-1 | 37,340 | 20,992 | 16,348 | 3,006 | 87.514 | 1 | pass |
|
||||
| approved-model-columns-2 | 56,148 | 36,096 | 20,052 | 3,298 | 97.466 | 2 | pass |
|
||||
| approved-model-all-2 | 88,340 | 52,480 | 35,860 | 3,084 | 113.847 | 2 | pass |
|
||||
|
||||
Both pairs pass every selected producer fact, exact SQL/schema/null/tagged
|
||||
cells, observed response Evidence IDs, target-byte preservation and profile
|
||||
cleanup. The grader checks producer observations, not independent geometry
|
||||
semantics. Actual events contain only the chosen MCP tool and agent messages;
|
||||
no shell or other agent execution appears. Usage is the final aggregate actually
|
||||
reported by the CLI, one usage record per trial; cached input remains included
|
||||
in total input. Output includes 853 / 357 / 523 / 664 reasoning tokens.
|
||||
|
||||
Total input falls **57.63% / 36.44%**, uncached input **54.14% / 44.08%**.
|
||||
Time is mixed: columns are **4.17% slower** in pair 1 and **14.39% faster** in
|
||||
pair 2. Output is 10.77% lower / 6.94% higher. Repeated calls remain counted;
|
||||
pair 1 has two all-column calls versus one selected call, while pair 2 has two
|
||||
calls on both sides. This supports scoped token savings from using the existing
|
||||
selector, not a dependable speedup or a new API requirement. Client context and
|
||||
usage are observed separately from raw SDK reply bytes.
|
||||
|
||||
Model payload CPU seconds / kernel peaks are 7.596 / 763,838,464;
|
||||
7.772 / 761,143,296; 8.409 / 762,597,376; and 11.759 / 774,025,216. CPU is mixed
|
||||
and memory differences are small. Payload cleanup/limit observations pass;
|
||||
missing terminal counters keep every strict whole-resource grade false.
|
||||
|
||||
The original blocked attempts remain separate and never provide comparison
|
||||
denominators. They each have one failed call, no SQLite result and failed facts:
|
||||
|
||||
| Blocked trial | Input | Cached | Uncached | Output | Seconds |
|
||||
| --------------- | -----: | -----: | -------: | -----: | ------: |
|
||||
| model-all-1 | 35,290 | 29,952 | 5,338 | 798 | 59.881 |
|
||||
| model-columns-1 | 35,356 | 22,016 | 13,340 | 670 | 44.764 |
|
||||
| model-columns-2 | 35,034 | 20,992 | 14,042 | 639 | 30.007 |
|
||||
| model-all-2 | 35,104 | 20,992 | 14,112 | 673 | 25.132 |
|
||||
|
||||
**Decision: no production/API change.** Existing column selection materially
|
||||
reduces unnecessary delivery and actual model input while retaining the complete
|
||||
requested schema and exact cells. Existing documentation already recommends
|
||||
table/column selection. No additional summary, schema-scope default change,
|
||||
automatic field omission, plugin layer or cross-session cache is justified by
|
||||
these results. Preserve the negative/mixed time, output and CPU findings.
|
||||
|
||||
Real one-shot CLI inventory, all-column sample and exact-column sample match
|
||||
the corresponding full MCP normalized results, subjects, parameters and IDs;
|
||||
the source digest remains unchanged. An independent Python binding using SQLite
|
||||
3.40.1 opens this immutable artifact read-only and agrees with the exact stored
|
||||
table SQL, eight declared columns and selected tagged sample cells. This is a
|
||||
second engine/binding observation of this file, not a general row-order or
|
||||
geometry-semantics guarantee. The six existing context/driver regression tests
|
||||
pass; Python compilation and JavaScript syntax checks pass. No new product test
|
||||
or production implementation is added for this measurement-only finding.
|
||||
|
||||
Manual review tightens the SQLite grader to distinguish missing fields from
|
||||
explicit null and preserve JSON scalar types. Regrading all four stored model
|
||||
answers still passes every selected fact; deliberate omission of `wal` and a
|
||||
boolean substituted for numeric `row_limit` both fail. Original transcripts,
|
||||
usage and previous review receipts stay intact; no model trial is rerun.
|
||||
|
||||
Heavy tasks remain serial under one CPU, unchanged 3 GiB early stop / 4 GiB hard
|
||||
memory, no swap, owned task scopes and host guards. SDK payload observations
|
||||
verify cleanup and zero phase limit events; final whole-scope counters remain
|
||||
unknown. This does not establish live/WAL atomicity, all SQLite applications,
|
||||
other runtimes/platforms or geospatial geometry semantics. Other packet/signal
|
||||
and capture lanes remain incremental follow-ups using their existing controls.
|
||||
@@ -0,0 +1,175 @@
|
||||
"""Distinct capture/cleanup failures; simulated CLI is not real-model evidence."""
|
||||
|
||||
import importlib.util
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import runpy
|
||||
import sys
|
||||
from pathlib import Path
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
from codex_context import capture_tool_context, selected_event
|
||||
|
||||
|
||||
class ManagedCoverageTests(unittest.TestCase):
|
||||
def test_verifies_extra_counts_and_preserves_unknown_fields(self):
|
||||
review = runpy.run_path(str(Path(__file__).with_name('review-model.py')))
|
||||
project = review['scope_with_verified_counts']
|
||||
expected = {'state': 'partial', 'issues': ['unreturned rows remain unknown']}
|
||||
original = {**expected, 'module_ref_count': 0, 'pinvoke_import_count': 2}
|
||||
scope, checks = project(original, expected, {'module_ref_count': 0, 'pinvoke_import_count': 2})
|
||||
self.assertEqual(scope, expected)
|
||||
self.assertTrue(all(checks.values()))
|
||||
self.assertEqual(original['pinvoke_import_count'], 2)
|
||||
for count in [3, '2', True]:
|
||||
scope, checks = project({**expected, 'pinvoke_import_count': count}, expected, {'pinvoke_import_count': 2})
|
||||
self.assertFalse(checks['pinvoke_import_count'])
|
||||
_, checks = project({**expected, 'module_ref_count': False}, {**expected, 'module_ref_count': 0}, {'module_ref_count': 0})
|
||||
self.assertFalse(checks['module_ref_count'])
|
||||
scope, _ = project({**expected, 'unsupported_claim': True}, expected, {})
|
||||
self.assertNotEqual(scope, expected)
|
||||
scope, _ = project({'state': 'partial', 'module_ref_count': 0}, expected, {'module_ref_count': 0})
|
||||
self.assertNotEqual(scope, expected)
|
||||
ambiguous = review['named_scope']({'native_boundaries': expected, 'native_boundary_declarations': {'state': 'complete'}}, ['native_boundaries', 'native_boundary_declarations'])
|
||||
scope, _ = project(ambiguous, expected, {'module_ref_count': 0})
|
||||
self.assertNotEqual(scope, expected)
|
||||
|
||||
|
||||
class ContextCaptureTests(unittest.TestCase):
|
||||
def test_filters_ambient_account_and_nested_usage_state(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
home = root / 'home'
|
||||
sessions = home / 'sessions'
|
||||
sessions.mkdir(parents=True)
|
||||
output = root / 'output'
|
||||
output.mkdir()
|
||||
events = [
|
||||
{'type': 'world_state', 'payload': {'state': {'env': 'ambient-marker'}}},
|
||||
{'type': 'session_meta', 'payload': {'creator_account_id': 'account-marker'}},
|
||||
{'type': 'response_item', 'payload': {
|
||||
'type': 'custom_tool_call_output', 'call_id': 'original-call',
|
||||
'output': 'caller-selected evidence and paths',
|
||||
'internal_chat_message_metadata_passthrough': 'private-marker'}},
|
||||
{'type': 'token_usage_record', 'payload': {
|
||||
'response_id': 'original-response', 'usage': {
|
||||
'input_tokens': 12, 'cached_input_tokens': 4,
|
||||
'environment': 'nested-ambient-marker'},
|
||||
'transport_auth': 'auth-marker'}},
|
||||
]
|
||||
(sessions / 'rollout.jsonl').write_text(''.join(json.dumps(x) + '\n' for x in events))
|
||||
(home / 'auth.json').write_text('copied-credential-marker')
|
||||
receipt = capture_tool_context(home, output)
|
||||
text = (output / 'codex-tool-context.jsonl').read_text()
|
||||
self.assertEqual(receipt['status'], 'captured')
|
||||
self.assertEqual(receipt['retained_events'], 2)
|
||||
for marker in ['ambient-marker', 'account-marker', 'private-marker',
|
||||
'auth-marker', 'copied-credential-marker']:
|
||||
self.assertNotIn(marker, text)
|
||||
self.assertIn('caller-selected evidence and paths', text)
|
||||
self.assertIn('original-response', text)
|
||||
self.assertEqual((output / 'codex-tool-context.jsonl').stat().st_mode & 0o777, 0o600)
|
||||
|
||||
def test_missing_rollouts_are_unknown_and_malformed_lines_remain_partial(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
output = root / 'missing'
|
||||
output.mkdir()
|
||||
self.assertEqual(capture_tool_context(root / 'home', output)['status'], 'unavailable')
|
||||
sessions = root / 'home' / 'sessions'
|
||||
sessions.mkdir(parents=True)
|
||||
(sessions / 'rollout.jsonl').write_text('not-json\n' + json.dumps({
|
||||
'type': 'event_msg', 'payload': {'type': 'token_count', 'info': None}}) + '\n')
|
||||
partial = root / 'partial'
|
||||
partial.mkdir()
|
||||
receipt = capture_tool_context(root / 'home', partial)
|
||||
self.assertEqual(receipt['status'], 'partial')
|
||||
self.assertEqual(receipt['errors'][0]['line'], 1)
|
||||
self.assertEqual(receipt['retained_events'], 1)
|
||||
with self.assertRaises(ValueError):
|
||||
selected_event({'type': 'token_usage_record', 'payload': {'usage': {'input_tokens': True}}})
|
||||
with self.assertRaises(ValueError):
|
||||
selected_event({'type': {'environment': 'unexpected'}, 'payload': {}})
|
||||
with self.assertRaises(ValueError):
|
||||
selected_event({'type': 'response_item', 'timestamp': {'environment': 'unexpected'}, 'payload': {}})
|
||||
unknown = selected_event({'type': 'event_msg', 'payload': {'type': 'token_count', 'info': {'model_context_window': None}}})
|
||||
self.assertIsNone(unknown['payload']['info']['model_context_window'])
|
||||
|
||||
def test_capture_failure_still_removes_trial_credentials_and_profile(self):
|
||||
spec = importlib.util.spec_from_file_location('agent_trial', Path(__file__).with_name('agent-trial.py'))
|
||||
agent = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(agent)
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
runtime = root / 'runtime'
|
||||
(runtime / 'scripts').mkdir(parents=True)
|
||||
(runtime / 'scripts' / 'rea.mjs').write_text('')
|
||||
(runtime / 'package-lock.json').write_text('{}')
|
||||
auth_home = root / 'source-auth'
|
||||
auth_home.mkdir()
|
||||
(auth_home / 'auth.json').write_text('synthetic-credential')
|
||||
question = root / 'question.txt'
|
||||
question.write_text('Observe the declared fixture.')
|
||||
output = root / 'trial'
|
||||
child = Mock(stdin=io.StringIO())
|
||||
child.wait.return_value = 0
|
||||
arguments = ['agent-trial.py', '--runtime-root', str(runtime), '--revision', 'fixture',
|
||||
'--prompt', str(question), '--output', str(output), '--capture-tool-context']
|
||||
with patch.dict(os.environ, {'CODEX_HOME': str(auth_home)}, clear=True), \
|
||||
patch.object(sys, 'argv', arguments), \
|
||||
patch.object(agent.shutil, 'which', return_value='/fixture/node'), \
|
||||
patch.object(agent.subprocess, 'check_output', return_value='v24.18.0\n'), \
|
||||
patch.object(agent.subprocess, 'Popen', return_value=child), \
|
||||
patch.object(agent, 'capture_tool_context', side_effect=OSError('fixture capture failure')), \
|
||||
patch('builtins.print'):
|
||||
with self.assertRaisesRegex(OSError, 'fixture capture failure'):
|
||||
agent.main()
|
||||
self.assertFalse(list(output.glob('profile-*')))
|
||||
self.assertTrue(json.loads((output / 'cleanup.json').read_text())['profile_removed'])
|
||||
self.assertEqual((auth_home / 'auth.json').read_text(), 'synthetic-credential')
|
||||
|
||||
def test_refuses_foreign_session_directory_and_symlink_records(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
home = root / 'home'
|
||||
home.mkdir()
|
||||
foreign = root / 'foreign'
|
||||
foreign.mkdir()
|
||||
(foreign / 'rollout.jsonl').write_text('private marker')
|
||||
(home / 'sessions').symlink_to(foreign, target_is_directory=True)
|
||||
output = root / 'directory-result'
|
||||
output.mkdir()
|
||||
receipt = capture_tool_context(home, output)
|
||||
self.assertEqual(receipt['status'], 'partial')
|
||||
self.assertEqual(receipt['retained_events'], 0)
|
||||
(home / 'sessions').unlink()
|
||||
(home / 'sessions').mkdir()
|
||||
(home / 'sessions' / 'linked.jsonl').symlink_to(foreign / 'rollout.jsonl')
|
||||
output = root / 'file-result'
|
||||
output.mkdir()
|
||||
receipt = capture_tool_context(home, output)
|
||||
self.assertEqual(receipt['status'], 'partial')
|
||||
self.assertEqual(receipt['errors'][0]['reason'], 'not_an_owned_regular_file')
|
||||
self.assertEqual((foreign / 'rollout.jsonl').read_text(), 'private marker')
|
||||
|
||||
def test_guidance_requires_selected_tools_before_acquiring_a_profile(self):
|
||||
spec = importlib.util.spec_from_file_location('agent_trial', Path(__file__).with_name('agent-trial.py'))
|
||||
agent = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(agent)
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
output = root / 'trial'
|
||||
arguments = ['agent-trial.py', '--runtime-root', str(root), '--revision', 'fixture',
|
||||
'--prompt', str(root / 'question'), '--output', str(output), '--guide-enabled-tools']
|
||||
with patch.object(sys, 'argv', arguments), patch.object(sys, 'stderr', io.StringIO()):
|
||||
with self.assertRaises(SystemExit) as raised:
|
||||
agent.main()
|
||||
self.assertEqual(raised.exception.code, 2)
|
||||
self.assertFalse(output.exists())
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,103 @@
|
||||
"""Receipt semantics; real systemd/cgroup workflows are verified separately."""
|
||||
|
||||
import json
|
||||
import importlib.util
|
||||
import io
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from resource_payload import read_observation
|
||||
|
||||
|
||||
def receipt():
|
||||
return {
|
||||
'phase': 'payload-exited-observer-alive', 'ownership_token': 'own-unit',
|
||||
'payload_exit_code': 3, 'observer_pid': 100, 'remaining_processes': [100],
|
||||
'payload_cleanup_verified': True, 'memory_peak_bytes': 123,
|
||||
'control_group': 'user.slice/own-unit.scope',
|
||||
'resource_events': {'memory.events': {'high': 0, 'max': 0, 'oom': 0, 'oom_kill': 0},
|
||||
'pids.events': {'max': 0}},
|
||||
'cpu_stat': {'usage_usec': 1000},
|
||||
}
|
||||
|
||||
|
||||
class PayloadObservationTests(unittest.TestCase):
|
||||
def test_observer_capacity_failure_precedes_any_process_acquisition(self):
|
||||
spec = importlib.util.spec_from_file_location('resource_run', Path(__file__).with_name('resource-run.py'))
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
arguments = ['resource-run.py', '--observe-payload-exit', '--task-ceiling', '1', '--', 'payload']
|
||||
with patch.object(sys, 'argv', arguments), patch.object(sys, 'stderr', io.StringIO()), \
|
||||
patch.object(module.subprocess, 'Popen') as spawn:
|
||||
with self.assertRaises(SystemExit) as failure:
|
||||
module.main()
|
||||
self.assertEqual(failure.exception.code, 2)
|
||||
spawn.assert_not_called()
|
||||
|
||||
def test_preserves_nonzero_status_kernel_peak_and_nonzero_events(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / 'receipt.json'
|
||||
value = receipt()
|
||||
value['resource_events']['memory.events']['oom'] = 1
|
||||
path.write_text(json.dumps(value))
|
||||
observed = read_observation(path, 'own-unit')
|
||||
self.assertEqual(observed['payload_exit_code'], 3)
|
||||
self.assertEqual(observed['memory_peak_bytes'], 123)
|
||||
self.assertEqual(observed['resource_events']['memory.events']['oom'], 1)
|
||||
|
||||
def test_rejects_missing_malformed_wrong_owner_and_inconsistent_cleanup(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / 'receipt.json'
|
||||
self.assertIsNone(read_observation(path, 'own-unit'))
|
||||
path.write_text('incomplete-json')
|
||||
self.assertIsNone(read_observation(path, 'own-unit'))
|
||||
cases = []
|
||||
for key, invalid in [('ownership_token', 'other-unit'), ('observer_pid', True),
|
||||
('memory_peak_bytes', -1), ('remaining_processes', [100, 101]),
|
||||
('payload_cleanup_verified', 'true'), ('phase', 'scope-exited')]:
|
||||
value = receipt()
|
||||
value[key] = invalid
|
||||
cases.append(value)
|
||||
value = receipt()
|
||||
del value['resource_events']['memory.events']['oom']
|
||||
cases.append(value)
|
||||
value = receipt()
|
||||
value['cpu_stat']['usage_usec'] = False
|
||||
cases.append(value)
|
||||
for value in cases:
|
||||
with self.subTest(value=value):
|
||||
path.write_text(json.dumps(value))
|
||||
self.assertIsNone(read_observation(path, 'own-unit'))
|
||||
|
||||
def test_retains_lingering_actor_facts_without_certifying_cleanup(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / 'receipt.json'
|
||||
value = receipt()
|
||||
value['remaining_processes'].append(101)
|
||||
value['payload_cleanup_verified'] = False
|
||||
path.write_text(json.dumps(value))
|
||||
observed = read_observation(path, 'own-unit')
|
||||
self.assertEqual(observed['remaining_processes'], [100, 101])
|
||||
self.assertFalse(observed['payload_cleanup_verified'])
|
||||
|
||||
def test_excludes_unrelated_metadata_and_foreign_symlinks(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / 'receipt.json'
|
||||
value = receipt()
|
||||
value['environment'] = {'ambient': 'excluded'}
|
||||
value['resource_events']['unrelated'] = {'ambient': 'excluded'}
|
||||
path.write_text(json.dumps(value))
|
||||
observed = read_observation(path, 'own-unit')
|
||||
self.assertNotIn('environment', observed)
|
||||
self.assertNotIn('unrelated', observed['resource_events'])
|
||||
link = Path(directory) / 'linked.json'
|
||||
link.symlink_to(path)
|
||||
self.assertIsNone(read_observation(link, 'own-unit'))
|
||||
self.assertEqual(json.loads(path.read_text()), value)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,60 @@
|
||||
# Selected unknown retrieval
|
||||
|
||||
`list_unknowns` now accepts optional paging. Omitted limits still return all
|
||||
matching heads. Selection happens before payload cloning in the ledger owner;
|
||||
MCP returns complete selected records and the total matching count.
|
||||
|
||||
## Bounded workflow comparison
|
||||
|
||||
The task is to retrieve the first ten complete current unknown heads, in stable
|
||||
ID order, from a session containing 200 heads. The complete-list workflow reads
|
||||
all heads and selects ten locally. The selected-page workflow requests
|
||||
`{"offset":0,"limit":10}`. Both use one real SDK MCP call through
|
||||
`Client`/`InMemoryTransport`, with the same imported session and no provider.
|
||||
All six executions returned identical selected records and the expected total.
|
||||
|
||||
These are two workflows on the candidate implementation, **not** a historical
|
||||
before/after binary comparison. The candidate was built from `f09bb8a99` plus
|
||||
the paging changes. Runtime file-table SHA-256 for the preserved complete-list
|
||||
copy: `06a457dac90912dceba88f945573897372701f6c6b2e0934d7028023b4646648`.
|
||||
Node 24.18.0, locked MCP SDK 2.3.1, tokenx 1.3.0, macOS arm64.
|
||||
|
||||
Flameox 0.2.9 ran three randomized paired blocks, seed 731, with a 30-second
|
||||
and 1-GiB limit per execution. Each fresh process performed a first read and
|
||||
five identical warm reads. The semantic oracle checked the selected answer
|
||||
SHA-256, `c18d718a71323eb53712040b4b09ffebfc3497000d1ff8f58d57665638077310`.
|
||||
Every execution and oracle passed.
|
||||
|
||||
| Measurement | Complete list | Selected page |
|
||||
| ----------------------------------------- | ------------: | ------------: |
|
||||
| Calls / errors per task | 1 / 0 | 1 / 0 |
|
||||
| Serialized response bytes | 646,855 | 32,603 |
|
||||
| Estimated request tokens | 12 | 21 |
|
||||
| Estimated response tokens | 151,097 | 7,647 |
|
||||
| Estimated full catalog tokens | 764,512 | 764,512 |
|
||||
| Median first-read elapsed milliseconds | 1,084.05 | 1,082.65 |
|
||||
| Median warm-read elapsed milliseconds | 3.943 | 0.426 |
|
||||
| Median first-read CPU milliseconds | 1,333.23 | 1,328.47 |
|
||||
| Median warm-read CPU milliseconds | 5.012 | 0.591 |
|
||||
| Median whole-process elapsed milliseconds | 1,752.40 | 1,664.42 |
|
||||
| Median peak process-family MiB | 598.59 | 554.09 |
|
||||
|
||||
Read timings include the MCP call and answer validation. First-read timings
|
||||
exclude startup, imports, session preparation, handshake and catalog discovery;
|
||||
the whole-process elapsed measurement includes those, all six reads, token
|
||||
estimation and cleanup. Whole-process CPU was not measured. Peak RSS includes
|
||||
preparation and imports; this workload has no child processes. Host caches were
|
||||
not flushed. Three pairs and fifteen warm reads per workflow describe this
|
||||
fixture only; they do not establish broad speed or memory improvements.
|
||||
|
||||
Token counts estimate serialized request, response and catalog JSON, including
|
||||
both MCP response representations. They are not measured model tokens. Host
|
||||
schema injection, prompts, reasoning, retries and actual agent behavior are
|
||||
unmeasured. The full catalog cost is unchanged between these workflows and can
|
||||
dominate a first task. Smaller selected delivery does not remove that cost.
|
||||
|
||||
Native Flameox evidence was preserved locally as
|
||||
`14ea4c0ef6595a1b89645299d4e184a582d5e975f7916294d68c1f1b312608d0`,
|
||||
analysis `f66e125179623591ce00b1ccf0d917c2e90512de55db1061269c042d65e1125d`.
|
||||
An earlier 512-MiB capture exceeded its memory limit and is excluded from these
|
||||
measurements. No production provider or model trial was run.
|
||||
@@ -0,0 +1,504 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { createHash } from "node:crypto";
|
||||
import { execFile } from "node:child_process";
|
||||
import { createReadStream } from "node:fs";
|
||||
import {
|
||||
access,
|
||||
chmod,
|
||||
lstat,
|
||||
mkdir,
|
||||
readFile,
|
||||
readdir,
|
||||
rm,
|
||||
writeFile,
|
||||
} from "node:fs/promises";
|
||||
import { isAbsolute, join } from "node:path";
|
||||
import { setTimeout as delay } from "node:timers/promises";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { parseArgs, promisify } from "node:util";
|
||||
import { z } from "zod";
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
|
||||
const { values } = parseArgs({
|
||||
options: {
|
||||
"runtime-root": { type: "string" },
|
||||
revision: { type: "string" },
|
||||
path: { type: "string" },
|
||||
"peer-path": { type: "string" },
|
||||
providers: { type: "string" },
|
||||
output: { type: "string" },
|
||||
},
|
||||
});
|
||||
for (const key of ["runtime-root", "path", "peer-path", "providers", "output"])
|
||||
assert.ok(isAbsolute(values[key] ?? ""), "Supply --" + key + " ABS");
|
||||
assert.match(values.revision ?? "", /^[a-f0-9]{40}$/u);
|
||||
assert.equal(process.platform, "linux");
|
||||
assert.equal(process.arch, "x64");
|
||||
assert.notEqual(
|
||||
process.getuid(),
|
||||
0,
|
||||
"Permission faults require a non-root user",
|
||||
);
|
||||
const runtime = values["runtime-root"],
|
||||
lab = values.output;
|
||||
const providers = z
|
||||
.record(z.string(), z.string())
|
||||
.parse(JSON.parse(await readFile(values.providers, "utf8")));
|
||||
const sha = (bytes) => createHash("sha256").update(bytes).digest("hex");
|
||||
const fileSha = async (path) => {
|
||||
const hash = createHash("sha256");
|
||||
for await (const part of createReadStream(path)) hash.update(part);
|
||||
return hash.digest("hex");
|
||||
};
|
||||
const sourceBefore = await fileSha(new URL(import.meta.url));
|
||||
const targetBefore = await fileSha(values.path);
|
||||
const peerBefore = await fileSha(values["peer-path"]);
|
||||
const corpus = JSON.parse(
|
||||
await readFile(new URL("./corpus.json", import.meta.url)),
|
||||
);
|
||||
assert.ok(corpus.artifacts.some((x) => x.sha256 === targetBefore));
|
||||
assert.equal(
|
||||
corpus.artifacts.find((x) => x.id === "sqlite.wasm").sha256,
|
||||
peerBefore,
|
||||
);
|
||||
const load = (path) => import(pathToFileURL(join(runtime, "dist", path)));
|
||||
const { toolContract } = await load("contracts/toolContracts.js");
|
||||
const { analysisErrorProjectionSchema } = await load(
|
||||
"contracts/errorSchemas.js",
|
||||
);
|
||||
const { parseEvidence } = await load("domain/evidence.js");
|
||||
const { evmoleInterfaceReplySchema } = await load(
|
||||
"evm/EvmoleInterfaceAdapter.js",
|
||||
);
|
||||
const {
|
||||
observeOwnedProcessLineage,
|
||||
observeProcessStartIdentity,
|
||||
systemProcessOwnershipHost: host,
|
||||
} = await load("process/ProcessOwnershipObservation.js");
|
||||
const { verifyNoTokenOwnedProcesses } = await load(
|
||||
"process/ProcessOwnership.js",
|
||||
);
|
||||
const { createVerifierRun, completeVerifierRun } = await import(
|
||||
pathToFileURL(join(runtime, "scripts/lib/verifier-run.mjs"))
|
||||
);
|
||||
const run = createVerifierRun();
|
||||
await mkdir(lab, { mode: 0o700 });
|
||||
const runtimeFiles = [];
|
||||
const fingerprint = async (directory) => {
|
||||
for (const entry of await readdir(directory, { withFileTypes: true })) {
|
||||
const path = join(directory, entry.name);
|
||||
if (entry.isDirectory()) await fingerprint(path);
|
||||
else if (entry.isFile())
|
||||
runtimeFiles.push({
|
||||
path: path.slice(runtime.length + 1),
|
||||
sha256: await fileSha(path),
|
||||
});
|
||||
else throw new Error("Runtime fingerprint must not follow symlinks");
|
||||
}
|
||||
};
|
||||
await fingerprint(join(runtime, "dist"));
|
||||
await fingerprint(join(runtime, "bridge"));
|
||||
runtimeFiles.sort((a, b) => (a.path < b.path ? -1 : a.path > b.path ? 1 : 0));
|
||||
await writeFile(join(lab, "runtime-files.json"), JSON.stringify(runtimeFiles), {
|
||||
flag: "wx",
|
||||
mode: 0o600,
|
||||
});
|
||||
const work = join(lab, "temporary");
|
||||
await mkdir(work, { mode: 0o700 });
|
||||
const wrapper = join(lab, "gated-prlimit.mjs");
|
||||
// Only gate command completion AFTER unchanged EVMole has produced its reply.
|
||||
// This does not measure interruption inside its synchronous WASM computation.
|
||||
await writeFile(
|
||||
wrapper,
|
||||
`#!${process.execPath}
|
||||
import { spawn } from "node:child_process";
|
||||
import { access, appendFile, readFile, rename, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { setTimeout as delay } from "node:timers/promises";
|
||||
const gate = process.env.REA_VERIFY_EVM_GATE;
|
||||
await appendFile(join(gate, "launches.log"), process.cwd() + "\\n", { mode: 0o600 });
|
||||
const child = spawn("/usr/bin/prlimit", process.argv.slice(2), { stdio: "inherit" });
|
||||
const code = await new Promise((resolve, reject) => {
|
||||
child.once("error", reject);
|
||||
child.once("exit", (code, signal) => signal ? reject(new Error(signal)) : resolve(code));
|
||||
});
|
||||
if (code !== 0) process.exit(code ?? 1);
|
||||
if (await readFile(join(gate, "mode"), "utf8") === "hold") {
|
||||
await writeFile(join(gate, "ready.pending.json"), JSON.stringify({
|
||||
pid: process.pid, parent_pid: process.ppid, run_id: process.env.REA_PROCESS_RUN_ID,
|
||||
root: process.cwd(), worker_exit: code,
|
||||
}), { flag: "wx", mode: 0o600 });
|
||||
await rename(join(gate, "ready.pending.json"), join(gate, "ready.json"));
|
||||
const end = Date.now() + 15000;
|
||||
for (;;) {
|
||||
try { await access(join(gate, "release")); break; }
|
||||
catch (error) { if (error.code !== "ENOENT") throw error; }
|
||||
if (Date.now() >= end) throw new Error("Verification gate exceeded 15 seconds");
|
||||
await delay(25);
|
||||
}
|
||||
}
|
||||
`,
|
||||
{ flag: "wx", mode: 0o700 },
|
||||
);
|
||||
const markerSchema = z.strictObject({
|
||||
pid: z.number().int().positive(),
|
||||
parent_pid: z.number().int().positive(),
|
||||
run_id: z.string().regex(/^rea-evm-interface-[a-f0-9-]+$/u),
|
||||
root: z.string(),
|
||||
worker_exit: z.literal(0),
|
||||
});
|
||||
const roots = async () =>
|
||||
(await readdir(work)).filter((x) => x.startsWith("rea-evm-interface-"));
|
||||
const gateMode = async (mode) => {
|
||||
await rm(join(lab, "ready.json"), { force: true });
|
||||
await rm(join(lab, "ready.pending.json"), { force: true });
|
||||
await rm(join(lab, "release"), { force: true });
|
||||
await writeFile(join(lab, "mode"), mode, { mode: 0o600 });
|
||||
};
|
||||
await gateMode("off");
|
||||
const client = new Client({ name: "rea-real-evm-lifecycle", version: "1" });
|
||||
const environment = {
|
||||
...process.env,
|
||||
...providers,
|
||||
TMPDIR: work,
|
||||
REA_EVM_PRLIMIT_COMMAND: wrapper,
|
||||
REA_VERIFY_EVM_GATE: lab,
|
||||
};
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [join(runtime, "scripts/rea.mjs"), "mcp"],
|
||||
env: environment,
|
||||
stderr: "pipe",
|
||||
});
|
||||
const rows = [];
|
||||
let fault;
|
||||
const textValue = (reply) =>
|
||||
JSON.parse(reply.content.find((x) => x.type === "text").text);
|
||||
const errorValue = (reply) => {
|
||||
assert.equal(reply.isError, true);
|
||||
return analysisErrorProjectionSchema.parse(textValue(reply).error);
|
||||
};
|
||||
const evidenceValue = (reply, name) => {
|
||||
assert.notEqual(reply.isError, true, JSON.stringify(reply));
|
||||
return parseEvidence(
|
||||
toolContract(name).outputSchema.parse(reply.structuredContent),
|
||||
);
|
||||
};
|
||||
const request = (path = values.path, signal) =>
|
||||
client.callTool(
|
||||
{
|
||||
name: "inspect_evm_interface",
|
||||
arguments: { path, encoding: "hex" },
|
||||
},
|
||||
{ timeout: 45000, ...(signal === undefined ? {} : { signal }) },
|
||||
);
|
||||
const waitFor = async (probe, message, milliseconds = 10000) => {
|
||||
const end = Date.now() + milliseconds;
|
||||
while (Date.now() < end) {
|
||||
const value = await probe();
|
||||
if (value) return value;
|
||||
await delay(50);
|
||||
}
|
||||
throw new Error(message);
|
||||
};
|
||||
const observeGate = async (expectedSuccess) => {
|
||||
const marker = await waitFor(async () => {
|
||||
try {
|
||||
return markerSchema.parse(
|
||||
JSON.parse(await readFile(join(lab, "ready.json"))),
|
||||
);
|
||||
} catch (error) {
|
||||
if (error.code === "ENOENT") return;
|
||||
throw error;
|
||||
}
|
||||
}, "No completed real worker at the pending command gate");
|
||||
assert.equal(marker.root.startsWith(work + "/rea-evm-interface-"), true);
|
||||
const rootStat = await lstat(marker.root);
|
||||
assert.equal(rootStat.isDirectory(), true);
|
||||
assert.equal(rootStat.uid, process.getuid());
|
||||
const processRow = (await host.listProcesses()).find(
|
||||
(x) => x.pid === marker.pid,
|
||||
);
|
||||
assert.ok(processRow, "Gated owned launcher already exited");
|
||||
const ownership = {
|
||||
runId: marker.run_id,
|
||||
leaderPid: marker.pid,
|
||||
processGroupId: processRow.processGroupId,
|
||||
expectedParentPid: marker.parent_pid,
|
||||
};
|
||||
const lineage = await observeOwnedProcessLineage(ownership);
|
||||
assert.equal(lineage.status, "verified", JSON.stringify(lineage));
|
||||
const identity = await observeProcessStartIdentity(
|
||||
marker.pid,
|
||||
undefined,
|
||||
marker.run_id,
|
||||
);
|
||||
assert.equal(identity?.state, "readable");
|
||||
const rawBytes = await readFile(join(marker.root, "reply.json"));
|
||||
const raw = JSON.parse(rawBytes);
|
||||
assert.equal(raw.ok, expectedSuccess);
|
||||
if (raw.ok) {
|
||||
assert.equal(raw.version, "0.9.3");
|
||||
evmoleInterfaceReplySchema.parse(raw.raw);
|
||||
} else assert.equal(raw.reason, "format");
|
||||
return {
|
||||
marker,
|
||||
ownership,
|
||||
lineage,
|
||||
identity,
|
||||
root_mode: rootStat.mode & 0o777,
|
||||
worker_reply_sha256: sha(rawBytes),
|
||||
worker_reply: raw,
|
||||
};
|
||||
};
|
||||
const confirmCleanup = async (launch, baseline, absentRoot) =>
|
||||
waitFor(async () => {
|
||||
const result = await verifyNoTokenOwnedProcesses(
|
||||
launch.marker.run_id,
|
||||
undefined,
|
||||
baseline,
|
||||
{
|
||||
leaderPid: launch.ownership.leaderPid,
|
||||
processGroupId: launch.ownership.processGroupId,
|
||||
},
|
||||
);
|
||||
if (!result.cleaned) return;
|
||||
if (absentRoot && (await roots()).length !== 0) return;
|
||||
return result;
|
||||
}, "Owned EVM cleanup not verified");
|
||||
const peer = async () =>
|
||||
evidenceValue(
|
||||
await client.callTool(
|
||||
{
|
||||
name: "inspect_wasm_artifact",
|
||||
arguments: {
|
||||
path: values["peer-path"],
|
||||
glue_paths: [],
|
||||
candidate_paths: [],
|
||||
detail: "summary",
|
||||
},
|
||||
},
|
||||
{ timeout: 45000 },
|
||||
),
|
||||
"inspect_wasm_artifact",
|
||||
);
|
||||
const restore = async () => {
|
||||
if (fault === undefined) return;
|
||||
await chmod(fault.path, fault.mode);
|
||||
fault = undefined;
|
||||
};
|
||||
const verifyWithCleanup = async (verify, cleanups) => {
|
||||
const failures = [];
|
||||
let value;
|
||||
try {
|
||||
value = await verify();
|
||||
} catch (error) {
|
||||
failures.push(error);
|
||||
}
|
||||
for (const cleanup of cleanups) {
|
||||
try {
|
||||
await cleanup();
|
||||
} catch (error) {
|
||||
failures.push(error);
|
||||
}
|
||||
}
|
||||
if (failures.length > 0)
|
||||
throw new AggregateError(failures, "Verification or cleanup failed");
|
||||
return value;
|
||||
};
|
||||
const verifyLateCancellation = async (original) => {
|
||||
const baseline = await host.captureBaseline();
|
||||
await gateMode("hold");
|
||||
const controller = new AbortController();
|
||||
let settled = false;
|
||||
const pending = request(values.path, controller.signal).then(
|
||||
(reply) => {
|
||||
settled = true;
|
||||
return { reply };
|
||||
},
|
||||
(error) => {
|
||||
settled = true;
|
||||
return {
|
||||
error: { name: error.name, code: error.code, message: error.message },
|
||||
};
|
||||
},
|
||||
);
|
||||
let launch;
|
||||
try {
|
||||
launch = await observeGate(true);
|
||||
assert.deepEqual(launch.worker_reply.raw, original.raw_result);
|
||||
assert.deepEqual(launch.worker_reply.bytecode, {
|
||||
sha256: original.normalized_result.bytecode.sha256,
|
||||
bytes: original.normalized_result.bytecode.bytes,
|
||||
hex: original.normalized_result.bytecode.hex,
|
||||
});
|
||||
assert.equal(settled, false, "Request completed before SDK cancellation");
|
||||
} finally {
|
||||
controller.abort(
|
||||
new Error("Late real EVM worker cancellation verification"),
|
||||
);
|
||||
await pending;
|
||||
}
|
||||
const canceled = await pending;
|
||||
assert.equal(canceled.error?.code, "REQUEST_TIMEOUT");
|
||||
const cleanup = await confirmCleanup(launch, baseline, true);
|
||||
await assert.rejects(access(launch.marker.root), { code: "ENOENT" });
|
||||
await gateMode("off");
|
||||
const recovered = evidenceValue(await request(), "inspect_evm_interface");
|
||||
assert.deepEqual(recovered.normalized_result, original.normalized_result);
|
||||
return {
|
||||
scenario: "late-cancellation",
|
||||
launch,
|
||||
canceled,
|
||||
cleanup,
|
||||
recovery_evidence_id: recovered.evidence_id,
|
||||
};
|
||||
};
|
||||
await verifyWithCleanup(async () => {
|
||||
await client.connect(transport);
|
||||
const catalog = await client.listTools();
|
||||
assert.equal(
|
||||
catalog.tools.filter((x) => x.name === "inspect_evm_interface").length,
|
||||
1,
|
||||
);
|
||||
const original = evidenceValue(await request(), "inspect_evm_interface");
|
||||
assert.equal(original.subject.digest.sha256, targetBefore);
|
||||
assert.equal(original.provider.version, "evmole@0.9.3");
|
||||
const cli = await promisify(execFile)(
|
||||
process.execPath,
|
||||
[
|
||||
join(runtime, "scripts/rea.mjs"),
|
||||
"inspect-evm-interface",
|
||||
values.path,
|
||||
"hex",
|
||||
"--json",
|
||||
],
|
||||
{ env: environment, timeout: 45000, maxBuffer: 32 * 1024 * 1024 },
|
||||
);
|
||||
const cliEvidence = parseEvidence(JSON.parse(cli.stdout));
|
||||
assert.deepEqual(cliEvidence.normalized_result, original.normalized_result);
|
||||
assert.deepEqual(cliEvidence.parameters, original.parameters);
|
||||
assert.deepEqual(cliEvidence.subject, original.subject);
|
||||
await writeFile(join(lab, "cli.json"), cli.stdout, {
|
||||
flag: "wx",
|
||||
mode: 0o600,
|
||||
});
|
||||
await writeFile(join(lab, "original.json"), JSON.stringify(original), {
|
||||
flag: "wx",
|
||||
mode: 0o600,
|
||||
});
|
||||
rows.push(await verifyLateCancellation(original));
|
||||
|
||||
const malformed = join(lab, "odd-nibble.hex");
|
||||
await writeFile(malformed, "0x0\n", { flag: "wx", mode: 0o600 });
|
||||
for (const scenario of ["success", "format-failure"]) {
|
||||
const baseline = await host.captureBaseline();
|
||||
await gateMode("hold");
|
||||
const pending = request(scenario === "success" ? values.path : malformed);
|
||||
let launch;
|
||||
await verifyWithCleanup(async () => {
|
||||
launch = await observeGate(scenario === "success");
|
||||
fault = { path: launch.marker.root, mode: launch.root_mode };
|
||||
await chmod(fault.path, 0o500);
|
||||
await writeFile(join(lab, "release"), "release", {
|
||||
flag: "wx",
|
||||
mode: 0o600,
|
||||
});
|
||||
const failed = errorValue(await pending);
|
||||
assert.equal(failed.code, "cleanup_incomplete");
|
||||
assert.deepEqual(failed.details.resources, [launch.marker.root]);
|
||||
assert.match(failed.details.diagnostics.reason, /EACCES/u);
|
||||
const previous = failed.details.diagnostics.previous_error;
|
||||
if (scenario === "success") assert.equal(previous, null);
|
||||
else assert.equal(previous.category, "invalid_input");
|
||||
const cleanup = await confirmCleanup(launch, baseline, false);
|
||||
assert.deepEqual(await roots(), [
|
||||
launch.marker.root.slice(work.length + 1),
|
||||
]);
|
||||
await gateMode("off");
|
||||
const launchLogBefore = await readFile(join(lab, "launches.log"));
|
||||
const retryBeforeRepair = errorValue(await request());
|
||||
assert.equal(retryBeforeRepair.code, "cleanup_incomplete");
|
||||
assert.deepEqual(retryBeforeRepair.details.resources, [
|
||||
launch.marker.root,
|
||||
]);
|
||||
assert.deepEqual(
|
||||
await readFile(join(lab, "launches.log")),
|
||||
launchLogBefore,
|
||||
);
|
||||
const peerEvidence = await peer();
|
||||
assert.equal(peerEvidence.subject.digest.sha256, peerBefore);
|
||||
await restore();
|
||||
const recovered = evidenceValue(await request(), "inspect_evm_interface");
|
||||
assert.deepEqual(recovered.normalized_result, original.normalized_result);
|
||||
await assert.rejects(access(launch.marker.root), { code: "ENOENT" });
|
||||
assert.deepEqual(await roots(), []);
|
||||
rows.push({
|
||||
scenario: "cleanup-retry-after-" + scenario,
|
||||
launch,
|
||||
failed,
|
||||
cleanup,
|
||||
retry_before_permission_repair: retryBeforeRepair,
|
||||
launch_log_sha256_before_retry: sha(launchLogBefore),
|
||||
peer_evidence_id: peerEvidence.evidence_id,
|
||||
recovery_evidence_id: recovered.evidence_id,
|
||||
});
|
||||
}, [
|
||||
restore,
|
||||
() => writeFile(join(lab, "release"), "release", { mode: 0o600 }),
|
||||
() => pending,
|
||||
]);
|
||||
}
|
||||
}, [
|
||||
restore,
|
||||
() => writeFile(join(lab, "release"), "release", { mode: 0o600 }),
|
||||
() => client.close(),
|
||||
() => transport.close(),
|
||||
]);
|
||||
assert.deepEqual(await roots(), []);
|
||||
const finalRun = await completeVerifierRun(run);
|
||||
assert.equal(
|
||||
finalRun.process_lineage.status,
|
||||
"verified",
|
||||
JSON.stringify(finalRun),
|
||||
);
|
||||
assert.deepEqual(finalRun.process_lineage.descendants, []);
|
||||
assert.equal(await fileSha(values.path), targetBefore);
|
||||
assert.equal(await fileSha(values["peer-path"]), peerBefore);
|
||||
assert.equal(await fileSha(new URL(import.meta.url)), sourceBefore);
|
||||
const runtimeTable = JSON.stringify(runtimeFiles);
|
||||
runtimeFiles.length = 0;
|
||||
await fingerprint(join(runtime, "dist"));
|
||||
await fingerprint(join(runtime, "bridge"));
|
||||
runtimeFiles.sort((a, b) => (a.path < b.path ? -1 : a.path > b.path ? 1 : 0));
|
||||
assert.equal(
|
||||
JSON.stringify(runtimeFiles),
|
||||
runtimeTable,
|
||||
"Runtime changed during verification",
|
||||
);
|
||||
await writeFile(
|
||||
join(lab, "summary.json"),
|
||||
JSON.stringify({
|
||||
revision: values.revision,
|
||||
verifier_run: finalRun,
|
||||
verifier_sha256: sourceBefore,
|
||||
runtime_file_table_sha256: sha(JSON.stringify(runtimeFiles)),
|
||||
cli_default_parity: true,
|
||||
fixture_sha256: await fileSha(wrapper),
|
||||
artifact: { path: values.path, sha256: targetBefore },
|
||||
peer: { path: values["peer-path"], sha256: peerBefore },
|
||||
scope:
|
||||
"Linux x64; actual EVMole reply before fixture-held command completion; no mid-WASM interruption claim",
|
||||
rows,
|
||||
source_bytes_unchanged: true,
|
||||
target_bytes_unchanged: true,
|
||||
peer_bytes_unchanged: true,
|
||||
owned_runtime_roots_removed: true,
|
||||
runtime_files_unchanged: true,
|
||||
}),
|
||||
{ flag: "wx", mode: 0o600 },
|
||||
);
|
||||
console.log(
|
||||
JSON.stringify({ pass: true, scenarios: rows.map((x) => x.scenario) }),
|
||||
);
|
||||
@@ -0,0 +1,167 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { execFile } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import { isAbsolute, resolve } from "node:path";
|
||||
import { parseArgs, promisify } from "node:util";
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
|
||||
const { values } = parseArgs({
|
||||
options: {
|
||||
"runtime-root": { type: "string" },
|
||||
revision: { type: "string" },
|
||||
bundle: { type: "string" },
|
||||
output: { type: "string" },
|
||||
},
|
||||
});
|
||||
for (const key of ["runtime-root", "bundle", "output"])
|
||||
assert.ok(values[key] && isAbsolute(values[key]), `Supply --${key} ABS`);
|
||||
assert.ok(values.revision);
|
||||
await mkdir(values.output, { mode: 0o700 });
|
||||
const sha = (bytes) => createHash("sha256").update(bytes).digest("hex");
|
||||
const bytes = await readFile(values.bundle);
|
||||
const original = JSON.parse(bytes.toString("utf8"));
|
||||
const parents = original.records.filter(
|
||||
(record) => record.operation === "inspect_evm_interface",
|
||||
);
|
||||
assert.ok(parents.length > 0, "Select a bundle containing old EVM Evidence");
|
||||
const entrypoint = resolve(values["runtime-root"], "scripts/rea.mjs");
|
||||
const environment = {
|
||||
...process.env,
|
||||
REA_LOG_LEVEL: "silent",
|
||||
NODE_OPTIONS: "--max-old-space-size=512",
|
||||
// If a view accidentally starts analysis, its required limiter is unavailable.
|
||||
REA_EVM_PRLIMIT_COMMAND: resolve(values.output, "absent-prlimit"),
|
||||
};
|
||||
const client = new Client({ name: "rea-old-evm-retention", version: "1" });
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [entrypoint, "mcp"],
|
||||
env: environment,
|
||||
stderr: "pipe",
|
||||
});
|
||||
const definitions = new Map();
|
||||
const calls = [];
|
||||
const call = async (name, args) => {
|
||||
const definition = definitions.get(name);
|
||||
assert.ok(definition, `Missing named contract ${name}`);
|
||||
const reply = await client.callTool(
|
||||
{ name, arguments: args },
|
||||
{ toolDefinition: definition, timeout: 60_000 },
|
||||
);
|
||||
await writeFile(
|
||||
resolve(values.output, `call-${calls.length + 1}.json`),
|
||||
JSON.stringify(reply),
|
||||
{ flag: "wx", mode: 0o600 },
|
||||
);
|
||||
assert.notEqual(reply.isError, true, JSON.stringify(reply));
|
||||
const text = JSON.parse(reply.content.find((x) => x.type === "text").text);
|
||||
assert.deepEqual(reply.structuredContent, text);
|
||||
calls.push({ name, arguments: args, evidence_id: text.evidence_id ?? null });
|
||||
return text;
|
||||
};
|
||||
const checked = [];
|
||||
try {
|
||||
await client.connect(transport);
|
||||
transport.stderr?.resume();
|
||||
for (const definition of (await client.listTools()).tools)
|
||||
definitions.set(definition.name, definition);
|
||||
await call("import_evidence_bundle", { path: values.bundle });
|
||||
const imported = (await call("get_evidence_bundle", {})).result.records;
|
||||
for (const parent of parents) {
|
||||
assert.deepEqual(
|
||||
imported.find((record) => record.evidence_id === parent.evidence_id),
|
||||
parent,
|
||||
);
|
||||
const source = {
|
||||
kind: "retained-evidence",
|
||||
evidence_id: parent.evidence_id,
|
||||
};
|
||||
const view = await call("inspect_analysis_view", {
|
||||
source,
|
||||
view: { kind: "summary" },
|
||||
});
|
||||
const { hex, ...bytecode } = parent.normalized_result.bytecode;
|
||||
const { stdout, ...diagnostics } = parent.normalized_result.diagnostics;
|
||||
assert.deepEqual(view.normalized_result.summary, {
|
||||
...parent.normalized_result,
|
||||
bytecode,
|
||||
diagnostics: { ...diagnostics, stdout_code_units: stdout.length },
|
||||
});
|
||||
assert.equal(view.normalized_result.parent_evidence_id, parent.evidence_id);
|
||||
assert.notEqual(view.evidence_id, parent.evidence_id);
|
||||
assert.deepEqual(view.evidence_links, [parent.evidence_id]);
|
||||
for (const [facet, expected] of [
|
||||
["functions", parent.normalized_result.functions.slice(0, 2)],
|
||||
["bytecode_hex", { text: hex.slice(0, 64), unit: "utf16-code-units" }],
|
||||
]) {
|
||||
const page = await call("inspect_analysis_view", {
|
||||
source,
|
||||
view: {
|
||||
kind: "evm",
|
||||
facet,
|
||||
offset: 0,
|
||||
limit: facet === "functions" ? 2 : 64,
|
||||
},
|
||||
});
|
||||
assert.deepEqual(page.normalized_result.item, expected);
|
||||
assert.equal(
|
||||
page.normalized_result.parent_evidence_id,
|
||||
parent.evidence_id,
|
||||
);
|
||||
assert.deepEqual(page.evidence_links, [parent.evidence_id]);
|
||||
}
|
||||
const inlinePath = resolve(values.output, `inline-${checked.length}.json`);
|
||||
await writeFile(
|
||||
inlinePath,
|
||||
JSON.stringify({
|
||||
source: { kind: "inline", evidence: parent },
|
||||
view: { kind: "summary" },
|
||||
}),
|
||||
{ flag: "wx", mode: 0o600 },
|
||||
);
|
||||
const result = await promisify(execFile)(
|
||||
process.execPath,
|
||||
[entrypoint, "inspect-analysis-view", inlinePath, "--json"],
|
||||
{ env: environment, timeout: 60_000, maxBuffer: 16 * 1024 * 1024 },
|
||||
);
|
||||
const inline = JSON.parse(result.stdout);
|
||||
assert.equal(
|
||||
inline.normalized_result.parent_evidence_id,
|
||||
parent.evidence_id,
|
||||
);
|
||||
assert.deepEqual(
|
||||
inline.normalized_result.summary,
|
||||
view.normalized_result.summary,
|
||||
);
|
||||
checked.push(parent.evidence_id);
|
||||
}
|
||||
const path = resolve(values.output, "exported.json");
|
||||
await call("export_evidence_bundle", { path });
|
||||
const exported = JSON.parse(await readFile(path, "utf8"));
|
||||
for (const record of original.records)
|
||||
assert.deepEqual(
|
||||
exported.records.find((x) => x.evidence_id === record.evidence_id),
|
||||
record,
|
||||
);
|
||||
assert.deepEqual(await readFile(values.bundle), bytes);
|
||||
} finally {
|
||||
await client.close();
|
||||
}
|
||||
const report = {
|
||||
runtime_revision: values.revision,
|
||||
original_bundle_sha256: sha(bytes),
|
||||
checked_original_evidence_ids: checked,
|
||||
original_records_and_source_bytes_unchanged: true,
|
||||
exact_summaries_pages_and_cli_inline_pass: true,
|
||||
calls,
|
||||
scope:
|
||||
"Old EVM Evidence through actual public SDK/CLI with an unavailable analysis limiter; original inferred facts, explicit unknowns, parent identities and export records preserved. Caller supplies the frozen runtime revision; this is migration verification, not a performance pair.",
|
||||
};
|
||||
await writeFile(
|
||||
resolve(values.output, "summary.json"),
|
||||
JSON.stringify(report, null, 2),
|
||||
{ flag: "wx", mode: 0o600 },
|
||||
);
|
||||
process.stdout.write(JSON.stringify(report) + "\n");
|
||||
@@ -0,0 +1,95 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { createHash } from "node:crypto";
|
||||
import { readFile, writeFile } from "node:fs/promises";
|
||||
import { isAbsolute } from "node:path";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { parseArgs } from "node:util";
|
||||
|
||||
const { values } = parseArgs({
|
||||
options: Object.fromEntries(
|
||||
["artifact", "artifact-json", "evidence", "keccak-module", "output"].map(
|
||||
(key) => [key, { type: "string" }],
|
||||
),
|
||||
),
|
||||
});
|
||||
assert.ok(values.artifact, "Select the exact corpus artifact ID");
|
||||
for (const key of ["artifact-json", "evidence", "keccak-module", "output"])
|
||||
assert.ok(isAbsolute(values[key] ?? ""), `${key} requires an absolute path`);
|
||||
const corpus = JSON.parse(
|
||||
await readFile(new URL("./corpus.json", import.meta.url), "utf8"),
|
||||
);
|
||||
const selected = corpus.artifacts.find(({ id }) => id === values.artifact);
|
||||
assert.ok(selected?.provenance.compiler_artifact_sha256);
|
||||
const source = await readFile(values["artifact-json"]);
|
||||
const hash = (bytes) => createHash("sha256").update(bytes).digest("hex");
|
||||
assert.equal(hash(source), selected.provenance.compiler_artifact_sha256);
|
||||
assert.equal(source.length, selected.provenance.compiler_artifact_bytes);
|
||||
const artifact = JSON.parse(source);
|
||||
const carrier = Buffer.from(artifact.deployedBytecode + "\n");
|
||||
assert.equal(hash(carrier), selected.sha256);
|
||||
assert.equal(carrier.length, selected.bytes);
|
||||
const { keccak_256 } = (await import(pathToFileURL(values["keccak-module"])))
|
||||
.default;
|
||||
assert.equal(
|
||||
typeof keccak_256,
|
||||
"function",
|
||||
"Supply js-sha3's Ethereum Keccak implementation",
|
||||
);
|
||||
const canonicalType = (input) =>
|
||||
input.type.startsWith("tuple")
|
||||
? `(${input.components.map(canonicalType).join(",")})${input.type.slice(5)}`
|
||||
: input.type;
|
||||
const selectors = artifact.abi
|
||||
.filter((entry) => entry.type === "function")
|
||||
.map((entry) => {
|
||||
const signature = `${entry.name}(${entry.inputs.map(canonicalType).join(",")})`;
|
||||
return { signature, selector: `0x${keccak_256(signature).slice(0, 8)}` };
|
||||
});
|
||||
const reply = JSON.parse(await readFile(values.evidence, "utf8"));
|
||||
const evidence = reply.structuredContent ?? reply;
|
||||
assert.equal(evidence.operation, "inspect_evm_interface");
|
||||
assert.equal(evidence.subject.digest.sha256, selected.sha256);
|
||||
assert.equal(evidence.normalized_result.artifact.sha256, selected.sha256);
|
||||
assert.equal(
|
||||
evidence.normalized_result.bytecode.sha256,
|
||||
selected.decoded_sha256,
|
||||
);
|
||||
assert.equal(
|
||||
hash(Buffer.from(evidence.normalized_result.bytecode.hex, "hex")),
|
||||
selected.decoded_sha256,
|
||||
);
|
||||
assert.equal(evidence.normalized_result.bytecode.bytes, selected.decoded_bytes);
|
||||
assert.equal(evidence.normalized_result.evidence_kind, "inferred");
|
||||
assert.equal(evidence.normalized_result.discovery_completeness, "unknown");
|
||||
const observed = evidence.normalized_result.functions
|
||||
.filter((entry) => entry.dispatch === "abi")
|
||||
.map((entry) => entry.selector);
|
||||
assert.deepEqual(
|
||||
observed.toSorted(),
|
||||
selectors.map((entry) => entry.selector).toSorted(),
|
||||
);
|
||||
await writeFile(
|
||||
values.output,
|
||||
JSON.stringify(
|
||||
{
|
||||
artifact: selected.id,
|
||||
compiler_artifact_sha256: hash(source),
|
||||
keccak_module_sha256: hash(await readFile(values["keccak-module"])),
|
||||
evidence_id: evidence.evidence_id,
|
||||
selector_set_matches_published_abi: true,
|
||||
selectors,
|
||||
scope:
|
||||
"Independent published ABI-derived selector oracle; no deployed authenticity, inferred argument/mutability equivalence or runtime verification is established.",
|
||||
},
|
||||
null,
|
||||
2,
|
||||
) + "\n",
|
||||
{ flag: "wx", mode: 0o600 },
|
||||
);
|
||||
process.stdout.write(
|
||||
JSON.stringify({
|
||||
artifact: selected.id,
|
||||
selector_count: selectors.length,
|
||||
selector_set_matches_published_abi: true,
|
||||
}) + "\n",
|
||||
);
|
||||
@@ -0,0 +1,395 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import {
|
||||
access,
|
||||
chmod,
|
||||
lstat,
|
||||
mkdir,
|
||||
readFile,
|
||||
readdir,
|
||||
writeFile,
|
||||
} from "node:fs/promises";
|
||||
import { dirname, join, isAbsolute } from "node:path";
|
||||
import { parseArgs } from "node:util";
|
||||
import { createReadStream } from "node:fs";
|
||||
import { setTimeout as delay } from "node:timers/promises";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { z } from "zod";
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
const { values } = parseArgs({
|
||||
options: {
|
||||
"runtime-root": { type: "string" },
|
||||
revision: { type: "string" },
|
||||
path: { type: "string" },
|
||||
providers: { type: "string" },
|
||||
output: { type: "string" },
|
||||
"disabled-analyzers": {
|
||||
type: "string",
|
||||
default: "Decompiler Switch Analysis",
|
||||
},
|
||||
"timeout-ms": { type: "string", default: "900000" },
|
||||
},
|
||||
});
|
||||
for (const key of ["runtime-root", "path", "providers", "output"])
|
||||
assert.ok(values[key] && isAbsolute(values[key]), "Supply --" + key + " ABS");
|
||||
assert.match(values.revision ?? "", /^[a-f0-9]{40}$/u);
|
||||
assert.equal(process.platform, "linux");
|
||||
assert.notEqual(
|
||||
process.getuid(),
|
||||
0,
|
||||
"Permission fault injection requires a non-root user",
|
||||
);
|
||||
const timeout = Number(values["timeout-ms"]);
|
||||
assert.ok(Number.isSafeInteger(timeout) && timeout > 0);
|
||||
const lab = values.output,
|
||||
runtime = values["runtime-root"];
|
||||
const { ghidraDisabledAnalyzersSchema } = await import(
|
||||
pathToFileURL(runtime + "/dist/config/ghidraDisabledAnalyzers.js")
|
||||
);
|
||||
assert.ok(
|
||||
ghidraDisabledAnalyzersSchema.parse(values["disabled-analyzers"]) !==
|
||||
undefined,
|
||||
"Select at least one analyzer to disable",
|
||||
);
|
||||
await mkdir(lab, { mode: 0o700 });
|
||||
const providerBytes = await readFile(values.providers);
|
||||
const providers = JSON.parse(providerBytes.toString("utf8"));
|
||||
assert.ok(
|
||||
providers !== null &&
|
||||
!Array.isArray(providers) &&
|
||||
typeof providers === "object",
|
||||
);
|
||||
assert.ok(Object.values(providers).every((x) => typeof x === "string"));
|
||||
const verifierSource = await readFile(new URL(import.meta.url));
|
||||
const {
|
||||
observeOwnedProcessLineage,
|
||||
observeProcessStartIdentity,
|
||||
systemProcessOwnershipHost: host,
|
||||
} = await import(
|
||||
pathToFileURL(runtime + "/dist/process/ProcessOwnershipObservation.js")
|
||||
);
|
||||
const { verifyNoTokenOwnedProcesses } = await import(
|
||||
pathToFileURL(runtime + "/dist/process/ProcessOwnership.js")
|
||||
);
|
||||
const target = values.path;
|
||||
const sha = (x) => createHash("sha256").update(x).digest("hex");
|
||||
const fileSha = async (path) => {
|
||||
const hash = createHash("sha256");
|
||||
for await (const part of createReadStream(path)) hash.update(part);
|
||||
return hash.digest("hex");
|
||||
};
|
||||
const before = await fileSha(target);
|
||||
const rows = [];
|
||||
const roots = async (work) =>
|
||||
(await readdir(work))
|
||||
.filter((x) => x.startsWith("rea-ghidra-"))
|
||||
.map((x) => join(work, x));
|
||||
const ownershipSchema = z
|
||||
.object({
|
||||
run_id: z.string().uuid(),
|
||||
pid: z.number().int().positive(),
|
||||
process_group_id: z.number().int().positive(),
|
||||
parent_pid: z.number().int().positive(),
|
||||
launcher: z.string().min(1),
|
||||
endpoint_path: z.string().min(1),
|
||||
ownership_kind: z.literal("posix-process-group"),
|
||||
})
|
||||
.passthrough();
|
||||
const manifest = async (root) =>
|
||||
ownershipSchema.parse(
|
||||
JSON.parse(await readFile(join(root, "ownership.json"), "utf8")),
|
||||
);
|
||||
const verifyWithCleanup = async (verify, cleanups) => {
|
||||
const failures = [];
|
||||
let result;
|
||||
try {
|
||||
result = await verify();
|
||||
} catch (error) {
|
||||
failures.push(error);
|
||||
}
|
||||
for (const cleanup of cleanups) {
|
||||
try {
|
||||
await cleanup();
|
||||
} catch (error) {
|
||||
failures.push(error);
|
||||
}
|
||||
}
|
||||
if (failures.length > 0)
|
||||
throw new AggregateError(
|
||||
failures,
|
||||
"Verification or resource cleanup failed",
|
||||
);
|
||||
return result;
|
||||
};
|
||||
const restoreProject = async (project, mode) => {
|
||||
if (project === undefined || mode === undefined) return;
|
||||
try {
|
||||
await chmod(project, mode);
|
||||
} catch (error) {
|
||||
if (error.code !== "ENOENT") throw error;
|
||||
}
|
||||
};
|
||||
const unreadyLaunch = async (work) => {
|
||||
for (const root of await roots(work)) {
|
||||
let record;
|
||||
try {
|
||||
record = await manifest(root);
|
||||
} catch (error) {
|
||||
if (error.code === "ENOENT") continue;
|
||||
throw error;
|
||||
}
|
||||
const lineage = await observeOwnedProcessLineage(owned(record));
|
||||
if (lineage.status !== "verified") continue;
|
||||
const identity = await observeProcessStartIdentity(
|
||||
record.pid,
|
||||
undefined,
|
||||
record.run_id,
|
||||
);
|
||||
if (identity?.state !== "readable") continue;
|
||||
try {
|
||||
await access(record.endpoint_path);
|
||||
} catch (error) {
|
||||
if (error.code !== "ENOENT") throw error;
|
||||
return { root, manifest: record, lineage, start_identity: identity };
|
||||
}
|
||||
}
|
||||
};
|
||||
const owned = (m) => ({
|
||||
runId: m.run_id,
|
||||
leaderPid: m.pid,
|
||||
processGroupId: m.process_group_id,
|
||||
expectedParentPid: m.parent_pid,
|
||||
expectedCommand: m.launcher,
|
||||
});
|
||||
const cancelStartup = async ({ client, defs, work, call }) => {
|
||||
const baseline = await host.captureBaseline();
|
||||
const controller = new AbortController();
|
||||
const pattern = "rea-lifecycle-" + randomUUID();
|
||||
let settled = false;
|
||||
const pending = client
|
||||
.callTool(
|
||||
{ name: "search_procedures", arguments: { pattern } },
|
||||
{
|
||||
toolDefinition: defs.get("search_procedures"),
|
||||
timeout,
|
||||
signal: controller.signal,
|
||||
},
|
||||
)
|
||||
.then(
|
||||
(reply) => {
|
||||
settled = true;
|
||||
return { reply };
|
||||
},
|
||||
(error) => {
|
||||
settled = true;
|
||||
return {
|
||||
error: {
|
||||
name: error.name,
|
||||
message: error.message,
|
||||
code: error.code,
|
||||
},
|
||||
};
|
||||
},
|
||||
);
|
||||
let launch;
|
||||
try {
|
||||
const deadline = Date.now() + 30000;
|
||||
while (Date.now() < deadline && !settled) {
|
||||
launch = await unreadyLaunch(work);
|
||||
if (launch) break;
|
||||
await delay(50);
|
||||
}
|
||||
assert.ok(launch, "No token-verified live JVM before endpoint readiness");
|
||||
assert.equal(settled, false);
|
||||
} finally {
|
||||
controller.abort(new Error("Preset startup cancellation"));
|
||||
await pending;
|
||||
}
|
||||
const canceled = await pending;
|
||||
assert.ok(canceled.error, "Pending SDK request did not cancel");
|
||||
await writeFile(work + "/cancellation.json", JSON.stringify(canceled), {
|
||||
flag: "wx",
|
||||
mode: 0o600,
|
||||
});
|
||||
let cleanup;
|
||||
const deadline = Date.now() + 30000;
|
||||
while (Date.now() < deadline) {
|
||||
cleanup = await verifyNoTokenOwnedProcesses(
|
||||
launch.manifest.run_id,
|
||||
undefined,
|
||||
baseline,
|
||||
{
|
||||
leaderPid: launch.manifest.pid,
|
||||
processGroupId: launch.manifest.process_group_id,
|
||||
},
|
||||
);
|
||||
const liveRoots = await roots(work);
|
||||
if (cleanup.cleaned && liveRoots.length === 0) break;
|
||||
await delay(100);
|
||||
}
|
||||
assert.equal(cleanup.cleaned, true, JSON.stringify(cleanup));
|
||||
assert.deepEqual(await roots(work), []);
|
||||
await assert.rejects(access(dirname(launch.manifest.endpoint_path)), {
|
||||
code: "ENOENT",
|
||||
});
|
||||
const afterCancel = (await call("binary_session")).result;
|
||||
assert.equal(afterCancel.open, true);
|
||||
assert.notEqual(afterCancel.analysis_activity.status, "busy");
|
||||
return { launch, cleanup, afterCancel, pattern };
|
||||
};
|
||||
for (const profile of ["default", "disabled"]) {
|
||||
const work = lab + "/" + profile;
|
||||
await mkdir(work, { mode: 0o700 });
|
||||
const client = new Client({
|
||||
name: "rea-real-preset-lifecycle",
|
||||
version: "1",
|
||||
});
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [runtime + "/scripts/rea.mjs", "mcp"],
|
||||
env: {
|
||||
...process.env,
|
||||
...providers,
|
||||
NODE_OPTIONS: "--max-old-space-size=512",
|
||||
TMPDIR: work,
|
||||
REA_GHIDRA_DISABLED_ANALYZERS:
|
||||
profile === "default" ? "" : values["disabled-analyzers"],
|
||||
},
|
||||
stderr: "pipe",
|
||||
});
|
||||
const defs = new Map();
|
||||
let count = 0;
|
||||
const call = async (name, args = {}, error = false) => {
|
||||
const definition = defs.get(name);
|
||||
assert.ok(definition, "Missing named contract " + name);
|
||||
const reply = await client.callTool(
|
||||
{ name, arguments: args },
|
||||
{ toolDefinition: definition, timeout },
|
||||
);
|
||||
await writeFile(
|
||||
work + "/call-" + ++count + ".json",
|
||||
JSON.stringify(reply),
|
||||
{ flag: "wx", mode: 0o600 },
|
||||
);
|
||||
const value =
|
||||
reply.structuredContent ??
|
||||
JSON.parse(reply.content.find((x) => x.type === "text").text);
|
||||
assert.equal(reply.isError === true, error, JSON.stringify(value));
|
||||
return value;
|
||||
};
|
||||
let project, mode;
|
||||
const row = await verifyWithCleanup(async () => {
|
||||
await client.connect(transport);
|
||||
transport.stderr?.resume();
|
||||
for (const t of (await client.listTools()).tools) defs.set(t.name, t);
|
||||
await call("open_binary", { path: target, provider_id: "ghidra" });
|
||||
const selectedProfile = (await call("binary_session")).result
|
||||
.analysis_provider_binding.analysis_profile;
|
||||
const { launch, cleanup, afterCancel, pattern } = await cancelStartup({
|
||||
client,
|
||||
defs,
|
||||
work,
|
||||
call,
|
||||
});
|
||||
const recovered = await call("search_procedures", { pattern });
|
||||
assert.deepEqual(recovered.normalized_result, []);
|
||||
const activeRoots = await roots(work);
|
||||
assert.equal(activeRoots.length, 1);
|
||||
const active = await manifest(activeRoots[0]);
|
||||
const activeIdentity = await observeProcessStartIdentity(
|
||||
active.pid,
|
||||
undefined,
|
||||
active.run_id,
|
||||
);
|
||||
assert.equal(activeIdentity?.state, "readable");
|
||||
assert.notEqual(activeIdentity.identity, launch.start_identity.identity);
|
||||
const activeLineage = await observeOwnedProcessLineage(owned(active));
|
||||
assert.equal(activeLineage.status, "verified");
|
||||
const activeBaseline = await host.captureBaseline();
|
||||
project = join(activeRoots[0], "project");
|
||||
const info = await lstat(project);
|
||||
assert.ok(info.isDirectory() && !info.isSymbolicLink());
|
||||
assert.equal(
|
||||
info.uid,
|
||||
process.getuid(),
|
||||
"Fault must affect only this user-owned project",
|
||||
);
|
||||
mode = info.mode & 0o777;
|
||||
await chmod(project, 0o500);
|
||||
const refusal = await verifyWithCleanup(
|
||||
() => call("close_binary", {}, true),
|
||||
[() => restoreProject(project, mode)],
|
||||
);
|
||||
assert.equal(refusal.error.code, "cleanup_incomplete");
|
||||
assert.ok(refusal.error.details.resources.includes(activeRoots[0]));
|
||||
await writeFile(work + "/cleanup-refusal.json", JSON.stringify(refusal), {
|
||||
flag: "wx",
|
||||
mode: 0o600,
|
||||
});
|
||||
assert.ok(
|
||||
(await roots(work)).includes(activeRoots[0]),
|
||||
"Failed cleanup resource ownership disappeared",
|
||||
);
|
||||
await call("close_binary");
|
||||
assert.deepEqual(await roots(work), []);
|
||||
await assert.rejects(access(dirname(active.endpoint_path)), {
|
||||
code: "ENOENT",
|
||||
});
|
||||
const retried = await verifyNoTokenOwnedProcesses(
|
||||
active.run_id,
|
||||
undefined,
|
||||
activeBaseline,
|
||||
{ leaderPid: active.pid, processGroupId: active.process_group_id },
|
||||
);
|
||||
assert.equal(retried.cleaned, true, JSON.stringify(retried));
|
||||
assert.equal((await call("binary_session")).result.open, false);
|
||||
assert.equal(await fileSha(target), before);
|
||||
return {
|
||||
profile,
|
||||
selected_profile: selectedProfile,
|
||||
source_sha256: before,
|
||||
actual_pending_startup_canceled: true,
|
||||
verified_launch: launch,
|
||||
token_cleanup: cleanup,
|
||||
post_cancellation_activity: afterCancel.analysis_activity,
|
||||
recovery_preserves_selected_target: true,
|
||||
logical_run_id_after_recovery: active.run_id,
|
||||
recovered_ownership: active,
|
||||
new_verified_process_generation_after_recovery: activeIdentity,
|
||||
real_permission_cleanup_failure: refusal,
|
||||
retry_cleanup: retried,
|
||||
source_bytes_unchanged: true,
|
||||
};
|
||||
}, [() => restoreProject(project, mode), () => client.close()]);
|
||||
rows.push(row);
|
||||
await writeFile(
|
||||
lab + "/progress.json",
|
||||
JSON.stringify(rows, null, 2) + "\n",
|
||||
{ mode: 0o600 },
|
||||
);
|
||||
process.stdout.write(JSON.stringify({ profile, status: "passed" }) + "\n");
|
||||
}
|
||||
assert.deepEqual(
|
||||
await readFile(new URL(import.meta.url)),
|
||||
verifierSource,
|
||||
"Verifier source changed during execution",
|
||||
);
|
||||
await writeFile(
|
||||
lab + "/summary.json",
|
||||
JSON.stringify(
|
||||
{
|
||||
caller_supplied_runtime_revision: values.revision,
|
||||
verifier_source_sha256: sha(verifierSource),
|
||||
provider_settings_sha256: sha(providerBytes),
|
||||
source_path: target,
|
||||
source_sha256: before,
|
||||
rows,
|
||||
scope:
|
||||
"Actual public MCP on Linux: token/start-identity verified pending Ghidra startup cancellation, fresh work under the selected target, controlled permission failure in an owned temporary project, retained-resource close retry and read-only token sweeps. Original input is unchanged. Runtime revision is caller-supplied; this is lifecycle verification, not paired performance or general OS coverage.",
|
||||
},
|
||||
null,
|
||||
2,
|
||||
) + "\n",
|
||||
{ flag: "wx", mode: 0o600 },
|
||||
);
|
||||
@@ -0,0 +1,387 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { execFile } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import { createReadStream } from "node:fs";
|
||||
import { mkdir, readFile, readdir, writeFile } from "node:fs/promises";
|
||||
import { isAbsolute, resolve } from "node:path";
|
||||
import { performance } from "node:perf_hooks";
|
||||
import { parseArgs, promisify, isDeepStrictEqual } from "node:util";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
|
||||
const { values } = parseArgs({
|
||||
options: {
|
||||
"runtime-root": { type: "string" },
|
||||
revision: { type: "string" },
|
||||
path: { type: "string" },
|
||||
providers: { type: "string" },
|
||||
strace: { type: "string" },
|
||||
output: { type: "string" },
|
||||
procedure: { type: "string" },
|
||||
"timeout-ms": { type: "string", default: "900000" },
|
||||
"disabled-analyzers": {
|
||||
type: "string",
|
||||
default: "Decompiler Switch Analysis",
|
||||
},
|
||||
},
|
||||
});
|
||||
for (const key of ["runtime-root", "path", "providers", "strace", "output"])
|
||||
assert.ok(values[key] && isAbsolute(values[key]), `Supply --${key} ABS`);
|
||||
assert.match(values.revision ?? "", /^[a-f0-9]{40}$/u);
|
||||
assert.equal(process.platform, "linux", "This verifier requires Linux strace");
|
||||
const timeout = Number(values["timeout-ms"]);
|
||||
assert.ok(Number.isSafeInteger(timeout) && timeout > 0);
|
||||
assert.ok(values["disabled-analyzers"].trim().length > 0);
|
||||
const verifierSource = await readFile(new URL(import.meta.url));
|
||||
await mkdir(values.output, { mode: 0o700 });
|
||||
const fileSha = async (path) => {
|
||||
const hash = createHash("sha256");
|
||||
for await (const chunk of createReadStream(path)) hash.update(chunk);
|
||||
return hash.digest("hex");
|
||||
};
|
||||
const before = await fileSha(values.path);
|
||||
const providerBytes = await readFile(values.providers);
|
||||
const providers = JSON.parse(providerBytes.toString("utf8"));
|
||||
assert.ok(
|
||||
providers !== null &&
|
||||
!Array.isArray(providers) &&
|
||||
typeof providers === "object",
|
||||
);
|
||||
assert.ok(Object.values(providers).every((value) => typeof value === "string"));
|
||||
const entrypoint = resolve(values["runtime-root"], "scripts/rea.mjs");
|
||||
const { parseAnalysisSnapshot } = await import(
|
||||
pathToFileURL(
|
||||
resolve(values["runtime-root"], "dist/domain/analysisSnapshot.js"),
|
||||
)
|
||||
);
|
||||
const { parseEvidence } = await import(
|
||||
pathToFileURL(resolve(values["runtime-root"], "dist/domain/evidence.js"))
|
||||
);
|
||||
const { createVerifierRun, completeVerifierRun } = await import(
|
||||
pathToFileURL(resolve(values["runtime-root"], "scripts/lib/verifier-run.mjs"))
|
||||
);
|
||||
const run = createVerifierRun();
|
||||
const snapshotPath = resolve(values.output, "default.snapshot.json");
|
||||
const changedSnapshotPath = resolve(values.output, "changed.snapshot.json");
|
||||
const tracedArgs = (work, args) => [
|
||||
"-ff",
|
||||
"-s",
|
||||
"4096",
|
||||
"-e",
|
||||
"trace=execve,execveat",
|
||||
"-o",
|
||||
resolve(work, "exec.trace"),
|
||||
process.execPath,
|
||||
entrypoint,
|
||||
...args,
|
||||
];
|
||||
const headlessLaunches = async (work) => {
|
||||
const files = (await readdir(work)).filter((name) =>
|
||||
name.startsWith("exec.trace."),
|
||||
);
|
||||
assert.ok(files.length > 0, "No actual syscall trace was recorded");
|
||||
const lines = (
|
||||
await Promise.all(
|
||||
files.map((name) => readFile(resolve(work, name), "utf8")),
|
||||
)
|
||||
).flatMap((text) => text.split("\n"));
|
||||
// This checks the complete known Ghidra launch argument, not arbitrary target execution.
|
||||
return lines.filter(
|
||||
(line) =>
|
||||
line.includes("ghidra.app.util.headless.AnalyzeHeadless") &&
|
||||
line.endsWith("= 0"),
|
||||
).length;
|
||||
};
|
||||
const environmentFor = (work, disabled) => ({
|
||||
...process.env,
|
||||
...providers,
|
||||
NODE_OPTIONS: "--max-old-space-size=512",
|
||||
TMPDIR: work,
|
||||
REA_PROCESS_RUN_ID: run.run_id,
|
||||
REA_GHIDRA_DISABLED_ANALYZERS: disabled,
|
||||
});
|
||||
const exactTaskFacts = (actual, expected) => {
|
||||
assert.deepEqual(actual.normalized_result, expected.normalized_result);
|
||||
assert.deepEqual(actual.subject, expected.subject);
|
||||
assert.deepEqual(actual.analysis_profile, expected.analysis_profile);
|
||||
assert.deepEqual(actual.parameters, expected.parameters);
|
||||
};
|
||||
let selected = values.procedure,
|
||||
original,
|
||||
profile,
|
||||
snapshotBytes;
|
||||
const rows = [];
|
||||
for (const scenario of [
|
||||
"default",
|
||||
"same-profile",
|
||||
"changed-profile",
|
||||
"restored-default",
|
||||
]) {
|
||||
const work = resolve(values.output, scenario);
|
||||
await mkdir(work, { mode: 0o700 });
|
||||
const environment = environmentFor(
|
||||
work,
|
||||
scenario === "changed-profile" ? values["disabled-analyzers"] : "",
|
||||
);
|
||||
const client = new Client({
|
||||
name: "rea-preset-snapshot-verifier",
|
||||
version: "1",
|
||||
});
|
||||
const transport = new StdioClientTransport({
|
||||
command: values.strace,
|
||||
args: tracedArgs(work, ["mcp"]),
|
||||
env: environment,
|
||||
stderr: "pipe",
|
||||
});
|
||||
const definitions = new Map();
|
||||
const calls = [];
|
||||
const call = async (name, args = {}, expectedError = false) => {
|
||||
const definition = definitions.get(name);
|
||||
assert.ok(definition, `Missing named contract ${name}`);
|
||||
const start = performance.now();
|
||||
const reply = await client.callTool(
|
||||
{ name, arguments: args },
|
||||
{ toolDefinition: definition, timeout },
|
||||
);
|
||||
await writeFile(
|
||||
resolve(work, `call-${calls.length + 1}.json`),
|
||||
JSON.stringify(reply),
|
||||
{ flag: "wx", mode: 0o600 },
|
||||
);
|
||||
const value =
|
||||
reply.structuredContent ??
|
||||
JSON.parse(reply.content.find((item) => item.type === "text").text);
|
||||
calls.push({
|
||||
name,
|
||||
arguments: args,
|
||||
elapsed_ms: performance.now() - start,
|
||||
is_error: reply.isError === true,
|
||||
evidence_id: value.evidence_id ?? null,
|
||||
});
|
||||
assert.equal(reply.isError === true, expectedError, JSON.stringify(value));
|
||||
return value;
|
||||
};
|
||||
try {
|
||||
await client.connect(transport);
|
||||
transport.stderr?.resume();
|
||||
for (const definition of (await client.listTools()).tools)
|
||||
definitions.set(definition.name, definition);
|
||||
if (scenario === "changed-profile") {
|
||||
const rejected = await call(
|
||||
"open_binary",
|
||||
{
|
||||
path: values.path,
|
||||
provider_id: "ghidra",
|
||||
snapshot_path: snapshotPath,
|
||||
},
|
||||
true,
|
||||
);
|
||||
assert.equal(rejected.error.code, "evidence_integrity_mismatch");
|
||||
assert.match(JSON.stringify(rejected.error), /profile_mismatch/u);
|
||||
assert.equal((await call("binary_session")).result.open, false);
|
||||
assert.equal(await headlessLaunches(work), 0);
|
||||
await client.ping();
|
||||
await call("open_binary", { path: values.path, provider_id: "ghidra" });
|
||||
const changed = (await call("binary_session")).result
|
||||
.analysis_provider_binding.analysis_profile;
|
||||
assert.notEqual(changed.digest, profile.digest);
|
||||
assert.notEqual(
|
||||
changed.parameters.analyzer_preset,
|
||||
profile.parameters.analyzer_preset,
|
||||
);
|
||||
const fact = parseEvidence(
|
||||
await call("analyze_function", { procedure: selected }),
|
||||
);
|
||||
assert.deepEqual(fact.analysis_profile, changed);
|
||||
await call("close_binary", { snapshot_path: changedSnapshotPath });
|
||||
rows.push({
|
||||
scenario,
|
||||
wrong_snapshot_rejected_before_headless_launch: true,
|
||||
selected_profile: changed,
|
||||
source_snapshot_unchanged: true,
|
||||
selected_function_facts_match_default: isDeepStrictEqual(
|
||||
fact.normalized_result,
|
||||
original.normalized_result,
|
||||
),
|
||||
limitations: fact.limitations,
|
||||
calls,
|
||||
});
|
||||
} else {
|
||||
if (scenario === "restored-default") {
|
||||
const rejected = await call(
|
||||
"open_binary",
|
||||
{
|
||||
path: values.path,
|
||||
provider_id: "ghidra",
|
||||
snapshot_path: changedSnapshotPath,
|
||||
},
|
||||
true,
|
||||
);
|
||||
assert.equal(rejected.error.code, "evidence_integrity_mismatch");
|
||||
assert.match(JSON.stringify(rejected.error), /profile_mismatch/u);
|
||||
assert.equal((await call("binary_session")).result.open, false);
|
||||
assert.equal(await headlessLaunches(work), 0);
|
||||
}
|
||||
await call("open_binary", {
|
||||
path: values.path,
|
||||
provider_id: "ghidra",
|
||||
...(scenario === "default" ? {} : { snapshot_path: snapshotPath }),
|
||||
});
|
||||
if (selected === undefined) {
|
||||
const procedures = (await call("list_procedures")).normalized_result;
|
||||
const observed = procedures.find(
|
||||
(row) =>
|
||||
row.procedure?.external === false && row.procedure.thunk === false,
|
||||
);
|
||||
assert.ok(
|
||||
observed,
|
||||
"No provider-observed internal procedure; select --procedure explicitly",
|
||||
);
|
||||
selected = observed.address;
|
||||
}
|
||||
const fact = parseEvidence(
|
||||
await call("analyze_function", { procedure: selected }),
|
||||
);
|
||||
const actualProfile = (await call("binary_session")).result
|
||||
.analysis_provider_binding.analysis_profile;
|
||||
assert.equal(actualProfile.parameters.analyzer_preset, "ghidra-default");
|
||||
if (scenario === "default") {
|
||||
original = fact;
|
||||
profile = actualProfile;
|
||||
await call("close_binary", { snapshot_path: snapshotPath });
|
||||
snapshotBytes = await readFile(snapshotPath);
|
||||
const snapshot = parseAnalysisSnapshot(
|
||||
JSON.parse(snapshotBytes.toString("utf8")),
|
||||
);
|
||||
assert.ok(
|
||||
snapshot.entries.some(
|
||||
(entry) => entry.operation === "analyze_function",
|
||||
),
|
||||
"The actual query must be stored, not merely equal after re-analysis",
|
||||
);
|
||||
assert.ok(
|
||||
snapshot.evidence_bundle.records.some(
|
||||
(record) => record.evidence_id === original.evidence_id,
|
||||
),
|
||||
);
|
||||
} else {
|
||||
assert.deepEqual(actualProfile, profile);
|
||||
exactTaskFacts(fact, original);
|
||||
assert.ok(
|
||||
fact.limitations.some((limitation) =>
|
||||
limitation.includes("local REA analysis snapshot"),
|
||||
),
|
||||
"No producer cache-reuse marker",
|
||||
);
|
||||
await call("close_binary");
|
||||
}
|
||||
rows.push({
|
||||
scenario,
|
||||
selected_profile: actualProfile,
|
||||
original_evidence_id: original.evidence_id,
|
||||
returned_evidence_id: fact.evidence_id,
|
||||
limitations: fact.limitations,
|
||||
calls,
|
||||
});
|
||||
}
|
||||
assert.equal((await call("binary_session")).result.open, false);
|
||||
} finally {
|
||||
await client.close();
|
||||
}
|
||||
const launches = await headlessLaunches(work);
|
||||
assert.equal(
|
||||
launches,
|
||||
scenario === "default" || scenario === "changed-profile" ? 1 : 0,
|
||||
);
|
||||
rows.at(-1).headless_launches = launches;
|
||||
assert.equal(await fileSha(values.path), before);
|
||||
assert.deepEqual(await readFile(snapshotPath), snapshotBytes);
|
||||
await writeFile(
|
||||
resolve(values.output, "progress.json"),
|
||||
JSON.stringify(rows, null, 2),
|
||||
{ mode: 0o600 },
|
||||
);
|
||||
process.stdout.write(
|
||||
JSON.stringify({
|
||||
scenario,
|
||||
status: "passed",
|
||||
headless_launches: launches,
|
||||
}) + "\n",
|
||||
);
|
||||
}
|
||||
const cliWork = resolve(values.output, "cli");
|
||||
await mkdir(cliWork, { mode: 0o700 });
|
||||
const cliStart = performance.now();
|
||||
const { stdout, stderr } = await promisify(execFile)(
|
||||
values.strace,
|
||||
tracedArgs(cliWork, [
|
||||
"function",
|
||||
values.path,
|
||||
selected,
|
||||
"--provider",
|
||||
"ghidra",
|
||||
"--snapshot",
|
||||
snapshotPath,
|
||||
"--json",
|
||||
]),
|
||||
{
|
||||
env: environmentFor(cliWork, ""),
|
||||
timeout,
|
||||
maxBuffer: 16 * 1024 * 1024,
|
||||
},
|
||||
);
|
||||
await writeFile(resolve(cliWork, "reply.json"), stdout, {
|
||||
flag: "wx",
|
||||
mode: 0o600,
|
||||
});
|
||||
await writeFile(resolve(cliWork, "stderr.log"), stderr, {
|
||||
flag: "wx",
|
||||
mode: 0o600,
|
||||
});
|
||||
const cli = parseEvidence(JSON.parse(stdout));
|
||||
exactTaskFacts(cli, original);
|
||||
assert.equal(cli.evidence_id, original.evidence_id);
|
||||
assert.deepEqual(await readFile(snapshotPath), snapshotBytes);
|
||||
assert.equal(await headlessLaunches(cliWork), 0);
|
||||
assert.equal(await fileSha(values.path), before);
|
||||
const lineage = await completeVerifierRun(run);
|
||||
assert.equal(lineage.process_lineage.status, "verified");
|
||||
assert.deepEqual(lineage.process_lineage.descendants, []);
|
||||
const report = {
|
||||
caller_supplied_runtime_revision: values.revision,
|
||||
verifier_source_sha256: createHash("sha256")
|
||||
.update(verifierSource)
|
||||
.digest("hex"),
|
||||
source_sha256: before,
|
||||
source_path: values.path,
|
||||
provider_settings_sha256: createHash("sha256")
|
||||
.update(providerBytes)
|
||||
.digest("hex"),
|
||||
original_evidence_id: original.evidence_id,
|
||||
source_snapshot_sha256: createHash("sha256")
|
||||
.update(snapshotBytes)
|
||||
.digest("hex"),
|
||||
selected_procedure: selected,
|
||||
original_record_and_source_bytes_unchanged: true,
|
||||
rows,
|
||||
cli: {
|
||||
elapsed_ms: performance.now() - cliStart,
|
||||
returned_evidence_id: cli.evidence_id,
|
||||
exact_original_task_facts: true,
|
||||
headless_launches: 0,
|
||||
},
|
||||
lineage,
|
||||
scope:
|
||||
"Actual Ghidra public SDK and CLI, explicit default/disabled profiles and immutable JSON snapshot reuse. Revision is caller-supplied. Selection/discovery and all verification calls are included; timings are not paired performance evidence. No engine database cache, target execution or annotation is requested. Final descendant observation is point-in-time; syscall audit covers the known headless launch command.",
|
||||
};
|
||||
assert.deepEqual(
|
||||
await readFile(new URL(import.meta.url)),
|
||||
verifierSource,
|
||||
"Verifier source changed during execution",
|
||||
);
|
||||
await writeFile(
|
||||
resolve(values.output, "summary.json"),
|
||||
JSON.stringify(report, null, 2),
|
||||
{ flag: "wx", mode: 0o600 },
|
||||
);
|
||||
@@ -0,0 +1,69 @@
|
||||
# Windows empty-profile startup verification
|
||||
|
||||
Follow-up to [#2345](https://github.com/morluto/rea/pull/2345), including
|
||||
@gefrezim's diagnosis of the import-time Windows home lookup.
|
||||
|
||||
## Source and boundary
|
||||
|
||||
Baseline: main `172407787f1842dedbcd27b5385177595f786ce4`. Verification used
|
||||
the real Windows 11 10.0.22631.0 host, official Windows x64 Node 24.18.0,
|
||||
locked production dependencies and REA's compiled public CLI / SDK 2.3.1 MCP
|
||||
server. No IDA or Ghidra engine was used. Runtime copies and compact receipts
|
||||
live outside the checkout; no account profile path or ambient environment is
|
||||
recorded here.
|
||||
|
||||
The tested dispatcher SHA-256 is
|
||||
`526105418604cc6e5a202a6275b700814e27450bfaffd45e7510530ba669de8d`.
|
||||
Only the dispatcher changed in the deployed runtime. It treats an empty Windows
|
||||
`USERPROFILE` as unset before dependency imports. Nonempty selections continue
|
||||
to reach the existing home resolver; no additional lookup or fallback owner is
|
||||
introduced.
|
||||
|
||||
## Real Windows observations
|
||||
|
||||
With `HOME` empty in each case, the baseline produced:
|
||||
|
||||
| USERPROFILE selection | `--version` | `--help` | MCP discovery |
|
||||
| --------------------------- | ----------------------- | ----------------------- | ------------- |
|
||||
| Unset | Exit 0 | Exit 0 | Healthy |
|
||||
| Empty | Exit 1, `uv_os_homedir` | Exit 1, `uv_os_homedir` | Healthy |
|
||||
| Nonempty test-owned profile | Exit 0 | Exit 0 | Healthy |
|
||||
|
||||
The corrected CLI passed all six invocations with no stderr. Version output
|
||||
remained 6 bytes and help output 15,497 bytes. Real setup dry runs with empty
|
||||
and nonempty profile selections both exited 0, planned their configuration
|
||||
under the expected home, preserved existing configuration bytes and created no
|
||||
configuration under the working directory. No installation was performed.
|
||||
|
||||
MCP was already healthy in the baseline; this change does not claim to repair
|
||||
an MCP startup failure. Complete baseline discovery was 187 tools and 2,545,790
|
||||
serialized bytes, SHA-256
|
||||
`1e6bbca7f62852134eb703ea098f5a302d1fe333b8433475429f6fc1b1aba18f`.
|
||||
All three corrected MCP controls also passed with no stderr and the same
|
||||
catalogue bytes and digest.
|
||||
|
||||
The existing `homeDirectoryFallback` acceptance test exercises the import-time
|
||||
failure through the public setup dry run. Adding it to the Windows curated lane
|
||||
makes that regression execute on the affected platform. No duplicate startup
|
||||
test was retained. Nine related tests passed, including the existing MCP doctor
|
||||
and curated-lane path checks; Linux passing alone does not prove Windows behavior.
|
||||
Review caught a missing package-scope registration for the newly listed test;
|
||||
the original CI shard failed that ownership invariant. The path is now also
|
||||
registered in `scripts/ci/scopes.mjs`, and all 84 scope checks plus the two
|
||||
curated-path checks pass. Runtime source and its tested digest are unchanged.
|
||||
|
||||
## Efficiency and resource limits
|
||||
|
||||
Heavy host probes ran serially at BelowNormal priority, with one CPU and a
|
||||
512 MiB CLI/server heap setting (256 MiB for the probe parent). Heap settings do
|
||||
not bound total process memory. An exit-time `process.resourceUsage()` observer
|
||||
recorded individual corrected CLI invocations: CPU time ranged from 7.968 to
|
||||
9.874 seconds and peak working set from 299,956 to 301,852 KiB. These figures
|
||||
exclude the probe parent and any observer overhead outside the CLI process.
|
||||
The six observed wall times ranged from 8.656 to 10.961 seconds.
|
||||
|
||||
The baseline did not use that resource observer, and cold/warm caches were not
|
||||
normalized. These observations establish functional behavior and bounded probe
|
||||
scope; no CPU, memory or latency improvement is claimed. The fix adds one
|
||||
constant-time environment check, no filesystem lookup, engine call, retry or
|
||||
cache. Output bytes are not model tokens; actual model usage was not measured.
|
||||
@@ -0,0 +1,107 @@
|
||||
# Windows runtime admission retry verification
|
||||
|
||||
Scope: [#2257](https://github.com/morluto/rea/issues/2257). Related initial
|
||||
diagnosis: [#2266](https://github.com/morluto/rea/pull/2266).
|
||||
|
||||
## Revisions and real input
|
||||
|
||||
The baseline is `4f4f6cd7b5d7a400b90772a138d7b9f942c2a556`.
|
||||
The candidate was developed from
|
||||
`4773c152bc5f299fa682575bb8d1efea7587c7bc`; relevant runtime, native and
|
||||
dependency-lock sources are identical between those two revisions.
|
||||
These measurements precede integration of subsequent main changes.
|
||||
|
||||
The host ran Windows 11 10.0.22631.0, official Windows x64 Node 24.18.0,
|
||||
and locked production dependencies. The real application input was the official
|
||||
npm 11.16.0 launcher distribution's `bin` directory. REA inspected that source;
|
||||
it did not execute the application as an analysis target. Runtime copies,
|
||||
raw replies and test executables remain outside the checkout. No IDA or Ghidra
|
||||
engine was involved in this native-runtime verification.
|
||||
|
||||
## Correctness and recovery
|
||||
|
||||
A test-owned directory handle requesting `FILE_ADD_FILE` produced an actual
|
||||
kernel sharing violation (Win32 32) during native runtime-parent admission.
|
||||
Releasing that handle restored admission. Before the fix, public CLI analysis
|
||||
reported a permanent `unsupported_provider` failure with `retryable: false`.
|
||||
The corrected CLI preserves `capability_unavailable` and the reason, with
|
||||
`category: unavailable` and `retryable: true`.
|
||||
|
||||
Normal CLI analysis and post-conflict recovery produced the same canonical
|
||||
normalized result before and after the correction: 535,733 compact JSON bytes,
|
||||
SHA-256 `a24d3c2d9897cccc5ba70e9ee4dce03da88f033a939fe87457136fc5b3a3e2db`.
|
||||
The complete successful CLI reply remained 537,124 bytes.
|
||||
|
||||
A single real SDK 2.3.1 stdio MCP connection first completed analysis, then
|
||||
encountered the same kernel conflict after native authority caching, then
|
||||
recovered successfully when the handle was released. Its final receipt passed
|
||||
and found no residual runtime directories. The native producer supplies
|
||||
`runtimeAllocation: "not-started"` only when parent admission failed before
|
||||
directory creation. The adapter requires that fact and exact numeric codes
|
||||
32/33; permission failures, opaque error text and unknown allocation state
|
||||
remain nonretryable. Incomplete cleanup also prevents retry.
|
||||
|
||||
The native verifier additionally reproduces a real sharing conflict using an
|
||||
owned runtime's DELETE lease, verifies the original error and allocation fact,
|
||||
and confirms no child directory was allocated. This case runs unconditionally
|
||||
in the Windows native verifier, without the optional process-boundary fixture.
|
||||
The full verifier passed with 20 positive controls. Its optional independently
|
||||
built process-boundary fixture was not supplied in this run.
|
||||
|
||||
The related focused suite passed 99 tests across seven files, including worker
|
||||
ownership, cancellation, cleanup, error projection, loader behavior and adapter
|
||||
boundaries. Source-map and WABT acquisition-failure tests use injected failures;
|
||||
they establish disposition propagation, not real engine execution.
|
||||
|
||||
Thirty baseline native self-tests passed with no residual directories.
|
||||
Win32 5 from #1701/#2196 was not reproduced on this host; these results do not
|
||||
establish its cause or resolution.
|
||||
|
||||
## Delivery and resource measurements
|
||||
|
||||
One final same-connection MCP trial used one CPU per Node process, BelowNormal
|
||||
priority, a 512 MiB client heap setting and a 768 MiB server heap setting.
|
||||
Heap settings do not bound total process or family memory.
|
||||
|
||||
| Stage | Observed wall time | Serialized reply bytes |
|
||||
| ---------------------------------- | -----------------: | ---------------------: |
|
||||
| Connect | 11,351 ms | Not recorded |
|
||||
| Complete tool discovery, 187 tools | 744 ms | 2,498,686 |
|
||||
| Normal analysis | 12,887 ms | 1,126,797 |
|
||||
| Sharing failure | 265 ms | 878 |
|
||||
| Recovery on the same connection | 5,578 ms | 1,126,797 |
|
||||
|
||||
The earlier warm failure reply was 1,218 bytes. Successful reply sizes and the
|
||||
catalogue digest were unchanged within the tested source comparison:
|
||||
`7bdadddc909f4b2b093f0b3da9f79fafdac879c9d3c147938b8fbce1b52cd726`.
|
||||
These are delivery bytes, not model tokens. Model-visible discovery handling,
|
||||
actual model token usage and total model task cost were not measured.
|
||||
|
||||
A targeted process-generation sampler collected 343 observations over
|
||||
35,630 ms. Its aggregate observed CPU time was a **lower bound** of 26,766 ms:
|
||||
process exits can hide the final CPU tail. Its peak sampled working-set sum was
|
||||
727,244,800 bytes (about 694 MiB). This is a sampled sum, can double-count shared
|
||||
pages and is not an exact physical-memory peak. Only the test-owned Node and
|
||||
directory-holder processes were included; the observer itself was excluded.
|
||||
|
||||
This is one controlled functional trial, with no paired CPU/memory baseline.
|
||||
Cold/warm filesystem caches and host noise were not normalized. No speedup,
|
||||
CPU reduction, memory reduction or model-token savings are claimed. The change
|
||||
adds no analysis retry loop or cache and performs no additional engine calls.
|
||||
Existing successful native authority caching remains intact; failed readiness
|
||||
is retried by a later caller attempt. Retry becomes useful advice only after
|
||||
the transient condition clears and cleanup permits another attempt.
|
||||
|
||||
## Integration replay
|
||||
|
||||
After rebasing onto main `a2d8b94dd`, runtime candidate
|
||||
`2c7a09eaf3ffe26667b40081f1cf0fd2ce44a146` again passed the 99 related tests,
|
||||
static and documentation gates, real Windows CLI and same-connection MCP
|
||||
normal/conflict/recovery sequence. Both workflows found no residual directories;
|
||||
the CLI normalized-result digest above still matched. The MCP success and error
|
||||
reply sizes remained 1,126,797 and 878 bytes respectively. Upstream Android/SAZ
|
||||
contract changes increased complete discovery to 2,517,904 bytes, digest
|
||||
`d14588eab7404ad129907eb34a45a0db0d43233c4de829f76babd59723bed199`.
|
||||
This later catalogue is not evidence of a delivery-size change caused by the
|
||||
runtime-admission fix. Resource sampling above belongs to the earlier pinned
|
||||
candidate; aggregate resources were not resampled for this integration replay.
|
||||
+1
-1
@@ -167,7 +167,7 @@ main() {
|
||||
[ "$installed_version" = "$version" ] || fail "installed version $installed_version does not match $version. Reinstall the requested version, then retry."
|
||||
|
||||
printf 'REA %s is installed.\n' "$version"
|
||||
if [ "$start_setup" = true ] && [ -r /dev/tty ] && [ -w /dev/tty ] && [ -t 1 ]; then
|
||||
if [ "$start_setup" = true ] && [ -r /dev/tty ] && [ -w /dev/tty ] && [ -t 1 ] && [ -t 2 ]; then
|
||||
printf 'Starting REA setup...\n'
|
||||
"$install_bin" setup </dev/tty >/dev/tty
|
||||
else
|
||||
|
||||
@@ -0,0 +1,480 @@
|
||||
GNU LIBRARY GENERAL PUBLIC LICENSE
|
||||
Version 2, June 1991
|
||||
|
||||
Copyright (C) 1991 Free Software Foundation, Inc.
|
||||
<https://fsf.org/>
|
||||
Everyone is permitted to copy and distribute verbatim copies
|
||||
of this license document, but changing it is not allowed.
|
||||
|
||||
[This is the first released version of the library GPL. It is
|
||||
numbered 2 because it goes with version 2 of the ordinary GPL.]
|
||||
|
||||
Preamble
|
||||
|
||||
The licenses for most software are designed to take away your
|
||||
freedom to share and change it. By contrast, the GNU General Public
|
||||
Licenses are intended to guarantee your freedom to share and change
|
||||
free software--to make sure the software is free for all its users.
|
||||
|
||||
This license, the Library General Public License, applies to some
|
||||
specially designated Free Software Foundation software, and to any
|
||||
other libraries whose authors decide to use it. You can use it for
|
||||
your libraries, too.
|
||||
|
||||
When we speak of free software, we are referring to freedom, not
|
||||
price. Our General Public Licenses are designed to make sure that you
|
||||
have the freedom to distribute copies of free software (and charge for
|
||||
this service if you wish), that you receive source code or can get it
|
||||
if you want it, that you can change the software or use pieces of it
|
||||
in new free programs; and that you know you can do these things.
|
||||
|
||||
To protect your rights, we need to make restrictions that forbid
|
||||
anyone to deny you these rights or to ask you to surrender the rights.
|
||||
These restrictions translate to certain responsibilities for you if
|
||||
you distribute copies of the library, or if you modify it.
|
||||
|
||||
For example, if you distribute copies of the library, whether gratis
|
||||
or for a fee, you must give the recipients all the rights that we gave
|
||||
you. You must make sure that they, too, receive or can get the source
|
||||
code. If you link a program with the library, you must provide
|
||||
complete object files to the recipients so that they can relink them
|
||||
with the library, after making changes to the library and recompiling
|
||||
it. And you must show them these terms so they know their rights.
|
||||
|
||||
Our method of protecting your rights has two steps: (1) copyright
|
||||
the library, and (2) offer you this license which gives you legal
|
||||
permission to copy, distribute and/or modify the library.
|
||||
|
||||
Also, for each distributor's protection, we want to make certain
|
||||
that everyone understands that there is no warranty for this free
|
||||
library. If the library is modified by someone else and passed on, we
|
||||
want its recipients to know that what they have is not the original
|
||||
version, so that any problems introduced by others will not reflect on
|
||||
the original authors' reputations.
|
||||
|
||||
Finally, any free program is threatened constantly by software
|
||||
patents. We wish to avoid the danger that companies distributing free
|
||||
software will individually obtain patent licenses, thus in effect
|
||||
transforming the program into proprietary software. To prevent this,
|
||||
we have made it clear that any patent must be licensed for everyone's
|
||||
free use or not licensed at all.
|
||||
|
||||
Most GNU software, including some libraries, is covered by the ordinary
|
||||
GNU General Public License, which was designed for utility programs. This
|
||||
license, the GNU Library General Public License, applies to certain
|
||||
designated libraries. This license is quite different from the ordinary
|
||||
one; be sure to read it in full, and don't assume that anything in it is
|
||||
the same as in the ordinary license.
|
||||
|
||||
The reason we have a separate public license for some libraries is that
|
||||
they blur the distinction we usually make between modifying or adding to a
|
||||
program and simply using it. Linking a program with a library, without
|
||||
changing the library, is in some sense simply using the library, and is
|
||||
analogous to running a utility program or application program. However, in
|
||||
a textual and legal sense, the linked executable is a combined work, a
|
||||
derivative of the original library, and the ordinary General Public License
|
||||
treats it as such.
|
||||
|
||||
Because of this blurred distinction, using the ordinary General
|
||||
Public License for libraries did not effectively promote software
|
||||
sharing, because most developers did not use the libraries. We
|
||||
concluded that weaker conditions might promote sharing better.
|
||||
|
||||
However, unrestricted linking of non-free programs would deprive the
|
||||
users of those programs of all benefit from the free status of the
|
||||
libraries themselves. This Library General Public License is intended to
|
||||
permit developers of non-free programs to use free libraries, while
|
||||
preserving your freedom as a user of such programs to change the free
|
||||
libraries that are incorporated in them. (We have not seen how to achieve
|
||||
this as regards changes in header files, but we have achieved it as regards
|
||||
changes in the actual functions of the Library.) The hope is that this
|
||||
will lead to faster development of free libraries.
|
||||
|
||||
The precise terms and conditions for copying, distribution and
|
||||
modification follow. Pay close attention to the difference between a
|
||||
"work based on the library" and a "work that uses the library". The
|
||||
former contains code derived from the library, while the latter only
|
||||
works together with the library.
|
||||
|
||||
Note that it is possible for a library to be covered by the ordinary
|
||||
General Public License rather than by this special one.
|
||||
|
||||
GNU LIBRARY GENERAL PUBLIC LICENSE
|
||||
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
|
||||
|
||||
0. This License Agreement applies to any software library which
|
||||
contains a notice placed by the copyright holder or other authorized
|
||||
party saying it may be distributed under the terms of this Library
|
||||
General Public License (also called "this License"). Each licensee is
|
||||
addressed as "you".
|
||||
|
||||
A "library" means a collection of software functions and/or data
|
||||
prepared so as to be conveniently linked with application programs
|
||||
(which use some of those functions and data) to form executables.
|
||||
|
||||
The "Library", below, refers to any such software library or work
|
||||
which has been distributed under these terms. A "work based on the
|
||||
Library" means either the Library or any derivative work under
|
||||
copyright law: that is to say, a work containing the Library or a
|
||||
portion of it, either verbatim or with modifications and/or translated
|
||||
straightforwardly into another language. (Hereinafter, translation is
|
||||
included without limitation in the term "modification".)
|
||||
|
||||
"Source code" for a work means the preferred form of the work for
|
||||
making modifications to it. For a library, complete source code means
|
||||
all the source code for all modules it contains, plus any associated
|
||||
interface definition files, plus the scripts used to control compilation
|
||||
and installation of the library.
|
||||
|
||||
Activities other than copying, distribution and modification are not
|
||||
covered by this License; they are outside its scope. The act of
|
||||
running a program using the Library is not restricted, and output from
|
||||
such a program is covered only if its contents constitute a work based
|
||||
on the Library (independent of the use of the Library in a tool for
|
||||
writing it). Whether that is true depends on what the Library does
|
||||
and what the program that uses the Library does.
|
||||
|
||||
1. You may copy and distribute verbatim copies of the Library's
|
||||
complete source code as you receive it, in any medium, provided that
|
||||
you conspicuously and appropriately publish on each copy an
|
||||
appropriate copyright notice and disclaimer of warranty; keep intact
|
||||
all the notices that refer to this License and to the absence of any
|
||||
warranty; and distribute a copy of this License along with the
|
||||
Library.
|
||||
|
||||
You may charge a fee for the physical act of transferring a copy,
|
||||
and you may at your option offer warranty protection in exchange for a
|
||||
fee.
|
||||
|
||||
2. You may modify your copy or copies of the Library or any portion
|
||||
of it, thus forming a work based on the Library, and copy and
|
||||
distribute such modifications or work under the terms of Section 1
|
||||
above, provided that you also meet all of these conditions:
|
||||
|
||||
a) The modified work must itself be a software library.
|
||||
|
||||
b) You must cause the files modified to carry prominent notices
|
||||
stating that you changed the files and the date of any change.
|
||||
|
||||
c) You must cause the whole of the work to be licensed at no
|
||||
charge to all third parties under the terms of this License.
|
||||
|
||||
d) If a facility in the modified Library refers to a function or a
|
||||
table of data to be supplied by an application program that uses
|
||||
the facility, other than as an argument passed when the facility
|
||||
is invoked, then you must make a good faith effort to ensure that,
|
||||
in the event an application does not supply such function or
|
||||
table, the facility still operates, and performs whatever part of
|
||||
its purpose remains meaningful.
|
||||
|
||||
(For example, a function in a library to compute square roots has
|
||||
a purpose that is entirely well-defined independent of the
|
||||
application. Therefore, Subsection 2d requires that any
|
||||
application-supplied function or table used by this function must
|
||||
be optional: if the application does not supply it, the square
|
||||
root function must still compute square roots.)
|
||||
|
||||
These requirements apply to the modified work as a whole. If
|
||||
identifiable sections of that work are not derived from the Library,
|
||||
and can be reasonably considered independent and separate works in
|
||||
themselves, then this License, and its terms, do not apply to those
|
||||
sections when you distribute them as separate works. But when you
|
||||
distribute the same sections as part of a whole which is a work based
|
||||
on the Library, the distribution of the whole must be on the terms of
|
||||
this License, whose permissions for other licensees extend to the
|
||||
entire whole, and thus to each and every part regardless of who wrote
|
||||
it.
|
||||
|
||||
Thus, it is not the intent of this section to claim rights or contest
|
||||
your rights to work written entirely by you; rather, the intent is to
|
||||
exercise the right to control the distribution of derivative or
|
||||
collective works based on the Library.
|
||||
|
||||
In addition, mere aggregation of another work not based on the Library
|
||||
with the Library (or with a work based on the Library) on a volume of
|
||||
a storage or distribution medium does not bring the other work under
|
||||
the scope of this License.
|
||||
|
||||
3. You may opt to apply the terms of the ordinary GNU General Public
|
||||
License instead of this License to a given copy of the Library. To do
|
||||
this, you must alter all the notices that refer to this License, so
|
||||
that they refer to the ordinary GNU General Public License, version 2,
|
||||
instead of to this License. (If a newer version than version 2 of the
|
||||
ordinary GNU General Public License has appeared, then you can specify
|
||||
that version instead if you wish.) Do not make any other change in
|
||||
these notices.
|
||||
|
||||
Once this change is made in a given copy, it is irreversible for
|
||||
that copy, so the ordinary GNU General Public License applies to all
|
||||
subsequent copies and derivative works made from that copy.
|
||||
|
||||
This option is useful when you wish to copy part of the code of
|
||||
the Library into a program that is not a library.
|
||||
|
||||
4. You may copy and distribute the Library (or a portion or
|
||||
derivative of it, under Section 2) in object code or executable form
|
||||
under the terms of Sections 1 and 2 above provided that you accompany
|
||||
it with the complete corresponding machine-readable source code, which
|
||||
must be distributed under the terms of Sections 1 and 2 above on a
|
||||
medium customarily used for software interchange.
|
||||
|
||||
If distribution of object code is made by offering access to copy
|
||||
from a designated place, then offering equivalent access to copy the
|
||||
source code from the same place satisfies the requirement to
|
||||
distribute the source code, even though third parties are not
|
||||
compelled to copy the source along with the object code.
|
||||
|
||||
5. A program that contains no derivative of any portion of the
|
||||
Library, but is designed to work with the Library by being compiled or
|
||||
linked with it, is called a "work that uses the Library". Such a
|
||||
work, in isolation, is not a derivative work of the Library, and
|
||||
therefore falls outside the scope of this License.
|
||||
|
||||
However, linking a "work that uses the Library" with the Library
|
||||
creates an executable that is a derivative of the Library (because it
|
||||
contains portions of the Library), rather than a "work that uses the
|
||||
library". The executable is therefore covered by this License.
|
||||
Section 6 states terms for distribution of such executables.
|
||||
|
||||
When a "work that uses the Library" uses material from a header file
|
||||
that is part of the Library, the object code for the work may be a
|
||||
derivative work of the Library even though the source code is not.
|
||||
Whether this is true is especially significant if the work can be
|
||||
linked without the Library, or if the work is itself a library. The
|
||||
threshold for this to be true is not precisely defined by law.
|
||||
|
||||
If such an object file uses only numerical parameters, data
|
||||
structure layouts and accessors, and small macros and small inline
|
||||
functions (ten lines or less in length), then the use of the object
|
||||
file is unrestricted, regardless of whether it is legally a derivative
|
||||
work. (Executables containing this object code plus portions of the
|
||||
Library will still fall under Section 6.)
|
||||
|
||||
Otherwise, if the work is a derivative of the Library, you may
|
||||
distribute the object code for the work under the terms of Section 6.
|
||||
Any executables containing that work also fall under Section 6,
|
||||
whether or not they are linked directly with the Library itself.
|
||||
|
||||
6. As an exception to the Sections above, you may also compile or
|
||||
link a "work that uses the Library" with the Library to produce a
|
||||
work containing portions of the Library, and distribute that work
|
||||
under terms of your choice, provided that the terms permit
|
||||
modification of the work for the customer's own use and reverse
|
||||
engineering for debugging such modifications.
|
||||
|
||||
You must give prominent notice with each copy of the work that the
|
||||
Library is used in it and that the Library and its use are covered by
|
||||
this License. You must supply a copy of this License. If the work
|
||||
during execution displays copyright notices, you must include the
|
||||
copyright notice for the Library among them, as well as a reference
|
||||
directing the user to the copy of this License. Also, you must do one
|
||||
of these things:
|
||||
|
||||
a) Accompany the work with the complete corresponding
|
||||
machine-readable source code for the Library including whatever
|
||||
changes were used in the work (which must be distributed under
|
||||
Sections 1 and 2 above); and, if the work is an executable linked
|
||||
with the Library, with the complete machine-readable "work that
|
||||
uses the Library", as object code and/or source code, so that the
|
||||
user can modify the Library and then relink to produce a modified
|
||||
executable containing the modified Library. (It is understood
|
||||
that the user who changes the contents of definitions files in the
|
||||
Library will not necessarily be able to recompile the application
|
||||
to use the modified definitions.)
|
||||
|
||||
b) Accompany the work with a written offer, valid for at
|
||||
least three years, to give the same user the materials
|
||||
specified in Subsection 6a, above, for a charge no more
|
||||
than the cost of performing this distribution.
|
||||
|
||||
c) If distribution of the work is made by offering access to copy
|
||||
from a designated place, offer equivalent access to copy the above
|
||||
specified materials from the same place.
|
||||
|
||||
d) Verify that the user has already received a copy of these
|
||||
materials or that you have already sent this user a copy.
|
||||
|
||||
For an executable, the required form of the "work that uses the
|
||||
Library" must include any data and utility programs needed for
|
||||
reproducing the executable from it. However, as a special exception,
|
||||
the source code distributed need not include anything that is normally
|
||||
distributed (in either source or binary form) with the major
|
||||
components (compiler, kernel, and so on) of the operating system on
|
||||
which the executable runs, unless that component itself accompanies
|
||||
the executable.
|
||||
|
||||
It may happen that this requirement contradicts the license
|
||||
restrictions of other proprietary libraries that do not normally
|
||||
accompany the operating system. Such a contradiction means you cannot
|
||||
use both them and the Library together in an executable that you
|
||||
distribute.
|
||||
|
||||
7. You may place library facilities that are a work based on the
|
||||
Library side-by-side in a single library together with other library
|
||||
facilities not covered by this License, and distribute such a combined
|
||||
library, provided that the separate distribution of the work based on
|
||||
the Library and of the other library facilities is otherwise
|
||||
permitted, and provided that you do these two things:
|
||||
|
||||
a) Accompany the combined library with a copy of the same work
|
||||
based on the Library, uncombined with any other library
|
||||
facilities. This must be distributed under the terms of the
|
||||
Sections above.
|
||||
|
||||
b) Give prominent notice with the combined library of the fact
|
||||
that part of it is a work based on the Library, and explaining
|
||||
where to find the accompanying uncombined form of the same work.
|
||||
|
||||
8. You may not copy, modify, sublicense, link with, or distribute
|
||||
the Library except as expressly provided under this License. Any
|
||||
attempt otherwise to copy, modify, sublicense, link with, or
|
||||
distribute the Library is void, and will automatically terminate your
|
||||
rights under this License. However, parties who have received copies,
|
||||
or rights, from you under this License will not have their licenses
|
||||
terminated so long as such parties remain in full compliance.
|
||||
|
||||
9. You are not required to accept this License, since you have not
|
||||
signed it. However, nothing else grants you permission to modify or
|
||||
distribute the Library or its derivative works. These actions are
|
||||
prohibited by law if you do not accept this License. Therefore, by
|
||||
modifying or distributing the Library (or any work based on the
|
||||
Library), you indicate your acceptance of this License to do so, and
|
||||
all its terms and conditions for copying, distributing or modifying
|
||||
the Library or works based on it.
|
||||
|
||||
10. Each time you redistribute the Library (or any work based on the
|
||||
Library), the recipient automatically receives a license from the
|
||||
original licensor to copy, distribute, link with or modify the Library
|
||||
subject to these terms and conditions. You may not impose any further
|
||||
restrictions on the recipients' exercise of the rights granted herein.
|
||||
You are not responsible for enforcing compliance by third parties to
|
||||
this License.
|
||||
|
||||
11. If, as a consequence of a court judgment or allegation of patent
|
||||
infringement or for any other reason (not limited to patent issues),
|
||||
conditions are imposed on you (whether by court order, agreement or
|
||||
otherwise) that contradict the conditions of this License, they do not
|
||||
excuse you from the conditions of this License. If you cannot
|
||||
distribute so as to satisfy simultaneously your obligations under this
|
||||
License and any other pertinent obligations, then as a consequence you
|
||||
may not distribute the Library at all. For example, if a patent
|
||||
license would not permit royalty-free redistribution of the Library by
|
||||
all those who receive copies directly or indirectly through you, then
|
||||
the only way you could satisfy both it and this License would be to
|
||||
refrain entirely from distribution of the Library.
|
||||
|
||||
If any portion of this section is held invalid or unenforceable under any
|
||||
particular circumstance, the balance of the section is intended to apply,
|
||||
and the section as a whole is intended to apply in other circumstances.
|
||||
|
||||
It is not the purpose of this section to induce you to infringe any
|
||||
patents or other property right claims or to contest validity of any
|
||||
such claims; this section has the sole purpose of protecting the
|
||||
integrity of the free software distribution system which is
|
||||
implemented by public license practices. Many people have made
|
||||
generous contributions to the wide range of software distributed
|
||||
through that system in reliance on consistent application of that
|
||||
system; it is up to the author/donor to decide if he or she is willing
|
||||
to distribute software through any other system and a licensee cannot
|
||||
impose that choice.
|
||||
|
||||
This section is intended to make thoroughly clear what is believed to
|
||||
be a consequence of the rest of this License.
|
||||
|
||||
12. If the distribution and/or use of the Library is restricted in
|
||||
certain countries either by patents or by copyrighted interfaces, the
|
||||
original copyright holder who places the Library under this License may add
|
||||
an explicit geographical distribution limitation excluding those countries,
|
||||
so that distribution is permitted only in or among countries not thus
|
||||
excluded. In such case, this License incorporates the limitation as if
|
||||
written in the body of this License.
|
||||
|
||||
13. The Free Software Foundation may publish revised and/or new
|
||||
versions of the Library General Public License from time to time.
|
||||
Such new versions will be similar in spirit to the present version,
|
||||
but may differ in detail to address new problems or concerns.
|
||||
|
||||
Each version is given a distinguishing version number. If the Library
|
||||
specifies a version number of this License which applies to it and
|
||||
"any later version", you have the option of following the terms and
|
||||
conditions either of that version or of any later version published by
|
||||
the Free Software Foundation. If the Library does not specify a
|
||||
license version number, you may choose any version ever published by
|
||||
the Free Software Foundation.
|
||||
|
||||
14. If you wish to incorporate parts of the Library into other free
|
||||
programs whose distribution conditions are incompatible with these,
|
||||
write to the author to ask for permission. For software which is
|
||||
copyrighted by the Free Software Foundation, write to the Free
|
||||
Software Foundation; we sometimes make exceptions for this. Our
|
||||
decision will be guided by the two goals of preserving the free status
|
||||
of all derivatives of our free software and of promoting the sharing
|
||||
and reuse of software generally.
|
||||
|
||||
NO WARRANTY
|
||||
|
||||
15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO
|
||||
WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW.
|
||||
EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR
|
||||
OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY
|
||||
KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE
|
||||
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
||||
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE
|
||||
LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME
|
||||
THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
|
||||
|
||||
16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN
|
||||
WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY
|
||||
AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU
|
||||
FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR
|
||||
CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE
|
||||
LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING
|
||||
RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A
|
||||
FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF
|
||||
SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
|
||||
DAMAGES.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
How to Apply These Terms to Your New Libraries
|
||||
|
||||
If you develop a new library, and you want it to be of the greatest
|
||||
possible use to the public, we recommend making it free software that
|
||||
everyone can redistribute and change. You can do so by permitting
|
||||
redistribution under these terms (or, alternatively, under the terms of the
|
||||
ordinary General Public License).
|
||||
|
||||
To apply these terms, attach the following notices to the library. It is
|
||||
safest to attach them to the start of each source file to most effectively
|
||||
convey the exclusion of warranty; and each file should have at least the
|
||||
"copyright" line and a pointer to where the full notice is found.
|
||||
|
||||
<one line to give the library's name and a brief idea of what it does.>
|
||||
Copyright (C) <year> <name of author>
|
||||
|
||||
This library is free software; you can redistribute it and/or
|
||||
modify it under the terms of the GNU Library General Public
|
||||
License as published by the Free Software Foundation; either
|
||||
version 2 of the License, or (at your option) any later version.
|
||||
|
||||
This library is distributed in the hope that it will be useful,
|
||||
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
||||
Library General Public License for more details.
|
||||
|
||||
You should have received a copy of the GNU Library General Public
|
||||
License along with this library; if not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
Also add information on how to contact you by electronic and paper mail.
|
||||
|
||||
You should also get your employer (if you work as a programmer) or your
|
||||
school, if any, to sign a "copyright disclaimer" for the library, if
|
||||
necessary. Here is a sample; alter the names:
|
||||
|
||||
Yoyodyne, Inc., hereby disclaims all copyright interest in the
|
||||
library `Frob' (a library for tweaking knobs) written by James Random Hacker.
|
||||
|
||||
<signature of Moe Ghoul>, 1 April 1990
|
||||
Moe Ghoul, President of Vice
|
||||
|
||||
That's all there is to it!
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user