Compare commits

..
Author SHA1 Message Date
N0zoM1z0 561877ad76 fix(process)!: version literal scenario replacement semantics
Keep caller-selected replacement text literal with a callback, preserving
empty-search positions instead of changing their split/join behavior.
Commit the replacement interpretation for captures using custom patterns so
legacy records cannot report this producer correction as a target change.

BREAKING CHANGE: Captures with custom normalization patterns use
literal-replacements-v1 and cannot be compared with legacy unversioned
pattern captures. Capture both sides with the same replacement semantics;
legacy captures remain readable and captures without patterns are unaffected.
2026-10-10 16:10:02 +08:00
N0zoM1z0 022c8b6f53 chore(process): integrate literal normalization with current main 2026-10-10 15:45:26 +08:00
9defed9725 perf(mcp): stop advertising placeholder input descriptions (#1479)
Input schema presentation gave every undescribed property a generic
"Value for <name>." sentence. That only restates the property name, yet
it covered 1,958 properties: 70 KB of the 522 KB of advertised input
schemas that clients load into model context. Authored descriptions,
referenced descriptions, and the name-pattern sentences that add meaning
(maximums, digests, paths, Evidence IDs) remain; other properties now
advertise no description.

The schema validity test now rejects placeholder descriptions instead of
requiring a description on every property.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 00:32:29 -07:00
5f6bb7a5ba perf(apple): state each NIB object's class, label, and ID once (#1477)
The NIBArchive projection synthesizes customClass, label, and objectID so
its records share the ibtool dictionary parser. The parser then returned
those values as class_name, name, and the object ID, and the graph node
kept the synthesized copies in attributes as well. Attributes now omit a
key whose value its canonical field already carries; differing authored
values stay. Transmission.app decode_interface_builder output shrinks
from 6.45 MB to 6.14 MB.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 00:31:00 -07:00
morluto 2596f8da14 test(android): retry verified cleanup during fixture teardown 2026-10-10 15:29:42 +08:00
morluto b12d0c6361 test(artifacts): respect destination filesystem case sensitivity 2026-10-10 15:29:42 +08:00
N0zoM1z0 cbcba2902d docs(website): align Blog community links with the site 2026-10-10 15:28:13 +08:00
N0zoM1z0 07d416505c docs(website): finalize the Touhou essay for publication 2026-10-10 15:28:13 +08:00
N0zoM1z0 3ea0233342 docs(website): simplify the Blog introduction 2026-10-10 15:28:13 +08:00
N0zoM1z0 cf6957e796 feat(website): link the Blog from the homepage introduction 2026-10-10 15:28:13 +08:00
N0zoM1z0 ed200a2302 docs(website): emphasize the reconstruction essay takeaways 2026-10-10 15:28:13 +08:00
N0zoM1z0 7df2050606 docs(website): write the Touhou reconstruction essay 2026-10-10 15:28:13 +08:00
N0zoM1z0 130ffdf6bc docs(website): outline the Touhou reconstruction workflow 2026-10-10 15:28:13 +08:00
N0zoM1z0 b09d02d93c feat(website): add a blog for methods and project notes 2026-10-10 15:28:13 +08:00
1f0775684b fix(artifacts): keep case-distinct archive names in inventory (#1480)
ZIP member names are case-sensitive. Inventory was folding them with a
filesystem case rule, so shrunk APKs never became evidence for the Android
application graph. Logical registration now keeps both spellings and records
an English Unicode case-folding limitation. Extraction fails only when the
destination directory already holds the other spelling.

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-10-10 15:21:16 +08:00
7296f2b5bf perf(javascript)!: stop repeating the semantic identifier strategy per fact (#1474)
Every semantic graph node and relation carried an identifier_strategy
object whose schema admits exactly one value per record type. The
constant made up about 9% of analyze_javascript_application output
(Obsidian app.asar: 12.04 MB to 10.96 MB), and semantic traces repeated
it on every returned fact.

The node_id and relation_id schema descriptions now state how each
identifier is derived.

BREAKING CHANGE: Semantic graph nodes and relations no longer include
identifier_strategy, and relation IDs no longer hash that constant.
Evidence recorded by earlier versions does not validate against the new
semantic graph schema.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 00:13:34 -07:00
3c7b176a3d perf(artifacts): deliver the inspected artifact graph once (#1473)
inspect_artifact nested the complete inventory Evidence and then restated
every artifact node and occurrence as an observation and every edge as a
derived relationship. No caller reads those copies; comparison and tests
use the nested inventory Evidence. On Transmission.app the result shrinks
from 2.46 MB to 1.10 MB.

Observations keep the root manifest and integrity findings.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 00:12:17 -07:00
aa23ec883f feat(clients): register Qwen Code during setup (#1415)
* feat(clients): register Qwen Code during setup

* fix(clients): honor Qwen home and shared skill discovery

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-10 15:12:09 +08:00
keemsisiandGitHub 934ffd28d1 perf(javascript): derive the application graph ID once
createJavaScriptApplicationGraph hashed the whole graph to derive its
graph_id, then parsed it with the public schema, whose invariant check
hashed the same content again to verify that ID. Validate the content
with every other invariant first, then derive graph_id once from the
validated copy. The public schema still verifies graph_id for stored
and untrusted graphs, and created graphs are unchanged.
2026-10-10 15:09:18 +08:00
79f6756239 docs(website): link the Discord community and REA on X (#1469)
Add an X link beside the GitHub and Discord header icons on every content
page, name the footer links "Join the community" and "Follow us on X", and
close the homepage with a "Join the community" section.

On narrow screens, pad the footer so the floating Top link no longer
covers its last row of links.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 15:00:09 +08:00
YsopwnableandGitHub 78e1308f49 docs(process): clarify normalized terminal line representation 2026-10-10 14:58:15 +08:00
YsopwnableandGitHub 4e3d335439 test(android): retain incomplete cleanup failure reasons (#1443)
Include the full stop disposition in the fixture assertion message, so incomplete owned teardown retains its typed reason in Vitest failure reporters. Preserve the existing cleanup requirement without retries or relaxed ownership checks.

Seven process-boundary tests, full serial typecheck/lint/duplication/format checks, and 19 final-head CI checks pass after integrating the combined-main fixture fix. A controlled failure-rendering probe verifies that the explicit message preserves the reason. The original CI cleanup failure remains unreproduced; this is diagnostic work with a synthetic JADX producer, not a root-cause or real-engine fix.

Refs #1441, #1446, #1448, #1065.
2026-10-10 14:37:49 +08:00
YsopwnableandGitHub ea92328775 fix(mcp): validate doctor probe against its discovered output contract (#1461)
Validate the target-free probe against the named definition discovered from the complete production catalog. Preserve initialized server and inventory observations when the SDK rejects the probe, and retain the schema failure reason, absolute deadline and owned-child cleanup.

The negative stdio replay regression fails on original main and passes the fix; actual production-boundary fault injection confirms rejection. Nine doctor/help tests, serial full static checks and real small-target CLI/MCP parity pass. Successful doctor reports remain byte-identical. No performance or model-token reduction is claimed.

Closes #1454. Refs #1065, #723, #1433.
2026-10-10 14:33:57 +08:00
morlutoandGitHub 28d3cb545c Merge pull request #1463 from morluto/codex/fix-skill-source-discovery
fix(skills): expose authored skill to repository discovery
2026-10-10 14:28:42 +08:00
b118f04492 perf(apple): stop repeating the archive path in Interface Builder evidence (#1460)
Every Interface Builder node and edge evidence record prefixed its
description with the archive's relative path, which artifact_path in the
same record already states. Bundles carry thousands of objects across
localized nibs, so the repeated path made up 6-10% of
decode_interface_builder output (XLD: 7.38 MB to 6.62 MB).

Descriptions now name only the decoded item, such as "object 81" or
"outlet o_quality".

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 23:21:24 -07:00
ef96657d6b perf(javascript): hash each semantic evidence context once (#1457)
The semantic graph's evidence-context registry canonicalized the full
provenance context three times for every node, relation, unknown and
fingerprint: once to hash its ID and twice more to compare against the
stored row. A file has only a few distinct contexts, so nearly all of
that work repeated a known answer.

Interning now keys contexts by their canonical text. Each fact
canonicalizes once, and only a new context is hashed and checked for a
digest collision. IDs and output are unchanged; Obsidian's 334 MB
Evidence is byte-identical and analysis drops from about 47.0 s to
44.4 s.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 23:21:07 -07:00
morluto 696fb834b8 fix(skills): expose authored skill to repository discovery 2026-10-10 14:20:22 +08:00
2781cad201 perf(process): stop padding rendered terminal lines to full width (#1450)
Rendered frames padded every visible row with trailing spaces to the
terminal width. Blank rows cost a full row each and every frame repeats
them, so the padding made up 14-39% of capture Evidence (2 KB of sparse
output produced 228 KB). The padding carried no information: columns is
recorded and serialized_state keeps the exact terminal state.

Lines are now trimmed after normalization, so line.padEnd(columns)
restores the previous value. The comparison contract commits
rendered_line_format so older padded captures do not compare as the
same contract.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 23:18:13 -07:00
openhands 96318b66a0 fix(process): normalize scenario replacements literally
capture_process_scenario declares normalization patterns as literal text, but\nreplaceAll expands \$&\, \$\`, and \$1\ in the replacement as a replacement\npattern. A caller normalizing text that contains a dollar sign silently gets\nduplicated, spliced, or dropped content in the recorded Evidence instead of\nthe requested literal replacement.\n\nUse split/join, which is literal on both sides and is already the idiom used\nfor exact-literal scenario rules in browserScenarioNormalization.ts.
2026-10-10 11:36:10 +05:30
YsopwnableandGitHub c89c8f5318 docs(mcp): record real Windows first-query recovery 2026-10-10 14:05:56 +08:00
1ec490f361 fix(javascript): stop replaying earlier writes for each object property write
Repeated property writes to one object made member-mutation collection
cubic. Each write evaluated its target binding to rule out a primitive,
and every evaluation folded all earlier mutation paths, copying every
slot once per path. 2,000 writes to one object took 27 s and 4,000 took
203 s.

- A recorded write always leaves a non-primitive binding value, so only
  the first write to a binding needs the primitive check.
- Apply mutation paths per slot in one pass with the same ordering
  semantics instead of one full slot copy per path.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com>
2026-10-10 14:04:31 +08:00
YsopwnableandGitHub 66863b609f test(javascript): reuse comparison node fixtures
Use the established typed graph fixture constructor while retaining right-hand comparison provenance assertions. Restore the unchanged duplicate-code gate on combined main.

Closes #1446. Refs #1434, #1443, #1065.
2026-10-10 14:02:02 +08:00
morlutoandGitHub 817efaca9a Merge pull request #1391 from marnixpluim/fix/electron-import-aliases
fix(javascript): recognize Electron APIs through renamed bindings
2026-10-10 13:57:24 +08:00
morluto 0d2082d36b fix(javascript): recognize unchanged var and let Electron aliases 2026-10-10 13:56:40 +08:00
morlutoandGitHub f1231d4a2c Merge pull request #1382 from BaaRai/fix/cross-branch-multiblock-callee
fix(ghidra): keep the cross-format branch callee multi-block
2026-10-10 13:50:51 +08:00
morlutoandGitHub a957f397ad Merge pull request #1303 from kvalls/codex/windows-ghidra-native-dlls
feat(ghidra): support native PE DLLs on Windows
2026-10-10 13:50:11 +08:00
YsopwnableandGitHub e09cfe9b59 fix(javascript): invalidate references stored through property targets (#1445)
* fix(javascript): invalidate references stored through property targets

* refactor(javascript): share rest keys and return-shape fixtures
2026-10-10 13:47:15 +08:00
morluto ea1dbda1c7 test(ghidra): share result projection client setup 2026-10-10 13:47:06 +08:00
morluto 7633dd8d5d fix(ghidra): publish Windows verification success after cleanup 2026-10-10 13:43:01 +08:00
morluto 941dc425ca chore: merge main into Windows DLL support branch 2026-10-10 13:42:54 +08:00
33b9b31a6f perf(javascript): index change-graph nodes once per comparison
Resolve each compared graph node through a per-side index while preserving right-hand evidence selection and complete comparison results.

Refs #1065.

Co-authored-by: keemsisi <7439375+keemsisi@users.noreply.github.com>
2026-10-10 13:21:13 +08:00
Kaoru0822andGitHub eeb83f6581 perf(cli): defer Playwright and isomorphic-git until a command needs them
Load browser composition after CLI scenario validation and load Git support only for local reference trees with Git metadata. Preserve default outputs and actual command behavior.

Refs #1065.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-10 13:16:44 +08:00
YsopwnableandGitHub bb6fa1c6f5 docs(process): verify independent full-report hashing budgets (#1440) 2026-10-10 13:13:05 +08:00
YsopwnableandGitHub ce16f7bc95 test(native): verify macOS provenance through real CLI and MCP
Verify five native operations through the public CLI and real SDK stdio MCP using a source-built macOS fixture. Retain normalized provenance, response parity and unchanged input bytes.

Refs #1430, #1065, #723.
2026-10-10 13:05:12 +08:00
YsopwnableandGitHub 0a14ca41b4 fix(javascript): preserve effects through yielded array references (#1437) 2026-10-10 13:00:11 +08:00
YsopwnableandGitHub dc5af6d68f perf(mcp): reuse output validators across catalog refreshes (#1433)
Advertise content- and dialect-bound identities for self-contained output schemas so the SDK can reuse unchanged Ajv validators after complete tools/list refreshes. Preserve explicit IDs and relative reference bases.

Verify changed schemas, availability transitions, reconnects, numeric and selector constraints, explicit IDs, relative references, and advertised dialects through the pinned SDK boundaries.

Refs #1065, #723
2026-10-10 12:49:09 +08:00
0b9815b6e7 docs(skill): save CLI application Evidence and read a summary view (#1429)
The skill's CLI fallback printed the complete application Evidence to
stdout. A real ASAR produces hundreds of megabytes (334 MB for Obsidian),
which an agent then reads into its context. Save the Evidence to a file
and project a summary or module page with inspect-analysis-view instead.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 21:38:52 -07:00
6375ccaf04 perf(native): record macOS command provenance once in its Evidence (#1430)
inspect_macho, inspect_signature, inspect_plist, list_architectures and
demangle_swift already return their command provenance inside the
normalized result. The provider also copied the same array into
raw_result, so every response carried it twice. raw_result now stays
null, as docs/mcp-contracts.md describes for results without a distinct
provider representation.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 21:35:50 -07:00
c29931ce81 fix(javascript): scan direct returns without recursive stack growth
Use the shared AST child enumeration with an explicit stack, preserving return order and nested callable boundaries.

Refs #1065.

Co-authored-by: rudycelekli <47457359+rudycelekli@users.noreply.github.com>
Co-authored-by: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com>
2026-10-10 12:25:00 +08:00
YsopwnableandGitHub f4a0c1ce95 fix(javascript)!: avoid event payload copies in graph metadata (#1427)
Share the canonical string identity rule with endpoint projection, scope event keys to each emitter, and keep complete event/listener names in properties. Verify real CLI and stdio MCP with large names, registration/removal, distinct emitters, exact name seeds, reserved-prefix values, and unresolved dynamic names.

BREAKING CHANGE: New analyses use digest-based IDs for long or reserved-prefix event names and shorter event/listener display labels. Select IDs from the parent graph and read exact names from event_name properties. Stored graphs and their original IDs remain accepted for semantic tracing.
2026-10-10 12:20:05 +08:00
a9ee2360fd feat(native): inspect oversized retained function dossiers via bounded views
Add caller-selected native dossier facets with preserved Evidence identity, provenance, unknowns and transport recovery.

Refs #1247 and #1065.

Co-authored-by: jhuang-tw <77732008+jhuang-tw@users.noreply.github.com>
Co-authored-by: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com>
2026-10-10 12:18:50 +08:00
YsopwnableandGitHub 4d95f2b2e7 fix(validation): retain unmatched union correction details
Preserve actionable field paths and schema-declared variants for unknown or missing discriminators.

Closes #1421. Refs #1065.
2026-10-10 12:15:11 +08:00
28595276fa fix(javascript): bound member-mutation paths through reassignment loops
Fixes #1401. Refs #1065. Preserve exact member facts and equal-depth write effects while bounding the observed reassignment-loop expansion.

Validated with the JavaScript semantic/application suite, focused alias regressions, full static checks, real compiled CLI and SDK MCP Notion workflows, and all 20 checks on the final PR head.

Co-Authored-By: Marnix Pluim <286664873+marnixpluim@users.noreply.github.com>
Co-Authored-By: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com>
Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-10 12:03:13 +08:00
YsopwnableandGitHub 98022a8352 fix(javascript)!: avoid endpoint payload copies in graph metadata (#1423)
Keep exact endpoint observations and request properties while using compact display labels and collision-safe canonical value digests for long artifact-local endpoint keys. Verify real CLI/stdio MCP tracing and version comparison without truncating evidence.

BREAKING CHANGE: New analyses use compact endpoint labels and digest-based IDs for long or reserved-prefix endpoint keys. Select IDs from the parent graph and read full values from endpoint properties. Previously stored graphs retain their original identities and remain valid workflow inputs.
2026-10-10 12:02:24 +08:00
YsopwnableandGitHub 81cdafc132 docs: celebrate 50,000 GitHub stars 🎉 2026-10-10 03:45:42 +00:00
YsopwnableandGitHub 4d8ef3f350 perf(javascript): aggregate semantic family coverage once (#1410)
Reduce repeated semantic relation and unknown family scans while preserving exact complete Evidence and coverage.
2026-10-10 11:30:42 +08:00
ca3131b36c perf(cli): write compact JSON when output is not a terminal (#1413)
`--json` / `--format json` always indented its streamed result. Agents and
scripts read REA through pipes and redirects, where indentation carries no
information: on real application results it added 16-45% model tokens
(o200k proxy) and 25-73% bytes. The skill's CLI fallback uses `--json`.

Indent only when stdout is a TTY, as `gh --json` does; piped or redirected
JSON is compact. The JSON value, `jsonl`, `--filter-output`,
`--full-output` and the token-window paths are unchanged. `jq .` restores
indentation for files.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 20:29:48 -07:00
YsopwnableandGitHub de8fc67195 fix(javascript): retain shorter literal metadata encodings (#1408)
Choose the shorter complete JSON or digest key and string display label.
Short values keep their existing identities and labels, while large values
retain the payload-copy reduction without an arbitrary truncation limit.
2026-10-10 11:23:47 +08:00
YsopwnableandGitHub 8550913cda fix(javascript)!: stop copying literals into semantic metadata (#1407)
Commit literal values with canonical JSON SHA-256 in semantic role keys
and use a type label for strings. Preserve complete values, provenance and
literal queries while avoiding two payload copies in each string node.

Verify real compiled CLI/default stdio MCP, typed union seeds, empty
strings and trace continuation with a large source-owned fixture.

Refs #723

BREAKING CHANGE: Newly produced semantic literal node IDs use
value-sha256 role keys, and string labels read "string literal". Read exact
values from properties.value and obtain node IDs from the selected parent
graph. Previously stored Evidence and its original IDs remain valid.
2026-10-09 20:00:53 -07:00
1f0f728c63 perf(hopper): stop repeating Hopper replies as raw_result (#1405)
HopperProvider passed the bridge reply as rawResult and, for every
operation except address_to_file_offset, the same object as the result.
Every Hopper Evidence record (decompilation, procedure and string
listings, xrefs, ...) therefore carried its complete result twice, and
an MCP response four times counting its text copy.

raw_result is now null unless REA transformed the reply, which only the
file-offset mapping does; that case keeps Hopper's original reply.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 20:00:35 -07:00
7623e21ff0 perf(native): record pwntools diagnostics once in their Evidence (#1404)
inspect_binary_layout and inspect_recorded_crash passed the decoded report
as both result and rawResult, so every Evidence record carried the whole
report twice (and an MCP response four times, counting its text copy).
The decoded report is the only representation, so raw_result is now null,
as for other single-representation operations.

The real pwntools verifiers asserted the duplicate; they now assert null.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 20:00:11 -07:00
499e2f54ef perf(evidence): stop copying normalized results into raw_result (#1402)
createAnalysisExecution defaulted an omitted rawResult to a second
complete copy of the result. Every caller that had no distinct provider
representation therefore doubled its Evidence: decode_interface_builder,
inspect_keyed_archive, inspect_asset_catalog, trace_dylib_resolution,
inspect_native_dispatch_metadata, observe_native_calls,
observe_native_ui/capture_native_ui_scenario (including screenshots), and
small Ghidra/IDA lifecycle records. Sibling operations of the same
providers already recorded null.

An omitted rawResult now records null. Normalized results are unchanged.
MCP responses repeat Evidence in text and structured content, so each
response and its share of the 10 MiB transport budget shrink by the same
half; e.g. trace_dylib_resolution on a real app bundle drops from
3,217,119 to 1,612,524 bytes per representation.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 19:52:54 -07:00
6875d4a64c fix(cli): accept JSON files for web capture and screenshot comparisons (#1397)
compare-web-captures and compare-web-screenshots parsed their arguments as
inline JSON only, unlike every other JSON-input command. Screenshot
artifacts and scenario captures embed base64 PNG bytes, so a typical
viewport screenshot exceeds the command-line length limit (128 KiB per
argument on Linux, 1 MiB total on macOS) and could not be compared from
the CLI at all, while MCP accepts it. A file path was reported as
malformed JSON, and screenshot schema failures returned no issues.

Each argument now accepts inline JSON or a JSON file path through
parseCliJsonInput; file failures keep its path diagnostics, malformed
inline JSON keeps the field issue, and screenshot validation reports
projected schema issues. The tests share one CLI JSON process fixture.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 19:29:42 -07:00
2b6e9ec03f fix(browser): classify selected web trace file failures by cause (#1396)
trace_web_module_imports and trace_web_source_location reported a missing
or non-file manifest, source map or import map as artifact_operation_failed
("Artifact could not be read or written" / "unsafe or conflicting internal
path"). These are caller selections: report ENOENT/ENOTDIR and non-regular
files as invalid input on the selected field, and EACCES/EPERM as a host
access denial. A missing manifest-derived source file stays an artifact fault.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 19:29:31 -07:00
498cec042a fix(cli): stop advertising multi-letter single-dash option aliases (#1395)
Incur's `alias` map registers one-letter short flags and already accepts
kebab-case spellings of camelCase options. Twenty options registered their
kebab-case long name as an "alias", so help and completions advertised
flags such as `-case-sensitive` and `-relative-path` that the parser reads
as stacked short flags and rejects with "Unknown flag: -c".

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 19:29:20 -07:00
Minghang ChamlingandGitHub 5fc4d792ca fix(javascript): handle oversized inline source maps (#1386)
* fix(javascript): handle oversized inline source maps

* fix(javascript): avoid source map directive regex overflow
2026-10-10 10:24:38 +08:00
Claude e4a8e0b728 fix(javascript): recognize Electron APIs through renamed bindings
Electron recognition matched callee spellings, so a bundler rename such as
esbuild's `import { ipcMain as ipcMain2 } from "electron"`, or a destructured
`const { contextBridge: bridge } = require("electron")`, hid every IPC
handler, BrowserWindow, contextBridge and utility process behind it.

Resolve each call and construction root lexically to its Electron import or
require origin, and classify the callee under that export's name. Shadowed
locals keep their own meaning.

On Hermes Desktop v0.21.6, whose main process is one esbuild ESM bundle,
recognized main IPC handlers rise from 1 to 204, BrowserWindow constructions
from 0 to 11, and paired renderer transmissions from 1 to 201.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XPY2GmufMNGQKoEgRUx1pd
2026-10-10 00:44:32 +00:00
GRAB Racem e17290cd2a ci: re-run the suite on the corrected main checkpoint
b8a8190b restored the guarded-mutation uncertainty the failing test
encodes, so the shard should pass against the new merge base.
2026-10-10 02:34:52 +02:00
GRAB Racem 40f632a155 ci: re-run the suite after the main-branch shard flake
The failed shard is the JavaScriptSemanticGraphBuilder short-circuit
mutation test, which also failed on main run 38003459598 at the same
base commit (08ec2bf8) before this branch started CI, and passed on
main run 38003459600 minutes later at the same commit.
2026-10-10 02:06:15 +02:00
morluto 7aa4d768eb chore(release): sync corrected 6.3 checkpoint 2026-10-10 07:57:04 +08:00
morlutoandGitHub 74cf9e1401 Merge pull request #1383 from morluto/release-please--branches--release/6.3.0--components--rea-agents
chore(release/6.3.0): release rea-agents 6.3.0
2026-10-10 07:56:11 +08:00
morluto bf58eac4ef docs(release): restore complete 6.3 migration and ancestry notes 2026-10-10 07:42:17 +08:00
github-actions[bot]andGitHub 565f2efa03 chore(release/6.3.0): release rea-agents 6.3.0 2026-10-09 23:41:35 +00:00
morluto b8a8190b2f fix(javascript): retain conditional mutation ordering uncertainty 2026-10-10 07:40:25 +08:00
GRAB Racem e10c934d5a fix(ghidra): keep the cross-format branch callee multi-block
The cross-format oracle requires a callee whose recovered control flow
has more than one basic block, and rea_cross_branch provided it through
a symmetric if/else of tail calls. Clang 21 folds that shape into a
conditional negate plus one tail branch, a single basic block, so the
lane rejected a correct Ghidra analysis of the fixture.

Give the branch a loop whose trip count depends on a volatile global:
no compiler can eliminate the backedge, and the callee stays multi-block
at every optimization level on every toolchain.
2026-10-10 01:36:11 +02:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
08ec2bf82f chore(main): release rea-agents 6.3.0 (#1341)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-10 07:14:55 +08:00
kvalls be6863e9a1 feat(ghidra): support native PE DLLs on Windows 2026-10-09 16:11:13 +01:00
214 changed files with 7464 additions and 853 deletions
@@ -2,7 +2,7 @@
name: reverse-engineer-anything
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
metadata:
version: "34"
version: "35"
---
# REA
@@ -62,13 +62,20 @@ with a version-pinned `rea-agents` package or continue through the CLI.
A concrete CLI fallback for an operator-supplied JavaScript tree or ASAR is:
```bash
npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json
npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json > app-evidence.json
jq -c '{source: {kind: "inline", evidence: .}, view: {kind: "summary"}}' app-evidence.json > app-view.json
npx -y rea-agents@latest inspect-analysis-view app-view.json
```
No MCP registration or native engine is required. Read the returned Evidence,
graph, limitations, and unknowns with the same care as an MCP result; the CLI
returns the Evidence record directly. This fallback does not establish that
MCP is configured. Native CLI tasks still require their selected engine.
No MCP registration or native engine is required. The CLI returns the complete
Evidence record directly, and a real application can produce hundreds of
megabytes, so save it to a file rather than reading it into context. Start from
the summary view, then request a module page
(`{"kind": "page", "collection": "modules", "offset": 0, "limit": 32}`) or one
module the same way, or pass the saved Evidence inline to
`trace-application-feature`. Read graph, limitations, and unknowns with the
same care as an MCP result. This fallback does not establish that MCP is
configured. Native CLI tasks still require their selected engine.
## Route the target first
@@ -34,6 +34,14 @@ cross-references. Addresses and recovered pseudocode are analysis observations,
not original source. Provider unavailability and unsupported metadata remain
unknown rather than false.
For a completed `analyze_function` dossier, use `inspect_analysis_view` with
`view: {kind: "native", facet: "value_flow_operations", offset: 0, limit: 32}`
and the exact retained Evidence ID to read selected facts without reanalysis.
Native facets also include procedure, pseudocode, assembly, callers/callees,
references, and value-flow summaries. Pseudocode offsets and limits count UTF-16
code units; use the returned `next_offset` for continuation. Provider limitations
and unavailable facts remain explicit; the complete parent Evidence is retained.
To see which functions or Objective-C methods a Mach-O actually calls in one
run, use `observe_native_calls` with explicit breakpoints and a bounded
`duration_ms`/`max_events`. It launches a new process under LLDB, so the target
+1 -1
View File
@@ -28,7 +28,7 @@ intentionally not included. -->
- Provider, bridge, target-format, or platform compatibility:
- Evidence, artifact, provenance, or reconstruction contract:
- Process execution, authorization, cleanup, or containment impact:
- Generated metadata (`docs/public/product-catalog.json`), package, or installation impact (CI generates and validates ignored catalogs and `skills/`; edit authored instructions in `skill-src/` and never commit build outputs):
- Generated metadata (`docs/public/product-catalog.json`), package, or installation impact (CI generates and validates ignored catalogs and `skills/`; edit authored instructions in `.agents/skills/reverse-engineer-anything/` and never commit build outputs):
## Evidence and regression coverage
+20
View File
@@ -64,6 +64,14 @@ jobs:
env:
REA_ANALYSIS_PROVIDER: ghidra
run: npm run verify:ghidra:windows -- --x86 | Tee-Object -FilePath windows-ghidra-x86-proof.log
- name: Verify native x64 DLL Windows Ghidra operations
env:
REA_ANALYSIS_PROVIDER: ghidra
run: npm run verify:ghidra:windows -- --dll | Tee-Object -FilePath windows-ghidra-dll-proof.log
- name: Verify native x86 DLL Windows Ghidra operations
env:
REA_ANALYSIS_PROVIDER: ghidra
run: npm run verify:ghidra:windows -- --x86 --dll | Tee-Object -FilePath windows-ghidra-x86-dll-proof.log
- name: Verify ordinary-user CLI and MCP operations
env:
REA_ANALYSIS_PROVIDER: ghidra
@@ -72,6 +80,14 @@ jobs:
env:
REA_ANALYSIS_PROVIDER: ghidra
run: npm run verify:ghidra:windows:package -- --x86 | Tee-Object -FilePath windows-ghidra-x86-package-proof.log
- name: Verify ordinary-user x64 DLL CLI and MCP operations
env:
REA_ANALYSIS_PROVIDER: ghidra
run: npm run verify:ghidra:windows:package -- --dll | Tee-Object -FilePath windows-ghidra-dll-package-proof.log
- name: Verify ordinary-user x86 DLL CLI and MCP operations
env:
REA_ANALYSIS_PROVIDER: ghidra
run: npm run verify:ghidra:windows:package -- --x86 --dll | Tee-Object -FilePath windows-ghidra-x86-dll-package-proof.log
- uses: actions/upload-artifact@v4
if: always()
with:
@@ -81,4 +97,8 @@ jobs:
windows-ghidra-package-proof.log
windows-ghidra-x86-proof.log
windows-ghidra-x86-package-proof.log
windows-ghidra-dll-proof.log
windows-ghidra-x86-dll-proof.log
windows-ghidra-dll-package-proof.log
windows-ghidra-x86-dll-package-proof.log
retention-days: 30
+1 -1
View File
@@ -35,7 +35,7 @@ Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain.
Use relevant checks for the change. The complete local gate, `npm run check:pr`, is optional for broad changes or CI diagnosis. See [CONTRIBUTING.md](CONTRIBUTING.md) for development, generated-file ownership, and release conventions.
`docs/public/product-catalog.json`, `docs/verification/managed-conformance-*.json`, and `skills/` are ignored outputs. Edit source contracts and `skill-src/`, then regenerate as needed. `.cache/mcp-tool-catalog.json` is test metadata; runtime builds and source checking do not consume it. Never commit binaries, provider project documents, credentials, `dist/`, `node_modules/`, or local planning artifacts.
`docs/public/product-catalog.json`, `docs/verification/managed-conformance-*.json`, and `skills/` are ignored outputs. Edit source contracts and `.agents/skills/reverse-engineer-anything/`, then regenerate as needed. `.cache/mcp-tool-catalog.json` is test metadata; runtime builds and source checking do not consume it. Never commit binaries, provider project documents, credentials, `dist/`, `node_modules/`, or local planning artifacts.
## Tool and Test Changes
+1
View File
@@ -66,6 +66,7 @@
* **javascript:** resolve exported callables through lexical bindings ([#1351](https://github.com/morluto/rea/issues/1351)) ([a9f5dad](https://github.com/morluto/rea/commit/a9f5dad5a8f196c88f6305d8d958b355cc0b52c2))
* **javascript:** resolve query-only HTML references to their document ([55a46f7](https://github.com/morluto/rea/commit/55a46f7c7176075c9edda4fe61ed995f2ceadf6a))
* **javascript:** retain array identity after length mutations ([4bcea1a](https://github.com/morluto/rea/commit/4bcea1aa76e3e7ca665197181739c5e27a62c40a))
* **javascript:** retain conditional mutation ordering uncertainty ([b8a8190](https://github.com/morluto/rea/commit/b8a8190b2f30e377eee2d69d1ba31ea454fa70b9))
* **javascript:** retain distinct module provenance path components ([2bd7de6](https://github.com/morluto/rea/commit/2bd7de6f8ca62fc1d48334cfa7fdced461b7b153))
* **javascript:** retain unresolved value frontiers in provenance queries ([3e19875](https://github.com/morluto/rea/commit/3e19875c7fa9c61cff655bbe6046ddc4ab558533))
* **javascript:** store source map URLs once ([ed8fcae](https://github.com/morluto/rea/commit/ed8fcae94b699b62b44162f880faa077a36c455d))
+6 -2
View File
@@ -95,8 +95,12 @@ Pre-commit formats and lints staged files; pre-push runs `check:fast`.
`evidence:generate` prepares only the managed commitment and its runtime/skill
dependencies. The ignored outputs are `docs/public/product-catalog.json`,
`docs/verification/managed-conformance-*.json`, and `skills/`. Edit authored
skill instructions in `skill-src/`. CI validates generated outputs and retains
them as artifacts; do not commit them. Reviewed source metadata, including
skill instructions in `.agents/skills/reverse-engineer-anything/`, where Skills
can discover the public bundle directly from a clean checkout. `verify:package`
installs that bundle with Skills 1.7.2 using default repository-root discovery
and checks every installed file, alongside the packaged setup checks.
CI validates generated outputs and retains them as artifacts; do not commit
them. Reviewed source metadata, including
`src/generatedPackageMetadata.ts` and `docs/error-contract.schema.json`, remains
tracked and checked for freshness.
+1 -1
View File
@@ -307,7 +307,7 @@ Report vulnerabilities through [SECURITY.md](SECURITY.md).
## Star history
🎉 **40,000 GitHub stars — thank you!**
🎉 **50,000 GitHub stars — thank you!**
Thanks to everyone using REA, reporting bugs, requesting features, testing builds, and contributing fixes.
+1 -1
View File
@@ -258,7 +258,7 @@ npx rea-agents@latest setup
## سجل النجوم
🎉 **40,000 نجمة على GitHub — شكرًا لكم!**
🎉 **50,000 نجمة على GitHub — شكرًا لكم!**
شكرًا لكل من يستخدم REA، ويبلّغ عن الأخطاء، ويقترح ميزات، ويختبر إصدارات البناء، ويساهم بالإصلاحات.
+1 -1
View File
@@ -256,7 +256,7 @@ Melde Sicherheitslücken gemäß [SECURITY.md](SECURITY.md).
## Sternverlauf
🎉 **40.000 GitHub-Sterne – vielen Dank!**
🎉 **50.000 GitHub-Sterne – vielen Dank!**
Danke an alle, die REA nutzen, Fehler melden, Funktionswünsche äußern, Builds testen und Fehlerbehebungen beitragen.
+1 -1
View File
@@ -256,7 +256,7 @@ Informa de vulnerabilidades siguiendo [SECURITY.md](SECURITY.md).
## Historial de estrellas
🎉 **40.000 estrellas en GitHub: ¡gracias!**
🎉 **50.000 estrellas en GitHub: ¡gracias!**
Gracias a quienes usan REA, informan de errores, solicitan funciones, prueban compilaciones y contribuyen con correcciones.
+1 -1
View File
@@ -241,7 +241,7 @@ npx rea-agents@latest setup
## تاریخچهٔ ستاره‌ها
🎉 **۴۰٬۰۰۰ ستارهٔ GitHub — سپاسگزاریم!**
🎉 **۵۰٬۰۰۰ ستارهٔ GitHub — سپاسگزاریم!**
از همهٔ کسانی که از REA استفاده می‌کنند، خطاها را گزارش می‌دهند، قابلیت‌های جدید پیشنهاد می‌کنند، buildها را آزمایش می‌کنند و در بهبود پروژه مشارکت دارند سپاسگزاریم.
+1 -1
View File
@@ -257,7 +257,7 @@ Signalez les vulnérabilités via [SECURITY.md](SECURITY.md).
## Historique des étoiles
🎉 **40 000 étoiles GitHub — merci !**
🎉 **50 000 étoiles GitHub — merci !**
Merci à toutes les personnes qui utilisent REA, signalent des bugs, proposent des fonctionnalités, testent les builds et contribuent des correctifs.
+1 -1
View File
@@ -258,7 +258,7 @@ npx rea-agents@latest setup
## スター履歴
🎉 **GitHub スター 40,000 件、ありがとうございます!**
🎉 **GitHub スター 50,000 件、ありがとうございます!**
REA の利用、バグ報告、機能の要望、ビルドのテスト、修正への貢献に感謝します。
+1 -1
View File
@@ -258,7 +258,7 @@ npx rea-agents@latest setup
## 스타 기록
🎉 **GitHub 스타 40,000개, 감사합니다!**
🎉 **GitHub 스타 50,000개, 감사합니다!**
REA를 사용하고, 버그를 보고하고, 기능을 요청하고, 빌드를 테스트하고, 수정에 기여해 주신 모든 분께 감사드립니다.
+1 -1
View File
@@ -256,7 +256,7 @@ Zgłaszaj podatności zgodnie z [SECURITY.md](SECURITY.md).
## Historia gwiazdek
🎉 **40 000 gwiazdek na GitHubie — dziękujemy!**
🎉 **50 000 gwiazdek na GitHubie — dziękujemy!**
Dziękujemy wszystkim, którzy używają REA, zgłaszają błędy, proponują nowe funkcje, testują kompilacje i przesyłają poprawki.
+1 -1
View File
@@ -256,7 +256,7 @@ Relate vulnerabilidades conforme [SECURITY.md](SECURITY.md).
## Histórico de estrelas
🎉 **40.000 estrelas no GitHub — muito obrigado!**
🎉 **50.000 estrelas no GitHub — muito obrigado!**
Obrigado a todos que usam o REA, relatam bugs, sugerem funcionalidades, testam builds e contribuem com correções.
+1 -1
View File
@@ -256,7 +256,7 @@ npx rea-agents@latest setup
## История звёзд
🎉 **40 000 звёзд на GitHub — спасибо!**
🎉 **50 000 звёзд на GitHub — спасибо!**
Спасибо всем, кто использует REA, сообщает об ошибках, предлагает новые функции, тестирует сборки и помогает с исправлениями.
+1 -1
View File
@@ -256,7 +256,7 @@ npx rea-agents@latest setup
## ประวัติดาว
🎉 **40,000 ดาวบน GitHub — ขอบคุณทุกคน!**
🎉 **50,000 ดาวบน GitHub — ขอบคุณทุกคน!**
ขอบคุณทุกคนที่ใช้ REA รายงานบั๊ก เสนอฟีเจอร์ ทดสอบบิลด์ และช่วยแก้ไข
+1 -1
View File
@@ -256,7 +256,7 @@ Güvenlik açıklarını [SECURITY.md](SECURITY.md) üzerinden bildirin.
## Yıldız geçmişi
🎉 **GitHub'da 40.000 yıldız — teşekkürler!**
🎉 **GitHub'da 50.000 yıldız — teşekkürler!**
REA'yı kullanan, hata bildiren, özellik talep eden, derlemeleri test eden ve düzeltmelere katkıda bulunan herkese teşekkürler.
+1 -1
View File
@@ -256,7 +256,7 @@ npx rea-agents@latest setup
## Історія зірок
🎉 **40 000 зірок на GitHub — дякуємо!**
🎉 **50 000 зірок на GitHub — дякуємо!**
Дякуємо всім, хто використовує REA, повідомляє про помилки, пропонує нові функції, тестує збірки й допомагає з виправленнями.
+1 -1
View File
@@ -256,7 +256,7 @@ Báo cáo lỗ hổng theo [SECURITY.md](SECURITY.md).
## Lịch sử sao
🎉 **40.000 sao trên GitHub — xin cảm ơn!**
🎉 **50.000 sao trên GitHub — xin cảm ơn!**
Cảm ơn mọi người đã sử dụng REA, báo lỗi, đề xuất tính năng, kiểm thử bản dựng và đóng góp bản sửa lỗi.
+1 -1
View File
@@ -258,7 +258,7 @@ npx rea-agents@latest setup
## Star 歷史
🎉 **GitHub Star 達到 40,000 個,感謝大家!**
🎉 **GitHub Star 達到 50,000 個,感謝大家!**
感謝每一位使用 REA、報告錯誤、提出功能需求、測試建置和貢獻修復的朋友。
+1 -1
View File
@@ -258,7 +258,7 @@ npx rea-agents@latest setup
## Star 历史
🎉 **GitHub Star 达到 40,000 个,感谢大家!**
🎉 **GitHub Star 达到 50,000 个,感谢大家!**
感谢每一位使用 REA、报告错误、提出功能需求、测试构建和贡献修复的朋友。
+7 -2
View File
@@ -126,8 +126,13 @@ also stops target hashing before provider admission.
These operations do not execute the APK or provide split APK/AAB handling,
signature validation, full resource-table semantics, native-library analysis or
Android runtime capture. Existing artifact inventory/extraction tools can supply
archive evidence. Project that Evidence with the execution-free inventory tool:
Android runtime capture. An `.aab` suffix is a ZIP archive, not an APK, so
Android tools and the application graph do not treat it as a standalone APK.
Existing artifact inventory/extraction tools can supply archive evidence.
Inventory keeps ZIP names that differ only in case and records that a
case-insensitive destination cannot store both spellings. Extraction fails
only when that destination directory cannot keep both names. Project that
Evidence with the execution-free inventory tool:
```sh
rea project-android-application-graph '{"inventory_evidence":[<inventory_artifact Evidence>]}'
+1
View File
@@ -18,6 +18,7 @@ REA can attach to a user-owned Chrome-family browser through the Chrome DevTools
- `compare_web_captures` / `rea compare-web-captures` compares passive captures or exact-step-aligned browser scenarios. Scenario results commit reproducible literal normalization and expose alignment failures plus artifact-level action, screenshot, DOM, accessibility, URL, history, storage, and event differences. Missing or truncated evidence is never treated as equivalence.
- `capture_web_screenshot` / `rea capture-web-screenshot` returns a content-addressed visible-viewport PNG.
- `compare_web_screenshots` / `rea compare-web-screenshots` performs bounded local PNG pixel comparison without OCR or external services.
- The CLI comparison commands accept each capture, screenshot artifact or normalization policy as inline JSON or a JSON file path. Use files for screenshot artifacts and scenario captures, whose embedded PNG bytes can exceed the host's command-line length limit.
- Every surface has equivalent CLI and MCP contracts and returns Evidence provenance.
- MCP tools return complete results inline and include the Evidence ID. Session evidence can be exported with `export_evidence_bundle`.
+21 -1
View File
@@ -20,6 +20,23 @@ files when neither `--provider` nor `--snapshot` is supplied. See
[JavaScript artifact reconstruction](javascript-artifact-reconstruction.md)
for results, integrity checks and coverage.
The complete application Evidence can be hundreds of megabytes for a real
application. When an agent or a person reads the result, save it to a file and
project a summary, a module page or one module from the saved Evidence:
```bash
rea analyze-javascript-application /absolute/path/to/app --json > app-evidence.json
jq -c '{source: {kind: "inline", evidence: .}, view: {kind: "summary"}}' app-evidence.json > app-view.json
rea inspect-analysis-view app-view.json
```
For example, a 334 MB Obsidian application Evidence file yields a summary view
of about 10 KB. Replace the view with
`{"kind": "page", "collection": "modules", "offset": 0, "limit": 32}` for a
module page. See
[JavaScript application workflows](javascript-application-workflows.md) for
views, feature traces and comparisons that take the same saved Evidence.
## Native analysis
Configure [Hopper or Ghidra](installation.md#hopper), or the
@@ -179,7 +196,10 @@ user permissions; consult the chosen guide for host requirements and effects.
## Output and exit status
The default terminal format is TOON. Use `--json` when saving results for a
JSON consumer. Output selection and formatting do not change operation status.
JSON consumer. `--json` is indented on a terminal and compact when piped or
redirected, which keeps large results smaller for agents and scripts; pipe
through `jq .` for indented files. Output selection and formatting do not change
operation status.
| Status | Meaning |
| --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
+5 -2
View File
@@ -62,11 +62,14 @@ Mach-O target coverage. It requires `clang`, LLD, and `lld-link`; set
these tools before compilation. Keeping this matrix separate lets Linux
host/provider acceptance run with only the host compiler.
`npm run verify:ghidra:windows` and its `-- --x86` variant check the source-owned native x86 and x86-64 PE
fixture, all 25 read-only operations, target/snapshot/import SHA-256 linkage,
`npm run verify:ghidra:windows` checks source-owned native x86-64 PE applications;
`-- --x86` selects x86 and `-- --dll` selects DLLs, and the flags can be combined.
These lanes check all 25 read-only operations, target/snapshot/import SHA-256 linkage,
and project, endpoint, process, and runtime cleanup. The independent native
lane checks DACLs, handle admission, cancellation, and Job Object lifecycle.
`npm run verify:ghidra:windows:package` packs and installs REA into an isolated
prefix, then checks ordinary-user CLI/MCP operations against canonical contracts.
It accepts the same architecture and DLL flags and verifies the selected PE role
in the analysis profile.
All lanes require matching native controls; package startup alone establishes
only package compatibility.
+13 -6
View File
@@ -95,7 +95,7 @@ npx skills add morluto/rea --skill reverse-engineer-anything
This installs agent instructions and bundled references, not REA MCP
registration or analysis engines. Follow the skill's
[conditional connection guide](https://github.com/morluto/rea/blob/main/skill-src/reverse-engineer-anything/SKILL.md#connect-only-when-needed).
[conditional connection guide](https://github.com/morluto/rea/blob/main/.agents/skills/reverse-engineer-anything/SKILL.md#connect-only-when-needed).
Working tools can be used immediately. If tools are missing, inspect the current
client's registration with `doctor --client codex --json` (substitute its client
ID), then plan repairs with `setup --client codex --dry-run --json`. Show and
@@ -135,10 +135,17 @@ listed after the table because its connector is not one of these files:
| Antigravity | `antigravity` |
| GitHub Copilot CLI | `copilot_cli` |
| Command Code | `commandcode` |
| Qwen Code | `qwen_code` |
| VS Code | `vscode` |
| Grok Build | `grok_build` |
| OMP | `omp` |
For Qwen Code, setup registers `rea` in the `mcpServers` table of the
user-level `~/.qwen/settings.json`, or `$QWEN_HOME/settings.json` when configured.
Setup and doctor follow Qwen Code's tilde and working-directory-relative home
overrides. Qwen Code also discovers personal skills in `~/.agents/skills`,
where setup installs the bundled workflow.
For OMP, setup writes a `type: "stdio"` entry to the user-level
`~/.omp/agent/mcp.json`. It follows `PI_CONFIG_DIR`, an absolute
`PI_CODING_AGENT_DIR`, and the profile selected by `OMP_PROFILE` or
@@ -194,10 +201,10 @@ save verified paths for an existing Ghidra installation.
The bundled skill is installed where each selected client discovers personal
skills: Claude Code uses `~/.claude/skills` (or
`$CLAUDE_CONFIG_DIR/skills` when configured), while other supported clients
use the shared `~/.agents/skills` directory. A mixed selection plans both
paths. Selecting the skill without a client uses the shared directory. Setup
leaves existing skill copies in other locations untouched.
`$CLAUDE_CONFIG_DIR/skills` when configured), while other supported clients,
including Qwen Code, use the shared `~/.agents/skills` directory. A mixed selection
plans the relevant paths. Selecting the skill without a client uses the shared
directory. Setup leaves existing skill copies in other locations untouched.
`doctor --skill --json` verifies the selected copies against the bundled
instructions and references. Consumers should use `identity.skill.state`,
@@ -489,7 +496,7 @@ entry comments atomically and returns refreshed analysis. These session metadata
edits leave executable bytes unchanged and are discarded on close. GUI controls
require Hopper; Windows P0 remains read-only.
Windows P0 admits native x86 and x86-64 PE applications on fixed local NTFS volumes.
Windows P0 admits native x86 and x86-64 PE applications and DLLs on fixed local NTFS volumes.
The npm package bundles native Job Object ownership, protected private runtime
DACLs, and handle-based path admission; no separate addon installation is needed.
See the [Windows Ghidra P0 guide](windows-ghidra-p0.md) for verified scope and
+42 -2
View File
@@ -90,6 +90,44 @@ keep the result ambiguous. Runtime Evidence can later corroborate an exact
mapped candidate, but structural reachability, semantic influence, runtime
observation, and causal proof remain separate claims.
Semantic literal nodes keep their complete value in `properties.value`; literal
queries match that field. String labels use `string literal` when that is
shorter than the complete JSON value. A literal's `identity.role_key` commits
its canonical JSON value as `value-sha256:<digest>` when the digest key is
shorter; smaller values keep their existing JSON key. This keeps large values
out of identity and display metadata without expanding short values or
truncating their Evidence.
New analyses therefore produce different IDs for hashed literals from the
earlier payload-bearing role keys. Use IDs returned by the selected parent
graph and read literal values from `properties.value`, rather than decoding
role keys or labels. Previously stored graphs and their original IDs remain
valid inputs.
Endpoint nodes keep the exact endpoint in each observation's `properties.value`;
semantic request and response nodes keep it in `properties.endpoint`. Endpoint
identity keys use `value-sha256:<digest>` over the canonical JSON string when
that is shorter, or when the original value starts with the reserved digest
prefix. Other short endpoint keys remain unchanged. Endpoint labels use the
shorter of the exact value and `<kind> endpoint`; semantic request/response
labels use the shorter of the endpoint and operation method. These labels are
display metadata, not endpoint lookup values.
New analyses therefore change hashed endpoint IDs and longer endpoint labels.
Use IDs from the selected graph and read the complete endpoint properties;
previously stored graphs still support their original IDs. Literal endpoint
search, semantic tracing, and version comparison use the preserved values.
Event and listener nodes likewise preserve the complete event name in
`properties.event_name`. Event role keys use the same reserved-prefix string
digest rule, scoped to the emitter; repeated registrations, removals, and
dispatches keep one event per exact emitter/name pair. Long event labels use
`event`, and listener labels use the shorter of `<method>:<name>` and
`<method>:listener`. Dynamic names remain `null` with explicit uncertainty. New analyses
change hashed event IDs and longer event/listener labels; query the preserved
name or select IDs from the parent graph. Previously stored graphs keep their
original IDs and remain valid trace inputs.
## Version comparison
REA pairs entities only when a tier produces one unique candidate on each side.
@@ -145,8 +183,10 @@ Direct return expressions, including expression-bodied arrows, are evaluated
through an execution-free value lattice. Literal object fields and direct
return sites are represented in the result. Calls, dynamic spreads, computed
keys, and parser recovery remain partial or unknown.
Objects and arrays passed to calls, constructors, or tagged templates, or used
as method receivers, are not assumed unchanged after the invocation. Aliases
Objects and arrays passed to calls, constructors, or tagged templates, used
as method receivers, or stored through property targets are not assumed
unchanged afterward. Property stores conservatively retain reference escape
uncertainty rather than pretending to execute later writes. Aliases
and shared children in spread and rest copies retain that uncertainty; copied
primitive slots and unrelated containing properties remain known. Object rest
excludes consumed keys, array rest excludes the consumed prefix for known
+4
View File
@@ -82,6 +82,10 @@ x86-64 ELF and native Windows PE targets analyzed on Linux x64 with Ghidra
See [NativeAOT recovery](ghidra-nativeaot.md) for build/configuration, recovered
metadata, derived-memory provenance, and the real verification lane.
REA's experimental Windows Ghidra boundary admits native x86 and x86-64 PE
applications and DLLs for static analysis. This does not establish Windows
NativeAOT metadata-recovery coverage.
For NativeAOT analysis, preserve the selected native artifact's path and digest
and report recovered type data as provider observations. Linking those types
to native function addresses requires verified
+37 -6
View File
@@ -36,6 +36,13 @@ after SDK conversion and check advertised validation and actual calls.
Individual model APIs can impose additional nesting limits; complete producer
captures can exceed ten structural levels.
Self-contained output schemas advertise a content-bound `$id`, including their
declared dialect. SDK validators can reuse compiled schemas across complete
catalog refreshes and equivalent tool outputs; changing the schema changes its
identity. Explicit schema IDs and relative external reference bases are
preserved. This keeps every tool and validation rule in discovery and does not
change the SDK's catalog invalidation or availability checks.
`compare_web_captures` accepts exactly one of two input shapes:
- Passive: `before` and `after` each contain `inspection`, the complete
@@ -186,10 +193,27 @@ the current client, so it is not a fresh-start recovery. On the same connection:
This close/reopen flow was exercised on one real Linux stdio connection after a
controlled startup timeout, followed by a successful overview and function
analysis. It is not a Windows/macOS coverage claim. REA cleans only resources it
owns and never switches to another provider automatically. A provider timeout,
installation failure, or host permission denial needs its own reported recovery;
increasing a client deadline alone does not fix those failures.
analysis. A separate Windows x64 stdio run with Ghidra 12.1.4, JDK 21 and Node
22.19.0 exercised the same connection and caller-selected native PE: the default
60-second client request timed out, close succeeded, and reopening before a
240-second request completed the cold overview in about 163 seconds. The next
overview took about 19 ms. The cold and warm overviews reported two procedures
and two segments; the input digest was unchanged and no owned runtime root or
observed process-family member remained after final close. This was a small
1,024-byte fixture, so the longer request setting remains an example, not a
guarantee for larger targets. No macOS cold-start recovery was exercised.
With a supplied progress token, the Windows run received an operation-start
notification before timeout and start/completion notifications for the
successful retry. It received no intermediate import or auto-analysis progress.
These operation markers do not measure analysis work completed or extend the
client's deadline. Each new provider session still imports the target into a
fresh temporary project.
REA cleans only resources it owns and never switches to another provider
automatically. A provider timeout, installation failure, or host permission
denial needs its own reported recovery; increasing a client deadline alone does
not fix those failures.
## Tool results
@@ -198,7 +222,9 @@ Custom clients upgrading from 6.1 should follow the
Evidence-producing tools return the complete canonical Evidence record in both
text and structured content. Read `structuredContent.normalized_result` for the
operation result and `structuredContent.evidence_id` for its identity. The same
operation result and `structuredContent.evidence_id` for its identity.
`raw_result` holds a provider representation distinct from that result, such
as an upstream tool's original report, and is `null` when there is none. The same
record is retained in the session bundle. `analysis_profile` is always present:
a concrete profile object or `null`; either value participates in semantic
identity. Records omitting this field are rejected. Pass the returned Evidence
@@ -235,7 +261,12 @@ Ordinary responses keep their existing complete result contract.
## Retained application Evidence inputs
`inspect_analysis_view` projects a caller-selected view of already completed
`inspect_binary_layout` or `analyze_javascript_application` Evidence. Source is
`inspect_binary_layout`, `analyze_javascript_application`, or `analyze_function`
Evidence. Native views select procedure, pseudocode, assembly, callers/callees,
references, and high-pcode facets without starting a provider. Native offset
and limit default to 0 and 64; pseudocode uses UTF-16 code units and never
splits surrogate pairs, while other pages count rows. Unavailable facts and
provider limitations remain explicit. Source is
an exact same-session retained reference or portable inline Evidence. Views are
a summary, a layout mitigations or linkage facet, one section/symbol/module, or
a stable page with a caller-selected positive `limit`. Module pages include
+1 -1
View File
@@ -379,7 +379,7 @@ the tool fails with a `debugger-attach-denied` reason.
Install a Ghidra 12.1.x release and the 64-bit full JDK it declares, then configure REA to
use them. Ghidra analysis supports Linux x64/arm64 and macOS x64/arm64; each installation requires
the matching native decompiler. Experimental Windows x64 P0 admits native
x86-64 PE applications on local NTFS using bundled Job Object ownership,
x86 and x86-64 PE applications and DLLs on local NTFS using bundled Job Object ownership,
protected runtime DACLs, and handle-based path admission. See [Windows Ghidra P0](windows-ghidra-p0.md) and
[issue #527](https://github.com/morluto/rea/issues/527).
On Linux and macOS, `annotate_native_function` atomically edits a function name
+61 -2
View File
@@ -110,6 +110,23 @@ while comparisons and trace assertions use the normalized text. Retaining the
original adds at most `limits.output_bytes` of text, because it comes from the
same admitted PTY chunks rather than a second unbounded stream.
Rendered frame `lines` hold the visible rows after normalization, with trailing
U+0020 spaces removed. Blank rows and internal spaces remain. `columns` records
terminal cell width; normalization and Unicode can make string lengths differ
from that width. `serialized_state` holds terminal serialization after the
selected normalization. Original admitted PTY chunks are available through
`raw_data ?? data`, as described above. Captures commit this line format in their
comparison contract, so a capture whose lines were padded to full width does not
compare as the same contract.
Custom `normalization.patterns` replace literal text with the caller's literal
replacement, including dollar sequences such as `$&` and `$$`. An empty pattern
also matches the first and last string positions. Captures with custom patterns
commit `pattern_normalization_version: "literal-replacements-v1"`. Legacy captures
remain readable. Cross-version comparison is rejected; capture both sides with
the same replacement semantics. Captures without custom patterns keep their
existing comparison contract.
Port normalization recognizes explicit `port`, `tcp_port`, `udp_port`, and
`listen` fields, URL authorities, IP endpoints, and `localhost` endpoints, with
ports from 0 through 65535. A port may end a sentence, as in `port: 8080.`;
@@ -159,8 +176,11 @@ Trace assertions use coverage for the sources they select, so an assertion
about complete raw terminal output need not fail because rendered snapshots
were omitted. Captures missing `truncation_details` are rejected on import;
preserve them as historical files and recapture for current comparisons.
These diagnostics use the existing scenario
budgets; they do not introduce a separate file-hashing budget.
`limits.file_bytes` is already the cumulative whole-file hashing budget,
independent of `limits.output_bytes`, which bounds retained terminal content.
Each filesystem checkpoint starts with a fresh `file_bytes` budget across all
selected roots; it is not a per-file allowance. File contents are read in
64 KiB chunks for hashing and are not embedded in capture Evidence.
When the host withholds an unrelated process’s ownership token, REA leaves that
process untouched and records its PID and reason in `cleanup.unverified_processes`
@@ -176,6 +196,45 @@ as well. A newly started unreadable process can prevent verified cleanup even
after the selected command exits. REA preserves this uncertainty and leaves
that process untouched.
## Hash a report independently of terminal output
Select the producer's report directory and give `file_bytes` enough room for
the complete files you need, including other files in those roots. For example,
save this scenario and run `rea capture-process ./scenario.json --json > capture.json`:
```json
{
"executable": "node",
"arguments": [
"-e",
"const fs=require('node:fs');fs.mkdirSync('reports',{recursive:true});fs.writeFileSync('reports/final.json',JSON.stringify({status:'complete',data:'X'.repeat(131072)}));console.log('x'.repeat(8192));"
],
"working_directory": ".",
"filesystem_observation_paths": ["./reports"],
"limits": { "output_bytes": 1024, "file_bytes": 262144 }
}
```
The report exceeds the terminal budget but fits the independent 256 KiB hash
budget. Its complete SHA-256 appears in `normalized_result.files_after` at
`root_0:final.json`; check `truncation_details.filesystem_after` for omissions
and actual bytes hashed. The verbose terminal output is truncated without
preventing the report digest. The same scenario object is the MCP
`capture_process_scenario` input. REA never returns a prefix hash as a full-file
digest: an insufficient remaining budget produces a null digest and an
explicit omission reason.
A digest binds the complete bytes observed at that checkpoint, not the
producer's notion of a finished report. A stable periodic or running report
can also have a complete digest. Choose the final file and verify the
producer's documented completion marker, exit outcome and settlement
observations before calling it final. `after_settlement` names a filesystem
checkpoint, not a guarantee that an application finished successfully or
that an unobserved writer will never change the file. REA checks opened-file
identity/state around hashing and reports observed changes or unavailable
files; cancellation stops the hash. If an external sealer produces a manifest,
keep its completion claims and digest separate from REA's checkpoint facts.
## Compare two captures
Compare saved capture Evidence with:
+13 -2
View File
@@ -17,7 +17,7 @@ additional permission flag or degraded mode is required.
- Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+;
- an operator-installed Ghidra 12.1.x distribution (verified with 12.1.4);
- a 64-bit full JDK inside that installation's declared Java range (JDK 21 or newer, with no maximum, for current 12.1 releases);
- an explicit native, non-managed, non-DLL x86 or x86-64 PE application; and
- an explicit native, non-managed x86 or x86-64 PE application or DLL; and
- the 25 read-only Ghidra inventory, memory, and function-analysis operations.
Loaded memory reads and file offsets preserve Ghidra's observed source mappings.
@@ -38,6 +38,13 @@ REA selects its packaged bridge scripts by explicit path. Same-named files or
directories in the caller's working directory do not select a different
script or require users to clean that directory before analysis.
Native DLL analysis preserves the PE shared-library role in the analysis profile.
REA imports an immutable copy for static analysis; it does not load the library
into the analyst process or execute its entry routine. The source-owned DLL
fixtures have no entry routine or imports and expose two exported functions.
The same native snapshot, path-admission and cleanup requirements apply to EXEs
and DLLs.
## Configuration
```powershell
@@ -100,7 +107,7 @@ and artifact SHA-256 before loading the package-owned addon. Native failure
reasons remain distinct from a valid installation and an unsupported target.
PE header classification and SHA-256 come from the same open file. The Windows
provider admits native x86 and x86-64 PE applications and rejects unsupported roles,
provider admits native x86 and x86-64 PE applications and DLLs and rejects unsupported roles,
architectures, managed images, and malformed headers. The original selected
source coordinate is preserved for native admission.
@@ -159,7 +166,11 @@ artifact and manifest.
npm run verify:windows-native
npm run verify:ghidra:windows
npm run verify:ghidra:windows -- --x86
npm run verify:ghidra:windows -- --dll
npm run verify:ghidra:windows -- --x86 --dll
npm run verify:ghidra:windows:package -- --x86
npm run verify:ghidra:windows:package -- --dll
npm run verify:ghidra:windows:package -- --x86 --dll
npm run verify:ghidra:windows:package -- C:\fixtures\installed-rea C:\fixtures\sample.exe
```
+95 -60
View File
@@ -3,68 +3,103 @@ import { dirname, resolve } from "node:path";
import { fileURLToPath } from "node:url";
const root = resolve(dirname(fileURLToPath(import.meta.url)), "..");
// Source-owned static fixtures: no compiler, target loading or execution.
for (const architecture of ["x86_64", "x86"]) {
const x86 = architecture === "x86";
const output = resolve(
root,
"build",
"fixtures",
x86 ? "rea-ghidra-windows-x86.exe" : "rea-ghidra-windows.exe",
);
const bytes = Buffer.alloc(1024);
const peOffset = 0x80;
const optionalHeader = peOffset + 24;
const optionalHeaderSize = x86 ? 0xe0 : 0xf0;
const sectionTable = optionalHeader + optionalHeaderSize;
for (const dll of [false, true]) {
const x86 = architecture === "x86";
const output = resolve(
root,
"build",
"fixtures",
`rea-ghidra-windows${x86 ? "-x86" : ""}.${dll ? "dll" : "exe"}`,
);
const bytes = Buffer.alloc(1024);
const peOffset = 0x80;
const optionalHeader = peOffset + 24;
const optionalHeaderSize = x86 ? 0xe0 : 0xf0;
const sectionTable = optionalHeader + optionalHeaderSize;
const directories = optionalHeader + (x86 ? 96 : 112);
bytes.write("MZ", 0, "ascii");
bytes.writeUInt32LE(peOffset, 0x3c);
bytes.write("PE\0\0", peOffset, "binary");
bytes.writeUInt16LE(x86 ? 0x14c : 0x8664, peOffset + 4);
bytes.writeUInt16LE(1, peOffset + 6);
bytes.writeUInt16LE(optionalHeaderSize, peOffset + 20);
bytes.writeUInt16LE(x86 ? 0x102 : 0x22, peOffset + 22);
bytes.write("MZ", 0, "ascii");
bytes.writeUInt32LE(peOffset, 0x3c);
bytes.write("PE\0\0", peOffset, "binary");
bytes.writeUInt16LE(x86 ? 0x14c : 0x8664, peOffset + 4);
bytes.writeUInt16LE(1, peOffset + 6);
bytes.writeUInt16LE(optionalHeaderSize, peOffset + 20);
bytes.writeUInt16LE(
(x86 ? 0x102 : 0x22) | (dll ? 0x2000 : 0),
peOffset + 22,
);
bytes.writeUInt16LE(x86 ? 0x10b : 0x20b, optionalHeader);
bytes.writeUInt32LE(0x200, optionalHeader + 4);
bytes.writeUInt32LE(0x1000, optionalHeader + 16);
bytes.writeUInt32LE(0x1000, optionalHeader + 20);
if (x86) bytes.writeUInt32LE(0x400000, optionalHeader + 28);
else bytes.writeBigUInt64LE(0x1_4000_0000n, optionalHeader + 24);
bytes.writeUInt32LE(0x1000, optionalHeader + 32);
bytes.writeUInt32LE(0x200, optionalHeader + 36);
bytes.writeUInt16LE(6, optionalHeader + 40);
bytes.writeUInt16LE(6, optionalHeader + 48);
bytes.writeUInt32LE(0x2000, optionalHeader + 56);
bytes.writeUInt32LE(0x200, optionalHeader + 60);
bytes.writeUInt16LE(3, optionalHeader + 68);
bytes.writeUInt16LE(x86 ? 0x8140 : 0x8160, optionalHeader + 70);
if (x86) {
bytes.writeUInt32LE(0x10_0000, optionalHeader + 72);
bytes.writeUInt32LE(0x1000, optionalHeader + 76);
bytes.writeUInt32LE(0x10_0000, optionalHeader + 80);
bytes.writeUInt32LE(0x1000, optionalHeader + 84);
bytes.writeUInt32LE(16, optionalHeader + 92);
} else {
bytes.writeBigUInt64LE(0x10_0000n, optionalHeader + 72);
bytes.writeBigUInt64LE(0x1000n, optionalHeader + 80);
bytes.writeBigUInt64LE(0x10_0000n, optionalHeader + 88);
bytes.writeBigUInt64LE(0x1000n, optionalHeader + 96);
bytes.writeUInt32LE(16, optionalHeader + 108);
bytes.writeUInt16LE(x86 ? 0x10b : 0x20b, optionalHeader);
bytes.writeUInt32LE(0x200, optionalHeader + 4);
bytes.writeUInt32LE(dll ? 0 : 0x1000, optionalHeader + 16);
bytes.writeUInt32LE(0x1000, optionalHeader + 20);
if (x86) bytes.writeUInt32LE(0x400000, optionalHeader + 28);
else bytes.writeBigUInt64LE(0x1_4000_0000n, optionalHeader + 24);
bytes.writeUInt32LE(0x1000, optionalHeader + 32);
bytes.writeUInt32LE(0x200, optionalHeader + 36);
bytes.writeUInt16LE(6, optionalHeader + 40);
bytes.writeUInt16LE(6, optionalHeader + 48);
bytes.writeUInt32LE(0x2000, optionalHeader + 56);
bytes.writeUInt32LE(0x200, optionalHeader + 60);
bytes.writeUInt16LE(3, optionalHeader + 68);
bytes.writeUInt16LE(x86 ? 0x8140 : 0x8160, optionalHeader + 70);
if (x86) {
bytes.writeUInt32LE(0x10_0000, optionalHeader + 72);
bytes.writeUInt32LE(0x1000, optionalHeader + 76);
bytes.writeUInt32LE(0x10_0000, optionalHeader + 80);
bytes.writeUInt32LE(0x1000, optionalHeader + 84);
bytes.writeUInt32LE(16, optionalHeader + 92);
} else {
bytes.writeBigUInt64LE(0x10_0000n, optionalHeader + 72);
bytes.writeBigUInt64LE(0x1000n, optionalHeader + 80);
bytes.writeBigUInt64LE(0x10_0000n, optionalHeader + 88);
bytes.writeBigUInt64LE(0x1000n, optionalHeader + 96);
bytes.writeUInt32LE(16, optionalHeader + 108);
}
bytes.write(".text", sectionTable, "ascii");
bytes.writeUInt32LE(dll ? 0x180 : 0x16, sectionTable + 8);
bytes.writeUInt32LE(0x1000, sectionTable + 12);
bytes.writeUInt32LE(0x200, sectionTable + 16);
bytes.writeUInt32LE(0x200, sectionTable + 20);
bytes.writeUInt32LE(0x6000_0020, sectionTable + 36);
// EXE entry returns zero. DLL caller adds one to the callee's 42.
bytes.set(
dll
? [0xe8, 0x0b, 0, 0, 0, 0x83, 0xc0, 1, 0xc3]
: [0xe8, 0x0b, 0, 0, 0, 0x31, 0xc0, 0xc3],
0x200,
);
bytes.set([0xb8, 0x2a, 0, 0, 0, 0xc3], 0x210);
if (dll) {
// Export both functions; an entry-point-free library is discovered by exports.
bytes.writeUInt32LE(0x1040, directories);
bytes.writeUInt32LE(0x130, directories + 4);
const exports = 0x240;
bytes.writeUInt32LE(0x1100, exports + 12);
bytes.writeUInt32LE(1, exports + 16);
bytes.writeUInt32LE(2, exports + 20);
bytes.writeUInt32LE(2, exports + 24);
bytes.writeUInt32LE(0x1080, exports + 28);
bytes.writeUInt32LE(0x1088, exports + 32);
bytes.writeUInt32LE(0x1090, exports + 36);
bytes.writeUInt32LE(0x1000, 0x280);
bytes.writeUInt32LE(0x1010, 0x284);
bytes.writeUInt32LE(0x1120, 0x288);
bytes.writeUInt32LE(0x1140, 0x28c);
bytes.writeUInt16LE(0, 0x290);
bytes.writeUInt16LE(1, 0x292);
bytes.write("rea-fixture.dll\0", 0x300, "binary");
bytes.write("fixture_caller\0", 0x320, "binary");
bytes.write("fixture_return42\0", 0x340, "binary");
}
await mkdir(dirname(output), { recursive: true });
await writeFile(output, bytes);
if (!x86 && !dll) process.stdout.write(`${output}\n`);
}
bytes.write(".text", sectionTable, "ascii");
bytes.writeUInt32LE(0x16, sectionTable + 8);
bytes.writeUInt32LE(0x1000, sectionTable + 12);
bytes.writeUInt32LE(0x200, sectionTable + 16);
bytes.writeUInt32LE(0x200, sectionTable + 20);
bytes.writeUInt32LE(0x6000_0020, sectionTable + 36);
// entry calls the second function, returns zero; second returns 42.
bytes.set([0xe8, 0x0b, 0, 0, 0, 0x31, 0xc0, 0xc3], 0x200);
bytes.set([0xb8, 0x2a, 0, 0, 0, 0xc3], 0x210);
await mkdir(dirname(output), { recursive: true });
await writeFile(output, bytes);
if (!x86) process.stdout.write(`${output}\n`);
}
+1 -1
View File
@@ -17,7 +17,7 @@ const lock = JSON.parse(
await readFile(join(root, "package-lock.json"), "utf8"),
);
const skill = await readFile(
join(root, "skill-src/reverse-engineer-anything/SKILL.md"),
join(root, ".agents/skills/reverse-engineer-anything/SKILL.md"),
"utf8",
);
const skillVersion = /^\s{2}version:\s*"([^"]+)"\s*$/mu.exec(skill)?.[1];
+2 -2
View File
@@ -10,7 +10,7 @@ for (const argument of arguments_)
throw new Error(`Unknown skill metadata option: ${argument}`);
const root = join(dirname(fileURLToPath(import.meta.url)), "..");
const sourceRoot = join(root, "skill-src/reverse-engineer-anything");
const sourceRoot = join(root, ".agents/skills/reverse-engineer-anything");
const outputRoot = join(root, "skills/reverse-engineer-anything");
const check = arguments_.has("--check");
const current = (
@@ -25,7 +25,7 @@ const { PRODUCT_IDENTITY } = await import(
);
if (/^\s{2}(?:tool_count|catalog_digest):/mu.test(current))
throw new Error(
"Catalog metadata belongs in the generated skill, not skill-src",
"Catalog metadata belongs in the generated skill, not the authored source",
);
const versionLine = /^ {2}version: "[^"\r\n]+"$/mu;
if (!versionLine.test(current))
+1 -1
View File
@@ -1283,7 +1283,7 @@ async function inspect(
evidence.provider.version,
"pwntools@4.15.0;pyelftools@0.33;unicorn@2.1.2",
);
assert.deepEqual(evidence.normalized_result, evidence.raw_result);
assert.equal(evidence.raw_result, null);
assert.equal(evidence.normalized_result.diagnostics.truncated, false);
return evidence.normalized_result;
}
+1
View File
@@ -85,6 +85,7 @@ export async function verifyPackageEnvironment({
PI_CODING_AGENT_DIR: undefined,
PI_CONFIG_DIR: undefined,
PI_PROFILE: undefined,
QWEN_HOME: undefined,
OPENCODE_CONFIG: undefined,
XDG_CONFIG_HOME: join(home, ".config"),
PATH: `${fakeBin}:${process.env.PATH ?? ""}`,
+1 -1
View File
@@ -38,7 +38,7 @@ export async function verifyPackagePack({ root, workspace }) {
throw new Error("package did not expose both rea command entry points");
if (packedManifest.mcpName !== "io.github.morluto/rea")
throw new Error("package did not retain its MCP Registry ownership marker");
if (packedFiles.some((path) => path.startsWith("package/skill-src/")))
if (packedFiles.some((path) => path.startsWith("package/.agents/")))
throw new Error(
"package included authored skill sources instead of only the generated bundle",
);
+4
View File
@@ -17,6 +17,7 @@ import { verifyPackageMcp } from "./verify-package-mcp.mjs";
import { verifyPackagePack } from "./verify-package-pack.mjs";
import { verifyPackagePlatform } from "./verify-package-platform.mjs";
import { verifyPackageSetup } from "./verify-package-setup.mjs";
import { verifyRepositorySkill } from "./verify-repository-skill.mjs";
import { verifyManaged } from "./verify-package-managed.mjs";
import { verifyUnknownProvider } from "./verify-package-unknown-provider.mjs";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
@@ -34,9 +35,11 @@ let tarball;
let mcpStartup;
let mcpModuleLoading;
let update;
let repositorySkill;
let report;
try {
repositorySkill = await verifyRepositorySkill({ root, workspace });
({ tarball } = await verifyPackagePack({ root, workspace }));
const environmentData = await verifyPackageEnvironment({
root,
@@ -156,6 +159,7 @@ try {
configSymlinkLifecycle: supportedSetupHost,
skill: supportedSetupHost,
skillReferences: supportedSetupHost,
repositorySkill,
mcpTools: TOOL_CONTRACTS.length,
mcpPrompts: prompts.names.length,
promptCompletion: true,
+102 -21
View File
@@ -15,7 +15,7 @@ import {
import { tmpdir } from "node:os";
import { dirname, join, resolve } from "node:path";
import { promisify } from "node:util";
import { pathToFileURL } from "node:url";
import { fileURLToPath, pathToFileURL } from "node:url";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
@@ -24,10 +24,50 @@ if (process.platform !== "win32" || process.arch !== "x64")
throw new Error(
"Packaged Windows Ghidra acceptance requires a Windows x64 host.",
);
const workspace = await mkdtemp(join(tmpdir(), "rea-packaged-ghidra-"));
const exec = promisify(execFile);
const x86Fixture = process.argv[2] === "--x86";
const packageRootArgument = x86Fixture ? undefined : process.argv[2];
const arguments_ = process.argv.slice(2);
const internalWorkspace = arguments_[0] === "--internal-workspace";
const fixtureArguments = internalWorkspace ? arguments_.slice(2) : arguments_;
const x86Fixture = fixtureArguments.includes("--x86");
const dllFixture = fixtureArguments.includes("--dll");
const [packageRootArgument, targetArgument, reportArgument] =
fixtureArguments.filter(
(argument) => argument !== "--x86" && argument !== "--dll",
);
// Windows locks the loaded package-owned addon until the verifier exits.
// Keep installation cleanup in a parent that never loads that addon.
if (!internalWorkspace) {
const ownedWorkspace = await mkdtemp(join(tmpdir(), "rea-packaged-ghidra-"));
let result;
try {
result = await exec(
process.execPath,
[
fileURLToPath(import.meta.url),
"--internal-workspace",
ownedWorkspace,
...arguments_,
],
{ timeout: 900_000, maxBuffer: 8 * 1024 * 1024 },
);
} catch (cause) {
if (typeof cause.stdout === "string") process.stdout.write(cause.stdout);
if (typeof cause.stderr === "string") process.stderr.write(cause.stderr);
throw cause;
} finally {
await rm(ownedWorkspace, { recursive: true, force: true });
}
const report = JSON.parse(result.stdout);
report.workspaceCleanup = true;
report.ok = true;
if (reportArgument !== undefined)
await writeFile(reportArgument, `${JSON.stringify(report)}\n`);
process.stdout.write(`${JSON.stringify(report)}\n`);
process.stderr.write(result.stderr);
process.exit(0);
}
assert.ok(arguments_[1] !== undefined, "Missing verifier workspace.");
const workspace = resolve(arguments_[1]);
let packageRoot = resolve(packageRootArgument ?? ".");
// The default lane verifies the npm artifact in an isolated prefix. Explicit
// package roots support an already-installed artifact without a second install.
@@ -75,17 +115,17 @@ if (packageRootArgument === undefined) {
);
packageRoot = join(prefix, "node_modules", "rea-agents");
} catch (cause) {
await rm(workspace, { recursive: true, force: true });
throw cause;
throw new Error("Packaged Windows verifier installation failed.", {
cause,
});
}
}
const target = resolve(
process.argv[3] ??
(x86Fixture
? "build/fixtures/rea-ghidra-windows-x86.exe"
: "build/fixtures/rea-ghidra-windows.exe"),
targetArgument ??
`build/fixtures/rea-ghidra-windows${x86Fixture ? "-x86" : ""}.${dllFixture ? "dll" : "exe"}`,
);
let TOOL_CONTRACTS, native, token, sha256;
let parsedTarget;
try {
({ TOOL_CONTRACTS } = await import(
pathToFileURL(join(packageRoot, "dist/contracts/toolContracts.js"))
@@ -107,9 +147,25 @@ try {
);
const digest = (bytes) => createHash("sha256").update(bytes).digest("hex");
sha256 = digest(await readFile(target));
const { parseBinaryTarget } = await import(
pathToFileURL(join(packageRoot, "dist/application/BinaryTargetResolver.js"))
);
parsedTarget = await parseBinaryTarget(target);
assert.equal(parsedTarget.ok, true);
assert.equal(parsedTarget.value.format, "pe");
assert.equal(parsedTarget.value.managed, false);
if (targetArgument === undefined) {
assert.equal(
parsedTarget.value.architecture,
x86Fixture ? "x86" : "x86_64",
);
assert.equal(
parsedTarget.value.executableRole,
dllFixture ? "shared-library" : "application",
);
}
} catch (cause) {
await rm(workspace, { recursive: true, force: true });
throw cause;
throw new Error("Packaged Windows verifier prerequisites failed.", { cause });
}
const digest = (bytes) => createHash("sha256").update(bytes).digest("hex");
const runtimeParent = join(workspace, "runtime with spaces");
@@ -145,6 +201,8 @@ const report = {
architecture: "x64",
nonAdmin: true,
targetSha256: sha256,
targetArchitecture: parsedTarget.value.architecture,
targetRole: parsedTarget.value.executableRole,
cli: {},
mcpOperations: [],
nativeControls: "available",
@@ -208,8 +266,8 @@ try {
});
const call = async (name, arguments_) => {
report.lastOperation = name;
if (process.argv[4] !== undefined)
await writeFile(process.argv[4], `${JSON.stringify(report)}\n`);
if (reportArgument !== undefined)
await writeFile(reportArgument, `${JSON.stringify(report)}\n`);
const contract = TOOL_CONTRACTS.find((value) => value.name === name);
assert.ok(contract !== undefined, `Missing canonical contract: ${name}`);
const argumentsParsed = contract.inputSchema.parse(arguments_);
@@ -230,6 +288,25 @@ try {
await call("list_names", {});
const procedures = await call("list_procedures", {});
assert.ok(procedures.length > 0);
if (targetArgument === undefined) {
const calleeAddress = x86Fixture ? "0x401010" : "0x140001010";
const calleeCode = await call("procedure_pseudo_code", {
procedure: calleeAddress,
});
assert.match(calleeCode, /\b(?:42|0x2a)\b/iu);
if (dllFixture) {
const callerAddress = x86Fixture ? "0x401000" : "0x140001000";
assert.ok(
(
await call("procedure_callees", { procedure: callerAddress })
).includes(calleeAddress),
);
assert.match(
await call("procedure_pseudo_code", { procedure: callerAddress }),
/\+\s*1\b/u,
);
}
}
assert.ok((await call("list_segments", {})).length > 0);
await call("list_strings", {});
const procedure =
@@ -296,6 +373,12 @@ try {
"Packaged verification omitted an admitted operation.",
);
const session = await call("binary_session", {});
assert.equal(
session.analysis_provider_binding?.analysis_profile?.parameters
?.executable_role,
parsedTarget.value.executableRole,
"MCP profile must preserve the selected PE role.",
);
assert.equal(
session.analysis_provider_binding?.selection_source,
"auto-single-candidate",
@@ -326,9 +409,9 @@ try {
assert.deepEqual(await readdir(join(callerDirectory, name)), []);
report.callerScriptCollisionsPreserved = true;
report.runtimeCleanup = true;
report.ok = true;
if (process.argv[4] !== undefined)
await writeFile(process.argv[4], `${JSON.stringify(report)}\n`);
// Only the parent can attest success after removing the installation.
if (reportArgument !== undefined)
await writeFile(reportArgument, `${JSON.stringify(report)}\n`);
process.stdout.write(`${JSON.stringify(report)}\n`);
} catch (cause) {
report.failure = {
@@ -338,9 +421,7 @@ try {
.replaceAll(target, "<target>")
.replaceAll(workspace, "<workspace>"),
};
if (process.argv[4] !== undefined)
await writeFile(process.argv[4], `${JSON.stringify(report)}\n`);
if (reportArgument !== undefined)
await writeFile(reportArgument, `${JSON.stringify(report)}\n`);
throw cause;
} finally {
await rm(workspace, { recursive: true, force: true });
}
+21 -6
View File
@@ -62,21 +62,21 @@ if (
);
const root = resolve(fileURLToPath(new URL("..", import.meta.url)));
const architecture = process.argv[2] === "--x86" ? "x86" : "x86_64";
const architecture = process.argv.includes("--x86") ? "x86" : "x86_64";
const dllFixture = process.argv.includes("--dll");
const role = dllFixture ? "shared-library" : "application";
const targetPath = resolve(
root,
"build",
"fixtures",
architecture === "x86"
? "rea-ghidra-windows-x86.exe"
: "rea-ghidra-windows.exe",
`rea-ghidra-windows${architecture === "x86" ? "-x86" : ""}.${dllFixture ? "dll" : "exe"}`,
);
const target = await parseBinaryTarget(targetPath);
if (
!target.ok ||
target.value.format !== "pe" ||
target.value.architecture !== architecture ||
target.value.executableRole !== "application" ||
target.value.executableRole !== role ||
target.value.managed !== false
)
throw new Error(
@@ -89,6 +89,8 @@ const profile = await resolveGhidraAnalysisProfile(
);
if (!profile.ok || profile.value.profile === null)
throw new Error("Windows Ghidra profile could not be committed");
if (profile.value.profile.parameters.executable_role !== role)
throw new Error("Windows Ghidra profile lost the observed PE role");
const client = new GhidraClient({
platform: installation.platform,
@@ -162,6 +164,19 @@ try {
throw new Error(
"Windows fixture decompilation lost its known return value",
);
if (dllFixture) {
const callerAddress = architecture === "x86" ? "0x401000" : "0x140001000";
const callees = await functionOperation("procedure_callees", {
procedure: callerAddress,
});
if (!callees.includes(callee.address))
throw new Error("DLL export caller lost its controlled call reference");
const callerCode = await functionOperation("procedure_pseudo_code", {
procedure: callerAddress,
});
if (typeof callerCode !== "string" || !/\+\s*1\b/u.test(callerCode))
throw new Error("DLL export caller lost its addition of one");
}
await inventory("inspect_native_load_image", {});
const memory = await inventory("read_bytes", {
@@ -254,7 +269,7 @@ try {
limitations: [
"approved-non-sensitive-fixtures-only",
"windows-x64-local-ntfs-only",
"native-x86-and-x86-64-pe-applications-only",
"native-x86-and-x86-64-pe-applications-and-dlls-only",
"no-gui-or-mutation-authority",
],
};
+97
View File
@@ -0,0 +1,97 @@
import { cp, mkdir, readFile, readdir } from "node:fs/promises";
import { dirname, join, relative } from "node:path";
import { exec, pathExists } from "./lib/verify-package-core.mjs";
/** Exercise Skills' default root discovery without ignored build outputs. */
export async function verifyRepositorySkill({ root, workspace }) {
const snapshot = join(workspace, "repository");
const consumer = join(workspace, "skill-consumer");
const home = join(workspace, "skill-consumer-home");
const tracked = (
await exec(
"git",
["ls-files", "--cached", "--format=%(objectmode) %(path)", "-z"],
{ cwd: root },
)
).stdout
.split("\0")
.filter(Boolean);
for (const entry of tracked) {
const mode = entry.slice(0, 6);
const path = entry.slice(7);
const destination = join(snapshot, path);
// A GitHub clone without submodule initialization has only these directories.
if (mode === "160000") {
await mkdir(destination, { recursive: true });
continue;
}
await mkdir(dirname(destination), { recursive: true });
await cp(join(root, path), destination, { verbatimSymlinks: true });
}
if (await pathExists(join(snapshot, "skills")))
throw new Error("repository skill check must run without generated skills");
await mkdir(consumer, { recursive: true });
await mkdir(home, { recursive: true });
const skillsVersion = "1.7.2";
await exec(
"npm",
[
"exec",
"--yes",
"--package",
`skills@${skillsVersion}`,
"--",
"skills",
"add",
snapshot,
"--skill",
"reverse-engineer-anything",
"--agent",
"codex",
"--yes",
],
{
cwd: consumer,
env: {
...process.env,
HOME: home,
USERPROFILE: home,
XDG_CONFIG_HOME: join(home, ".config"),
CODEX_HOME: join(home, ".codex"),
DO_NOT_TRACK: "1",
SKILLS_NO_TELEMETRY: "1",
CI: "1",
},
timeout: 120_000,
},
);
const authoredRoot = join(
snapshot,
".agents/skills/reverse-engineer-anything",
);
const installedSkills = join(consumer, ".agents/skills");
const installedRoot = join(installedSkills, "reverse-engineer-anything");
const files = await filePaths(authoredRoot);
if (
JSON.stringify(await readdir(installedSkills)) !==
JSON.stringify(["reverse-engineer-anything"]) ||
JSON.stringify(await filePaths(installedRoot)) !== JSON.stringify(files)
)
throw new Error(
"Skills did not install exactly the selected public bundle",
);
for (const path of files) {
const authored = await readFile(join(authoredRoot, path));
if (!authored.equals(await readFile(join(installedRoot, path))))
throw new Error(`Skills changed authored skill content: ${path}`);
}
return { skillsVersion, files };
}
async function filePaths(root) {
return (await readdir(root, { recursive: true, withFileTypes: true }))
.filter((entry) => entry.isFile())
.map((entry) => relative(root, join(entry.parentPath, entry.name)))
.sort();
}
+3
View File
@@ -17,6 +17,7 @@ import {
import { verifyDyldEnvironment } from "./dyld-environment.mjs";
import { verifyDylibResolution } from "./macos-dylib-e2e.mjs";
import { verifyMachOOccurrences } from "./macho-occurrences.mjs";
import { verifyNativeProvenance } from "./native-provenance.mjs";
const exec = promisify(execFile);
@@ -69,6 +70,7 @@ const EXPECTED_BUNDLES = [
let report;
try {
const { app, executable } = await buildMacosBundleFixture(root);
const nativeProvenance = await verifyNativeProvenance(app, executable);
const machoOccurrences = await verifyMachOOccurrences(app, executable);
const zip = join(root, "MacFixture.zip");
await exec("/usr/bin/ditto", ["-c", "-k", "--keepParent", app, zip]);
@@ -166,6 +168,7 @@ try {
containers: containers.map(({ format }) => format),
bundles: EXPECTED_BUNDLES.length,
dmg_detached: true,
native_provenance: nativeProvenance,
macho_occurrences: machoOccurrences,
dylib_resolution: dylibs,
dyld_environment: dyldEnvironment,
@@ -0,0 +1,71 @@
import assert from "node:assert/strict";
import { createHash } from "node:crypto";
import { readFile } from "node:fs/promises";
import { toolContract } from "../../../dist/contracts/toolContracts.js";
import {
artifactCliEvidence,
withArtifactMcp,
} from "../../lib/artifact-e2e.mjs";
/** Inspect the compiled bundle with real native utilities through both adapters. */
export async function verifyNativeProvenance(app, executable) {
const original = await readFile(executable);
const sha256 = createHash("sha256").update(original).digest("hex");
const symbols = ["$s4main3FooV", "plain_symbol"];
const operations = [
["inspect_macho", []],
["inspect_signature", []],
["inspect_plist", []],
["list_architectures", []],
["demangle_swift", symbols],
];
const observations = [];
await withArtifactMcp(app, async (client) => {
const definitions = new Map(
(await client.listTools()).tools.map((tool) => [tool.name, tool]),
);
for (const [operation, arguments_] of operations) {
const cli = toolContract(operation).outputSchema.parse(
await artifactCliEvidence(
operation.replaceAll("_", "-"),
app,
arguments_,
),
);
const response = await client.callTool(
{
name: operation,
arguments: operation === "demangle_swift" ? { symbols } : {},
},
{ toolDefinition: definitions.get(operation) },
);
assert.notEqual(response.isError, true, JSON.stringify(response));
const mcp = toolContract(operation).outputSchema.parse(
response.structuredContent,
);
assert.equal(cli.raw_result, null);
assert.equal(mcp.raw_result, null);
assert.ok(cli.normalized_result.provenance.length > 0);
assert.deepEqual(mcp.normalized_result, cli.normalized_result);
assert.deepEqual(
JSON.parse(response.content.find((item) => item.type === "text").text),
response.structuredContent,
);
await client.ping();
observations.push({
operation,
commands: cli.normalized_result.provenance.length,
cli_bytes: Buffer.byteLength(JSON.stringify(cli)),
mcp_bytes: Buffer.byteLength(JSON.stringify(response)),
});
}
});
assert.deepEqual(await readFile(executable), original);
return {
real_native_utilities: true,
target_sha256: sha256,
bytes_unchanged: true,
observations,
};
}
+3 -3
View File
@@ -54,15 +54,15 @@ try {
data: { ...evidence, normalized_result: { payload: [marker] } },
meta: metadata,
};
const skeleton = JSON.stringify(template, null, format === "json" ? 2 : 0);
// The destination is a redirected file, so both formats are compact.
const skeleton = JSON.stringify(template);
const encodedMarker = JSON.stringify(marker);
const markerIndex = skeleton.indexOf(encodedMarker);
assert.ok(markerIndex >= 0);
assert.equal(markerIndex, skeleton.lastIndexOf(encodedMarker));
const prefix = skeleton.slice(0, markerIndex);
const suffix = `${skeleton.slice(markerIndex + encodedMarker.length)}\n`;
const indentation = prefix.slice(prefix.lastIndexOf("\n") + 1);
const separator = format === "json" ? `,\n${indentation}` : ",";
const separator = ",";
const expectedHash = createHash("sha256");
expectedHash.update(prefix);
for (let index = 0; index < count; index += 1) {
+1 -1
View File
@@ -80,7 +80,7 @@ export async function connectRecordedCrash({ entrypoint, environment }) {
evidence.confidence,
debuggerContext ? "derived" : "observed",
);
assert.deepEqual(evidence.raw_result, report);
assert.equal(evidence.raw_result, null);
assert.equal(evidence.subject.local_path, path);
assert.equal(report.artifact.path, path);
assert.equal(
+22
View File
@@ -0,0 +1,22 @@
import { expect, it } from "vitest";
import { createAnalysisExecution } from "./AnalysisProvider.js";
const PROVIDER = { id: "fixture", name: "Fixture", version: "1" };
it("records no raw result unless the provider supplies a distinct one", () => {
const result = { entries: [{ value: "normalized" }] };
expect(createAnalysisExecution(result, PROVIDER)).toMatchObject({
result,
rawResult: null,
});
expect(
createAnalysisExecution(result, PROVIDER, {
rawResult: { stdout: "raw" },
}).rawResult,
).toEqual({ stdout: "raw" });
expect(
createAnalysisExecution(result, PROVIDER, { rawResult: null }).rawResult,
).toBeNull();
});
+8 -2
View File
@@ -41,7 +41,13 @@ export interface AnalysisExecution {
readonly subject: EvidenceSubjectTarget | null;
}
/** Build a validated atomic provider observation at an adapter boundary. */
/**
* Build a validated atomic provider observation at an adapter boundary.
*
* Pass `rawResult` only for a provider representation distinct from
* `result`. Omitting it records `null` rather than a second complete copy of
* the normalized result in every Evidence record and tool response.
*/
export const createAnalysisExecution = (
result: unknown,
provider: ProviderIdentity,
@@ -56,7 +62,7 @@ export const createAnalysisExecution = (
result: jsonValueSchema.parse(result),
rawResult:
options.rawResult === undefined
? jsonValueSchema.parse(result)
? null
: jsonValueSchema.parse(options.rawResult),
provider,
...(options.analysisProfile === undefined
@@ -91,6 +91,7 @@ export const readClientRegistrationStatuses = async (
PI_CODING_AGENT_DIR: options.environment.PI_CODING_AGENT_DIR,
PI_CONFIG_DIR: options.environment.PI_CONFIG_DIR,
PI_PROFILE: options.environment.PI_PROFILE,
QWEN_HOME: options.environment.QWEN_HOME,
SAND_DATA_ROOT: options.environment.SAND_DATA_ROOT,
XDG_CONFIG_HOME: options.environment.XDG_CONFIG_HOME,
},
+2 -2
View File
@@ -2,8 +2,6 @@ import fs from "node:fs";
import { lstat, readFile } from "node:fs/promises";
import { join, resolve } from "node:path";
import { resolveRef } from "isomorphic-git";
export type ReferenceSourceVcsInfo =
| {
readonly kind: "git";
@@ -42,6 +40,8 @@ export const readReferenceSourceVcs = async (
};
const resolveSourceHead = async (root: string): Promise<string> => {
// isomorphic-git is loaded on first use so CLI and MCP startup skip it.
const { resolveRef } = await import("isomorphic-git");
const dotgit = join(root, ".git");
if (!(await lstat(dotgit)).isFile())
return resolveRef({ fs, dir: root, ref: "HEAD" });
+30 -1
View File
@@ -1,4 +1,4 @@
import { isAbsolute, join } from "node:path";
import { isAbsolute, join, resolve } from "node:path";
import { lstatSync } from "node:fs";
/** One supported client configuration location. */
@@ -35,6 +35,7 @@ interface ClientPathContext {
readonly PI_CODING_AGENT_DIR?: string | undefined;
readonly PI_CONFIG_DIR?: string | undefined;
readonly PI_PROFILE?: string | undefined;
readonly QWEN_HOME?: string | undefined;
readonly SAND_DATA_ROOT?: string | undefined;
readonly XDG_CONFIG_HOME?: string | undefined;
};
@@ -85,6 +86,25 @@ const claudeCodeMarkerDirectory = ({ home, env }: ClientPathContext): string =>
const codexDirectory = ({ home, env }: ClientPathContext): string =>
env.CODEX_HOME ?? join(home, ".codex");
/** Match Qwen Code's home override, including tilde and cwd-relative paths. */
const qwenCodeDirectory = ({ home, env }: ClientPathContext): string => {
const override = env.QWEN_HOME;
if (!override) return join(home, ".qwen");
const expanded =
override === "~"
? home
: override.startsWith("~/") || override.startsWith("~\\")
? join(
home,
...override
.slice(2)
.split(/[/\\]+/)
.filter(Boolean),
)
: override;
return isAbsolute(expanded) ? expanded : resolve(expanded);
};
const grokDirectory = ({ home, env }: ClientPathContext): string =>
env.GROK_HOME ?? join(home, ".grok");
@@ -272,6 +292,14 @@ export const SUPPORTED_CLIENT_DEFINITIONS = [
markerPath: commandCodeDirectory,
format: "commandcode",
},
{
name: "qwen_code",
displayName: "Qwen Code",
configPath: (context: ClientPathContext) =>
join(qwenCodeDirectory(context), "settings.json"),
markerPath: qwenCodeDirectory,
format: "json",
},
{
name: "vscode",
displayName: "VS Code",
@@ -323,6 +351,7 @@ export const supportedClients = (
PI_CODING_AGENT_DIR: process.env.PI_CODING_AGENT_DIR,
PI_CONFIG_DIR: process.env.PI_CONFIG_DIR,
PI_PROFILE: process.env.PI_PROFILE,
QWEN_HOME: process.env.QWEN_HOME,
SAND_DATA_ROOT: process.env.SAND_DATA_ROOT,
XDG_CONFIG_HOME: process.env.XDG_CONFIG_HOME,
},
@@ -2,6 +2,8 @@ import { expect, it } from "vitest";
import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "../../contracts/javascript/javascriptRuntimeReconciliationExample.js";
import { createEvidence, parseEvidence } from "../../domain/evidence.js";
import { functionDossierSchema } from "../../domain/hopperValues.js";
import { ghidraFunctionDossier } from "../../domain/ghidraValues.fixture.js";
import {
analysisViewJavaScriptAnalysisWithSource,
analysisViewJavaScriptEvidence,
@@ -9,6 +11,96 @@ import {
} from "../../../tests/fixtures/analysisView.js";
import { inspectAnalysisView } from "./AnalysisViewService.js";
it.each(["/fixtures/unknown.exe", ""])(
"preserves recorded empty native identity fields: path=%j",
(path) => {
const original = functionDossierSchema.parse(ghidraFunctionDossier());
const parent = createEvidence(
{ path, format: "pe", sha256: "c".repeat(64) },
{ id: "ghidra", name: "Ghidra", version: "12.1.4" },
{
operation: "analyze_function",
parameters: {},
result: {
...original,
procedure: {
...original.procedure,
address: "",
body: { available: false, reason: "Not recorded" },
},
},
},
);
const result = inspectAnalysisView({
source: { kind: "inline", evidence: parent },
view: {
kind: "native",
facet: "procedure",
offset: 0,
limit: 64,
},
});
if (!result.ok) throw result.error;
expect(result.value.normalized_result).toMatchObject({
procedure_address: "",
artifact: { path },
item: { address: "" },
});
expect(result.value.locations).toEqual(
path.length > 0 ? [{ kind: "artifact-path", path }] : [],
);
},
);
it("projects a small native view from an authenticated >10 MiB retained dossier", () => {
const original = functionDossierSchema.parse(ghidraFunctionDossier());
const parent = createEvidence(
{ path: "/fixtures/large.exe", format: "pe", sha256: "c".repeat(64) },
{ id: "ghidra", name: "Ghidra", version: "12.1.4" },
{
operation: "analyze_function",
parameters: { address: "0x401000" },
result: { ...original, pseudocode: "X".repeat(11 * 1024 * 1024) },
limitations: ["Synthetic fixture"],
},
);
const input = {
source: {
kind: "retained-evidence" as const,
evidence_id: parent.evidence_id,
},
view: {
kind: "native" as const,
facet: "pseudocode" as const,
offset: 0,
limit: 128,
},
};
const view = inspectAnalysisView(input, (id) =>
id === parent.evidence_id ? parent : undefined,
);
if (!view.ok) throw view.error;
expect(view.value.evidence_links).toEqual([parent.evidence_id]);
expect(view.value.normalized_result).toMatchObject({
kind: "native",
parent_operation: "analyze_function",
parent_evidence_id: parent.evidence_id,
procedure_address: "0x401000",
artifact: { path: "/fixtures/large.exe", sha256: "c".repeat(64) },
coverage: { total: 11 * 1024 * 1024, examined: 128, next_offset: 128 },
});
expect(view.value.locations).toEqual([
{ kind: "artifact-path", path: "/fixtures/large.exe" },
{ kind: "address", address: "0x401000" },
]);
expect(JSON.stringify(view.value).length).toBeLessThan(12000);
expect(JSON.stringify(parent).length).toBeGreaterThan(10 * 1024 * 1024);
expect(inspectAnalysisView(input, () => undefined)).toMatchObject({
ok: false,
error: { _tag: "EvidenceIntegrityError" },
});
});
it("projects inline layout Evidence and retains a derived view record", () => {
const parent = analysisViewLayoutEvidence();
const result = inspectAnalysisView({
@@ -53,6 +53,13 @@ export const inspectAnalysisViewValidated = (
operation: parent.operation,
normalizedResult: parent.normalized_result,
limitations: parent.limitations,
artifact:
parent.subject === null
? null
: {
path: parent.subject.local_path,
sha256: parent.subject.digest.sha256,
},
},
input.view,
);
@@ -126,7 +133,16 @@ const createAnalysisViewEvidence = (
confidence: "derived",
authority: parent.authority,
limitations: result.limitations,
locations: [{ kind: "artifact-path", path: result.artifact.path }],
locations: [
...(result.artifact.path.length > 0
? [{ kind: "artifact-path" as const, path: result.artifact.path }]
: []),
...(result.kind === "native" &&
result.procedure_address !== null &&
result.procedure_address.length > 0
? [{ kind: "address" as const, address: result.procedure_address }]
: []),
],
evidenceLinks: [parent.evidence_id],
});
};
@@ -5,6 +5,7 @@ import {
BINARY_LAYOUT_TEST_PROVIDER,
} from "../../../tests/fixtures/binaryDiagnostics/layout.js";
import { AnalysisCapabilityUnavailableError } from "../../domain/analysisErrorCore.js";
import { parseEvidence } from "../../domain/evidence.js";
import { err, ok } from "../../domain/result.js";
it.each([
@@ -28,6 +29,19 @@ it.each([
},
);
it("records the decoded layout once in its Evidence", async () => {
const layout = binaryLayoutFixture();
const service = new BinaryLayoutService({
identity: BINARY_LAYOUT_TEST_PROVIDER,
inspect: () => Promise.resolve(ok(layout)),
});
const result = await service.inspect({ path: layout.artifact.path });
if (!result.ok) throw result.error;
const evidence = parseEvidence(result.value);
expect(evidence.normalized_result).toEqual(layout);
expect(evidence.raw_result).toBeNull();
});
it("preserves a provider's actionable unsupported reason", async () => {
const failure = new AnalysisCapabilityUnavailableError(
"fixture-layout",
@@ -78,7 +78,8 @@ export class BinaryLayoutService {
operation: OPERATION,
parameters: input.data,
result: value,
rawResult: value,
// The decoded report is the only representation; do not repeat it.
rawResult: null,
confidence: "observed",
limitations: value.limitations,
locations: [{ kind: "artifact-path", path: value.artifact.path }],
@@ -24,6 +24,7 @@ it("labels an envelope containing debugger-derived maps as derived", async () =>
const evidence = parseEvidence(result.value);
expect(evidence.confidence).toBe("derived");
expect(evidence.normalized_result).toEqual(fixture);
expect(evidence.raw_result).toBeNull();
});
it.each([
@@ -83,7 +83,8 @@ export class RecordedCrashService {
operation: OPERATION,
parameters: input.data,
result: value,
rawResult: value,
// The decoded report is the only representation; do not repeat it.
rawResult: null,
confidence:
value.debugger.status === "available" ? "derived" : "observed",
limitations: value.limitations,
@@ -1,5 +1,6 @@
import { createHash } from "node:crypto";
import { compactStringIdentityKey } from "../../domain/compactStringIdentity.js";
import type { ApplicationNode } from "../../domain/javascript/javascriptApplicationGraphSchemas.js";
import type { JavaScriptStaticAnalysis } from "../../domain/javascript/javascriptStaticAnalysisTypes.js";
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
@@ -162,16 +163,15 @@ const addEndpoint = (
const source =
sourceNodeFor(context, input.file.path, input.value.module_key) ??
input.asset;
const key = compactStringIdentityKey(input.value.value);
const label = `${input.value.kind} endpoint`;
const node = context.accumulator.addNode({
kind: "endpoint",
identity: artifactLocalIdentity(
input.file.sha256,
input.value.kind,
input.value.value,
),
identity: artifactLocalIdentity(input.file.sha256, input.value.kind, key),
observations: [
{
label: input.value.value,
label:
input.value.value.length <= label.length ? input.value.value : label,
properties: {
endpoint_kind: input.value.kind,
value: input.value.value,
@@ -1,3 +1,4 @@
import { compactStringIdentityKey } from "../../domain/compactStringIdentity.js";
import { createJavaScriptSemanticGraphUnknown } from "../../domain/javascript/javascriptSemanticGraph.js";
import type { JavaScriptSemanticGraphNode } from "../../domain/javascript/javascriptSemanticGraphSchemas.js";
import type {
@@ -21,7 +22,7 @@ export const projectSemanticEvents = (
const eventKey =
operation.eventName === null
? operation.eventId
: `${operation.emitterKey}\u0000${operation.eventName}`;
: `${operation.emitterKey}\u0000${compactStringIdentityKey(operation.eventName)}`;
const event =
eventNodes.get(eventKey) ?? addEventNode(context, operation, eventKey);
if (event === null) continue;
@@ -59,7 +60,11 @@ const addEventNode = (
kind: "event",
roleKey: `event:${eventKey}`,
location: operation.eventName === null ? operation.location : null,
label: operation.eventName,
label:
operation.eventName === null ||
operation.eventName.length <= "event".length
? operation.eventName
: "event",
functionNodeId: null,
properties: {
emitter_key: operation.emitterKey,
@@ -78,7 +83,9 @@ const addListenerNode = (
label:
operation.eventName === null
? operation.method
: `${operation.method}:${operation.eventName}`,
: operation.eventName.length <= "listener".length
? `${operation.method}:${operation.eventName}`
: `${operation.method}:listener`,
functionNodeId:
operation.ownerCallableId === null
? null
@@ -175,7 +175,21 @@ export const createJavaScriptSemanticGraphProjection =
): Value => {
bindSemanticGraphApplicationNodes(state, applicationGraph);
if (state.roots.size === 0) addFallbackRoot(rootArtifactSha256, state);
const relations = [...state.relations.values()];
const unknowns = [...state.unknowns.values()];
type Family = (typeof JAVASCRIPT_SEMANTIC_RELATION_FAMILIES)[number];
const retainedByFamily = new Map<Family, number>();
const unknownIdsByFamily = new Map<Family, string[]>();
for (const relation of relations) {
const family = JAVASCRIPT_SEMANTIC_RELATION_FAMILY[relation.relation];
retainedByFamily.set(family, (retainedByFamily.get(family) ?? 0) + 1);
}
for (const unknown of unknowns) {
const identifiers = unknownIdsByFamily.get(unknown.family);
if (identifiers === undefined)
unknownIdsByFamily.set(unknown.family, [unknown.unknown_id]);
else identifiers.push(unknown.unknown_id);
}
const graph = factory({
schema: "JavaScriptSemanticRelationGraph",
root_artifact_sha256: rootArtifactSha256,
@@ -183,7 +197,7 @@ export const createJavaScriptSemanticGraphProjection =
root_node_ids: [...state.roots],
evidence_contexts: state.evidenceContexts.contexts,
nodes: [...state.nodes.values()],
relations: [...state.relations.values()],
relations,
fingerprints,
unknowns,
coverage: {
@@ -204,15 +218,9 @@ export const createJavaScriptSemanticGraphProjection =
families: JAVASCRIPT_SEMANTIC_RELATION_FAMILIES.map((family) => ({
family,
status: semanticFamilyStatus(family),
retained_relations: [...state.relations.values()].filter(
(relation) =>
JAVASCRIPT_SEMANTIC_RELATION_FAMILY[relation.relation] ===
family,
).length,
retained_relations: retainedByFamily.get(family) ?? 0,
omitted_relations: truncatedFiles > 0 ? null : 0,
unknown_ids: unknowns
.filter((unknown) => unknown.family === family)
.map(({ unknown_id: identifier }) => identifier),
unknown_ids: unknownIdsByFamily.get(family) ?? [],
})),
},
limitations: [
@@ -137,7 +137,11 @@ const addRequestNode = (
kind: operation.kind === "request" ? "request" : "response",
roleKey: operation.requestId,
location: operation.location,
label: operation.endpoint ?? operation.method,
label:
operation.endpoint !== null &&
operation.endpoint.length <= operation.method.length
? operation.endpoint
: operation.method,
functionNodeId:
operation.ownerCallableId === null
? null
@@ -2,6 +2,7 @@ import {
semanticContainer,
semanticPropertyPointer,
} from "../../domain/javascript/javascriptSemanticSlots.js";
import { digestCanonicalValue } from "../../domain/canonicalDigest.js";
import type { JavaScriptSemanticBinding } from "../../domain/javascript/javascriptSemanticIr.js";
import type {
JavaScriptSemanticProperty,
@@ -153,16 +154,27 @@ const addLiteralNode = (
binding: JavaScriptSemanticBinding,
value: string | number | boolean | null,
role: string,
) =>
retainSemanticGraphNode(context.state, context.file, {
) => {
const encodedValue = JSON.stringify(value);
const valueDigest = `value-sha256:${digestCanonicalValue(value, "Semantic literal")}`;
const valueKey =
encodedValue.length <= valueDigest.length ? encodedValue : valueDigest;
const stringLabel = "string literal";
return retainSemanticGraphNode(context.state, context.file, {
kind: "literal",
roleKey: `literal:${binding.bindingId}:${role}:${JSON.stringify(value)}`,
// Identity needs an exact value commitment, not another payload copy.
// Keep the complete literal in properties for queries and Evidence export.
roleKey: `literal:${binding.bindingId}:${role}:${valueKey}`,
location: binding.definitions[0]?.location ?? null,
label: JSON.stringify(value),
label:
typeof value === "string" && encodedValue.length > stringLabel.length
? stringLabel
: encodedValue,
functionNodeId:
context.bindingNodes.get(binding.bindingId)?.function_node_id ?? null,
properties: { value },
});
};
/** Create one canonical slot for a root binding and exact static property path. */
export const semanticPropertySlot = (
+157 -11
View File
@@ -1,9 +1,24 @@
import { posix } from "node:path";
import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js";
import { ArtifactReaderFailure } from "./ArtifactReader.js";
const DRIVE_OR_UNC = /^(?:[A-Za-z]:|\\|\/\/)/u;
/**
* English Unicode case folding used only to describe logical names a
* case-insensitive destination might collapse. It is not a filesystem case table.
*/
const ENGLISH_UNICODE_CASE_NOTE =
"This comparison is not a filesystem observation.";
/** Inventory limitation when logical names collide under English Unicode case folding. */
export const ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION = `Logical archive names collide under English (en-US) Unicode case folding; a case-insensitive destination cannot store both spellings. ${ENGLISH_UNICODE_CASE_NOTE}`;
/** User-facing prefix for a destination that cannot keep two logical spellings. */
export const DESTINATION_CASE_COLLISION_PREFIX =
"Destination filesystem cannot store both ";
/** Normalize an untrusted logical archive path without touching filesystem. */
export const normalizeArtifactPath = (input: string): string => {
if (
@@ -30,7 +45,7 @@ export const normalizeArtifactPath = (input: string): string => {
return normalized;
};
/** Reject exact, Unicode-normalized, case-folded, and prefix collisions. */
/** Reject exact and same-spelling prefix collisions. Case-distinct names stay distinct. */
export class ArtifactPathRegistry {
readonly #root: PathTrieNode = { kind: undefined, children: new Map() };
@@ -43,16 +58,10 @@ export class ArtifactPathRegistry {
"path",
`Artifact prefix conflict: ${path}`,
);
const foldedPart = part.toLocaleLowerCase("en-US");
let child = node.children.get(foldedPart);
if (child !== undefined && child.spelling !== part)
throw new ArtifactReaderFailure(
"path",
`Artifact path collision: ${path} differs only in case from ${[...parts.slice(0, index), child.spelling].join("/")}`,
);
let child = node.children.get(part);
if (child === undefined) {
child = { spelling: part, kind: undefined, children: new Map() };
node.children.set(foldedPart, child);
child = { kind: undefined, children: new Map() };
node.children.set(part, child);
}
node = child;
if (index === parts.length - 1) {
@@ -73,7 +82,144 @@ export class ArtifactPathRegistry {
}
interface PathTrieNode {
readonly spelling?: string;
kind: "file" | "directory" | "symlink" | "slice" | undefined;
readonly children: Map<string, PathTrieNode>;
}
interface CaseNode {
readonly bySpelling: Map<string, CaseNode>;
readonly spellingsByFold: Map<string, string[]>;
readonly terminals: string[];
}
const emptyCaseNode = (): CaseNode => ({
bySpelling: new Map(),
spellingsByFold: new Map(),
terminals: [],
});
const englishCaseFold = (part: string): string =>
part.toLocaleLowerCase("en-US");
/**
* Record English (en-US) Unicode case collisions on each affected occurrence.
* Returns the inventory limitation when any logical names collide.
*/
export const annotateEnglishUnicodeCaseCollisions = (
occurrences: readonly {
readonly logical_path: string;
limitations: string[];
}[],
): string | undefined => {
const collisions = englishUnicodeCaseCollisions(
occurrences.map(({ logical_path }) => logical_path),
);
if (collisions.size === 0) return undefined;
for (const occurrence of occurrences) {
const others = collisions.get(occurrence.logical_path);
if (others === undefined || others.length === 0) continue;
occurrence.limitations.push(
`Logical path ${occurrence.logical_path} collides under English (en-US) Unicode case folding with ${others.join(", ")}; a case-insensitive destination cannot store both. ${ENGLISH_UNICODE_CASE_NOTE}`,
);
}
return ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION;
};
/**
* Name the directory-listing spelling that differs from the requested segment
* only by English (en-US) Unicode case. Exact matches are the same logical name.
*/
export const destinationCaseCollisionMessage = (
requestedPath: string,
existingNames: readonly string[],
): string | undefined => {
const slash = requestedPath.lastIndexOf("/");
const segment = slash < 0 ? requestedPath : requestedPath.slice(slash + 1);
if (existingNames.includes(segment)) return undefined;
const folded = englishCaseFold(segment);
const existing = existingNames
.filter((name) => englishCaseFold(name) === folded)
.sort(compareUnicodeCodePoints)[0];
if (existing === undefined) return undefined;
const parent = slash < 0 ? "" : requestedPath.slice(0, slash + 1);
return `${DESTINATION_CASE_COLLISION_PREFIX}${requestedPath} and ${parent}${existing}; inventory retains both logical names.`;
};
const englishUnicodeCaseCollisions = (
paths: readonly string[],
): ReadonlyMap<string, readonly string[]> => {
const root = emptyCaseNode();
for (const path of paths) {
if (path.length === 0 || path === ".") continue;
let node = root;
for (const part of path.split("/")) {
const folded = englishCaseFold(part);
const spellings = node.spellingsByFold.get(folded);
if (spellings === undefined) node.spellingsByFold.set(folded, [part]);
else if (!spellings.includes(part)) spellings.push(part);
let child = node.bySpelling.get(part);
if (child === undefined) {
child = emptyCaseNode();
node.bySpelling.set(part, child);
}
node = child;
}
node.terminals.push(path);
}
const collisions = new Map<string, Set<string>>();
recordCaseCollisions(root, collisions);
return new Map(
[...collisions.entries()].map(([path, others]) => [
path,
[...others].sort(compareUnicodeCodePoints),
]),
);
};
const recordCaseCollisions = (
node: CaseNode,
collisions: Map<string, Set<string>>,
): void => {
for (const spellings of node.spellingsByFold.values())
if (spellings.length > 1) noteFoldedSpellings(node, spellings, collisions);
for (const child of node.bySpelling.values())
recordCaseCollisions(child, collisions);
};
const noteFoldedSpellings = (
node: CaseNode,
spellings: readonly string[],
collisions: Map<string, Set<string>>,
): void => {
const groups = spellings.map((spelling) => {
const child = node.bySpelling.get(spelling);
return child === undefined ? [] : terminalsUnder(child);
});
for (const [index, group] of groups.entries())
noteCaseGroup(
group,
groups.flatMap((candidate, candidateIndex) =>
candidateIndex === index ? [] : candidate,
),
collisions,
);
};
const noteCaseGroup = (
group: readonly string[],
others: readonly string[],
collisions: Map<string, Set<string>>,
): void => {
for (const path of group) {
const set = collisions.get(path) ?? new Set<string>();
for (const other of others) if (other !== path) set.add(other);
collisions.set(path, set);
}
};
const terminalsUnder = (node: CaseNode): readonly string[] => {
const paths = [...node.terminals];
for (const child of node.bySpelling.values())
paths.push(...terminalsUnder(child));
return paths;
};
+33 -2
View File
@@ -4,6 +4,7 @@ import {
lstat,
mkdir,
open,
readdir,
realpath,
rm,
type FileHandle,
@@ -14,6 +15,7 @@ import { streamChunkToBuffer } from "./StreamBytes.js";
import {
ArtifactPathRegistry,
destinationCaseCollisionMessage,
normalizeArtifactPath,
} from "./ArtifactPaths.js";
import { ArtifactReaderFailure } from "./ArtifactReader.js";
@@ -134,7 +136,8 @@ export class SafeOutputTree {
constants.O_WRONLY |
constants.O_NOFOLLOW,
0o600,
).catch((cause: unknown) => {
).catch(async (cause: unknown) => {
await throwIfDestinationCaseCollision(dirname(destination), path, cause);
throw new ArtifactReaderFailure(
"path",
`Could not exclusively create extraction path: ${path}`,
@@ -245,17 +248,28 @@ export class SafeOutputTree {
if (fileName === undefined)
throw new ArtifactReaderFailure("path", "Invalid extraction path");
let current = this.#outputRoot;
let logicalParent = "";
for (const part of parts) {
const parentDirectory = current;
const logicalPath =
logicalParent.length === 0 ? part : `${logicalParent}/${part}`;
current = join(current, part);
let created = true;
await mkdir(current, { mode: 0o700 }).catch((cause: unknown) => {
if (!isAlreadyExists(cause)) throw cause;
created = false;
});
if (!created)
await throwIfDestinationCaseCollision(parentDirectory, logicalPath);
const metadata = await lstat(current);
if (!metadata.isDirectory() || metadata.isSymbolicLink())
if (!metadata.isDirectory() || metadata.isSymbolicLink()) {
await throwIfDestinationCaseCollision(parentDirectory, logicalPath);
throw new ArtifactReaderFailure(
"path",
`Unsafe extraction parent: ${relativePath}`,
);
}
logicalParent = logicalPath;
}
return join(current, fileName);
}
@@ -330,6 +344,23 @@ const isNotFound = (cause: unknown): boolean =>
const isAlreadyExists = (cause: unknown): boolean =>
cause instanceof Error && "code" in cause && cause.code === "EEXIST";
/** Fail when this directory already holds another spelling of the requested segment. */
const throwIfDestinationCaseCollision = async (
parentDirectory: string,
logicalPath: string,
cause?: unknown,
): Promise<void> => {
const names = await readdir(parentDirectory).catch(() => undefined);
if (names === undefined) return;
const message = destinationCaseCollisionMessage(logicalPath, names);
if (message === undefined) return;
throw new ArtifactReaderFailure(
"path",
message,
cause === undefined ? undefined : { cause },
);
};
const abortIfNeeded = (signal?: AbortSignal): void => {
if (signal?.aborted === true)
throw new ArtifactReaderFailure(
+16 -7
View File
@@ -73,12 +73,12 @@ export const scanCanonicalArtifactInventory = async (
try {
if (reader instanceof AsarArtifactReader)
await reader.prepareContainer(rootDigest?.sha256, options.signal);
const { nodes, occurrences, pendingContradictions } = await scanReader(
reader,
ownedReaders,
options.signal,
integrity,
);
const {
nodes,
occurrences,
pendingContradictions,
caseCollisionLimitation,
} = await scanReader(reader, ownedReaders, options.signal, integrity);
outcome = {
kind: "completed",
snapshot: await buildInventorySnapshot({
@@ -91,6 +91,7 @@ export const scanCanonicalArtifactInventory = async (
nodes,
occurrences,
pendingContradictions,
caseCollisionLimitation,
}),
};
} catch (cause: unknown) {
@@ -130,6 +131,7 @@ interface SnapshotBuildInput {
readonly nodes: Map<string, ArtifactNode>;
readonly occurrences: MutableOccurrence[];
readonly pendingContradictions: PendingIntegrityContradiction[];
readonly caseCollisionLimitation: string | undefined;
}
const buildInventorySnapshot = async (
@@ -202,7 +204,12 @@ const buildInventorySnapshot = async (
edges: orderedEdges,
provenance: reader?.provenance() ?? [],
integrity_contradictions: integrityContradictions,
limitations: buildLimitations(rootFormat, reader, integrityContradictions),
limitations: buildLimitations(
rootFormat,
reader,
integrityContradictions,
input.caseCollisionLimitation,
),
};
};
@@ -310,8 +317,10 @@ const buildLimitations = (
rootFormat: ArtifactOccurrence["artifact_format"],
reader: ArtifactReader | undefined,
integrityContradictions: readonly IntegrityContradiction[],
caseCollisionLimitation: string | undefined,
): string[] => [
...inventoryLimitations(rootFormat, reader),
...(caseCollisionLimitation === undefined ? [] : [caseCollisionLimitation]),
...(integrityContradictions.length === 0
? []
: [
+14 -2
View File
@@ -1,6 +1,7 @@
import { isAbsolute } from "node:path";
import {
annotateEnglishUnicodeCaseCollisions,
ArtifactPathRegistry,
normalizeArtifactPath,
} from "../ArtifactPaths.js";
@@ -64,10 +65,16 @@ export const scanReader = async (
readonly nodes: Map<string, ArtifactNode>;
readonly occurrences: MutableOccurrence[];
readonly pendingContradictions: PendingIntegrityContradiction[];
readonly caseCollisionLimitation: string | undefined;
}> => {
const { nodes, occurrences, pendingContradictions } = emptyScan();
if (reader === undefined)
return { nodes, occurrences, pendingContradictions };
return {
nodes,
occurrences,
pendingContradictions,
caseCollisionLimitation: undefined,
};
const context: ScanContext = {
reader,
signal,
@@ -90,7 +97,12 @@ export const scanReader = async (
context.occurrenceByPath,
context.expandedContainerIds,
)?.occurrence_id ?? null;
return { nodes, occurrences, pendingContradictions };
return {
nodes,
occurrences,
pendingContradictions,
caseCollisionLimitation: annotateEnglishUnicodeCaseCollisions(occurrences),
};
};
const visitArtifactEntries = async (
@@ -25,6 +25,7 @@ export class LocalWebScriptArtifacts implements WebScriptArtifactPort {
async load(input: SelectedWebScriptInput, options?: ExecutionOptions) {
let field: readonly (string | number)[] = ["manifest_path"];
let targetPath = input.manifest_path;
let callerSelected = true;
try {
const manifestBytes = await readWebArtifactJson(
input.manifest_path,
@@ -58,6 +59,7 @@ export class LocalWebScriptArtifacts implements WebScriptArtifactPort {
const root = resolve(dirname(input.manifest_path));
const sourcePath = join(root, "files", portable);
targetPath = sourcePath;
callerSelected = false;
await assertContained(root, sourcePath);
const sourceBytes = await readStableArtifact(
sourcePath,
@@ -99,7 +101,7 @@ export class LocalWebScriptArtifacts implements WebScriptArtifactPort {
return err(
webArtifactReadError(
cause,
{ operation: this.operation, field, targetPath },
{ operation: this.operation, field, targetPath, callerSelected },
options?.signal,
),
);
+37
View File
@@ -3,6 +3,7 @@ import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js";
import { readStableArtifact } from "../../artifacts/readStableArtifact.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import {
AnalysisAccessDeniedError,
AnalysisCancelledError,
AnalysisInputError,
} from "../../domain/analysisErrorCore.js";
@@ -15,6 +16,8 @@ export interface WebArtifactReadContext {
readonly operation: "trace_web_module_imports" | "trace_web_source_location";
readonly field: readonly (string | number)[];
readonly targetPath: string;
/** Whether the caller named this file, rather than a manifest deriving it. */
readonly callerSelected: boolean;
}
/** A malformed producer document, distinct from host I/O failures. */
@@ -54,6 +57,10 @@ export const webArtifactReadError = (
): AnalysisError => {
if (signal?.aborted === true)
return new AnalysisCancelledError(context.operation);
const selection = context.callerSelected
? selectedFileError(cause, context)
: undefined;
if (selection !== undefined) return selection;
if (cause instanceof ArtifactReaderFailure)
return new ArtifactOperationError(
context.operation,
@@ -91,3 +98,33 @@ export const webArtifactReadError = (
},
});
};
/**
* Report a missing, non-regular or unreadable caller-selected file as the
* caller's selection or a host permission denial, not an artifact fault.
*/
const selectedFileError = (
cause: unknown,
context: WebArtifactReadContext,
): AnalysisError | undefined => {
const selected = (message: string) =>
new AnalysisInputError(context.operation, { cause }, [
{ path: context.field, reason: "invalid_value", message },
]);
if (cause instanceof ArtifactReaderFailure)
return cause.reason === "path" ? selected(cause.message) : undefined;
const code =
cause instanceof Error && "code" in cause ? String(cause.code) : "";
if (code === "EACCES" || code === "EPERM")
return new AnalysisAccessDeniedError(
context.operation,
context.targetPath,
code,
{ cause },
);
if (code === "ENOENT" || code === "ENOTDIR")
return selected(
`Selected file could not be read (${code}): ${context.targetPath}`,
);
return undefined;
};
@@ -48,6 +48,7 @@ export class LocalWebSourceLocationArtifacts implements WebSourceLocationArtifac
operation: "trace_web_source_location",
field: ["source_map", "path"],
targetPath: input.source_map.path,
callerSelected: true,
},
options?.signal,
),
@@ -52,6 +52,7 @@ export class LocalWebModuleArtifacts implements WebModuleArtifactPort {
operation: "trace_web_module_imports",
field: ["import_map", "path"],
targetPath: selected.path,
callerSelected: true,
},
options?.signal,
),
+1 -1
View File
@@ -18,7 +18,7 @@ export const registerAnalysisViewCommands = (
): void => {
cli.command(CLI_COMMANDS.inspectAnalysisView, {
description:
"Project a selected view of completed layout or JavaScript application Evidence JSON",
"Project a selected view of completed layout, JavaScript application or native function Evidence JSON",
args: z.object({
inputJson: z.string().describe("Inline workflow JSON or JSON file path"),
}),
-9
View File
@@ -123,7 +123,6 @@ const registerAssetCatalogCommand = (
.default(1_000)
.describe("Maximum records to return"),
}),
alias: {},
run: ({ args, options }) =>
logCliCommand(logger, CLI_COMMANDS.inspectAssetCatalog, () =>
runProviderAnalysis(
@@ -170,11 +169,6 @@ const registerInterfaceBuilderCommand = (
.default(40_000)
.describe("Maximum decoded connections"),
}),
alias: {
maxDocuments: "max-documents",
maxObjects: "max-objects",
maxConnections: "max-connections",
},
run: ({ args, options }) =>
logCliCommand(logger, "decode-interface-builder", () =>
runProviderAnalysis(
@@ -233,9 +227,6 @@ const registerInspectionCommand = (
.default("fail")
.describe("Behavior when declared artifact integrity does not match"),
}),
alias: {
integrityPolicy: "integrity-policy",
},
run: ({ args, options }) =>
logCliCommand(logger, "inspect-artifact", () =>
runProviderAnalysis(
+78 -36
View File
@@ -15,13 +15,14 @@ import {
type AnalysisInputIssue,
} from "../domain/analysisErrorCore.js";
import { browserCaptureComparisonInputSchema } from "../domain/browserCaptureComparison.js";
import { projectInputIssues } from "../domain/inputIssueProjection.js";
import { discoverWebMcpToolsInputSchema } from "../domain/webMcpDiscovery.js";
import {
captureWebScreenshotInputSchema,
compareWebScreenshotsInputSchema,
} from "../domain/webScreenshot.js";
import type { JsonValue } from "../domain/jsonValue.js";
import { safeParseJson } from "../domain/safeJson.js";
import { parseCliJsonInput } from "../cliJsonInput.js";
import type { Logger } from "pino";
import { CLI_COMMANDS } from "../cliCommandNames.js";
@@ -79,46 +80,39 @@ const registerCaptureDiff = (
description:
"Compare two normalized passive captures or recorded browser scenarios",
args: z.object({
beforeJson: z.string().describe("Earlier normalized web capture JSON"),
afterJson: z.string().describe("Later normalized web capture JSON"),
beforeJson: z
.string()
.describe("Earlier normalized web capture JSON or JSON file path"),
afterJson: z
.string()
.describe("Later normalized web capture JSON or JSON file path"),
}),
options: z.object({
normalizationJson: z
.string()
.default('{"rules":[]}')
.describe(
"Recorded literal normalization policy for browser scenarios",
"Recorded literal normalization policy JSON or JSON file path for browser scenarios",
),
}),
run: ({ args, options }) =>
logCliCommand(logger, "compare-web-captures", async () => {
const before = safeParseJson(args.beforeJson);
const after = safeParseJson(args.afterJson);
const normalization = safeParseJson(options.normalizationJson);
if (!before.ok)
return inputError("compare_web_captures", [
invalidJsonIssue("before"),
]);
if (!after.ok)
return inputError("compare_web_captures", [
invalidJsonIssue("after"),
]);
if (!normalization.ok)
return inputError("compare_web_captures", [
invalidJsonIssue("normalization"),
]);
const inputs = await parseJsonArguments("compare_web_captures", {
before: args.beforeJson,
after: args.afterJson,
normalization: options.normalizationJson,
});
if (!inputs.ok) return inputs.error;
const { before, after, normalization } = inputs.value;
const scenarioComparison =
browserCaptureComparisonInputSchema.safeParse({
before_scenario: before.value,
after_scenario: after.value,
normalization: normalization.value,
before_scenario: before,
after_scenario: after,
normalization,
});
const parsed = scenarioComparison.success
? scenarioComparison
: browserCaptureComparisonInputSchema.safeParse({
before: before.value,
after: after.value,
});
: browserCaptureComparisonInputSchema.safeParse({ before, after });
if (!parsed.success) return inputError("compare_web_captures");
const result = await compareWebCaptureEvidence(
new CdpBrowserProvider(),
@@ -168,8 +162,12 @@ const registerScreenshotDiff = (
cli.command(CLI_COMMANDS.compareWebScreenshots, {
description: "Compare two self-verifying PNG artifact JSON values",
args: z.object({
beforeJson: z.string().describe("Earlier screenshot artifact JSON"),
afterJson: z.string().describe("Later screenshot artifact JSON"),
beforeJson: z
.string()
.describe("Earlier screenshot artifact JSON or JSON file path"),
afterJson: z
.string()
.describe("Later screenshot artifact JSON or JSON file path"),
}),
options: z.object({
channelThreshold: z
@@ -182,12 +180,27 @@ const registerScreenshotDiff = (
}),
run: ({ args, options }) =>
logCliCommand(logger, "compare-web-screenshots", async () => {
const parsed = compareWebScreenshotsInputSchema.safeParse({
before: parseJson(args.beforeJson),
after: parseJson(args.afterJson),
channel_threshold: options.channelThreshold,
const operation = "compare_web_screenshots";
const inputs = await parseJsonArguments(operation, {
before: args.beforeJson,
after: args.afterJson,
});
if (!parsed.success) return inputError("compare_web_screenshots");
if (!inputs.ok) return inputs.error;
const { before, after } = inputs.value;
const input = {
before,
after,
channel_threshold: options.channelThreshold,
};
const parsed = compareWebScreenshotsInputSchema.safeParse(input);
if (!parsed.success)
return browserCliError(
new AnalysisInputError(
operation,
{ cause: parsed.error },
projectInputIssues(parsed.error.issues, input),
),
);
const result = await compareWebScreenshotEvidence(
new CdpBrowserProvider(),
parsed.data,
@@ -197,11 +210,40 @@ const registerScreenshotDiff = (
});
};
const parseJson = (value: string): unknown => {
const parsed = safeParseJson(value);
return parsed.ok ? parsed.value : undefined;
/**
* Parse each named inline JSON argument or local JSON file, like other
* JSON-input commands. Screenshot artifacts and scenario captures embed PNG
* bytes that can exceed the host's command-line length limit.
*/
const parseJsonArguments = async (
operation: string,
fields: Readonly<Record<string, string>>,
): Promise<
| { readonly ok: true; readonly value: Readonly<Record<string, unknown>> }
| { readonly ok: false; readonly error: JsonValue }
> => {
const values: Record<string, unknown> = {};
for (const [field, value] of Object.entries(fields)) {
const parsed = await parseCliJsonInput(value, operation);
if (!parsed.ok)
return isFileInputFailure(parsed.error)
? parsed
: {
ok: false,
error: inputError(operation, [invalidJsonIssue(field)]),
};
values[field] = parsed.value;
}
return { ok: true, value: values };
};
/** File read and decode failures name the selected path; inline JSON does not. */
const isFileInputFailure = (error: JsonValue): boolean =>
typeof error === "object" &&
error !== null &&
!Array.isArray(error) &&
"input_path" in error;
const invalidJsonIssue = (field: string): AnalysisInputIssue => ({
path: [field],
reason: "invalid_format",
-5
View File
@@ -54,7 +54,6 @@ const registerOverviewCommands = (
options: overviewOptions.extend({
integrityPolicy: javascriptApplicationOptions.shape.integrityPolicy,
}),
alias: { integrityPolicy: "integrity-policy" },
run: async ({ args, options }) => {
// Route JavaScript targets exactly like analyze-javascript-application,
// including typed cancellation; cancellation wraps logging so its exit
@@ -274,9 +273,6 @@ const registerTraceCommand = (
"target-format": formatSelectionOption,
provider: providerSelectionOption,
}),
alias: {
caseSensitive: "case-sensitive",
},
run: ({ args, options }) =>
logCliCommand(logger, "trace", () =>
runDirectAnalysis(
@@ -455,7 +451,6 @@ const registerSearchCommand = (
"target-format": formatSelectionOption,
provider: providerSelectionOption,
}),
alias: { caseSensitive: "case-sensitive" },
run: ({ args, options }) =>
logCliCommand(logger, "search", () =>
runDirectAnalysis(
-13
View File
@@ -157,13 +157,6 @@ export const registerCoreNativeCommands = (
"target-format": formatSelectionOption,
provider: providerSelectionOption,
}),
alias: {
maxDepth: "max-depth",
maxFunctions: "max-functions",
maxCallSites: "max-call-sites",
maxNodes: "max-nodes",
maxEdges: "max-edges",
},
run: ({ args, options }) =>
logCliCommand(logger, CLI_COMMANDS.traceNativeValues, () =>
runDirectAnalysis(
@@ -283,7 +276,6 @@ const registerNativeDispatchMetadataCommand = (
"target-format": formatSelectionOption,
provider: providerSelectionOption,
}),
alias: { maxRecords: "max-records" },
run: async ({ args, options }) =>
logCliCommand(logger, "inspect-native-dispatch-metadata", () =>
runDirectAnalysis(
@@ -343,11 +335,6 @@ const registerNativeUiActionCommand = (
"target-format": formatSelectionOption,
provider: providerSelectionOption,
}),
alias: {
maxDepth: "max-depth",
maxNodes: "max-nodes",
maxEdges: "max-edges",
},
run: async ({ args, options }) =>
logCliCommand(logger, "trace-native-ui-action", () =>
runDirectAnalysis(
-1
View File
@@ -190,7 +190,6 @@ const registerJavaScriptApplicationCommand = (
path: z.string().describe("ASAR or extracted application path"),
}),
options: javascriptApplicationOptions,
alias: { integrityPolicy: "integrity-policy" },
run: ({ args, options }) =>
withCommandCancellation((signal) =>
logCliCommand(logger, CLI_COMMANDS.analyzeJavaScriptApplication, () =>
+15 -3
View File
@@ -4,16 +4,27 @@ import { bufferedJsonParts, jsonParts } from "../domain/jsonSerialization.js";
type JsonOutputFormat = "json" | "jsonl";
/** Write ordinary JSON without constructing a document-sized string. */
/**
* Write ordinary JSON without constructing a document-sized string.
*
* `json` is indented for a terminal reader. Piped or redirected output, which
* agents and scripts consume, is compact: indentation adds no information and
* measured 16-45% more model tokens on real results. The value is identical.
*/
export const writeJsonOutput = async (
value: unknown,
destination: Writable,
format: JsonOutputFormat = "json",
): Promise<void> => {
for (const part of bufferedJsonParts(jsonOutputParts(value, format)))
for (const part of bufferedJsonParts(
jsonOutputParts(value, format, isTerminal(destination)),
))
await writePart(destination, part);
};
const isTerminal = (destination: Writable): boolean =>
"isTTY" in destination && destination.isTTY === true;
const writePart = (destination: Writable, part: string): Promise<void> =>
new Promise((resolve, reject) => {
const onError = (cause: unknown): void => reject(cause);
@@ -39,6 +50,7 @@ const writePart = (destination: Writable, part: string): Promise<void> =>
export function* jsonOutputParts(
value: unknown,
format: JsonOutputFormat,
pretty = true,
): Generator<string> {
if (format === "jsonl" && Array.isArray(value)) {
for (const item of value) {
@@ -47,6 +59,6 @@ export function* jsonOutputParts(
}
return;
}
yield* jsonParts(value, format === "json");
yield* jsonParts(value, pretty && format === "json");
yield "\n";
}
-1
View File
@@ -148,7 +148,6 @@ const registerMaintenanceCommands = (
.default(false)
.describe("Also remove REA caches and state"),
}),
alias: { purgeData: "purge-data" },
run: ({ options }) =>
logCliCommand(
logger,
-5
View File
@@ -65,10 +65,6 @@ export const registerUtilityCommands = (
.default(500)
.describe("Maximum accessibility nodes per capture"),
}),
alias: {
windowId: "window-id",
maxNodes: "max-nodes",
},
run: ({ args, options }) =>
logCliCommand(logger, command, async () => {
const parameters = {
@@ -172,7 +168,6 @@ const registerNativeCommands = (
"Plist path relative to the app root (default: Contents/Info.plist)",
),
}),
alias: { relativePath: "relative-path" },
run: ({ args, options }) =>
logCliCommand(logger, "inspect-plist", () =>
runProviderAnalysis(
+3 -1
View File
@@ -2,7 +2,6 @@ import { Cli } from "incur";
import { z } from "zod";
import { captureBrowserScenario } from "./application/BrowserScenarioCaptureService.js";
import { createBrowserScenarioProvider } from "./composition/browserScenario.js";
import { CLI_COMMANDS } from "./cliCommandNames.js";
import { parseCliJsonInput, resolveCliJsonPaths } from "./cliJsonInput.js";
import { logCliCommand } from "./cliLogging.js";
@@ -44,6 +43,9 @@ export const registerBrowserScenarioCommands = (
projectInputIssues(scenario.error.issues, input.value),
),
);
// Load Playwright only for this command, not during CLI startup.
const { createBrowserScenarioProvider } =
await import("./composition/browserScenario.js");
const result = await captureBrowserScenario(
createBrowserScenarioProvider(environment),
scenario.data,
+17 -2
View File
@@ -8,14 +8,14 @@ import { evidenceResultOf } from "./toolOutputSchemaPrimitives.js";
const RETAINED_EXAMPLE_ID = `ev_${"a".repeat(64)}`;
/** Selected views of already completed layout and JavaScript analysis Evidence. */
/** Selected views of already completed layout, JavaScript and native function Evidence. */
export const ANALYSIS_VIEW_TOOL_CONTRACTS = [
{
name: "inspect_analysis_view",
...toolContractMetadata("inspect_analysis_view"),
kind: "application",
description:
"Inspect a selected view of completed binary-layout or JavaScript application Evidence without repeating analysis. Use an exact same-session retained evidence_id or portable inline Evidence. Select a summary, layout mitigations/linkage facet, one section/symbol/module, or a stable page with a positive limit. Module pages contain JavaScript assets, bundled modules and source modules; use node_id when paths are ambiguous or unavailable. Returns selected facts inline with artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the retained Evidence when needed.",
"Inspect selected views of completed binary-layout, JavaScript application or native analyze_function Evidence without repeating analysis. Use an exact same-session retained evidence_id or portable inline Evidence. Select a summary, layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Native views select procedure, pseudocode, assembly, callers/callees, references and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.",
inputSchema: inspectAnalysisViewInputSchema,
outputSchema: evidenceResultOf(analysisViewResultSchema),
examples: [
@@ -43,6 +43,21 @@ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [
},
},
},
{
title: "Page native high-pcode operations from retained Evidence",
input: {
source: {
kind: "retained-evidence",
evidence_id: RETAINED_EXAMPLE_ID,
},
view: {
kind: "native",
facet: "value_flow_operations",
offset: 0,
limit: 32,
},
},
},
{
title: "Page module identities from retained application Evidence",
input: {
+1 -1
View File
@@ -63,7 +63,7 @@ const artifact = <
export const ARTIFACT_TOOL_CONTRACTS = [
artifact(
"inspect_artifact",
"Inspect an active archive or application package in one call. Returns the complete content-addressed artifact graph, source Evidence, observations, relationships, integrity contradictions, hypotheses, unexplored branches, limitations, and next probes inline. Read-only DMG mounting is used automatically on supported macOS hosts. Integrity mismatches fail by default; record-and-continue keeps mismatches explicitly untrusted. It does not extract files; use extract_artifact to materialize its regular contents.",
"Inspect an active archive or application package in one call. Returns the complete content-addressed artifact graph once, as the nested inventory Evidence, with root and integrity observations, integrity contradictions, hypotheses, unexplored branches, limitations, and next probes inline. Read-only DMG mounting is used automatically on supported macOS hosts. Integrity mismatches fail by default; record-and-continue keeps mismatches explicitly untrusted. It does not extract files; use extract_artifact to materialize its regular contents.",
artifactInventoryInputSchema,
),
artifact(
+10 -10
View File
@@ -59,7 +59,7 @@ const referencedDescription = (
const describeSchema = (
schema: unknown,
root: Readonly<Record<string, unknown>>,
describeProperty: (property: string) => string,
describeProperty: (property: string) => string | undefined,
): unknown => {
if (!isObject(schema)) return schema;
return Object.fromEntries(
@@ -70,16 +70,16 @@ const describeSchema = (
Object.fromEntries(
Object.entries(value).map(([name, child]) => {
const described = describeSchema(child, root, describeProperty);
if (key !== "properties" || !isObject(described))
return [name, described];
const description =
referencedDescription(root, described) ??
describeProperty(name);
return [
name,
key === "properties" && isObject(described)
? {
...described,
description:
referencedDescription(root, described) ??
describeProperty(name),
}
: described,
description === undefined
? described
: { ...described, description },
];
}),
),
@@ -223,7 +223,7 @@ const flattenRootObjectUnion = (
/** Describe canonical input fields and present the root as one object. */
export const presentInputJsonSchema = (
root: Record<string, unknown>,
describeProperty: (property: string) => string,
describeProperty: (property: string) => string | undefined,
): Record<string, unknown> => {
const flattened = flattenRootObjectUnion(root);
const described = describeSchema(flattened, flattened, describeProperty);
+6 -1
View File
@@ -97,9 +97,14 @@ describe("advertised input references", () => {
before: { description: "Exact byte range in the selected artifact." },
after: { description: "Exact byte range in the selected artifact." },
metadata: { default: literal, examples: [literal] },
constructor: { description: "Value for constructor." },
},
});
// Inherited object keys never supply a description.
expect(inline).not.toHaveProperty([
"properties",
"constructor",
"description",
]);
const ajv =
target === "draft-2020-12"
? new Ajv2020({ strict: false })
@@ -1,11 +1,69 @@
import { Ajv } from "ajv";
import { Ajv2020 } from "ajv/dist/2020.js";
import { AjvJsonSchemaValidator } from "@modelcontextprotocol/client/validators/ajv";
import { expect, it } from "vitest";
import { z } from "zod";
import { toolContract } from "./toolContracts.js";
import { toolOutputSchemaWithMetadata } from "./toolSchemaMetadata.js";
it("keeps dialect-specific validators distinct after JSON schema serialization", () => {
const contract = {
...toolContract("binary_session"),
outputSchema: z.strictObject({ value: z.number().int().min(1).max(8) }),
};
const output = toolOutputSchemaWithMetadata(contract);
const provider = new AjvJsonSchemaValidator();
for (const target of ["draft-2020-12", "draft-07"] as const) {
const projected = output["~standard"].jsonSchema.output({ target });
const parsed = z
.record(z.string(), z.unknown())
.parse(JSON.parse(JSON.stringify(projected)));
const validate = provider.getValidator(parsed);
expect(validate({ value: 8 }).valid).toBe(true);
expect(validate({ value: 9 }).valid).toBe(false);
expect(validate({ value: 1.5 }).valid).toBe(false);
}
});
it("preserves explicit schema identity and relative reference resolution", () => {
const value = z.string();
const canonical = z.strictObject({ value });
const contract = {
...toolContract("binary_session"),
outputSchema: canonical,
};
const projected = toolOutputSchemaWithMetadata(contract)[
"~standard"
].jsonSchema.output({
target: "draft-2020-12",
libraryOptions: {
override: (context: {
readonly zodSchema: unknown;
readonly jsonSchema: Record<string, unknown>;
}) => {
if (context.zodSchema !== value) return;
delete context.jsonSchema.type;
context.jsonSchema.$ref = "child.json";
},
},
});
const ajv = new Ajv2020({ strict: false });
ajv.addSchema({ $id: "child.json", type: "string", minLength: 3 });
const validate = ajv.compile(projected);
expect(validate({ value: "abc" })).toBe(true);
expect(validate({ value: "ab" })).toBe(false);
const identified = toolOutputSchemaWithMetadata({
...contract,
outputSchema: canonical.meta({ $id: "https://example.test/result" }),
})["~standard"].jsonSchema.output({ target: "draft-2020-12" });
expect(identified.$id).toBe("https://example.test/result");
const explicit = ajv.compile(identified);
expect(explicit({ value: "abc" })).toBe(true);
expect(explicit({ value: 1 })).toBe(false);
});
it.each(["draft-2020-12", "draft-07"] as const)(
"preserves refined numeric bounds and strict repeated objects through %s references",
(target) => {
+43 -6
View File
@@ -2,6 +2,7 @@ import { z } from "zod";
import type { ToolContract } from "./toolContractTypes.js";
import { presentInputJsonSchema } from "./inputSchemaPresentation.js";
import { digestCanonicalValue } from "../domain/canonicalDigest.js";
const PROPERTY_DESCRIPTIONS: Readonly<Record<string, string>> = {
addresses: "Ordered provider-normalized procedure addresses to analyze.",
@@ -69,20 +70,55 @@ export const toolInputSchemaWithMetadata = <Contract extends ToolContract>(
},
);
/** Share repeated output definitions without changing canonical validation. */
/** Share output definitions and content-bound validator identities on the wire. */
export const toolOutputSchemaWithMetadata = <Contract extends ToolContract>(
contract: Contract,
): Contract["outputSchema"] =>
withAdvertisedJsonSchema(
contract.outputSchema,
"output",
(project) => (options) =>
project({
(project) => (options) => {
const shared = project({
...options,
libraryOptions: { reused: "ref", ...options.libraryOptions },
}),
});
// An explicit ID owns the reference base. Relative external references
// also depend on that base, so leave those projections unchanged.
if ("$id" in shared || hasRelativeSchemaReference(shared)) return shared;
return {
...shared,
$id: `urn:rea:tool-output:sha256:${digestCanonicalValue(
{ target: options.target, schema: shared },
"Tool output schema",
)}`,
};
},
);
const hasRelativeSchemaReference = (
schema: Record<string, unknown>,
): boolean => {
const pending: unknown[] = [schema];
const visited = new WeakSet<object>();
while (pending.length > 0) {
const current = pending.pop();
if (current === null || typeof current !== "object" || visited.has(current))
continue;
visited.add(current);
for (const [key, value] of Object.entries(current)) {
if (
["$id", "$ref", "$dynamicRef", "$recursiveRef"].includes(key) &&
typeof value === "string" &&
!value.startsWith("#") &&
!URL.canParse(value)
)
return true;
if (value !== null && typeof value === "object") pending.push(value);
}
}
return false;
};
const withAdvertisedJsonSchema = <Schema extends z.ZodType>(
canonical: Schema,
io: "input" | "output",
@@ -128,7 +164,7 @@ const memoizeByTarget = (
};
};
const fallbackPropertyDescription = (property: string): string => {
const fallbackPropertyDescription = (property: string): string | undefined => {
const explicit = Object.hasOwn(PROPERTY_DESCRIPTIONS, property)
? PROPERTY_DESCRIPTIONS[property]
: undefined;
@@ -156,5 +192,6 @@ const fallbackPropertyDescription = (property: string): string => {
return `Local filesystem ${words} selected for this operation.`;
if (property.startsWith("is_") || property.startsWith("has_"))
return `Whether ${words}.`;
return `Value for ${words}.`;
// A generic sentence would only restate the property name in every schema.
return undefined;
};
+20 -3
View File
@@ -191,7 +191,7 @@ export const analysisErrorUserMessage = (error: AnalysisError): string => {
const standardMessage = standardErrorMessage(error._tag);
if (standardMessage !== undefined) return standardMessage;
if (error instanceof ArtifactOperationError)
return artifactMessage(error.operation, error.reason);
return artifactMessage(error.operation, error.reason, error.detail);
if (error instanceof EvidenceReferenceError)
return error.reason === "missing"
? `Evidence ${error.evidenceId} is not retained in this session. Supply complete inline Evidence, re-run its producer, or import its bundle before using this reference.`
@@ -329,13 +329,13 @@ const START_FAILURE_TAGS: ReadonlySet<AnalysisErrorTag> = new Set([
const artifactMessage = (
operation: ArtifactOperationError["operation"],
reason: ArtifactOperationError["reason"],
detail?: string,
): string => {
if (reason === "cancelled")
return "Artifact operation was cancelled. Start it again when ready.";
if (reason === "limit")
return "Artifact is too large to process safely. Narrow the requested path or use a smaller artifact.";
if (reason === "path")
return "Artifact contains an unsafe or conflicting internal path. Inspect the reported path and correct the artifact before retrying.";
if (reason === "path") return artifactPathMessage(detail);
if (reason === "unavailable")
return "Artifact processing is unavailable for the current target or host. Check artifact support and required tools.";
if (reason === "integrity" && operation === "analyze_javascript_application")
@@ -345,6 +345,23 @@ const artifactMessage = (
return "Artifact could not be read or written. Check file access and try again.";
};
const artifactPathMessage = (detail: string | undefined): string => {
// Kept in step with DESTINATION_CASE_COLLISION_PREFIX. Domain does not import
// the artifact adapter that produces the detail.
if (
detail !== undefined &&
detail.startsWith("Destination filesystem cannot store both ")
)
return detail;
if (
detail !== undefined &&
(detail.startsWith("Artifact path is absolute or unsafe:") ||
detail.startsWith("Artifact path is not normalized:"))
)
return "Artifact contains an unsafe internal path. Inspect the reported path and correct the artifact before retrying.";
return "Artifact contains a conflicting internal path. Inspect the reported path before retrying.";
};
const evidenceFileMessage = ({
operation,
reason,
+184 -13
View File
@@ -1,6 +1,8 @@
import { expect, it } from "vitest";
import { resolveJavaScriptSourceMapReference } from "../javascript/javascriptSourceMapPaths.js";
import { functionDossierSchema } from "../hopperValues.js";
import { ghidraFunctionDossier } from "../ghidraValues.fixture.js";
import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "../../contracts/javascript/javascriptRuntimeReconciliationExample.js";
import {
analysisViewBindJavaScriptGraphs,
@@ -48,21 +50,190 @@ const javascriptParent = (
limitations: JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE.limitations,
});
it("accepts caller-selected page sizes and rejects malformed bounds", () => {
const input = {
source: { kind: "retained-evidence", evidence_id: `ev_${"a".repeat(64)}` },
view: { kind: "page", collection: "sections", offset: 0, limit: 500 },
};
expect(inspectAnalysisViewInputSchema.safeParse(input).success).toBe(true);
for (const limit of [0, -1, 0.5, Infinity, Number.MAX_SAFE_INTEGER + 1])
expect(
inspectAnalysisViewInputSchema.safeParse({
...input,
view: { ...input.view, limit },
}).success,
).toBe(false);
const nativeParent = (
result: unknown = ghidraFunctionDossier(),
): AnalysisViewParent => ({
evidenceId: "ev_" + "b".repeat(64),
operation: "analyze_function",
normalizedResult: result,
limitations: ["Retained provider limitation"],
artifact: { path: "/fixtures/example.exe", sha256: "a".repeat(64) },
});
it("projects native procedure and bounded assembly and high-pcode with exact parent identity", () => {
const parent = nativeParent();
const procedure = projectAnalysisView(parent, {
kind: "native",
facet: "procedure",
offset: 0,
limit: 64,
});
if (!procedure.ok) throw procedure.error;
expect(procedure.value).toMatchObject({
kind: "native",
parent_operation: "analyze_function",
artifact: parent.artifact,
item: { address: "0x401000" },
});
expect(procedure.value.view_digest).not.toBe(parent.evidenceId.slice(3));
const assembly = projectAnalysisView(parent, {
kind: "native",
facet: "assembly",
offset: 0,
limit: 1,
});
if (!assembly.ok) throw assembly.error;
expect(assembly.value).toMatchObject({
coverage: { examined: 1, total: 2, next_offset: 1 },
item: ["0x401000: CALL 0x401020"],
});
const flow = projectAnalysisView(parent, {
kind: "native",
facet: "value_flow_operations",
offset: 0,
limit: 1,
});
if (!flow.ok) throw flow.error;
expect(flow.value).toMatchObject({ item: [{ opcode: "COPY" }] });
});
it("rejects invalid native parents, unsupported source operations and singleton offsets", () => {
const view = {
kind: "native",
facet: "assembly",
offset: 0,
limit: 4,
} as const;
expect(projectAnalysisView(nativeParent({}), view)).toMatchObject({
ok: false,
error: { _tag: "AnalysisInputError" },
});
expect(
projectAnalysisView({ ...nativeParent(), artifact: null }, view),
).toMatchObject({
ok: false,
error: { _tag: "AnalysisInputError" },
});
expect(projectAnalysisView(layoutParent(), view)).toMatchObject({
ok: false,
error: { _tag: "AnalysisInputError" },
});
expect(projectAnalysisView(javascriptParent(), view)).toMatchObject({
ok: false,
error: { _tag: "AnalysisInputError" },
});
expect(
projectAnalysisView(nativeParent(), {
kind: "native",
facet: "procedure",
offset: 1,
limit: 2,
}),
).toMatchObject({ ok: false, error: { _tag: "AnalysisInputError" } });
});
it("preserves native API limitations and unavailable residual unknowns", () => {
const original = functionDossierSchema.parse(ghidraFunctionDossier());
const view = {
kind: "native",
facet: "procedure",
offset: 0,
limit: 64,
} as const;
const unavailable = projectAnalysisView(
nativeParent({
...original,
native_api: {
available: false,
reason: "No decompiler",
residual_unknowns: ["Calling convention unresolved"],
},
}),
view,
);
if (!unavailable.ok) throw unavailable.error;
expect(unavailable.value.unknowns).toContain("Calling convention unresolved");
const available = projectAnalysisView(nativeParent(), view);
if (!available.ok) throw available.error;
if (!original.native_api?.available) throw new Error("expected API fixture");
expect(available.value.limitations).toEqual(
expect.arrayContaining(original.native_api.limitations),
);
});
it("reports the malformed native dossier field", () => {
const result = projectAnalysisView(
nativeParent({
...functionDossierSchema.parse(ghidraFunctionDossier()),
pseudocode: 12,
}),
{
kind: "native",
facet: "pseudocode",
offset: 0,
limit: 64,
},
);
expect(result).toMatchObject({
ok: false,
error: {
issues: expect.arrayContaining([
expect.objectContaining({
path: ["source", "normalized_result", "pseudocode"],
}),
]),
},
});
});
it("native pseudocode pages do not split surrogate pairs", () => {
const original = functionDossierSchema.parse(ghidraFunctionDossier());
const parent = nativeParent({ ...original, pseudocode: "ab😀cd" });
expect(
projectAnalysisView(parent, {
kind: "native",
facet: "pseudocode",
offset: 3,
limit: 2,
}),
).toMatchObject({ ok: false, error: { _tag: "AnalysisInputError" } });
const safe = projectAnalysisView(parent, {
kind: "native",
facet: "pseudocode",
offset: 2,
limit: 3,
});
if (!safe.ok) throw safe.error;
expect(safe.value).toMatchObject({
item: { text: "😀c", unit: "utf16-code-units" },
coverage: { examined: 3, next_offset: 5 },
});
});
it.each([
{ kind: "page", collection: "sections", offset: 0, limit: 500 },
{ kind: "native", facet: "assembly", offset: 0, limit: 500 },
])(
"accepts caller-selected page sizes and rejects malformed bounds: $kind",
(view) => {
const input = {
source: {
kind: "retained-evidence",
evidence_id: `ev_${"a".repeat(64)}`,
},
view,
};
expect(inspectAnalysisViewInputSchema.safeParse(input).success).toBe(true);
for (const limit of [0, -1, 0.5, Infinity, Number.MAX_SAFE_INTEGER + 1])
expect(
inspectAnalysisViewInputSchema.safeParse({
...input,
view: { ...input.view, limit },
}).success,
).toBe(false);
},
);
it("projects layout summary, facet, item, and stable pages", () => {
const parent = layoutParent();
const layout = analysisViewLayoutFixture();
+60 -3
View File
@@ -12,6 +12,7 @@ import { jsonObjectSchema, jsonValueSchema } from "../jsonValue.js";
import { err, ok, type Result } from "../result.js";
import type { AnalysisError } from "../analysisErrorBase.js";
import { binaryLayoutSchema } from "../native/binaryLayout.js";
import { functionDossierSchema } from "../hopperValues.js";
import {
parseOwnedJavaScriptApplicationAnalysisSteps,
type JavaScriptApplicationAnalysisResult,
@@ -19,6 +20,7 @@ import {
import { analysisInputErrorFromIssues } from "../inputIssueProjection.js";
import { projectBinaryLayoutView } from "./binaryLayoutView.js";
import { projectJavaScriptApplicationView } from "./javascriptApplicationView.js";
import { projectNativeFunctionView } from "./nativeFunctionView.js";
/** Public name of the selected-view workflow. */
export const INSPECT_ANALYSIS_VIEW_OPERATION = "inspect_analysis_view" as const;
@@ -73,12 +75,35 @@ const pageViewSchema = z.strictObject({
limit: z.number().int().positive(),
});
/** Bounded native function facets of already authenticated analyze_function Evidence. */
const nativeViewSchema = z.strictObject({
kind: z.literal("native"),
facet: z.enum([
"procedure",
"pseudocode",
"assembly",
"callers",
"callees",
"incoming_references",
"outgoing_references",
"value_flow_summary",
"value_flow_operations",
"value_flow_def_use",
"value_flow_effects",
"value_flow_parameters",
"value_flow_parameter_uses",
]),
offset: z.number().int().nonnegative().default(0),
limit: z.number().int().positive().default(64),
});
/** Caller-selected projection of already completed analysis Evidence. */
export const analysisViewRequestSchema = z.discriminatedUnion("kind", [
summaryViewSchema,
facetViewSchema,
itemViewSchema,
pageViewSchema,
nativeViewSchema,
]);
/** Inline Evidence or an exact same-session retained reference. */
@@ -108,7 +133,9 @@ const coverageSchema = z.strictObject({
});
const artifactSchema = z.strictObject({
path: z.string().min(1),
path: z
.string()
.describe("Exact recorded artifact path, including an empty string."),
sha256: digestSchema,
});
@@ -150,6 +177,7 @@ const viewResultBase = {
parent_operation: z.enum([
"inspect_binary_layout",
"analyze_javascript_application",
"analyze_function",
]),
parent_digest: digestSchema,
view_digest: digestSchema,
@@ -185,6 +213,13 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [
items: z.array(jsonValueSchema),
...viewResultBase,
}),
z.strictObject({
kind: z.literal("native"),
view: nativeViewSchema,
procedure_address: z.string().nullable(),
item: jsonValueSchema,
...viewResultBase,
}),
]);
export type AnalysisViewRequest = z.output<typeof analysisViewRequestSchema>;
@@ -200,6 +235,7 @@ export interface AnalysisViewParent {
readonly operation: string;
readonly normalizedResult: unknown;
readonly limitations: readonly string[];
readonly artifact?: { readonly path: string; readonly sha256: string } | null;
}
/** Construct a typed input failure for one selected-view constraint. */
@@ -216,7 +252,11 @@ export type UnsignedAnalysisView =
>
| Omit<Extract<AnalysisViewResult, { readonly kind: "facet" }>, "view_digest">
| Omit<Extract<AnalysisViewResult, { readonly kind: "item" }>, "view_digest">
| Omit<Extract<AnalysisViewResult, { readonly kind: "page" }>, "view_digest">;
| Omit<Extract<AnalysisViewResult, { readonly kind: "page" }>, "view_digest">
| Omit<
Extract<AnalysisViewResult, { readonly kind: "native" }>,
"view_digest"
>;
/** Attach the content digest that must change when projected bytes change. */
export const sealAnalysisView = (
@@ -261,7 +301,7 @@ const unsupportedParent = (
new AnalysisUnsupportedTargetError(
INSPECT_ANALYSIS_VIEW_OPERATION,
parentArtifactPath(parent),
`Parent Evidence operation ${JSON.stringify(parent.operation)} is not inspect_binary_layout or analyze_javascript_application.`,
`Parent Evidence operation ${JSON.stringify(parent.operation)} is not inspect_binary_layout, analyze_javascript_application or analyze_function.`,
);
const parentArtifactPath = (parent: AnalysisViewParent): string => {
@@ -327,5 +367,22 @@ export const projectAnalysisView = (
);
return projected.ok ? ok(sealAnalysisView(projected.value)) : projected;
}
if (parent.operation === "analyze_function") {
const native = functionDossierSchema.safeParse(parent.normalizedResult);
if (!native.success)
return err(
analysisInputErrorFromIssues(
INSPECT_ANALYSIS_VIEW_OPERATION,
native.error.issues.map((issue) => ({
...issue,
path: ["source", "normalized_result", ...issue.path],
})),
{ source: { normalized_result: parent.normalizedResult } },
{ cause: native.error },
),
);
const projected = projectNativeFunctionView(parent, native.data, view);
return projected.ok ? ok(sealAnalysisView(projected.value)) : projected;
}
return err(unsupportedParent(parent));
};
@@ -152,6 +152,10 @@ export const projectBinaryLayoutView = (
view: AnalysisViewRequest,
): Result<UnsignedAnalysisView, AnalysisError> => {
const shared = parentFields(parent, layout);
if (view.kind === "native")
return err(
layoutIncompatible("Native views require analyze_function Evidence."),
);
if (view.kind === "summary")
return ok({
kind: "summary",
@@ -243,6 +243,10 @@ export const projectJavaScriptApplicationView = (
): Result<UnsignedAnalysisView, AnalysisError> => {
const shared = parentFields(parent, analysis);
const modules = moduleEntries(analysis.graph.nodes);
if (view.kind === "native")
return err(
jsIncompatible("Native views require analyze_function Evidence."),
);
if (view.kind === "summary")
return ok({
kind: "summary",
@@ -0,0 +1,202 @@
import type { AnalysisError } from "../analysisErrorBase.js";
import { uniqueSorted } from "../canonicalOrdering.js";
import type { FunctionDossier } from "../hopperValues.js";
import { jsonValueSchema } from "../jsonValue.js";
import { err, ok, type Result } from "../result.js";
import {
analysisViewInputError,
type AnalysisViewCoverage,
type AnalysisViewParent,
type AnalysisViewRequest,
type UnsignedAnalysisView,
} from "./analysisView.js";
import {
completeWithinViewCoverage,
pageViewCoverage,
} from "./analysisViewCoverage.js";
type Artifact = { readonly path: string; readonly sha256: string };
const viewError = (message: string): AnalysisError =>
analysisViewInputError([
{ path: ["view"], reason: "invalid_value", message },
]);
const parentFields = (
parent: AnalysisViewParent,
dossier: FunctionDossier,
artifact: Artifact,
) => {
const flow = dossier.native_value_flow;
const api = dossier.native_api;
const unknowns: string[] = [];
if (flow === null) unknowns.push("native_value_flow was not recorded");
else if (!flow.available)
unknowns.push("native_value_flow unavailable: " + flow.reason);
else if (flow.truncated)
unknowns.push(
"native_value_flow is truncated; omitted observations remain unknown",
);
if (api === null) unknowns.push("native_api was not recorded");
else if (!api.available)
unknowns.push(
"native_api unavailable: " + api.reason,
...api.residual_unknowns,
);
return {
parent_evidence_id: parent.evidenceId,
parent_operation: "analyze_function" as const,
parent_digest: parent.evidenceId.slice(3),
procedure_address: dossier.procedure.address,
artifact,
limitations: uniqueSorted([
...parent.limitations,
...dossier.limitations,
...(flow?.limitations ?? []),
...(api?.available ? api.limitations : []),
]),
unknowns: uniqueSorted(unknowns),
};
};
const pseudocodePage = (
body: string,
offset: number,
limit: number,
): Result<
{
readonly item: { readonly text: string; readonly unit: "utf16-code-units" };
readonly coverage: AnalysisViewCoverage;
},
AnalysisError
> => {
const start = Math.min(offset, body.length);
const high = (n: number) => n >= 0xd800 && n <= 0xdbff;
const low = (n: number) => n >= 0xdc00 && n <= 0xdfff;
if (
start > 0 &&
start < body.length &&
high(body.charCodeAt(start - 1)) &&
low(body.charCodeAt(start))
)
return err(viewError("Pseudocode offset splits a UTF-16 surrogate pair."));
let end = Math.min(start + limit, body.length);
if (
end > start &&
end < body.length &&
high(body.charCodeAt(end - 1)) &&
low(body.charCodeAt(end))
)
end--;
if (end === start && start < body.length)
return err(
viewError(
"Increase pseudocode limit to include the next complete Unicode character.",
),
);
return ok({
item: { text: body.slice(start, end), unit: "utf16-code-units" },
coverage: pageViewCoverage(start, end - start, body.length),
});
};
const nativeRows = (
dossier: FunctionDossier,
facet: Exclude<
Extract<AnalysisViewRequest, { readonly kind: "native" }>["facet"],
"procedure" | "pseudocode" | "value_flow_summary"
>,
) => {
const sourceFlow = dossier.native_value_flow;
const flow = sourceFlow?.available ? sourceFlow : undefined;
return {
assembly: dossier.assembly,
callers: dossier.callers,
callees: dossier.callees,
incoming_references: dossier.incoming_references,
outgoing_references: dossier.outgoing_references,
value_flow_operations: flow?.operations,
value_flow_def_use: flow?.def_use,
value_flow_effects: flow?.effects,
value_flow_parameters: flow?.parameters,
value_flow_parameter_uses: flow?.parameter_uses,
}[facet];
};
/** Project bounded native function facets from complete retained Evidence without reanalysis. */
export const projectNativeFunctionView = (
parent: AnalysisViewParent,
dossier: FunctionDossier,
view: AnalysisViewRequest,
): Result<UnsignedAnalysisView, AnalysisError> => {
if (view.kind !== "native")
return err(viewError("analyze_function Evidence requires a native view."));
const artifact = parent.artifact;
if (!artifact)
return err(
analysisViewInputError([
{
path: ["source", "subject"],
reason: "invalid_value",
message:
"Native function Evidence requires an artifact-bound subject.",
},
]),
);
const common = parentFields(parent, dossier, artifact);
const singleton = (item: unknown): UnsignedAnalysisView => ({
kind: "native",
view,
item: jsonValueSchema.parse(item),
coverage: completeWithinViewCoverage(1, 1),
...common,
});
const flow = dossier.native_value_flow;
if (view.facet === "procedure" || view.facet === "value_flow_summary") {
if (view.offset !== 0)
return err(viewError("Singleton native facets require offset 0."));
if (view.facet === "procedure") return ok(singleton(dossier.procedure));
const status =
flow === null
? { available: false, reason: "not recorded" }
: !flow.available
? { available: false, reason: flow.reason }
: {
available: true,
provenance: flow.provenance,
truncated: flow.truncated,
omitted_operations_lower_bound:
flow.omitted_operations_lower_bound,
known_omitted_inputs: flow.known_omitted_inputs,
known_omitted_edges: flow.known_omitted_edges,
operations: flow.operations.length,
def_use: flow.def_use.length,
effects: flow.effects.length,
parameters: flow.parameters.length,
parameter_uses: flow.parameter_uses.length,
};
return ok(singleton(status));
}
if (view.facet === "pseudocode") {
const page = pseudocodePage(dossier.pseudocode, view.offset, view.limit);
return page.ok
? ok({ kind: "native", view, ...page.value, ...common })
: page;
}
const rows = nativeRows(dossier, view.facet);
if (rows === undefined)
return ok(
singleton({
available: false,
reason: flow?.available === false ? flow.reason : "not recorded",
}),
);
const items = rows.slice(view.offset, view.offset + view.limit);
return ok({
kind: "native",
view,
item: jsonValueSchema.parse(items),
coverage: pageViewCoverage(view.offset, items.length, rows.length),
...common,
});
};
+9 -28
View File
@@ -125,6 +125,8 @@ export const projectAndroidApplication = (
"Manifest, resource, signing, and bytecode semantics require a dedicated Android provider; this projection reports exact inventory paths and hashes only.",
"Runtime families are inferred from inventory formats and paths; filename suffixes do not establish valid DEX or JVM class bytes.",
"Bridge candidates are path-based hypotheses, not decoded JNI declarations or observed runtime calls.",
"A bridge basis is inferred from the native path and is repeated for every managed component.",
"ZIP inventory cannot see the APK Signing Block, so an empty signing array does not mean the APK is unsigned.",
...(bridgeProjection.coverage.status === "partial"
? [
`Bridge candidate pairs exceeded the projection safety budget; ${bridgeProjection.coverage.omitted_candidates} hypotheses are omitted. Component arrays still include every component from the supplied inventory pages.`,
@@ -157,23 +159,12 @@ export const projectAndroidApplication = (
};
const classify = (all: readonly Component[]) => ({
manifests: all.filter(
({ path, format }) =>
format === "android-manifest" ||
/(?:^|\/)AndroidManifest\.xml$/u.test(path),
),
resources: all.filter(
({ path, format }) =>
format === "android-resources" || /(?:^|\/)resources\.arsc$/u.test(path),
),
dex: all.filter(
({ path, format }) =>
format === "dex" || /(?:^|\/)[^/]+\.dex$/iu.test(path),
),
jvm_classes: all.filter(
({ path, format }) =>
format === "jvm-class" || /(?:^|\/)[^/]+\.class$/iu.test(path),
manifests: all.filter(({ path }) =>
/(?:^|\/)AndroidManifest\.xml$/u.test(path),
),
resources: all.filter(({ path }) => /(?:^|\/)resources\.arsc$/u.test(path)),
dex: all.filter(({ path }) => /(?:^|\/)[^/]+\.dex$/iu.test(path)),
jvm_classes: all.filter(({ path }) => /(?:^|\/)[^/]+\.class$/iu.test(path)),
native_libraries: all.filter(
({ path, format }) =>
format === "elf" || /(?:^|\/)lib\/[^/]+\/[^/]+\.so$/iu.test(path),
@@ -189,19 +180,9 @@ const runtimeFamilies = (all: readonly Component[]) => {
const families = new Set<
AndroidApplicationProjectionResult["runtime_families"][number]
>();
if (
all.some(
({ path, format }) =>
format === "dex" || path.toLowerCase().endsWith(".dex"),
)
)
if (all.some(({ path }) => path.toLowerCase().endsWith(".dex")))
families.add("dalvik-art");
if (
all.some(
({ path, format }) =>
format === "jvm-class" || path.toLowerCase().endsWith(".class"),
)
)
if (all.some(({ path }) => path.toLowerCase().endsWith(".class")))
families.add("java-kotlin");
if (all.some(({ format }) => format === "elf")) families.add("native");
if (all.some(({ format }) => format === "javascript-bundle"))

Some files were not shown because too many files have changed in this diff Show More