Compare commits

..
Author SHA1 Message Date
Kaoru0822andClaude Opus 5.5 7f348657e6 test(javascript): describe the parameter's provenance without claiming shadowing
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 09:09:25 +00:00
Kaoru0822andClaude Opus 5.5 fd768524d8 test(javascript): expect unknown provenance for a shadowing parameter
c7a890d12 deliberately keeps parameter provenance unknown ("Parameter
provenance depends on caller arguments.") instead of the previous
implicit local fallback, but the structure test still expected the
shadowing `ipcRenderer` parameter to be local, so it fails on main and
in every PR's domain shard. The test's purpose, that the parameter does
not inherit the module alias it shadows, is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 08:51:21 +00:00
morluto c02fb0ea80 fix(native): rank Mach-O slices by parsed CPU subtype 2026-10-13 16:43:12 +08:00
96fc84f1fb fix(native): select one slice when universal Mach-O slices share a CPU type (#2040)
* fix(native): select one slice when universal Mach-O slices share a CPU type

On macOS 27 most system binaries are universal files whose slices share
CPU_TYPE_ARM64: arm64e + arm64e.x1 (Calculator, TextEdit), x86_64 + arm64e +
arm64e.x1 (/bin/ls), or arm64 + arm64e.x1 (ShortcutDroplet). Target admission
names slices by CPU family, so these targets have availableArchitectures
["arm64"] while the file holds two arm64 slices.

inspect_macho read that family count as a slice count and ran otool, nm,
dyld_info, dwarfdump and vtool without -arch. ShortcutDroplet failed with
"Malformed otool load-command output" (40 load commands against ncmds 20).
Calculator reported both slices' imports (4138 instead of 2069), nm's
"(for architecture ...)" header lines as exports, and no selected slice.
selectArchitecture now also filters when lipo lists more than one slice. Its
preference is unchanged: the family name, then arm64e, then the first
arm64e.<variant>.

inspect_native_dispatch_metadata matched FAT slices by CPU type only and
refused /bin/ls and TextEdit as "Ambiguous FAT architecture slice".
selectMachoSlice now ranks slices of the requested CPU type by base subtype in
the same order: arm64 (0, then 1) or x86_64 (3), then arm64e (2), then other
variants such as arm64e.x1 in FAT table order. It reports ambiguity only when
the top-ranked slices share a subtype.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(native): share Mach-O slice preference across inspections

* chore: preserve upstream process capture test during merge

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-13 16:36:23 +08:00
b69d20a300 fix(captures): name the violated HAR schema constraint (#2708)
HAR validation reported only the AJV keyword, so a capture missing
`log.version` failed with "HAR schema validation failed: required." and
the caller could not tell which property to supply. Include AJV's
constraint message, which names the missing property, expected type, or
pattern without restating captured values.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 16:22:24 +08:00
AvocadoandGitHub 11f8b76257 fix(apktool): accept --no-strings CLI option (#2707) 2026-10-13 16:20:03 +08:00
morluto 11f1d952e7 ci: raise duplication threshold to 3.5 percent 2026-10-13 16:18:53 +08:00
morluto a0317cf732 fix(ci): align FAT fixtures and consolidate duplicated workflow helpers
Declare the synthetic 4 KiB Mach-O slice alignment accurately while preserving the IDA thinning scenario. Share web cleanup aggregation and process-trace fixture facts instead of duplicating them across consumers and tests; keep the 3% duplication threshold unchanged.

Validation: npm run check:ci, test discovery, and 42 focused IDA, process-trace, web workflow and cleanup boundary tests. Duplication falls from 3.01% to 2.99%.
2026-10-13 16:16:31 +08:00
morluto 67c7a24e60 Merge remote-tracking branch 'origin/main' 2026-10-13 16:08:58 +08:00
morluto c7a890d129 fix(javascript): preserve unknown module-alias provenance
Replace the implicit local-provenance fallback with explicit locally constructed values. Keep opaque results, parameters and caught values unknown; follow assignment and sequence results while retaining exact module aliases. Clarify that alias provenance does not represent transitive data taint.

Validation: 26 focused provenance/module-link tests, 135 semantic and graph-builder tests, six application module-relationship boundary tests, and the fast typecheck/lint gate.
2026-10-13 16:08:41 +08:00
morluto b955b825fc fix(artifacts): retain read cleanup ownership through CLI and MCP shutdown
Keep failed stable-read handles and WABT runtime roots under retryable resource scopes. Preserve cancellation causes and primary errors, close the same service instances bound to MCP tools, and consolidate CLI operation/cleanup projection.

Validation: fast typecheck and lint, cached runtime build, docs checks, SDK schema validation, focused service and real-file cleanup tests, CLI cleanup regressions, and MCP lifecycle/shutdown workflows. Provider process fixtures simulate engine output; real WABT and Linux EVM execution are not established by this pass.
2026-10-13 16:08:41 +08:00
9cf49c7a78 fix(snapshot): classify non-object roots as invalid content (#2685)
Co-authored-by: jiangjiang248 <237556902+jiangjiang248@users.noreply.github.com>
2026-10-13 16:04:44 +08:00
morluto c923fb0dc9 fix(javascript): retain compiler-dependent module resolution facts
Compare import and require outcomes for static TypeScript references when compiler options are unknown. Preserve branch facts through worker IR and both graph projections; retain declaration fallbacks as candidates without changing explicitly selected paths.

Validation: 210 focused resolver and semantic tests; 21 production-boundary and source-extension tests. A bounded 2,000-reference probe adds about 16–17 ms wall time for dual resolution; full task token costs and real compiled application behavior remain unmeasured.
2026-10-13 15:58:25 +08:00
morluto ded199abcd Merge remote-tracking branch 'origin/main' 2026-10-13 15:25:24 +08:00
morluto ed881c6d2e fix(progress): measure delivery intervals with a monotonic clock 2026-10-13 15:23:56 +08:00
morluto eaa9616585 fix(javascript): preserve cancellation through recovery publication cleanup 2026-10-13 15:19:18 +08:00
bf34d0d2fc perf(native): add opt-in Mach-O summary delivery and retained pages (#2634)
* perf(native): add opt-in Mach-O summary delivery and retained pages

inspect_macho returns every import and export inline. For VS Code's
Electron Framework one MCP call carries 2.5 MB of text (about 828k o200k
tokens), and the same again as structuredContent; the MCP response
budget admits it. Agents asking for slice identity, UUID or dependency
counts paid that cost on every first call.

Add the established opt-in detail selection: omitted or complete keeps
the existing complete result and Evidence identity, while summary
retains the complete Evidence and returns slice identity with
per-collection returned/total/exhaustive counts (956 tokens).
inspect_analysis_view kind macho pages the original rows of any
collection or the command provenance, repeating the producer's own
total and exhaustive flag so page exhaustion is never read as absence.

Delivery detail never reaches the native tools or the source Evidence
ID: registerEvidenceTools gains a parametersForEvidence hook, so the
summary's parent is the same Evidence as a complete call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(mcp): refuse unretainable summary delivery before analysis

A server without Evidence retention (a fixed analysis source) ran the
complete inspect_macho or inspect_artifact analysis for detail summary
and only then refused it in summarizeRetainedAnalysis, paying the very
cost summary delivery exists to avoid. registerEvidenceTools gains a
preflight hook; both registrations check retention there, as the WASM,
EVM, managed, pcap and capture handlers already do before dispatch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(native): keep whole rows in Mach-O architecture and segment pages

The page item union listed plain-object row arrays, and zod returns the
first union member that parses while dropping unknown keys. Entrypoint
rows declare only file_offset, so every architecture and segment row
(which also carry a numeric file_offset) matched it and lost its name,
CPU fields, permissions and sections. Rows now use exact key sets, and
imports/exports share one member. The MCP test pages every facet of a
real /bin/ls and compares it with the unparsed complete rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 00:13:12 -07:00
morluto e802441bce Merge remote-tracking branch 'origin/main' 2026-10-13 15:12:34 +08:00
morluto e7882b34f4 fix(artifacts): honor cancellation at the output publication boundary 2026-10-13 15:12:17 +08:00
morluto 9049dbaf0c fix(process): preserve monotonic capture timing and observed wall dates 2026-10-13 15:12:08 +08:00
morluto 7fc0b5ec9c fix(mcp): join owned doctor shutdown without signaling saved PIDs 2026-10-13 15:12:08 +08:00
6a05248d46 perf(frida): evict captured messages without copying the whole queue (#2628)
Once a Frida session reached its 16 MiB capture budget, every new message
evicted the oldest one with Array#shift, re-serialized it to recover its
byte charge, and searched and spliced its per-script array. Each of
those is linear in the retained messages, so with about 188k small
messages retained the cost per message rose from 0.14 µs to 32 µs and
capped the main thread at roughly 31k messages per second, which a
hooked hot function can exceed.

Keep both queues as head-offset FIFOs that compact once half their slots
are evicted, and record each message's charge when it is admitted.
Arrival order already makes the globally oldest message the oldest one
of its script, so per-script eviction becomes a head check. Eviction is
now amortized O(1) at about 0.14 µs per message; retained messages,
their order and the truncation flag are unchanged.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 00:11:38 -07:00
20f51d7286 fix(native): name the ownership helper prerequisite in command failures (#2506)
* fix(native): name the ownership helper prerequisite in command failures

Every supervised native command launch needs the Darwin process-ownership
helper. When that helper cannot be compiled (relative REA_DARWIN_SWIFTC_PATH,
missing SDKROOT, a missing compiler, or an SDK/compiler mismatch), the
runner reduced the typed DarwinProcessOwnershipInspectionError to a bare
`io` failure. inspect_macho then reported `execution_failure` with no hint,
and inspect_artifact on any universal Mach-O said only "lipo could not
enumerate universal Mach-O slices".

The runner now keeps the inspection message as the failure's `detail` at
the launch boundary and carries it through xcrun resolution. The native
provider adds it to its diagnostics and the slice reader appends it to its
message, matching what the SQLite and capture providers already report.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(native): cover the provider's ownership helper diagnostics

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 00:10:58 -07:00
morluto 6bd2b28161 fix(browser): keep elapsed capture timing independent of wall-clock changes 2026-10-13 15:02:54 +08:00
morluto d9e9fb2273 fix(process): use monotonic time for provider execution budgets 2026-10-13 15:01:59 +08:00
morluto b953f96c0a fix(ghidra): share Java properties semantics with PSP extension admission 2026-10-13 15:00:14 +08:00
OlorinandGitHub 4276aaf18b feat(signals): discover installed decoders and option help (#2487)
* feat(signals): discover installed decoders and selected option help

* fix(signals): bound retained discovery data and preserve resource failures

* fix(signals): complete CLI help and portable decoder patterns

* test(signals): verify case-insensitive decoder help queries

* fix(signals): retain discovery operation across cleanup retries
2026-10-13 15:58:19 +09:00
morluto fe8d0bdd36 fix(android): reject malformed UTF-8 at the JADX protocol boundary 2026-10-13 14:56:04 +08:00
9c34c091b3 feat(captures): discover installed protocol and field definitions (#2496)
* feat(captures): discover installed protocol and field definitions

* test(captures): isolate the direct glossary oracle profile

* test(captures): accept padded kernel limit reports

* fix(captures): detach every retained glossary column

* fix(captures): describe every field discovery CLI option

* fix(captures): retain discovery operation across cleanup retries

---------

Co-authored-by: N0zoM1z0 <161784452+N0zoM1z0@users.noreply.github.com>
2026-10-13 15:55:25 +09:00
Rudy Mizrahi CelekliandGitHub b9ee921560 fix(process): preserve nested path identities during normalization (#2468) 2026-10-13 15:54:18 +09:00
15463bdbe6 fix(adb): count requested permissions that the device marks restricted (#2359)
Android 10 and 11 print restricted entries of the requested-permissions
list as "NAME: restricted=true". The parser took the first such line as
the end of the list, so the count stopped there while coverage still
read complete.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 14:52:48 +08:00
morluto 55e21a2eec Merge remote-tracking branch 'origin/main' 2026-10-13 14:51:59 +08:00
morluto f6775a3669 fix(setup): preserve current recovery snapshots and decoding failures 2026-10-13 14:51:55 +08:00
Rudy Mizrahi CelekliandGitHub a07ceaec36 fix(config): reject NUL bytes in Hopper loader arguments (#2474) 2026-10-13 14:51:29 +08:00
Rudy Mizrahi CelekliandGitHub 6a04f922e1 fix(javascript): parse explicit CommonJS wrapper syntax (#2475) 2026-10-13 14:51:23 +08:00
Rudy Mizrahi CelekliandGitHub e74e4e951b fix(screenshot): reject invalid PNG header dimensions (#2472) 2026-10-13 15:50:44 +09:00
68ecf3df8c fix(browser): exclude sensitive binary lengths from historical captures (#2477)
Co-authored-by: chAngE <172472630+chAng-L19@users.noreply.github.com>
2026-10-13 14:49:47 +08:00
ab62ee9dcf fix(windows): preserve caller affinity only within one processor group (#2202)
* fix(windows): preserve caller affinity only within one processor group

`spawnProcess` read the caller's affinity mask and applied it to the
suspended child unconditionally. A mask names processors relative to one
processor group, and Windows assigns each new process to a group of its
own choosing, so on hosts with more than 64 logical processors the child
is often placed in another group. `SetProcessAffinityMask` then rejects
the mask with ERROR_INVALID_PARAMETER (87) and the spawn fails, making
every owned provider process start intermittently on such hosts.

Affinity is a nicety, not an authority or isolation control, so it is now
applied only when the caller and the suspended child both occupy the same
single processor group. A child placed in another group keeps that group's
default affinity and the spawn proceeds. Job Object membership,
kill-on-owner-close policy, owner normalization and priority preservation
are unchanged, and the mask failure remains fatal within a shared group,
where it is not expected.

Single-group hosts, including every host with 64 or fewer logical
processors, are unaffected: `GetProcessGroupAffinity` reports one group
for both processes and the same mask is applied as before.

The optional process-boundary fixture gains an `affinity` observation,
and the native verification lane asserts that an owned spawn succeeds and
that the child's mask matches the caller's whenever both share a single
group.

Fixes #2181

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(windows): refuse to widen a pinned caller's affinity across groups

Addresses review feedback on the first commit.

An unconstrained caller loses nothing when Windows places an owned child
in another processor group, so that spawn proceeds with the group's
default. A caller deliberately pinned to fewer processors than its group
holds is a different case: silently accepting the default would hand the
provider tree the whole group, which `docs/installation.md` tells
operators to rely on affinity to prevent. Such a spawn now fails naming
the constraint instead. `GetProcessAffinityMask` already reports the
group's active set, which distinguishes the two without another query.

Selecting the caller's group at creation through
PROC_THREAD_ATTRIBUTE_GROUP_AFFINITY was tried first and rejected:
`CreateProcessW` fails with ERROR_INVALID_PARAMETER on a host with a
single processor group, both alongside PROC_THREAD_ATTRIBUTE_JOB_LIST and
on its own, so it cannot carry the ordinary path.

The boundary fixture's affinity observation now reports a process id's
own mask rather than its own, because a separate probe process is
assigned its own group and cannot stand in for the process that spawns
the child. Masks and the group's active set cross as fixed-width
hexadecimal strings: a mask can set bits above the safe-integer range,
where `JSON.parse` would round two different masks to one value.

The verification lane asserts the mask is inherited exactly when caller
and child share a group, and otherwise that the caller was unconstrained.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(windows): record what owned-process affinity does not preserve

A mask covering its group's whole active set does not distinguish a
caller the system placed in that group from one that selected the group
deliberately, so an owned child can still be placed elsewhere with its
processor count bounded and its group placement lost. Treating that case
as constrained instead is not available: on a multi-group host an
ordinary process is assigned a group by the system and its mask is that
group's active set, so refusing it would fail every owned spawn there.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-13 14:49:34 +08:00
2c6a4fac4f fix(process): accept a trace identifier the comparison result can represent (#2488)
The trace specification owns the declared identifier vocabulary and the
comparison result echoes those identifiers verbatim into raw_trace[].event_id,
matched_variant and diagnostic event_ids. The result module restated the shared
IDENTIFIER_PATTERN by hand with a 64-character cap, so a specification with a
longer identifier validated and then its own comparison result failed to parse:
compareProcessTraces threw a raw ZodError at the caller instead of returning a
comparison.

Export the identifier schema from the specification as the single owner and
reuse it in the evaluation module, removing the capped copy. IDENTIFIER_PATTERN
has no length bound and every other consumer uses it unbounded, so the
duplicate was the outlier. Removing the bound rather than adding one keeps the
advertised trace_spec input contract unchanged; identifiers of 64 characters or
fewer behave exactly as before.

Verified with 48 process tests, 24 boundary CLI/MCP trace tests, check:fast and
verify:test-discovery on macOS arm64. The regression test failed with the
ZodError before the change and passes after it.

Co-authored-by: 吴佳浩 <wujiahao@supremind.com>
2026-10-13 14:48:26 +08:00
Mykola DementievandGitHub c6acd19b6d fix(ida): stop the private input of a universal Mach-O at the end of the selected slice (#2291)
* test(ida): expect the private input of a universal Mach-O to end at the selected slice

Cover a universal file larger than one read chunk with another slice after the selected one, for both a middle slice and the first slice.

* fix(ida): stop the private input of a universal Mach-O at the end of the selected slice

A read chunk that starts after the slice produced a negative end offset, and Buffer.subarray counts a negative bound from the end, so the head of the following slice was written to the private input and its digest.
2026-10-13 14:48:17 +08:00
1a1b67c888 fix(javascript): preserve POSIX runtime path case on Windows (#2422)
Use POSIX path semantics for recorded POSIX roots when reconciling runtime Evidence on Windows. Preserve Windows mapping behavior and cover implicit and explicit file roots through the reconciliation service.

Assisted-by: OpenAI GPT-6 <noreply@openai.com>

Co-authored-by: Aurantium-47 <280805072+Aurantium-47@users.noreply.github.com>
2026-10-13 14:46:28 +08:00
Rohit GhumareandGitHub 12303812a8 fix(evidence): admit bundle imports across session transitions (#2303) 2026-10-13 14:45:03 +08:00
f6556ad2bc fix(pcap): order evidence ties by code point instead of host locale (#2509)
The pcap summary sorted equal-count endpoints and protocol stacks with
String#localeCompare, and Flutter build inspection sorted APK ABI
directory names the same way. localeCompare follows the host's ICU
locale, so the same capture produced differently ordered Evidence on
different hosts: under da_DK "aarp" sorts after "zz", under lt_LT
"syslog" sorts before "ssdp", and every locale puts "fe80::1" before
"fe80:1::1". Use the repository's compareUnicodeCodePoints, as the
other Evidence producers already do.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 14:44:09 +08:00
morluto 5f0f25ad61 Merge remote-tracking branch 'origin/main' 2026-10-13 14:43:23 +08:00
d73ed6fce2 fix(domain): preserve retryable capability failures through redaction (#2494)
redactExplicitFailure rebuilds each typed failure field by field so explicit
literals are removed while the projected code stays the same. The rebuild for
AnalysisCapabilityUnavailableError carried providerId, operation, reason and
userMessage but not retryable, and the constructor defaults that field to false.
Redacting a transient capability failure therefore made it permanent and changed
the remediation shown to the caller.

Pass the field through with the same conditional-spread shape used for
userMessage. The constructor recomputes retryable from cleanupIncomplete, and
cleanup is already part of the reconstructed context, so a value that was true
before redaction stays true and a false value stays absent.

Verified with 66 error projection, access-denied, redaction and capture-failure
tests, check:fast and verify:test-discovery on macOS arm64. The regression test
failed with "expected false to be true" before the change.

Co-authored-by: 吴佳浩 <wujiahao@supremind.com>
2026-10-13 14:43:10 +08:00
Vinícius AndradeandGitHub 60f771a11e fix(apple): limit already-loaded reuse to rpath dependencies (#2292)
* fix(apple): limit already-loaded reuse to rpath dependencies

dyld reuses an already-loaded image by install name only for @rpath/
dependencies (Loader.cpp:802-816) and evaluates other dependency forms
against their expanded path for each loader. When multiple images in
different directories depend on @loader_path/ libraries sharing an
install name, checking loaded images before candidate expansion caused
subsequent loaders to bind to the first loaded image's copy or appear
resolved when their own copy was absent.

Limit byInstallName registration and lookup to install names that start
with @rpath/, leaving caller-relative and literal dependencies to expand
and evaluate their own candidate paths.

Signed-off-by: Vinícius DS Andrade <95101505+viniciusdsandrade@users.noreply.github.com>

* test(apple): verify loader_path dependencies across isolated directories

Assert that multiple images depending on @loader_path/ libraries with
the same install name do not reuse already-loaded images across loaders.
When the second loader's helper is absent, its candidate path is absent
and dyld reports required-load-unresolved; when both helpers exist, each
loader resolves to its own directory.

Signed-off-by: Vinícius DS Andrade <95101505+viniciusdsandrade@users.noreply.github.com>

---------

Signed-off-by: Vinícius DS Andrade <95101505+viniciusdsandrade@users.noreply.github.com>
2026-10-13 14:40:43 +08:00
The one -NiuandGitHub 650237efb0 fix(setup): pin Hopper's macOS publisher identity before install (#2481)
The macOS installer accepted any bundle that passed `codesign --verify`
and whose `Info.plist` said `com.cryptic-apps.hopper-web-4`. An ad-hoc
signed bundle with that identifier satisfies both, so a substituted DMG
could be copied to ~/Applications and launched.

Verify a pinned code requirement instead, covering the sealed bundle
identifier and the signing team together, and report a distinct
`publisher_verification_failed` code so a vendor key change is not
confused with a corrupt package.
2026-10-13 14:40:04 +08:00
morluto 26bffb94bf perf(evidence): reuse authenticated comparisons through aggregation 2026-10-13 14:39:49 +08:00
7017c4d68c fix(artifacts): read every ZIP package family extraction is asked to open (#2493)
extract_artifact carried its own copy of the ZIP package format list with five
of the eight members the domain owner declares, and used it both as the guard
that refuses a format without an extraction reader and as the selector for the
reader itself. A .hap, .hsp or App Pack target was therefore refused with
"Artifact format has no extraction reader" even though the next branch builds a
ZipArtifactReader for exactly those formats, while inventory_artifact classifies
the same path through the owner and reads it.

Import isZipPackageFormat from the domain owner and delete the duplicate list
and its predicate, removing the second representation instead of reconciling it.
The reader branch now narrows to ZipPackageFormat, which is what the
ZipArtifactReader constructor already requires.

Verified with 41 boundary extraction, ZIP integrity and detection tests,
check:fast and verify:test-discovery on macOS arm64. Reverting only the source
change fails exactly the three added families and leaves the zip control green.

Co-authored-by: 吴佳浩 <wujiahao@supremind.com>
2026-10-13 14:39:29 +08:00
141d2095f5 test(hopper): repair Linux demo setup and native verification (#2638)
* fix(verification): align Linux Hopper setup lane with current contracts

* fix(verification): retain orphan checks on clean Linux runners

* fix(verification): detect Hopper through alternate launcher paths

* fix(verification): report identified Hopper leaks before proc errors

* fix(verification): distinguish Hopper string recognition from byte fidelity

* fix(verification): report full literal decoding coverage explicitly

---------

Co-authored-by: Luis Taffelli <86675342+LuisTaffelli@users.noreply.github.com>
2026-10-13 14:39:18 +08:00
morluto 1b953021ea fix(javascript): centralize module syntax and separate erased type dependencies 2026-10-13 14:39:06 +08:00
morluto e1cd292aa6 fix(adb): admit canonical device states and preserve malformed output 2026-10-13 14:38:39 +08:00
YsopwnableandGitHub 18831491cf fix(ci): ignore generated website translation staging (#2511)
Ignore generator-owned website translation staging output while preserving authored lint errors and complete diagnostics.

Verified with a real checker regression, a production translation build, complete coverage, developer/CI regression checks and native lanes.

Fixes #2508
2026-10-13 14:32:22 +08:00
260 changed files with 10059 additions and 1960 deletions
@@ -76,6 +76,9 @@ explicit path or endpoint and do not need it.
Facts retain all-recorded-events-before-selection scope. Other qlog layouts
and PCAP correlation are unsupported.
- User-owned Electron runtime already open: `list_electron_targets`.
- Packet-field discovery: `inspect_packet_fields` searches installed protocols
and field names/types before choosing fields for `inspect_packet_capture`.
Query/page the relevant records; registrations do not establish observed traffic.
- Recorded packet captures: `inspect_packet_capture` with selected TShark fields,
display filters and decode-as rules. Requires Linux, util-linux prlimit and `REA_TSHARK_COMMAND`; see
[capture guide](https://github.com/morluto/rea/blob/main/docs/packet-captures.md).
@@ -89,6 +92,9 @@ explicit path or endpoint and do not need it.
[packet summaries](https://github.com/morluto/rea/blob/main/docs/pcap-analysis.md).
Select an exact DNS question for address pages; page exhaustion does not prove
network completeness. This reads recordings and does not acquire traffic.
- Decoder discovery: `inspect_signal_decoders` searches installed decoder IDs
or selected decoder help, including channels and options. Query and page the
relevant details before choosing settings; this does not scan hardware.
- Recorded logic/signal traces: `decode_signal_recording` with explicit decoder,
channel and protocol settings. On Linux with util-linux prlimit, supply sigrok-cli 0.7.2 through
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/signal-recordings.md).
@@ -83,6 +83,14 @@ names, the native API record, and value-flow summaries. Pseudocode offsets and l
code units; use the returned `next_offset` for continuation. Provider limitations
and unavailable facts remain explicit; the complete parent Evidence is retained.
`inspect_macho` on a large framework returns megabytes of imports and exports.
Select `detail: "summary"` for slice identity, UUID and per-collection counts,
then page rows with `view: {kind: "macho", facet: "exports", offset: 0, limit: 64}`
and `normalized_result.parent_evidence_id`. Facets are entrypoints,
architectures, build_metadata, load_commands, dependencies, imports, exports,
segments and provenance. `source_exhaustive: false` means unreturned symbols
may exist.
To see which functions or Objective-C methods a Mach-O actually calls in one
run, use `observe_native_calls` with explicit breakpoints and a bounded
`duration_ms`/`max_events`. It launches a new process under LLDB, so the target
+1 -1
View File
@@ -51,7 +51,7 @@ jobs:
- run: sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y tshark util-linux python3
- run: npm ci
- run: npm run build:cached
- run: npm run test:focused -- src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts
- run: npm run test:focused -- tests/boundary/captures/fieldDiscovery.test.ts src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts
- run: node scripts/verify-packet-capture.mjs
env:
REA_TSHARK_COMMAND: /usr/bin/tshark
+1 -1
View File
@@ -44,7 +44,7 @@ jobs:
- run: sudo apt-get update && sudo apt-get install -y sigrok-cli util-linux python3
- run: npm ci
- run: npm run build:cached
- run: npm run test:focused -- src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts
- run: npm run test:focused -- tests/boundary/signals/decoderDiscovery.test.ts src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts
- run: node scripts/verify-signal-recording.mjs
env:
REA_SIGROK_COMMAND: /usr/bin/sigrok-cli
+2
View File
@@ -72,3 +72,5 @@ Thumbs.db
# Complete static website output, including generated language editions.
website/_site/
# Temporary translation output prepared before replacing the complete site.
website/.i18n-*/
+1 -1
View File
@@ -1,5 +1,5 @@
{
"threshold": 3.0,
"threshold": 3.5,
"reporters": ["consoleFull"],
"ignore": [
"node_modules/**",
+1 -1
View File
@@ -93,7 +93,7 @@ def project_record(state, secrets):
safe_bytes, credential_url = transport_value(value, path, transport_url)
content_base64 = base64.b64encode(value).decode("ascii")
digest = hashlib.sha256(value).hexdigest()
hidden = any(secret in value or secret in safe_bytes for secret in byte_secrets) or any(secret in content_base64 or secret in digest for secret in secrets)
hidden = any(secret in value or secret in safe_bytes for secret in byte_secrets) or any(secret in content_base64 or secret in digest or secret in str(len(value)) for secret in secrets)
safe_text = None
try:
safe_text = safe_bytes.decode("utf-8", errors="strict")
+1 -1
View File
@@ -174,7 +174,7 @@ Resolution follows dyld:
- `@rpath` tries each `LC_RPATH` of the loading image first, then those of the
image that loaded it, up to the process root.
- Dependencies load dependents-first in load-command order, and each image
loads once per process. A later request whose install name matches an
loads once per process. A later @rpath request whose install name matches an
already loaded image reuses it.
- Symlinks such as `Versions/Current` are followed inside the analyzed root.
An escape through `..` or an absolute link is `escapes-target`.
+4
View File
@@ -57,6 +57,10 @@ Hopper installation or any file write. Declining or pressing Ctrl-C makes no
changes. Agent configuration writes preserve unrelated entries and comments,
create backups, use atomic replacement, and verify their result. Setup and
uninstall retain an existing `.rea.backup` rather than replacing the first snapshot.
If that snapshot differs from the current configuration, setup previews and
preserves a verified `.rea.backup.<sha256>.recovery` copy of the current bytes.
Both snapshots remain available; a failed rollback reports the current recovery
copy's path and the restore failure.
Setup also retains valid environment overrides in REA's existing server entry,
such as `REA_MCP_INPUT_SCHEMA_PROFILE` and `REA_MCP_MAX_RESPONSE_BYTES`.
Detected provider settings replace only matching environment keys.
+9
View File
@@ -104,6 +104,15 @@ static inferences. Dynamic or ambiguous IPC is retained without a pairing edge.
Runtime entities retain passive-CDP authority, while a successful static/runtime
correspondence is a separately identified cross-layer inference.
Source module links retain their resolution mode. Static imports in `.ts` and
`.tsx` are `compiler-dependent` when compiler options have not been retained.
REA compares Node import and require resolution: a shared outcome remains
available with a limitation; divergent outcomes remain unavailable with both
conditional branches and their limitations. This models source relationships,
not verified emitted code or execution. A declaration file found only through
an emitted-extension fallback is retained as a candidate, not a runtime target;
an explicitly selected path keeps its original identity.
## Evidence and authority
Every observation and edge carries all of the following:
+8
View File
@@ -23,6 +23,14 @@ artifact digest and structural graph ID. Semantic tracing requires the semantic
relation graph and reports when it is unavailable rather than treating missing
data as an empty graph.
Explicit TypeScript `import type` and `export type` declarations remain source
facts and do not create runtime import or export relationships. Type export names
are reported separately from value export names. Inline `type` specifiers can
leave an empty import or re-export with side effects under
[`verbatimModuleSyntax`](https://www.typescriptlang.org/tsconfig/verbatimModuleSyntax.html);
REA retains those dependency candidates and reports the unavailable compiler
options rather than assuming the dependency is erased.
The semantic graph wire format stores shared non-location provenance once in
its required `evidence_contexts` table. Each node, relation, fingerprint, and
unknown retains its exact `evidence.location` and names the owning context with
+15 -2
View File
@@ -147,8 +147,8 @@ after target content was read.
## Retained application Evidence inputs
`analyze_javascript_application`, `inspect_binary_layout`, `inspect_artifact`, and
`inspect_wasm_artifact` accept `detail`:
`analyze_javascript_application`, `inspect_binary_layout`, `inspect_artifact`,
`inspect_wasm_artifact`, and `inspect_macho` accept `detail`:
`complete` (default) returns the complete analysis Evidence; `summary` records
that complete Evidence in the session and returns the derived
`inspect_analysis_view` summary Evidence instead. Its
@@ -218,6 +218,19 @@ attempting oversized delivery, and returns the same summary with an exact
`parent_evidence_id` for later export or facet reads. Delivery detail does not
reach WABT or change the original source Evidence ID.
Completed `inspect_macho` Evidence supports a summary and
`view: {kind: "macho", facet: "exports", offset: 0, limit: 64}`. Facets are
`entrypoints`, `architectures`, `build_metadata`, `load_commands`,
`dependencies`, `imports`, `exports`, `segments`, and `provenance`; pages return
the original rows. The summary keeps the selected slice, header fields and UUID,
and replaces each collection with its returned count, reported total and
`exhaustive` flag. Each page repeats the paged collection's `source_total` and
`source_exhaustive`; exhausting a non-exhaustive collection does not establish
that unreturned symbols are absent. Large frameworks return megabytes of
symbols (Electron Framework: 2.5 MB of imports and exports, 3.3 KB as a
summary), so select `detail: "summary"` on the first call. Delivery detail
never reaches the native tools or changes the source Evidence ID.
`trace_application_feature`, `trace_javascript_semantics`,
`compare_application_versions`, `compare_source_to_bundle`, and
`compare_javascript_export_shapes` accept complete inline application Evidence
+46
View File
@@ -128,3 +128,49 @@ Each provider admits one inspection at a time, including acquisition and cleanup
An occupied provider returns a retryable capacity error before acquiring another
file or worker. Closing the provider stops new admission and drains accepted work;
failed cleanup remains owned and can be retried.
## Discover protocols and field names
`inspect_packet_fields` / `inspect-packet-fields` queries the installed TShark
registration glossary before selecting fields for capture inspection:
```sh
rea inspect-packet-fields --query usb
rea inspect-packet-fields --kind fields --protocol usb --query idVendor
rea inspect-packet-fields --kind fields --protocol udp --all
```
`kind` defaults to `protocols`; `fields` selects field definitions. `protocol`
selects an exact protocol abbreviation (the protocol itself or its child fields).
`query` is a case-insensitive literal substring in name, abbreviation or field
description. `offset` skips matching records; `limit` defaults to 50. MCP
`limit: null` or CLI `--all` requests all remaining matches within the resource
budget. `--all` and `--limit` conflict. The window reports examined records of
the selected kind before filters, matching records, returned records and
`has_more`; it never counts capture packets.
Returned records preserve field type, parent protocol, display base, bitmask,
description and original glossary line number. Empty producer strings remain
empty; protocol records have null field-only properties. Repeated abbreviations
remain separate registrations. A listed field is an installation capability, not
proof that the field appears in a selected recording.
The adapter uses the documented `tshark -G fields` grammar under its existing
private empty profile and Linux process limits. TShark 4.6.9's full glossary was
measured at 22,303,197 bytes; discovery admits at most **32 MiB** combined command
output (capture inspection remains at **4 MiB**). Parsing walks lines once and
constructs objects only for the selected page. Returned strings are copied so
retained pages do not retain the complete glossary string. Before constructing a selected
record it charges six times the source UTF-8 bytes plus 512 bytes against an
**8 MiB** representation budget, covering JSON escaping and record bookkeeping.
If that budget would be exceeded, narrow the query or page size; results are not
silently truncated. This is a conservative admission budget, not measured heap
usage. Parent pipe buffers and producer per-process limits remain distinct.
Tool/limiter identity, version, stderr and selected arguments are retained in
Evidence. No capture, interface or device is opened. Discovery shares capture
analysis's admission slot and cleanup owner. Each call rereads the glossary;
installed plugins, libraries and global configuration remain trusted and are not
fully fingerprinted. There is no cache or snapshot-consistent pagination claim.
The real Linux lane checks public CLI/MCP queries against the direct glossary,
including USB vendor-ID typing, negative queries and paging.
+34
View File
@@ -106,3 +106,37 @@ One decode is admitted at a time, including acquisition and cleanup. Excess
requests receive a retryable capacity error before acquiring another file or
worker. Shutdown closes admission and drains accepted work; failed cleanup
remains owned and retryable.
## Discover decoders before choosing settings
`inspect_signal_decoders` / `inspect-signal-decoders` queries the **installed**
sigrok catalog without opening a recording or scanning devices:
```sh
rea inspect-signal-decoders --query uart
rea inspect-signal-decoders --decoder uart --query baud
rea inspect-signal-decoders --decoder i2c --offset 0 --limit 50
```
Omit `decoder` to search decoder IDs and names. Supply one exact decoder ID to
read its literal help, including required/optional channels, options, annotation
classes and documentation. `query` is a case-insensitive literal substring;
matching help retains original one-based line numbers. Empty query selects all.
`offset` skips matches; `limit` defaults to 50. MCP `limit: null` or CLI `--all`
returns all remaining matches; CLI `--all` and `--limit` conflict.
The window reports examined, matching and returned records and `has_more`.
A filtered empty page is not evidence that the installation has no decoders.
Discovery refuses more than 65,535 line separators before constructing record
objects (65,536-line representation budget), in addition to the 4 MiB output cap.
Exceeding that budget is a resource failure with captured output. Returned text
is copied so a retained page does not keep the complete producer output alive.
Discovery shares decoding's owned process, admission slot, resource limits,
cleanup retries and Evidence delivery. It records the executable and limiter
identities, version banner and warnings. Failed version checks and malformed
producer output preserve diagnostics. Help is producer text, not an independently
validated parameter schema. Each request probes again; it does not cache or claim
stable pages across installation changes. Libraries/decoder modules remain
trusted and are not fully fingerprinted. Catalog output includes a SHA-256 of
its executable, not a digest of the complete installation.
+6 -2
View File
@@ -153,8 +153,12 @@ the private runtime, isolating Ghidra preferences, logs, and caches.
Process creation assigns a suspended child atomically to an owned Job Object.
Membership and kill-on-owner-close policy are verified before resume.
Breakaway is disabled. Only the child's default file owner is normalized;
the caller token stays unchanged. Caller CPU affinity and priority are
preserved. Close, cancellation, timeout, startup failure, and provider failure
the caller token stays unchanged. Caller priority is preserved, and caller CPU
affinity whenever the caller and the child share a processor group. A child that
Windows places in another group keeps that group's default when the caller asked
for no bound of its own; a caller pinned to fewer processors than its group
holds is refused there rather than widened.
Close, cancellation, timeout, startup failure, and provider failure
terminate the job and verify all members have settled before runtime cleanup.
Cleanup removes admitted objects by handle and unlinks reparse entries without
following their targets. Abrupt owner death kills the job but can leave
+16 -4
View File
@@ -69,10 +69,22 @@ without this lane reports Windows native controls as unavailable.
limits only for its JavaScript helpers when running on Bun, which ignores
their V8 heap flags; Node.js launches are unchanged.
- Only the suspended child's default file owner is normalized. Caller token
privileges and ownership remain unchanged. Caller CPU affinity and process
priority are preserved in the child. Environment names are sorted and
deduplicated using Windows ordinal comparison, preserving their native Unicode
semantics and caller-selected values.
privileges and ownership remain unchanged. Caller process priority is
preserved in the child, and caller CPU affinity whenever both occupy the same
processor group. Windows chooses a new process's group itself and rejects a
mask belonging to another group, so a child placed elsewhere keeps that
group's default when the caller asked for no bound of its own. A caller
deliberately pinned to fewer processors than its group holds is a constraint
such a child cannot express, and is refused rather than widened to that
group's default. A mask covering its group's whole active set does not
distinguish a caller Windows placed in that group from one that selected the
group deliberately, so such a child can still be placed elsewhere: its
processor count stays bounded, its group placement is not preserved, and
neither that case nor the refusal above has a verification lane, because both
need a host whose processors span more than one group.
Environment names are sorted and deduplicated using Windows
ordinal comparison, preserving their native Unicode semantics and
caller-selected values.
- Cleanup walks locked directory objects, removes files by handle, and unlinks
reparse entries without traversing their targets. Failed removal retains the
root and ancestor leases for retry; a partially removed runtime accepts only
+29 -1
View File
@@ -68,6 +68,18 @@ static ULONGLONG jobMemoryLimit(napi_env env, napi_value value) {
return limit;
}
// An affinity mask is meaningful only inside one processor group, so only a
// process occupying exactly one group has a group a child can inherit. A
// process whose threads span groups, or whose membership cannot be read, has no
// single group.
static bool singleProcessorGroup(HANDLE process, USHORT& group) {
USHORT count = 1;
std::array<USHORT, 1> groups{};
if (!GetProcessGroupAffinity(process, &count, groups.data()) || count != 1) return false;
group = groups[0];
return true;
}
static void pipe(Handle& read, Handle& write, bool parentReads = true) {
SECURITY_ATTRIBUTES attributes{sizeof(SECURITY_ATTRIBUTES), nullptr, TRUE};
HANDLE rawRead, rawWrite;
@@ -168,11 +180,27 @@ std::unique_ptr<Process> spawnProcess(const std::wstring& command, const std::ws
TOKEN_OWNER owner{reinterpret_cast<TOKEN_USER*>(tokenUser.data())->User.Sid};
require(SetTokenInformation(token.get(), TokenOwner, &owner, sizeof(owner)),
"Set suspended process file owner failed", command);
// A mask names processors inside one processor group, and Windows assigns a
// new process to a group of its own choosing, so a mask belonging to another
// group is rejected (ERROR_INVALID_PARAMETER). An unconstrained caller, whose
// mask is its group's whole active set, asks for no bound the child's own
// group cannot give, so such a child keeps that group's default and the spawn
// proceeds. A narrower mask is a deliberate CPU constraint that a child in
// another group cannot express, and silently widening it would hand the
// provider tree the whole group, so that spawn fails naming the constraint.
DWORD_PTR affinity = 0, systemAffinity = 0;
require(GetProcessAffinityMask(GetCurrentProcess(), &affinity, &systemAffinity),
"Read caller process affinity failed", command);
if (affinity != 0)
USHORT callerGroup = 0, childGroup = 0;
const bool sharedGroup = singleProcessorGroup(GetCurrentProcess(), callerGroup) &&
singleProcessorGroup(result->process.get(), childGroup) &&
callerGroup == childGroup;
if (affinity != 0 && sharedGroup)
require(SetProcessAffinityMask(result->process.get(), affinity), "Preserve caller process affinity failed", command);
else
require(affinity == 0 || affinity == systemAffinity,
"Caller CPU affinity cannot bound an owned process in another processor group", command,
ERROR_INVALID_PARAMETER);
const DWORD priority = GetPriorityClass(GetCurrentProcess());
require(priority != 0 && SetPriorityClass(result->process.get(), priority),
"Preserve caller process priority failed", command);
@@ -45,6 +45,20 @@ class Generator:
assert "ows-secret" not in json.dumps(projected), projected
if isinstance(value, bytes):
assert projected["binary_fields"][-1]["state"] == "redacted", projected
body = b"A" * 12345
state = {"response": {"content": body}}
control = adapter.project_record(state, [])
field = control["binary_fields"][0]
assert field["state"] == "retained" and field["bytes"] == len(body), control
for literal in ["12345", "2345"]:
# Only the generated byte count matches this exclusion.
assert literal not in control["reported"]["response"]["content"]
assert literal not in field["content_base64"] and literal not in field["sha256"]
projected = adapter.project_record(state, [literal])
assert projected["reported"]["response"]["content"] is None, projected
assert projected["binary_fields"] == [{"pointer": "/response/content", "representation": "producer-bytes", "state": "redacted", "content_base64": None, "bytes": None, "sha256": None}], projected
assert {"pointer": "/response/content", "reason": "explicit-sensitive-value"} in projected["redactions"], projected
assert literal not in json.dumps(projected), projected
(root / "native-error-classifications.json").write_text(json.dumps({"passed": len(classifications) + 2}))
first = http.HTTPFlow(connection.Client(peername=("127.0.0.1", 1), sockname=("127.0.0.1", 2)), connection.Server(address=("example.test", 80)))
first.id = "original-producer-id"
+17 -23
View File
@@ -1,5 +1,8 @@
import { readFile, readdir, stat } from "node:fs/promises";
import { basename } from "node:path";
import { execFile } from "node:child_process";
import { promisify } from "node:util";
import { readdir, stat } from "node:fs/promises";
const execFileAsync = promisify(execFile);
/**
* Observe leftover demo processes on a dedicated clean Linux runner, including
@@ -14,6 +17,9 @@ export const assertCleanLinuxHopperRuntime = async (
: [process.env.HOPPER_LAUNCHER_PATH]),
],
) => {
const { stdout } = await execFileAsync("ps", ["-ax", "-o", "command="]);
if (/\/opt\/hopper\/bin\/Hopper|\bXvfb\b|hopper-demo-x11\.py/u.test(stdout))
throw new Error("Clean Linux runner retained a Hopper demo process");
const launchers = [];
for (const path of launcherPaths) {
try {
@@ -22,34 +28,18 @@ export const assertCleanLinuxHopperRuntime = async (
if (error?.code !== "ENOENT") throw error;
}
}
if (launchers.length === 0) return;
// /proc exposes the executed file even when argv[0] names a removed symlink.
// Compare file identity, not a display path or a process name.
let inspectionError;
for (const entry of await readdir("/proc", { withFileTypes: true })) {
if (!entry.isDirectory() || !/^\d+$/u.test(entry.name)) continue;
let retainedRuntime;
let executable;
try {
// procfs preserves argument boundaries. A caller merely mentioning a
// runtime in its script text or other arguments is not that runtime.
const arguments_ = (
await readFile(`/proc/${entry.name}/cmdline`, "utf8")
).split("\0");
retainedRuntime =
arguments_[0] === "/opt/hopper/bin/Hopper" ||
basename(arguments_[0] ?? "") === "Xvfb" ||
(basename(arguments_[0] ?? "") === "python3" &&
basename(arguments_[1] ?? "") === "hopper-demo-x11.py");
if (!retainedRuntime && launchers.length > 0) {
const executable = await stat(`/proc/${entry.name}/exe`, {
bigint: true,
});
retainedRuntime = launchers.some(
({ dev, ino }) => dev === executable.dev && ino === executable.ino,
);
}
executable = await stat(`/proc/${entry.name}/exe`, { bigint: true });
} catch (error) {
if (error?.code === "ENOENT" || error?.code === "ESRCH") continue;
// Other users' process facts may be unreadable. The setup lane launches
// Other users' executables may be unreadable. The setup lane launches
// providers as the caller; unreadable caller-owned processes fail closed.
if (error?.code === "EACCES") {
const owner = await stat(`/proc/${entry.name}`).catch((cause) => {
@@ -63,7 +53,11 @@ export const assertCleanLinuxHopperRuntime = async (
inspectionError ??= error;
continue;
}
if (retainedRuntime)
if (
launchers.some(
({ dev, ino }) => dev === executable.dev && ino === executable.ino,
)
)
throw new Error("Clean Linux runner retained a Hopper demo process");
}
if (inspectionError !== undefined) throw inspectionError;
+3 -4
View File
@@ -1,7 +1,3 @@
import type { z } from "zod";
import type { analysisStringSchema } from "../../src/domain/hopperValues.js";
type VerifierCall = (
name: string,
arguments_?: Record<string, unknown>,
@@ -29,3 +25,6 @@ export function verifyHopperStringObjects(
}[];
readonly displayEscapesExcludedFromMatching: true;
}>;
import type { z } from "zod";
import type { analysisStringSchema } from "../../src/domain/hopperValues.js";
+150 -3
View File
@@ -1,8 +1,9 @@
#!/usr/bin/env node
import assert from "node:assert/strict";
import { estimateTokenCount } from "tokenx";
import { execFile } from "node:child_process";
import { createHash } from "node:crypto";
import { readFile, writeFile, readdir } from "node:fs/promises";
import { readFile, writeFile, readdir, mkdir, rm } from "node:fs/promises";
import { isAbsolute, join } from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
@@ -91,6 +92,20 @@ const source = pcap(1, [
// LINKTYPE_BLUETOOTH_HCI_H4 = 187; H4 command packet, HCI Reset opcode 0x0c03, no parameters.
const bluetoothSource = pcap(187, [Buffer.from("01030c00", "hex")]);
let cases = 0;
const discoveryMeasurements = [];
const inheritedAddressSpace = (await readFile("/proc/self/limits", "utf8"))
.split("\n")
.find((line) => line.startsWith("Max address space "))
?.trim()
.match(/^Max address space\s+(\S+)\s+(\S+)\s+bytes$/u);
assert.ok(inheritedAddressSpace);
const expectedAddressSpace = Math.min(
1073741824,
...inheritedAddressSpace
.slice(1)
.map((value) => (value === "unlimited" ? Infinity : Number(value))),
);
try {
await writeFile(path, source);
await writeFile(bluetooth, bluetoothSource);
@@ -121,14 +136,16 @@ try {
.digest("hex"),
);
assert.equal(report.hardware_acquisition, "not-performed");
assert.equal(report.limits.address_space_bytes, 1073741824);
assert.equal(report.limits.address_space_bytes, expectedAddressSpace);
assert.equal(report.tool.configured_path, process.env.REA_TSHARK_COMMAND);
assert.equal(
report.resource_limiter.configured_path,
process.env.REA_TSHARK_PRLIMIT_COMMAND ?? "/usr/bin/prlimit",
);
assert.ok(
report.resource_limiter.arguments.includes("--as=1073741824:1073741824"),
report.resource_limiter.arguments.includes(
`--as=${expectedAddressSpace}:${expectedAddressSpace}`,
),
);
assert.equal(report.configuration, "private-empty-personal-profile");
const flags = Object.entries(input)
@@ -313,6 +330,133 @@ try {
true,
);
cases++;
const discovery = (await client.listTools()).tools.find(
({ name }) => name === "inspect_packet_fields",
);
assert.ok(discovery?.outputSchema);
assert.ok(ajv.validateSchema(discovery.inputSchema));
assert.ok(ajv.validateSchema(discovery.outputSchema));
const nativeProfile = join(root.path, "native-profile");
await mkdir(nativeProfile);
const native = (
await execute(process.env.REA_TSHARK_COMMAND, ["-G", "fields"], {
env: { ...environment, WIRESHARK_CONFIG_DIR: nativeProfile },
timeout: 30000,
maxBuffer: 32 * 1024 * 1024,
})
).stdout;
await rm(nativeProfile, { recursive: true });
const nativeLines = native.split(/\r?\n/);
if (nativeLines.at(-1) === "") nativeLines.pop();
for (const input of [
{ query: "usb" },
{ limit: 1 },
{ offset: 1, limit: 2 },
{ kind: "fields", protocol: "usb", query: "idVendor" },
{ kind: "fields", protocol: "udp", limit: null },
{ kind: "fields", query: "rea_no_such_field" },
]) {
const started = performance.now();
const response = await client.callTool({
name: discovery.name,
arguments: input,
});
assert.notEqual(response.isError, true, JSON.stringify(response));
assert.ok(
ajv.validate(discovery.outputSchema, response.structuredContent),
JSON.stringify(ajv.errors),
);
const evidence = parseEvidence(response.structuredContent);
const report = evidence.normalized_result;
const kind = input.kind ?? "protocols";
const query = (input.query ?? "").toLowerCase();
const expectedMatches = nativeLines.flatMap((line, index) => {
const columns = line.split("\t");
if ((kind === "protocols") !== (columns[0] === "P")) return [];
if (
input.protocol !== undefined &&
(columns[0] === "P" ? columns[2] : columns[4]) !== input.protocol
)
return [];
if (
![
columns[1],
columns[2],
...(columns[0] === "F" ? [columns[7]] : []),
].some((value) => value.toLowerCase().includes(query))
)
return [];
return [{ sourceLine: index + 1, columns }];
});
const offset = input.offset ?? 0;
const limit = input.limit === null ? undefined : (input.limit ?? 50);
const expectedPage = expectedMatches.slice(
offset,
limit === undefined ? undefined : offset + limit,
);
assert.deepEqual(
report.records.map((record) => record.source_line),
expectedPage.map((record) => record.sourceLine),
);
assert.deepEqual(report.output_window, {
examined: nativeLines.filter((line) =>
line.startsWith(kind === "protocols" ? "P\t" : "F\t"),
).length,
matching: expectedMatches.length,
returned: expectedPage.length,
has_more:
Math.max(0, expectedMatches.length - offset) > expectedPage.length,
});
discoveryMeasurements.push({
input,
elapsed_ms: +(performance.now() - started).toFixed(2),
response_bytes: Buffer.byteLength(JSON.stringify(response)),
estimated_tokens_one_tool_request_response: estimateTokenCount(
JSON.stringify(discovery) +
JSON.stringify(input) +
JSON.stringify(response),
),
});
for (const record of report.records) {
const columns = nativeLines[record.source_line - 1].split("\t");
assert.equal(record.name, columns[1]);
assert.equal(record.abbreviation, columns[2]);
assert.equal(record.field_type, columns[0] === "F" ? columns[3] : null);
assert.equal(
record.parent_protocol,
columns[0] === "F" ? columns[4] : null,
);
assert.equal(record.description, columns[0] === "F" ? columns[7] : null);
assert.equal(record.display_base, columns[0] === "F" ? columns[5] : null);
assert.equal(record.bitmask, columns[0] === "F" ? columns[6] : null);
}
if (input.query === "idVendor")
assert.ok(
report.records.some(
(row) =>
row.abbreviation === "usb.idVendor" &&
row.field_type === "FT_UINT16",
),
);
if (input.query === "usb")
assert.ok(report.records.some((row) => row.abbreviation === "usb"));
if (input.query === "rea_no_such_field")
assert.equal(report.output_window.matching, 0);
const flags = Object.entries(input).flatMap(([key, value]) =>
value === null ? ["--all"] : [`--${key}`, String(value)],
);
const cli = JSON.parse(
(
await execute(
process.execPath,
[entrypoint, "inspect-packet-fields", ...flags, "--format", "json"],
{ env: environment, timeout: 60000, maxBuffer: 16 * 1024 * 1024 },
)
).stdout,
);
assert.deepEqual(cli.normalized_result, report);
cases++;
}
await client.close();
assert.deepEqual((await readdir(root.path)).sort(), [
"bluetooth.pcap",
@@ -329,6 +473,9 @@ console.log(
{
status: "passed",
cases,
discovery_measurements: discoveryMeasurements,
measurement_scope:
"Warm MCP server, fresh TShark processes; tokenx estimates include one advertised tool/request/response, not billed usage. Model-driven selection, CPU and peak process-family memory are unmeasured; no cache/speedup claim.",
host: `${process.platform}/${process.arch}`,
scope:
"real TShark CLI/MCP; synthetic Ethernet/IPv4/UDP/DNS and Bluetooth HCI PCAP; no hardware acquisition",
+104 -2
View File
@@ -1,5 +1,6 @@
#!/usr/bin/env node
import assert from "node:assert/strict";
import { estimateTokenCount } from "tokenx";
import { execFile } from "node:child_process";
import { createHash } from "node:crypto";
import { readFile, writeFile, readdir } from "node:fs/promises";
@@ -68,6 +69,20 @@ for (const byte of Buffer.from("REA")) {
lines.push(`#${tick + 1000}`, "1!");
const source = lines.join("\n") + "\n";
let cases = 0;
const discoveryMeasurements = [];
const inheritedAddressSpace = (await readFile("/proc/self/limits", "utf8"))
.split("\n")
.find((line) => line.startsWith("Max address space "))
?.trim()
.match(/^Max address space\s+(\S+)\s+(\S+)\s+bytes$/u);
assert.ok(inheritedAddressSpace);
const expectedAddressSpace = Math.min(
1073741824,
...inheritedAddressSpace
.slice(1)
.map((value) => (value === "unlimited" ? Infinity : Number(value))),
);
try {
await writeFile(path, source);
await client.connect(transport);
@@ -104,14 +119,16 @@ try {
createHash("sha256").update(source).digest("hex"),
);
assert.equal(report.hardware_acquisition, "not-performed");
assert.equal(report.limits.address_space_bytes, 1073741824);
assert.equal(report.limits.address_space_bytes, expectedAddressSpace);
assert.equal(report.tool.configured_path, process.env.REA_SIGROK_COMMAND);
assert.equal(
report.resource_limiter.configured_path,
process.env.REA_SIGROK_PRLIMIT_COMMAND ?? "/usr/bin/prlimit",
);
assert.ok(
report.resource_limiter.arguments.includes("--as=1073741824:1073741824"),
report.resource_limiter.arguments.includes(
`--as=${expectedAddressSpace}:${expectedAddressSpace}`,
),
);
assert.equal(report.diagnostics, "");
const flags = Object.entries(input)
@@ -245,6 +262,88 @@ try {
});
assert.equal(invalid.isError, true);
cases++;
const discoveryTool = (await client.listTools()).tools.find(
({ name }) => name === "inspect_signal_decoders",
);
assert.ok(discoveryTool?.outputSchema);
assert.ok(ajv.validateSchema(discoveryTool.inputSchema));
assert.ok(ajv.validateSchema(discoveryTool.outputSchema));
const nativeHelp = (
await execute(
process.env.REA_SIGROK_COMMAND,
["--dont-scan", "--protocol-decoders=uart", "--show"],
{ env: environment, timeout: 30000, maxBuffer: 4 * 1024 * 1024 },
)
).stdout.split(/\r?\n/);
if (nativeHelp.at(-1) === "") nativeHelp.pop();
for (const input of [
{ query: "uart" },
{ limit: 1 },
{ offset: 1, limit: 2 },
{ decoder: "uart", query: "BaUd" },
{ decoder: "uart", offset: 50, limit: null },
{ query: "rea_no_such_decoder" },
]) {
const started = performance.now();
const response = await client.callTool({
name: discoveryTool.name,
arguments: input,
});
assert.notEqual(response.isError, true, JSON.stringify(response));
assert.ok(
ajv.validate(discoveryTool.outputSchema, response.structuredContent),
JSON.stringify(ajv.errors),
);
const evidence = parseEvidence(response.structuredContent);
const report = evidence.normalized_result;
discoveryMeasurements.push({
input,
elapsed_ms: +(performance.now() - started).toFixed(2),
response_bytes: Buffer.byteLength(JSON.stringify(response)),
estimated_tokens_one_tool_request_response: estimateTokenCount(
JSON.stringify(discoveryTool) +
JSON.stringify(input) +
JSON.stringify(response),
),
});
if (input.decoder) {
const matched = nativeHelp
.map((text, i) => ({ line_number: i + 1, text }))
.filter(({ text }) =>
text.toLowerCase().includes((input.query ?? "").toLowerCase()),
);
assert.deepEqual(
report.help_lines,
matched.slice(
input.offset ?? 0,
input.limit === null
? undefined
: (input.offset ?? 0) + (input.limit ?? 50),
),
);
} else if (input.query === "uart")
assert.ok(report.decoders.some(({ id }) => id === "uart"));
const flags = Object.entries(input).flatMap(([key, value]) =>
value === null ? ["--all"] : [`--${key}`, String(value)],
);
const cli = JSON.parse(
(
await execute(
process.execPath,
[entrypoint, "inspect-signal-decoders", ...flags, "--format", "json"],
{ env: environment, timeout: 60000, maxBuffer: 8 * 1024 * 1024 },
)
).stdout,
);
assert.deepEqual(cli.normalized_result, report);
cases++;
}
const unavailable = await client.callTool({
name: discoveryTool.name,
arguments: { decoder: "rea_no_such_decoder" },
});
assert.equal(unavailable.isError, true);
cases++;
await client.close();
assert.deepEqual((await readdir(root.path)).sort(), [
"paged.vcd",
@@ -260,6 +359,9 @@ console.log(
{
status: "passed",
cases,
discovery_measurements: discoveryMeasurements,
measurement_scope:
"Warm MCP server, fresh sigrok processes; tokenx estimates include one advertised tool/request/response, not billed model usage. No model-driven selection benchmark, CPU or peak process-family memory measurement; no cache or speedup claim.",
host: `${process.platform}/${process.arch}`,
scope:
"real sigrok UART VCD through compiled CLI/MCP; no physical acquisition",
+36
View File
@@ -662,6 +662,42 @@ setInterval(()=>{},1000);\n`,
unicodeNamesPreserved: true,
});
report.processes.windowsEnvironmentSemantics = true;
// Observed through this process's own id: a separate probe process would
// get its own processor group, so it cannot stand in for the process that
// spawns the child.
const callerAffinity = JSON.parse(
execFileSync(boundaryFixture, ["affinity", String(process.pid)], {
encoding: "utf8",
}),
);
const affinityChild = native.call("process_spawn", [
boundaryFixture,
`${quote(boundaryFixture)} affinity`,
workspace,
environment,
]);
const observedAffinity = await wait(affinityChild);
native.call("process_close", [affinityChild.handle]);
assert.equal(observedAffinity.exitCode, 0);
const childAffinity = JSON.parse(observedAffinity.output);
// Masks cross as hexadecimal strings because a mask can exceed the
// safe-integer range. Sharing one processor group makes the caller's mask
// applicable, so it must be inherited exactly. Windows placing the child in
// another group is only permitted when the caller asked for no bound of its
// own, which its group's whole active set establishes; a narrower mask has
// to fail the spawn rather than widen to that group's default.
const sharedGroup =
callerAffinity.singleProcessorGroup &&
childAffinity.singleProcessorGroup &&
callerAffinity.processorGroup === childAffinity.processorGroup;
if (sharedGroup)
assert.equal(childAffinity.affinityMask, callerAffinity.affinityMask);
else assert.equal(callerAffinity.affinityMask, callerAffinity.systemMask);
report.processes.callerAffinityBoundsOwnedProcessOrRefusesToWiden = true;
report.callerAffinity = {
...callerAffinity,
sharedGroupWithChild: sharedGroup,
};
}
const ownerFixture = join(workspace, "owner.mjs");
const ownerPidFile = join(workspace, "crash-child.pid");
+20 -2
View File
@@ -84,6 +84,24 @@ describe("parseAdbDevicesOutput", () => {
]);
});
it("accepts complete multiword states and rejects state prefixes", () => {
const output = [
"serial-no-permission no permissions usb:1-1",
"serial-bad-device deviceXYZ ",
"serial-bad-offline offline-ish",
"serial-bad-auth unauthorizedXYZ",
].join("\n");
const { devices, unparsedLines } = parseAdbDevicesOutput(output);
expect(devices.map(({ serial, state }) => [serial, state])).toEqual([
["serial-no-permission", "no permissions"],
]);
expect(unparsedLines).toEqual([
"serial-bad-device deviceXYZ ",
"serial-bad-offline offline-ish",
"serial-bad-auth unauthorizedXYZ",
]);
});
it("returns unrecognized lines instead of dropping them", () => {
const output = [
"List of devices attached",
@@ -194,10 +212,10 @@ describe("parsePackageListOutput", () => {
it("keeps unrecognized lines for partial coverage reporting", () => {
const { packages, unparsedLines } = parsePackageListOutput(
"package:/data/app/x/base.apk=com.ok\ngarbage line\n",
"package:/data/app/x/base.apk=com.ok\n garbage line \n",
);
expect(packages).toHaveLength(1);
expect(unparsedLines).toEqual(["garbage line"]);
expect(unparsedLines).toEqual([" garbage line "]);
});
});
+14 -25
View File
@@ -10,11 +10,15 @@ import {
adbHostFileName,
type AdbHostFileNameReason,
} from "./AdbHostFileName.js";
import {
ADB_DEVICE_STATES,
type AdbDeviceState,
} from "../domain/adb/adbDeviceAnalysis.js";
/** One device exactly as `adb devices -l` reports it. */
export interface AdbDeviceEntry {
readonly serial: string;
readonly state: string;
readonly state: AdbDeviceState;
readonly transport: "usb" | "tcp" | "unknown";
readonly product: string | null;
readonly model: string | null;
@@ -24,25 +28,6 @@ export interface AdbDeviceEntry {
readonly kind_basis: "emulator_serial_prefix" | "usb_transport" | "none";
}
/**
* Device states adb prints after the serial, longest first so two-word
* states match before their single-word prefixes. A line whose remainder
* starts with none of them is not a device line.
*/
const DEVICE_STATES = [
"no permissions",
"unauthorized",
"authorizing",
"offline",
"recovery",
"sideload",
"connecting",
"bootloader",
"device",
"unknown",
"host",
] as const;
const DEVICE_LINE = /^(\S+)\s+(.*)$/u;
const metadataValue = (remainder: string, key: string): string | null => {
@@ -73,11 +58,15 @@ export const parseAdbDevicesOutput = (
continue;
}
const match = DEVICE_LINE.exec(line);
const state = DEVICE_STATES.find((candidate) =>
match?.[2]?.startsWith(candidate),
);
const state = ADB_DEVICE_STATES.find((candidate) => {
const remainder = match?.[2];
if (remainder === undefined || !remainder.startsWith(candidate))
return false;
const next = remainder[candidate.length];
return next === undefined || /\s/u.test(next);
});
if (match === null || state === undefined) {
if (line.trim() !== "") unparsedLines.push(line);
if (rawLine.trim() !== "") unparsedLines.push(rawLine);
continue;
}
const serial = match[1]!;
@@ -192,7 +181,7 @@ export const parsePackageListOutput = (
if (line === "") continue;
const match = PACKAGE_LINE.exec(line);
if (match === null) {
unparsedLines.push(line);
unparsedLines.push(rawLine);
continue;
}
packages.push({
+74
View File
@@ -121,6 +121,52 @@ fi`,
]);
});
it("refuses device listings with an invalid state token", async () => {
const malformedLine = "serial deviceXYZ ";
const { binary } = await writeAdbStub(
() => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; exit 0; fi
if [ "$1" = "devices" ]; then
printf 'List of devices attached\\n${malformedLine}\\n'
exit 0
fi`,
);
const result = await providerOver(binary).execute({
operation: "list_adb_devices",
input: {},
});
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error._tag).toBe("AnalysisProtocolError");
expect(result.error.message).toContain(malformedLine);
});
it("lists rescue and detached transports alongside online devices", async () => {
const { binary } = await writeAdbStub(
() => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; exit 0; fi
if [ "$1" = "devices" ]; then
printf 'List of devices attached\\n'
printf 'serial-online device transport_id:1\\n'
printf 'serial-rescue rescue transport_id:2\\n'
printf 'serial-detached detached transport_id:3\\n'
exit 0
fi`,
);
const result = await evidenceResult(providerOver(binary), {
operation: "list_adb_devices",
input: {},
});
expect(result.devices).toEqual(
expect.arrayContaining([
expect.objectContaining({ serial: "serial-online", state: "device" }),
expect.objectContaining({ serial: "serial-rescue", state: "rescue" }),
expect.objectContaining({
serial: "serial-detached",
state: "detached",
}),
]),
);
});
it("inspects a device through the getprop whitelist", async () => {
const { binary } = await writeAdbStub(
() => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; exit 0; fi
@@ -580,6 +626,34 @@ it("projects package details from dumpsys package output", async () => {
});
});
// Android 10 and 11 append ": restricted=true" to hard- and soft-restricted
// entries of the requested-permissions list (Settings.dumpPackageLPr).
it("counts requested permissions that the device marks as restricted", async () => {
const { binary } = await writeAdbStub(
shellStub({
"dumpsys:package": `Packages:
Package [com.example] (abcdef):
userId=10234
versionName=1.2.3
requested permissions:
android.permission.INTERNET
android.permission.WRITE_EXTERNAL_STORAGE: restricted=true
android.permission.CAMERA
install permissions:
android.permission.INTERNET: granted=true
`,
}),
);
const result = await evidenceResult(providerOver(binary), {
operation: "inspect_adb_package",
input: { serial: "emulator-5554", package: "com.example" },
});
expect(result).toMatchObject({
requested_permissions_count: 3,
coverage: "complete",
});
});
it("reports an uninstalled package from dumpsys refusal output", async () => {
const { binary } = await writeAdbStub(
shellStub({ "dumpsys:package": "Unable to find package: com.absent\n" }),
+5 -1
View File
@@ -236,7 +236,11 @@ export const parseDumpsysPackageOutput = (
inPermissions = true;
continue;
}
if (inPermissions && /^\s{2,}[A-Za-z0-9._]+$/.test(line)) {
// Android 10 and 11 mark restricted entries as "NAME: restricted=true".
if (
inPermissions &&
/^\s{2,}[A-Za-z0-9._]+(?:: restricted=true)?$/u.test(line)
) {
permissions += 1;
continue;
}
+5 -3
View File
@@ -199,9 +199,11 @@ export class JadxMcpTransport implements Transport {
throw new Error(`JADX MCP frame exceeds ${MAX_FRAME_BYTES} bytes`);
this.#appendFrame(fragment, frameBytes);
if (newline === -1) return;
const line = this.#frameBuffer
.subarray(0, this.#frameLength)
.toString("utf8");
const line = new TextDecoder("utf-8", {
fatal: true,
// Preserve a leading BOM so the JSON-RPC parser can reject it.
ignoreBOM: true,
}).decode(this.#frameBuffer.subarray(0, this.#frameLength));
this.#frameBuffer = Buffer.alloc(0);
this.#frameLength = 0;
this.onmessage?.(deserializeMessage(line));
+2 -2
View File
@@ -397,7 +397,7 @@ const workflowAvailabilityFor = ({
remediation:
"On Linux x64, set absolute REA_PWNTOOLS_PYTHON to caller-supplied Python with pwntools 4.15.0. No active binary target is required.",
};
if (name === "inspect_packet_capture")
if (name === "inspect_packet_capture" || name === "inspect_packet_fields")
return policy.packetCaptureEnabled === true
? { reason: "available", remediation: null }
: {
@@ -405,7 +405,7 @@ const workflowAvailabilityFor = ({
remediation:
"On Linux, supply TShark 4.x via absolute REA_TSHARK_COMMAND and util-linux prlimit (REA_TSHARK_PRLIMIT_COMMAND, default /usr/bin/prlimit). No active binary target or hardware is required; the tool is checked at execution.",
};
if (name === "decode_signal_recording")
if (name === "decode_signal_recording" || name === "inspect_signal_decoders")
return policy.signalRecordingEnabled === true
? { reason: "available", remediation: null }
: {
+6 -2
View File
@@ -78,7 +78,9 @@ export const readJsonObjectFile = (
value === null ||
Array.isArray(value)
)
throw new TypeError("JSON document root must be an object");
return err(
invalidJson(operation, "JSON document root must be an object"),
);
for (const members of scans)
for (const [key, member] of members) {
if (!Object.hasOwn(value, key)) continue;
@@ -177,7 +179,9 @@ export const readJsonObjectFile = (
signal === undefined ? {} : { signal },
);
if (!rootObject)
throw new TypeError("JSON document root must be an object");
return err(
invalidJson(operation, "JSON document root must be an object"),
);
const digest = hash.digest("hex");
if (previousDigest !== undefined && digest !== previousDigest)
throw new RegularFileChangedError(path);
+23 -4
View File
@@ -2,12 +2,16 @@ import { createHash } from "node:crypto";
import { describe, expect, it } from "vitest";
import { installMacHopper, type MacHopperInstallHost } from "./MacHopper.js";
import {
installMacHopper,
type MacHopperBundleVerification,
type MacHopperInstallHost,
} from "./MacHopper.js";
class FakeMacHopperHost implements MacHopperInstallHost {
readonly archive = new TextEncoder().encode("hopper-dmg");
archiveOk = true;
valid = true;
verification: MacHopperBundleVerification = "verified";
existing = false;
mountSucceeds = true;
copySucceeds = true;
@@ -15,6 +19,7 @@ class FakeMacHopperHost implements MacHopperInstallHost {
badHash = false;
unmounts = 0;
cleanups = 0;
installs = 0;
opens = 0;
replaces = false;
@@ -47,7 +52,7 @@ class FakeMacHopperHost implements MacHopperInstallHost {
return Promise.resolve();
};
appBundle = () => Promise.resolve("/mounted/Hopper Disassembler.app");
validBundle = () => Promise.resolve(this.valid);
validBundle = () => Promise.resolve(this.verification);
destinationExists = () => Promise.resolve(this.existing);
installBundle = (
_source: string,
@@ -55,6 +60,7 @@ class FakeMacHopperHost implements MacHopperInstallHost {
_stage: string,
replaceExisting: boolean,
) => {
this.installs += 1;
this.replaces = replaceExisting;
return Promise.resolve(this.copySucceeds);
};
@@ -115,7 +121,7 @@ describe("macOS Hopper installation", () => {
it("cleans and detaches after bundle validation fails", async () => {
const host = new FakeMacHopperHost();
host.valid = false;
host.verification = "invalid_signature";
await expect(installMacHopper({}, host)).resolves.toEqual({
status: "failed",
reason: "bundle",
@@ -123,4 +129,17 @@ describe("macOS Hopper installation", () => {
expect(host.unmounts).toBe(1);
expect(host.cleanups).toBe(1);
});
it("reports a publisher mismatch without copying or launching the bundle", async () => {
const host = new FakeMacHopperHost();
host.verification = "unverified_publisher";
await expect(installMacHopper({}, host)).resolves.toEqual({
status: "failed",
reason: "publisher_verification_failed",
});
expect(host.installs).toBe(0);
expect(host.opens).toBe(0);
expect(host.unmounts).toBe(1);
expect(host.cleanups).toBe(1);
});
});
+52 -22
View File
@@ -24,7 +24,35 @@ const execFileAsync = promisify(execFile);
const RELEASES_URL =
"https://www.hopperapp.com/include/files-api.php?request=releases&public=true";
const DOWNLOAD_PREFIX = "https://www.hopperapp.com:443/downloader/public/";
const BUNDLE_IDENTIFIER = "com.cryptic-apps.hopper-web-4";
/**
* Apple publisher identity pinned for the official Hopper macOS bundle. The
* team identifier was captured from `Hopper-6.6.0-demo.dmg` at
* `https://www.hopperapp.com:443/downloader/public/Hopper-6.6.0-demo.dmg`
* (34846271 bytes, SHA-1 0be530f038249742590992199c7b042ea20ca955) with
* `codesign -dvvv`, which reported
* `Authority=Developer ID Application: Cryptic Apps (2AMA2753NF)`.
*/
export const HOPPER_PUBLISHER = Object.freeze({
bundleIdentifier: "com.cryptic-apps.hopper-web-4",
teamIdentifier: "2AMA2753NF",
});
/**
* Code requirement enforcing the bundle identifier and signing team together.
* The identifier is read from the sealed CodeDirectory, so it cannot be changed
* by editing `Info.plist` the way the previous string comparison could.
*/
const HOPPER_PUBLISHER_REQUIREMENT =
`= identifier "${HOPPER_PUBLISHER.bundleIdentifier}"` +
` and anchor apple generic` +
` and certificate leaf[subject.OU] = "${HOPPER_PUBLISHER.teamIdentifier}"`;
/** Outcome of reading a mounted bundle's signature and publisher identity. */
export type MacHopperBundleVerification =
| "verified"
| "invalid_signature"
| "unverified_publisher";
/** Stable network and command effects disclosed before macOS Hopper setup. */
export const macHopperInstallDisclosure = {
@@ -32,6 +60,7 @@ export const macHopperInstallDisclosure = {
commands: [
"hdiutil attach -readonly -nobrowse -mountpoint <private-mount> <verified-hopper.dmg>",
"codesign --verify --deep --strict <mounted-hopper.app>",
`codesign --verify --strict '-R${HOPPER_PUBLISHER_REQUIREMENT}' <mounted-hopper.app>`,
"ditto <mounted-hopper.app> <user-Applications-stage>",
"open <installed-hopper.app>",
] as const,
@@ -52,6 +81,7 @@ export type MacHopperInstallResult =
| "integrity"
| "mount"
| "bundle"
| "publisher_verification_failed"
| "destination_exists"
| "copy"
| "launcher_missing";
@@ -69,7 +99,7 @@ export interface MacHopperInstallHost {
mount(packagePath: string, mountPath: string): Promise<boolean>;
unmount(mountPath: string): Promise<void>;
appBundle(mountPath: string): Promise<string | undefined>;
validBundle(appPath: string): Promise<boolean>;
validBundle(appPath: string): Promise<MacHopperBundleVerification>;
destinationExists(path: string): Promise<boolean>;
installBundle(
source: string,
@@ -124,8 +154,12 @@ export const installMacHopper = async (
return { status: "failed", reason: "mount" };
mounted = true;
const source = await host.appBundle(mountPath);
if (source === undefined || !(await host.validBundle(source)))
if (source === undefined) return { status: "failed", reason: "bundle" };
const verification = await host.validBundle(source);
if (verification === "invalid_signature")
return { status: "failed", reason: "bundle" };
if (verification === "unverified_publisher")
return { status: "failed", reason: "publisher_verification_failed" };
const stage = `${destination}.rea-stage`;
if (
!(await host.installBundle(
@@ -222,25 +256,21 @@ export const systemMacHopperInstallHost = (
environment,
))
)
return false;
try {
const identifier = (
await execFileAsync(
"/usr/libexec/PlistBuddy",
[
"-c",
"Print :CFBundleIdentifier",
join(appPath, "Contents/Info.plist"),
],
{ env: environment },
)
).stdout.trim();
return identifier === BUNDLE_IDENTIFIER;
} catch (cause: unknown) {
// best-effort cleanup: optional bundle probing; unreadable means invalid.
void cause;
return false;
}
return "invalid_signature";
if (
!(await commandSucceeds(
"codesign",
[
"--verify",
"--strict",
`-R${HOPPER_PUBLISHER_REQUIREMENT}`,
appPath,
],
environment,
))
)
return "unverified_publisher";
return "verified";
},
async destinationExists(path) {
try {
+1 -1
View File
@@ -23,7 +23,7 @@ export const createProgressReporter = (
} = {},
): ProgressReporter => {
const minimumIntervalMs = options.minimumIntervalMs ?? 100;
const now = options.now ?? Date.now;
const now = options.now ?? (() => performance.now());
let lastCompleted = -1;
let lastSentAt = Number.NEGATIVE_INFINITY;
let sequence = 0;
@@ -12,12 +12,14 @@ import {
detectReferenceSourceLanguage,
} from "../domain/referenceSourceClassification.js";
import type { HistoricalSourceGraphInput } from "../domain/referenceSourceGraph.js";
import { parseReferenceSourceImports } from "../domain/referenceSourceImportParsing.js";
import {
parseReferenceSourceImports,
PARSEABLE_REFERENCE_SOURCE_LANGUAGES,
} from "../domain/referenceSourceImportParsing.js";
import type {
ReferenceSourceEntry,
ReferenceSourceRead,
} from "../reference/ReferenceSourceReaderTypes.js";
import { PARSEABLE_REFERENCE_SOURCE_LANGUAGES } from "./ReferenceSourceImportTypes.js";
import { typescriptEmittedSourceCandidates } from "./TypeScriptEmittedSourceCandidates.js";
export interface ParsedReferenceSourceEntries {
@@ -102,7 +104,11 @@ const resolveInternalSpecifier = (
]);
// Retain present runtime files first; substitute only a missing counterpart.
// Counterparts cover explicit filenames, not directory/index resolution.
candidates.push(...typescriptEmittedSourceCandidates(fromPath, normalized));
candidates.push(
...typescriptEmittedSourceCandidates(fromPath, normalized).map(
({ path }) => path,
),
);
const match = lookupPath.resolveCandidates(candidates);
return match === undefined
? { to: normalized, resolution: "unresolved" }
@@ -87,6 +87,3 @@ export const DEFAULT_REFERENCE_SOURCE_IGNORE_PATTERNS = [
".coverage",
"*.log",
] as const;
export const PARSEABLE_REFERENCE_SOURCE_LANGUAGES: ReadonlySet<string> =
new Set(["JavaScript", "TypeScript", "JSX", "TSX"]);
+40 -25
View File
@@ -19,6 +19,17 @@ export interface RegularFileReadOptions {
readonly resources?: ArtifactResourceScope;
}
export class InvalidUtf8FileError extends TypeError {
readonly _tag = "InvalidUtf8FileError";
constructor(
readonly path: string,
cause: unknown,
) {
super(`File is not valid UTF-8: ${path}`, { cause });
this.name = this._tag;
}
}
export type RegularFileReadOutcome<Value> =
| { readonly kind: "completed"; readonly value: Value }
| { readonly kind: "failed"; readonly cause: unknown };
@@ -142,7 +153,34 @@ const withRegularFileOwned = async <Value>(
export const readRegularFile = (
path: string,
options: RegularFileReadOptions = {},
): Promise<Buffer> =>
): Promise<Buffer> => readRegularFileContent(path, (bytes) => bytes, options);
/** Read valid UTF-8 without replacing source bytes or stripping an initial BOM. */
export const readRegularFileText = (
path: string,
options: RegularFileReadOptions = {},
): Promise<string> =>
readRegularFileContent(
path,
(bytes) => {
try {
return new TextDecoder("utf-8", {
fatal: true,
ignoreBOM: true,
}).decode(bytes);
} catch (cause: unknown) {
if (!(cause instanceof TypeError)) throw cause;
throw new InvalidUtf8FileError(path, cause);
}
},
options,
);
const readRegularFileContent = <Value>(
path: string,
decode: (bytes: Buffer) => Value,
options: RegularFileReadOptions,
): Promise<Value> =>
withRegularFile(
path,
async (handle, stats) => {
@@ -153,30 +191,7 @@ export const readRegularFile = (
);
if (bytes === undefined || bytes.length !== stats.size)
throw new RegularFileChangedError(path);
return bytes;
return decode(bytes);
},
options,
);
/** Read one admitted regular file as UTF-8 without waiting on a pipe. */
export const readRegularFileText = async (
path: string,
options: RegularFileReadOptions = {},
): Promise<string> => {
try {
return (await readRegularFile(path, options)).toString("utf8");
} catch (cause: unknown) {
if (
cause instanceof RegularFileCleanupFailure &&
cause.outcome.kind === "completed" &&
Buffer.isBuffer(cause.outcome.value)
)
throw new RegularFileCleanupFailure(
cause.path,
cause.owner,
{ kind: "completed", value: cause.outcome.value.toString("utf8") },
cause.cleanupCause,
);
throw cause;
}
};
+34
View File
@@ -54,6 +54,40 @@ describe("setup workflow", () => {
expect(result.remediation).toContain("could not be updated");
});
it("reports a failed config rollback with its destination and recovery backup", async () => {
const host = new FakeSetupHost();
host.hopper = "/Applications/Hopper";
host.clients = [{ name: "codex", configPath: "/fixture/codex.json" }];
host.clientResults.set("codex", {
status: "failed",
reason: "rollback",
backupPath: "/fixture/codex.json.rea.backup",
failure: {
kind: "restore",
path: "/fixture/codex.json",
detail: "permission denied",
},
});
const result = await runSetup(
{ ...options(true), clientIds: ["codex"] },
host,
);
expect(result.clients.codex).toEqual({
status: "failed",
reason: "rollback",
backupPath: "/fixture/codex.json.rea.backup",
failure: {
kind: "restore",
path: "/fixture/codex.json",
detail: "permission denied",
},
});
expect(result.remediation).toContain("permission denied");
expect(result.remediation).toContain("/fixture/codex.json.rea.backup");
});
it("explains skill installation recovery", async () => {
const host = new FakeSetupHost();
host.hopper = "/Applications/Hopper";
+160 -15
View File
@@ -19,6 +19,7 @@ import {
} from "./ClientConfigurationDocument.js";
import { dshDocument, DshCordisDocument } from "./DshCordisPatch.js";
import { constants as fsConstants } from "node:fs";
import { createHash } from "node:crypto";
import { copyFile, mkdir, rm, writeFile } from "node:fs/promises";
import { dirname } from "node:path";
import writeFileAtomic from "write-file-atomic";
@@ -29,6 +30,7 @@ import { npxRegistrationCommand } from "./ClientRegistrationIdentity.js";
import { MCP_STARTUP_POLICY } from "../mcpStartupPolicy.js";
import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js";
import {
readRegularFile,
readRegularFileText,
RegularFileCleanupFailure,
} from "./RegularFileRead.js";
@@ -40,6 +42,26 @@ import type {
} from "./SetupTypes.js";
import type { SetupClient } from "./SupportedClients.js";
export class ClientConfigurationCleanupRollbackFailure extends RegularFileCleanupFailure<string> {
constructor(
failure: RegularFileCleanupFailure<string>,
readonly configurationFailure: Extract<
ClientConfigurationResult,
{ readonly status: "failed"; readonly reason: "rollback" }
>,
) {
super(
failure.path,
failure.owner,
failure.outcome,
new AggregateError(
[failure.cleanupCause, new Error(configurationFailure.failure.detail)],
`Configuration readback cleanup and rollback failed: ${configurationFailure.failure.path}`,
),
);
}
}
/** Configure one supported client's native stdio MCP registration shape. */
export const configureClientConfiguration = (
client: SetupClient,
@@ -167,11 +189,23 @@ const persistClientConfiguration = async ({
}: ClientConfigurationUpdate): Promise<ClientConfigurationResult> => {
const backupPath =
original === undefined ? undefined : `${client.configPath}.rea.backup`;
if (
backupPath !== undefined &&
!(await preserveConfigBackup(transactionPath, backupPath))
)
return { status: "failed", reason: "backup" };
let recoveryBackupPath: string | undefined;
if (backupPath !== undefined && original !== undefined) {
const selection = await selectCurrentRecoveryBackup(backupPath, original);
if (!selection.ok) return { status: "failed", reason: "backup" };
if (
!(await preserveConfigBackup(transactionPath, selection.value.backupPath))
)
return { status: "failed", reason: "backup" };
const verified = await selectCurrentRecoveryBackup(backupPath, original);
if (
!verified.ok ||
verified.value.backupPath !== selection.value.backupPath ||
!verified.value.present
)
return { status: "failed", reason: "backup" };
recoveryBackupPath = verified.value.backupPath;
}
// A native OpenCode V2 entry replaces REA's legacy V1 entry, which would
// otherwise conflict with it.
const legacyPath = legacyClientServerPath(
@@ -239,19 +273,39 @@ const persistClientConfiguration = async ({
!clientConfigurationValuesEqual(readback.servers[name], entry),
)
) {
await restoreConfig(transactionPath, original);
return { status: "failed", reason: "readback" };
return rollbackOrReadbackFailure(
transactionPath,
original,
recoveryBackupPath,
);
}
} catch (cause: unknown) {
if (cause instanceof RegularFileCleanupFailure) throw cause;
if (cause instanceof RegularFileCleanupFailure) {
const rollback = await restoreConfig(transactionPath, original);
if (!rollback.ok)
throw new ClientConfigurationCleanupRollbackFailure(cause, {
status: "failed",
reason: "rollback",
...(recoveryBackupPath === undefined
? {}
: { backupPath: recoveryBackupPath }),
failure: rollback.error,
});
throw cause;
}
// Readback failure restores the transaction before reporting.
void cause;
await restoreConfig(transactionPath, original);
return { status: "failed", reason: "readback" };
return rollbackOrReadbackFailure(
transactionPath,
original,
recoveryBackupPath,
);
}
return {
status: "configured",
...(backupPath === undefined ? {} : { backupPath }),
...(recoveryBackupPath === undefined
? {}
: { backupPath: recoveryBackupPath }),
};
};
@@ -406,9 +460,18 @@ export const inspectClientConfiguration = async (
"The existing configuration is malformed. Repair it before setup applies any other changes.",
};
}
const backupSelection = await selectCurrentRecoveryBackup(
`${client.configPath}.rea.backup`,
original,
);
if (!backupSelection.ok)
return {
status: "invalid",
remediation: backupSelection.error.detail,
};
return {
status: "update",
backupPath: `${client.configPath}.rea.backup`,
backupPath: backupSelection.value.backupPath,
};
};
@@ -473,19 +536,101 @@ const preserveConfigBackup = async (
}
};
interface RecoveryBackupSelection {
readonly backupPath: string;
readonly present: boolean;
}
interface RecoveryBackupSelectionFailure {
readonly kind: "unreadable" | "occupied";
readonly path: string;
readonly detail: string;
}
const selectCurrentRecoveryBackup = async (
firstBackupPath: string,
original: string,
): Promise<Result<RecoveryBackupSelection, RecoveryBackupSelectionFailure>> => {
const expected = Buffer.from(original, "utf8");
let firstBackup: Buffer | undefined;
try {
firstBackup = await readRegularFile(firstBackupPath, {
symlinks: "reject",
});
} catch (cause: unknown) {
if (cause instanceof RegularFileCleanupFailure) throw cause;
if (!isMissing(cause))
return err({
kind: "unreadable",
path: firstBackupPath,
detail: `The recovery backup could not be checked: ${errorDetail(cause)}`,
});
}
if (firstBackup === undefined)
return ok({ backupPath: firstBackupPath, present: false });
if (firstBackup.equals(expected))
return ok({ backupPath: firstBackupPath, present: true });
const digest = createHash("sha256").update(original, "utf8").digest("hex");
const recoveryPath = `${firstBackupPath}.${digest}.recovery`;
let recovery: Buffer | undefined;
try {
recovery = await readRegularFile(recoveryPath, { symlinks: "reject" });
} catch (cause: unknown) {
if (cause instanceof RegularFileCleanupFailure) throw cause;
if (!isMissing(cause))
return err({
kind: "unreadable",
path: recoveryPath,
detail: `The recovery backup could not be checked: ${errorDetail(cause)}`,
});
}
if (recovery !== undefined && !recovery.equals(expected))
return err({
kind: "occupied",
path: recoveryPath,
detail: `The recovery path already contains different data: ${recoveryPath}`,
});
return ok({ backupPath: recoveryPath, present: recovery !== undefined });
};
const restoreConfig = async (
path: string,
original: string | undefined,
): Promise<void> => {
): Promise<
Result<
void,
{ readonly kind: "restore"; readonly path: string; readonly detail: string }
>
> => {
try {
if (original === undefined) await rm(path, { force: true });
else await writeFile(path, original, { encoding: "utf8", mode: 0o600 });
return ok(undefined);
} catch (cause: unknown) {
// The backup remains available for the remediation reported by setup.
void cause;
return err({ kind: "restore", path, detail: errorDetail(cause) });
}
};
const rollbackOrReadbackFailure = async (
targetPath: string,
original: string | undefined,
backupPath: string | undefined,
): Promise<ClientConfigurationResult> => {
const rollback = await restoreConfig(targetPath, original);
return rollback.ok
? { status: "failed", reason: "readback" }
: {
status: "failed",
reason: "rollback",
...(backupPath === undefined ? {} : { backupPath }),
failure: rollback.error,
};
};
const errorDetail = (cause: unknown): string =>
cause instanceof Error ? cause.message : String(cause);
/** Whether REA's entry matches, no legacy entry remains, and no client disable list suppresses it. */
const registrationCurrent = (
parsed: ClientConfigurationDocument,
+11 -7
View File
@@ -6,15 +6,19 @@ import type {
} from "./SetupTypes.js";
import type { SetupClient } from "./SupportedClients.js";
const failedConfigurationMessage = (
reason: "path" | "backup" | "write" | "readback",
export const failedConfigurationMessage = (
result: Extract<ClientConfigurationResult, { readonly status: "failed" }>,
): string => {
if (reason === "path")
if (result.reason === "path")
return "Agent configuration path could not be safely verified. Check its permissions and, if it is a symbolic link, verify that the link resolves to a regular file owned by the current user, then rerun setup.";
if (reason === "backup")
if (result.reason === "backup")
return "Agent configuration could not be backed up, so no change was made. Check file permissions, then rerun setup.";
if (reason === "write")
if (result.reason === "write")
return "Agent configuration could not be updated. Check file permissions, then rerun setup.";
if (result.reason === "rollback")
return result.backupPath === undefined
? `Agent configuration rollback failed at ${result.failure.path}: ${result.failure.detail}; the failed first-run file may remain. Remove or repair it before rerunning setup.`
: `Agent configuration rollback failed at ${result.failure.path}: ${result.failure.detail}. Inspect the preserved backup at ${result.backupPath} and restore the intended configuration, then rerun setup.`;
return "Agent configuration could not be verified after writing. Repair the configuration file or restore its `.rea.backup`, then rerun setup.";
};
@@ -40,12 +44,12 @@ export const configureDetectedClients = async (options: {
);
options.clients[client.name] = result;
if (result.status === "failed") {
failure ??= failedConfigurationMessage(result.reason);
failure ??= failedConfigurationMessage(result);
options.onProgress?.({
actionId,
label,
state: "failed",
detail: failedConfigurationMessage(result.reason),
detail: failedConfigurationMessage(result),
});
} else if (result.status === "configured") {
options.appliedActions.push(`configured_${client.name}`);
+7 -1
View File
@@ -33,9 +33,11 @@ import {
} from "./SetupSkill.js";
import {
clientConfigurationAligned,
ClientConfigurationCleanupRollbackFailure,
configureClientConfiguration,
inspectClientConfiguration,
} from "./SetupClientConfiguration.js";
import { failedConfigurationMessage } from "./SetupClients.js";
import { setupInstallFailure } from "./SetupInstallFailure.js";
import {
RegularFileCleanupFailure,
@@ -275,8 +277,12 @@ const runClientConfigurationOperation = async <Value>(
cleanup.outcome.kind === "failed"
? cleanup.outcome.cause
: cause.cleanupCause;
const rollbackMessage =
cause instanceof ClientConfigurationCleanupRollbackFailure
? ` ${failedConfigurationMessage(cause.configurationFailure)}`
: "";
throw configurationCleanupError(
`Could not safely access client configuration at ${cause.path}: ${errorMessage(primary)}`,
`Could not safely access client configuration at ${cause.path}: ${errorMessage(primary)}.${rollbackMessage}`,
cause,
cleanup.cleanup,
);
@@ -0,0 +1,23 @@
import { describe, expect, it } from "vitest";
import { HOPPER_PUBLISHER } from "./MacHopper.js";
import { setupInstallFailure } from "./SetupInstallFailure.js";
describe("macOS Hopper setup failure projection", () => {
it("projects a publisher mismatch onto its own code and recovery", () => {
expect(setupInstallFailure("publisher_verification_failed")).toEqual({
status: "failed",
code: "publisher_verification_failed",
remediation: `The mounted Hopper application is not signed by the publisher REA pins (${HOPPER_PUBLISHER.bundleIdentifier}, Apple team ${HOPPER_PUBLISHER.teamIdentifier}); nothing was copied or launched. Install Hopper yourself and set HOPPER_LAUNCHER_PATH to its launcher, then rerun rea setup.`,
});
});
it("keeps a broken bundle distinct from a publisher mismatch", () => {
expect(setupInstallFailure("bundle")).toEqual({
status: "failed",
code: "bundle_invalid",
remediation:
"The mounted package did not contain the expected Hopper application bundle. Retry setup; if it repeats, update REA before installing.",
});
});
});
+10
View File
@@ -1,3 +1,5 @@
import { HOPPER_PUBLISHER } from "./MacHopper.js";
export type SetupFailureCode =
| "unsupported_host"
| "download_failed"
@@ -10,6 +12,7 @@ export type SetupFailureCode =
| "destination_exists"
| "mount_failed"
| "bundle_invalid"
| "publisher_verification_failed"
| "copy_failed"
| "skill_install_failed"
| "cleanup_failed";
@@ -39,6 +42,7 @@ export type HopperInstallFailureReason =
| "destination_exists"
| "mount"
| "bundle"
| "publisher_verification_failed"
| "copy";
/** Map provider installer failures to stable setup codes and safe recovery. */
@@ -115,6 +119,12 @@ export const setupInstallFailure = (
remediation:
"The mounted package did not contain the expected Hopper application bundle. Retry setup; if it repeats, update REA before installing.",
};
case "publisher_verification_failed":
return {
status: "failed",
code: "publisher_verification_failed",
remediation: `The mounted Hopper application is not signed by the publisher REA pins (${HOPPER_PUBLISHER.bundleIdentifier}, Apple team ${HOPPER_PUBLISHER.teamIdentifier}); nothing was copied or launched. Install Hopper yourself and set HOPPER_LAUNCHER_PATH to its launcher, then rerun rea setup.`,
};
case "copy":
return {
status: "failed",
+1 -1
View File
@@ -320,5 +320,5 @@ const clientConfigurationDetail = (
? " If this plan installs Hopper, REA will add HOPPER_LAUNCHER_PATH using the launcher path reported by that installation."
: " The HOPPER_LAUNCHER_PATH value shown above is the current fallback; if this plan installs Hopper, REA will replace it with the launcher path reported by that installation."
: "";
return `Add the REA MCP registration for ${client}; preserve unrelated configuration. Command: ${command.map((part) => JSON.stringify(part)).join(" ")}.${environmentDetail}${hopperDetail}`;
return `Add the REA MCP registration for ${client}; preserve unrelated configuration. The backup path reflects the configuration inspected for this plan; if that file changes before execution, setup will select a recovery path for its current contents. Command: ${command.map((part) => JSON.stringify(part)).join(" ")}.${environmentDetail}${hopperDetail}`;
};
+10
View File
@@ -26,6 +26,16 @@ export type ClientConfigurationResult =
| {
readonly status: "failed";
readonly reason: "path" | "backup" | "write" | "readback";
}
| {
readonly status: "failed";
readonly reason: "rollback";
readonly backupPath?: string;
readonly failure: {
readonly kind: "restore";
readonly path: string;
readonly detail: string;
};
};
/** Read-only state used to block unsafe setup plans before any mutation. */
@@ -8,6 +8,11 @@ const EMITTED_SUFFIXES: ReadonlyArray<
[".jsx", [".tsx", ".d.ts"]],
];
export interface TypeScriptEmittedSourceCandidate {
readonly path: string;
readonly kind: "implementation" | "declaration";
}
/**
* Return TypeScript source counterparts for an explicit emitted filename.
* Callers retain exact runtime-file precedence and must not use these for
@@ -16,7 +21,7 @@ const EMITTED_SUFFIXES: ReadonlyArray<
export const typescriptEmittedSourceCandidates = (
sourcePath: string,
candidate: string,
): readonly string[] => {
): readonly TypeScriptEmittedSourceCandidate[] => {
if (!/\.(?:ts|tsx|mts|cts)$/u.test(sourcePath)) return [];
const emitted = EMITTED_SUFFIXES.find(([suffix]) =>
candidate.endsWith(suffix),
@@ -25,5 +30,8 @@ export const typescriptEmittedSourceCandidates = (
// Strip the matched suffix by length. posix.extname would miss dotfile
// names such as ".js", whose whole basename is the emitted suffix itself.
const stem = candidate.slice(0, candidate.length - emitted[0].length);
return emitted[1].map((suffix) => `${stem}${suffix}`);
return emitted[1].map((suffix) => ({
path: `${stem}${suffix}`,
kind: suffix.startsWith(".d.") ? "declaration" : "implementation",
}));
};
+10 -2
View File
@@ -25,7 +25,11 @@ import {
import { isOwnedClientRegistrationCommand } from "./ClientRegistrationIdentity.js";
import { dshDocument } from "./DshCordisPatch.js";
import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js";
import { readRegularFileText } from "./RegularFileRead.js";
import {
InvalidUtf8FileError,
RegularFileCleanupFailure,
readRegularFileText,
} from "./RegularFileRead.js";
import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
import {
manualRegistrationRemediation,
@@ -269,13 +273,17 @@ const readClientConfiguration = async (
try {
original = await fileSystem.readText(transactionPath);
} catch (cause: unknown) {
const readFailure =
cause instanceof RegularFileCleanupFailure ? cause.cause : cause;
return itemRead(
isMissing(cause)
? item(client.name, "skipped", "Configuration does not exist.")
: item(
client.name,
"failed",
"Configuration could not be read. Check file permissions, then rerun uninstall.",
readFailure instanceof InvalidUtf8FileError
? "Configuration is not valid UTF-8 and was not changed. Repair it, then rerun uninstall."
: "Configuration could not be read. Check file permissions, then rerun uninstall.",
),
);
}
+1
View File
@@ -22,6 +22,7 @@ export interface WebModuleArtifacts extends SelectedWebScriptArtifacts {
/** Read selected local artifacts without refetching website assets. */
export interface WebModuleArtifactPort {
close?(): Promise<void>;
load(
input: WebModuleTraceInput,
options?: ExecutionOptions,
+8 -1
View File
@@ -1,3 +1,4 @@
import { closeOwnedWebResources } from "./closeOwnedWebResources.js";
import { isAbsolute } from "node:path";
import { z } from "zod";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
@@ -70,7 +71,13 @@ export class WebModuleTraceService {
) {}
close(): Promise<void> {
return this.resolver.close?.() ?? Promise.resolve();
return closeOwnedWebResources(
[
this.artifacts.close?.bind(this.artifacts),
this.resolver.close?.bind(this.resolver),
],
"Web module trace cleanup failed",
);
}
async trace(
+2
View File
@@ -12,4 +12,6 @@ export interface WebNetworkCapturePort {
input: InspectWebNetworkCaptureInput,
options?: ExecutionOptions,
): Promise<Result<WebNetworkCapture, AnalysisError>>;
/** Retry retained reader/runtime cleanup when the owning service shuts down. */
close?(): Promise<void>;
}
@@ -27,6 +27,11 @@ const OPERATION = "inspect_web_network_capture";
export class WebNetworkCaptureService {
constructor(readonly port: WebNetworkCapturePort) {}
/** Release resources retained by the selected capture adapter. */
close(): Promise<void> {
return this.port.close?.() ?? Promise.resolve();
}
/** Return selected complete records inline, without inventing live browser attribution. */
async inspect(
rawInput: unknown,
+1
View File
@@ -8,6 +8,7 @@ import type { ExecutionOptions } from "./AnalysisProvider.js";
/** Read the caller-selected capture export and verify the selected source identity. */
export interface WebScriptArtifactPort {
close?(): Promise<void>;
load(
input: SelectedWebScriptInput,
options?: ExecutionOptions,
@@ -20,6 +20,7 @@ export interface WebSourceLocationArtifacts extends SelectedWebScriptArtifacts {
/** Replaceable artifact acquisition boundary for a caller-selected pairing. */
export interface WebSourceLocationArtifactPort {
close?(): Promise<void>;
load(
input: WebSourceLocationInput,
options?: ExecutionOptions,
+9 -1
View File
@@ -1,3 +1,4 @@
import { closeOwnedWebResources } from "./closeOwnedWebResources.js";
import { createHash } from "node:crypto";
import { isAbsolute } from "node:path";
import { z } from "zod";
@@ -71,7 +72,13 @@ export class WebSourceLocationService {
) {}
close(): Promise<void> {
return this.decoder.close?.() ?? Promise.resolve();
return closeOwnedWebResources(
[
this.artifacts.close?.bind(this.artifacts),
this.decoder.close?.bind(this.decoder),
],
"Web source-location cleanup failed",
);
}
/** Shared CLI/MCP workflow preserves raw declarations and every equal-position match. */
@@ -174,6 +181,7 @@ export class WebSourceLocationService {
);
}
}
const invalidOutput = (message: string): Result<never, AnalysisError> =>
err(new AnalysisOutputError(OPERATION, message));
const invalid = (
@@ -12,6 +12,7 @@ import type { Result } from "../../domain/result.js";
/** Stable read and PE decoding boundary, independent of an active disassembler session. */
export interface PeResourcesPort {
readonly identity: ProviderIdentity;
close?(): Promise<void>;
inspect(
input: InspectPeResourcesInput,
options?: ExecutionOptions,
@@ -21,6 +21,10 @@ import { err, ok, type Result } from "../../domain/result.js";
export class PeResourcesService {
constructor(readonly provider: PeResourcesPort) {}
close(): Promise<void> {
return this.provider.close?.() ?? Promise.resolve();
}
async inspect(
rawInput: unknown,
options?: ExecutionOptions,
@@ -1,3 +1,9 @@
import {
inspectPacketFieldsInputSchema,
packetFieldsCatalogSchema,
type InspectPacketFieldsInput,
} from "../../domain/captures/packetFields.js";
import type { JsonValue } from "../../domain/jsonValue.js";
import { isAbsolute } from "node:path";
import type {
AnalysisExecution,
@@ -24,6 +30,10 @@ export interface PacketCapturePort {
input: InspectPacketCaptureInput,
options?: ExecutionOptions,
): Promise<Result<AnalysisExecution, AnalysisError>>;
inspectFields(
input: InspectPacketFieldsInput,
options?: ExecutionOptions,
): Promise<Result<AnalysisExecution, AnalysisError>>;
close(): Promise<void>;
}
@@ -74,14 +84,58 @@ export class PacketCaptureService {
"Capture analyzer returned malformed data or changed the selected artifact identity/options.",
),
);
const evidence = createEvidence(execution.subject, execution.provider, {
operation,
parameters: input.data,
result: report.data,
confidence: "observed",
locations: execution.locations,
limitations: execution.limitations,
});
return this.#evidence(operation, input.data, execution, options);
}
async inspectFields(
raw: unknown,
options?: ExecutionOptions,
): Promise<Result<Evidence, AnalysisError>> {
const operation = "inspect_packet_fields";
if (options?.signal?.aborted)
return err(new AnalysisCancelledError(operation));
const input = inspectPacketFieldsInputSchema.safeParse(raw);
if (!input.success)
return err(
analysisInputErrorFromIssues(operation, input.error.issues, raw, {
cause: input.error,
}),
);
const decoded = await this.provider.inspectFields(input.data, options);
if (!decoded.ok) return decoded;
const execution = decoded.value;
const report = packetFieldsCatalogSchema.safeParse(execution.result);
if (
!report.success ||
JSON.stringify(report.data.selection) !== JSON.stringify(input.data) ||
execution.subject?.path !== report.data.tool.path ||
execution.subject.sha256 !== report.data.tool.sha256
)
return err(
new AnalysisOutputError(
operation,
"Field discovery returned malformed data or changed the selected installation/options.",
),
);
return this.#evidence(operation, input.data, execution, options);
}
#evidence(
operation: string,
parameters: Readonly<Record<string, JsonValue>>,
execution: AnalysisExecution,
options?: ExecutionOptions,
): Result<Evidence, AnalysisError> {
const evidence = createEvidence(
execution.subject ?? undefined,
execution.provider,
{
operation,
parameters,
result: execution.result,
confidence: "observed",
locations: execution.locations,
limitations: execution.limitations,
},
);
return options?.signal?.aborted
? err(
new AnalysisCancelledError(operation, {
+15
View File
@@ -0,0 +1,15 @@
/** Attempt every web workflow cleanup owner and report all failures. */
export const closeOwnedWebResources = async (
close: readonly ((() => Promise<void>) | undefined)[],
message: string,
): Promise<void> => {
const results = await Promise.allSettled(
close.flatMap((owner) =>
owner === undefined ? [] : [Promise.resolve().then(owner)],
),
);
const failures = results.flatMap((result) =>
result.status === "rejected" ? [result.reason] : [],
);
if (failures.length > 0) throw new AggregateError(failures, message);
};
@@ -809,6 +809,42 @@ it("keeps an active script message bucket after its last message is evicted", as
await manager.closeAll();
});
it("retains the newest contiguous messages across repeated budget evictions", async () => {
const { bindings, manager, started } = await startLocalAttachedSession();
expect(started.ok).toBe(true);
if (!started.ok) return;
bindings.device.session.nextPayload = { index: -1 };
const loaded = await manager.loadScript(started.value.sessionId, {
sourceKind: "inline",
source: "send({index: -1});",
});
expect(loaded.ok).toBe(true);
const script = bindings.device.session.scripts[0];
expect(script).toBeDefined();
if (script === undefined) return;
// About 500 of these fit the 16 MiB budget, so 2,000 sends evict and
// compact the queue several times.
const filler = "x".repeat(32_000);
for (let index = 0; index < 2_000; index += 1)
script.message.emit({ type: "send", payload: { index, filler } }, null);
const status = manager.status(started.value.sessionId);
expect(status?.messagesTruncated).toBe(true);
const indexes = (status?.messages ?? []).map(
(message) => (message as { payload: { index: number } }).payload.index,
);
expect(indexes.length).toBeGreaterThan(400);
expect(indexes.at(-1)).toBe(1_999);
expect(indexes).toEqual(
Array.from(
{ length: indexes.length },
(_, offset) => 2_000 - indexes.length + offset,
),
);
await manager.closeAll();
});
it("serializes a final remote release against a new same-endpoint session", async () => {
const bindings = new FakeDeviceManager();
let enteredRemoval: (() => void) | undefined;
+1
View File
@@ -12,6 +12,7 @@ import type { Result } from "../../domain/result.js";
/** Replaceable offline build-information reader with no target-execution authority. */
export interface GoBinaryPort {
readonly identity: ProviderIdentity;
close?(): Promise<void>;
inspect(
input: InspectGoBinaryInput,
options?: ExecutionOptions,
+4
View File
@@ -34,6 +34,10 @@ const architectureForSubject = (architecture: string) => {
export class GoBinaryService {
constructor(readonly provider: GoBinaryPort) {}
close(): Promise<void> {
return this.provider.close?.() ?? Promise.resolve();
}
/** Inspect one explicitly selected file without acquiring a binary session. */
async inspect(
rawInput: unknown,
@@ -1,6 +1,6 @@
import { createHash } from "node:crypto";
import { moduleKindForJavaScriptReference } from "./JavaScriptArtifactPathResolution.js";
import { moduleResolutionModeForJavaScriptReference } from "../../domain/javascript/javascriptModuleResolutionMode.js";
import { setImmediate } from "node:timers/promises";
import { resolveJavaScriptSourceMapReferences } from "../../domain/javascript/javascriptSourceMapPaths.js";
@@ -384,7 +384,10 @@ function* analyzeArtifactFileSteps<
analysis.parse_status === "complete" || analysis.parse_status === "partial"
? yield* analyzeParsedJavaScriptSemanticsSteps(
parsed,
moduleKindForJavaScriptReference(file.path, "static-import"),
moduleResolutionModeForJavaScriptReference(
file.path,
"static-import",
),
)
: null;
const projected =
@@ -66,6 +66,7 @@ export interface AstEdgeInput {
/** Inferred static relationship edge values. */
export interface InferenceEdgeInput extends AstEdgeInput {
readonly limitations?: readonly string[];
readonly relation:
| "imports"
| "loads"
@@ -162,6 +163,9 @@ export const addStaticInferenceEdge = (
range: input.range,
operation: "map-static-relationship",
coverage: input.coverage,
...(input.limitations === undefined
? {}
: { limitations: input.limitations }),
}),
});
};
@@ -347,7 +351,7 @@ export const resolveArtifactPath = (
specifier: string,
sourcePath: string,
files: ReadonlyMap<string, JavaScriptArtifactFile>,
moduleKind?: "import" | "require",
moduleKind?: "import" | "require" | "compiler-dependent",
): string | null =>
resolveArtifactPathByContext({
declaredPath: specifier,
@@ -12,7 +12,6 @@ import {
javascriptAnalysisCoverage,
linkElectronRoleToAsset,
moduleLookupKey,
resolveArtifactPath,
selfReferenceOmissions,
sourceNodeFor,
type JavaScriptArtifactGraphContext,
@@ -23,9 +22,11 @@ import {
staticInferenceEvidence,
} from "./JavaScriptArtifactGraphEvidence.js";
import {
moduleKindForJavaScriptReference,
artifactPathResolutionProperties,
resolveArtifactPathByContext,
type ArtifactPathResolution,
} from "./JavaScriptArtifactPathResolution.js";
import { moduleResolutionModeForJavaScriptReference } from "../../domain/javascript/javascriptModuleResolutionMode.js";
type StaticReference = JavaScriptStaticAnalysis["references"][number];
type StaticEndpoint = JavaScriptStaticAnalysis["endpoints"][number];
@@ -41,9 +42,12 @@ interface FindingInput<Value> {
}
interface ResolvedReference {
readonly path: string;
readonly file: JavaScriptArtifactFile;
readonly path: string | null;
readonly file: JavaScriptArtifactFile | null;
readonly node: ApplicationNode;
readonly limitations: readonly string[];
readonly resolutionStatus?: string;
readonly resolution?: ArtifactPathResolution;
}
/** Project imports, workers, roles, endpoints, storage, and source-map links. */
@@ -90,7 +94,7 @@ const addReference = (
const resolved =
input.value.specifier === null
? null
: resolveReference(context, input.file.path, input.value);
: resolveReference(context, input.file.path, input.value, input.coverage);
if (input.value.kind === "worker" || input.value.kind === "service-worker") {
const worker = context.accumulator.addNode({
kind: input.value.kind === "worker" ? "worker" : "service-worker",
@@ -126,7 +130,7 @@ const addReference = (
relation: "loads",
properties: { kind: input.value.kind },
});
if (resolved !== null)
if (resolved !== null && resolved.path !== null)
addStaticInferenceEdge(context, {
source: worker,
target: resolved.node,
@@ -135,6 +139,7 @@ const addReference = (
coverage: input.coverage,
relation: "maps_to",
properties: { resolved_path: resolved.path },
limitations: resolved.limitations,
});
return;
}
@@ -154,13 +159,25 @@ const addReference = (
file: input.file,
range: input.value.location,
coverage: input.coverage,
relation: resolved?.file.kind === "native-addon" ? "loads" : "imports",
relation: resolved?.file?.kind === "native-addon" ? "loads" : "imports",
properties: {
kind: input.value.kind,
specifier: input.value.specifier,
expression: input.value.expression,
resolution_mode:
input.value.kind === "static-import" ||
input.value.kind === "dynamic-import" ||
input.value.kind === "require"
? moduleResolutionModeForJavaScriptReference(
input.file.path,
input.value.kind,
)
: null,
resolved_path: resolved?.path ?? null,
resolution_status: resolved?.resolutionStatus ?? null,
...artifactPathResolutionProperties(resolved?.resolution),
},
...(resolved === null ? {} : { limitations: resolved.limitations }),
});
};
@@ -373,6 +390,7 @@ const resolveReference = (
context: JavaScriptArtifactGraphContext,
sourcePath: string,
reference: StaticReference,
coverage: JavaScriptArtifactGraphCoverage,
): ResolvedReference | null => {
const specifier = reference.specifier;
if (specifier === null) return null;
@@ -392,30 +410,70 @@ const resolveReference = (
const file = context.filesByPath.get(sourcePath);
return file === undefined
? null
: { path: `${sourcePath}#module:${specifier}`, file, node: module };
: {
path: `${sourcePath}#module:${specifier}`,
file,
node: module,
limitations: [],
};
}
// Worker scripts are URLs relative to the script, not module specifiers.
const path = isWorker
const resolution = isWorker
? resolveArtifactPathByContext({
declaredPath: specifier,
sourcePath,
context: "url-reference",
files: context.filesByPath,
}).resolved_path
: resolveArtifactPath(
specifier,
})
: resolveArtifactPathByContext({
declaredPath: specifier,
sourcePath,
context.filesByPath,
reference.kind === "require" ||
reference.kind === "static-import" ||
reference.kind === "dynamic-import"
? moduleKindForJavaScriptReference(sourcePath, reference.kind)
: undefined,
);
if (path === null) return null;
context: "module-specifier",
files: context.filesByPath,
...("require" === reference.kind ||
"static-import" === reference.kind ||
"dynamic-import" === reference.kind
? {
moduleKind: moduleResolutionModeForJavaScriptReference(
sourcePath,
reference.kind,
),
}
: {}),
});
const path = resolution.resolved_path;
if (path === null) {
if (isWorker) return null;
const source = context.filesByPath.get(sourcePath);
return source === undefined
? null
: {
path: null,
file: null,
node: unresolvedReferenceNode(
context,
source,
reference,
coverage,
resolution,
),
limitations: resolution.limitations,
resolutionStatus: resolution.resolution_status,
resolution,
};
}
const file = context.filesByPath.get(path);
const node = context.fileNodes.get(path);
return file === undefined || node === undefined ? null : { path, file, node };
return file === undefined || node === undefined
? null
: {
path,
file,
node,
limitations: resolution.limitations,
resolutionStatus: resolution.resolution_status,
resolution,
};
};
const resolveBundlerChunkReference = (
@@ -453,7 +511,7 @@ const resolveBundlerChunkReference = (
const node = unique[0];
return node === undefined
? null
: { path: `${sourcePath}#chunk:${key}`, file, node };
: { path: `${sourcePath}#chunk:${key}`, file, node, limitations: [] };
};
const unresolvedReferenceNode = (
@@ -461,6 +519,7 @@ const unresolvedReferenceNode = (
file: JavaScriptArtifactFile,
reference: StaticReference,
coverage: JavaScriptArtifactGraphCoverage,
resolution?: ArtifactPathResolution,
): ApplicationNode =>
context.accumulator.addNode({
kind: "javascript-module",
@@ -477,6 +536,10 @@ const unresolvedReferenceNode = (
specifier: reference.specifier,
expression: reference.expression,
resolution: "unresolved",
...(resolution === undefined
? {}
: { resolution_status: resolution.resolution_status }),
...artifactPathResolutionProperties(resolution),
},
evidence: staticInferenceEvidence({
sha256: file.sha256,
@@ -487,6 +550,7 @@ const unresolvedReferenceNode = (
confidence: "low",
limitations: [
"The static reference could not be resolved within the inventoried artifact.",
...(resolution?.limitations ?? []),
],
}),
},
@@ -1890,21 +1890,40 @@ describe("TypeScript NodeNext emitted-extension resolution", () => {
).toMatchObject({ resolution_status: "resolved", resolved_path: target });
},
);
});
describe("TypeScript declaration fallback ownership", () => {
const container = "container";
it.each([
["./dep.js", "src/dep.d.ts"],
["./cfg.mjs", "src/cfg.d.mts"],
["./legacy.cjs", "src/legacy.d.cts"],
] as const)(
"resolves %s to a declaration counterpart",
"retains %s as compile-time evidence without inventing a runtime target",
(specifier, target) => {
const files = fileMap([
file("src/main.ts", container),
file(target, container, "export const value = 1;"),
]);
for (const moduleKind of ["import", "require"] as const) {
expect(
resolve({
declaredPath: specifier,
sourcePath: "src/main.ts",
context: "module-specifier",
moduleKind,
files,
}),
).toMatchObject({
resolution_status: "unavailable",
resolved_path: null,
candidate_path: target,
});
}
expect(
resolve({
declaredPath: specifier,
declaredPath: "./" + target.slice(4),
sourcePath: "src/main.ts",
context: "module-specifier",
moduleKind: "import",
@@ -19,7 +19,13 @@ import {
classifyPackageImportTargetString,
lookupExactPackageImport,
} from "../../domain/javascript/javascriptPackageImportAliases.js";
import { typescriptEmittedSourceCandidates } from "../TypeScriptEmittedSourceCandidates.js";
import {
typescriptEmittedSourceCandidates,
type TypeScriptEmittedSourceCandidate,
} from "../TypeScriptEmittedSourceCandidates.js";
import { isRelativeJavaScriptModuleSpecifier } from "../../domain/javascript/javascriptModuleSpecifier.js";
import type { JavaScriptModuleResolutionMode } from "../../domain/javascript/javascriptModuleResolutionMode.js";
import type { JsonValue } from "../../domain/jsonValue.js";
type ArtifactPathResolutionContext =
| "package-entrypoint"
@@ -38,6 +44,16 @@ interface ArtifactPathResolutionBase {
readonly declared_path: string;
readonly resolution_context: ArtifactPathResolutionContext;
readonly limitations: readonly string[];
readonly conditional_resolutions?: readonly ConditionalModuleResolution[];
readonly candidate_path?: string;
}
export interface ConditionalModuleResolution {
readonly mode: "import" | "require";
readonly resolution_status: UnresolvedArtifactPathStatus | "resolved";
readonly resolved_path: string | null;
readonly candidate_path?: string;
readonly limitations: readonly string[];
}
/** Explicit outcome of artifact-confined path resolution. */
@@ -61,26 +77,13 @@ export interface ResolveArtifactPathInput {
readonly files: ReadonlyMap<string, JavaScriptArtifactFile>;
readonly htmlBaseHref?: string | null;
/** Import/require semantics; runtime resource paths omit this provenance. */
readonly moduleKind?: "import" | "require";
readonly moduleKind?: JavaScriptModuleResolutionMode;
}
/** Loader selected by a source-level JavaScript module reference. */
export const moduleKindForJavaScriptReference = (
sourcePath: string,
referenceKind: "static-import" | "dynamic-import" | "require",
resolutionMode?: "import" | "require",
): "import" | "require" =>
resolutionMode ??
(referenceKind === "require" ||
(referenceKind === "static-import" &&
posix.extname(sourcePath).toLowerCase() === ".cts")
? "require"
: "import");
/** Preserve an explicitly retained loader mode on graph relationships. */
export const moduleResolutionModeProperties = (
mode: "import" | "require" | undefined,
): Readonly<Record<string, "import" | "require">> =>
mode: JavaScriptModuleResolutionMode | undefined,
): Readonly<Record<string, JavaScriptModuleResolutionMode>> =>
mode === undefined ? {} : { resolution_mode: mode };
const EXTENSIONS = [
@@ -109,43 +112,116 @@ interface PackageFileTarget {
readonly url: URL;
}
type CandidateResolution =
| {
readonly resolvedPath: string;
readonly status: "resolved";
readonly limitations: readonly string[];
}
type CandidateResolution = {
readonly limitations: readonly string[];
readonly candidatePath?: string;
readonly conditionalResolutions?: readonly ConditionalModuleResolution[];
} & (
| { readonly resolvedPath: string; readonly status: "resolved" }
| {
readonly resolvedPath: null;
readonly status: UnresolvedArtifactPathStatus;
readonly limitations: readonly string[];
/** A truthy package main failed, so Node must not try ancestor packages. */
readonly stopsAncestorLookup?: true;
};
}
);
/** Resolve one declaration while preserving its original input identity. */
export const resolveArtifactPathByContext = (
input: ResolveArtifactPathInput,
): ArtifactPathResolution => {
const resolution = resolvePath(input);
const facts = {
declared_path: input.declaredPath,
resolution_context: input.context,
limitations: resolution.limitations,
...(resolution.candidatePath === undefined
? {}
: { candidate_path: resolution.candidatePath }),
...(resolution.conditionalResolutions === undefined
? {}
: { conditional_resolutions: resolution.conditionalResolutions }),
};
return resolution.status === "resolved"
? {
declared_path: input.declaredPath,
resolution_context: input.context,
...facts,
resolved_path: resolution.resolvedPath,
resolution_status: resolution.status,
limitations: resolution.limitations,
}
: {
declared_path: input.declaredPath,
resolution_context: input.context,
resolved_path: null,
resolution_status: resolution.status,
limitations: resolution.limitations,
};
: { ...facts, resolved_path: null, resolution_status: resolution.status };
};
/** Project retained conditional identities without flattening them into display text. */
export const artifactPathResolutionProperties = (
resolution: ArtifactPathResolution | undefined,
): Readonly<Record<string, JsonValue>> => ({
...(resolution?.candidate_path === undefined
? {}
: { candidate_path: resolution.candidate_path }),
...(resolution?.conditional_resolutions === undefined
? {}
: {
conditional_resolutions: resolution.conditional_resolutions.map(
(branch) => ({
mode: branch.mode,
resolution_status: branch.resolution_status,
resolved_path: branch.resolved_path,
limitations: [...branch.limitations],
...(branch.candidate_path === undefined
? {}
: { candidate_path: branch.candidate_path }),
}),
),
}),
});
const conditionalResolution = (
mode: "import" | "require",
result: CandidateResolution,
): ConditionalModuleResolution => ({
mode,
resolution_status: result.status,
resolved_path: result.resolvedPath,
limitations: result.limitations,
...(result.candidatePath === undefined
? {}
: { candidate_path: result.candidatePath }),
});
const resolvePath = (input: ResolveArtifactPathInput): CandidateResolution => {
if (input.moduleKind === "compiler-dependent") {
const imported = resolvePath({ ...input, moduleKind: "import" });
const required = resolvePath({ ...input, moduleKind: "require" });
const sameOutcome =
imported.status === required.status &&
imported.resolvedPath === required.resolvedPath &&
imported.candidatePath === required.candidatePath;
const limitation =
"Static TypeScript loader mode is compiler-dependent without retained compiler options. " +
(sameOutcome
? "Node import and require resolution agree on this outcome."
: "Node import and require resolution disagree; neither branch is an established target.");
return sameOutcome
? {
...imported,
limitations: [
...new Set([
...imported.limitations,
...required.limitations,
limitation,
]),
],
}
: {
resolvedPath: null,
status: "unavailable",
limitations: [limitation],
conditionalResolutions: [
conditionalResolution("import", imported),
conditionalResolution("require", required),
],
};
}
// Package import keys are literal lookup identities, not artifact paths.
// The selected target is admitted separately by resolveExportTarget.
if (
@@ -166,7 +242,7 @@ const resolvePath = (input: ResolveArtifactPathInput): CandidateResolution => {
);
const isModuleUrlPath =
input.context === "module-specifier" &&
(input.declaredPath.startsWith(".") ||
(isRelativeJavaScriptModuleSpecifier(input.declaredPath) ||
input.declaredPath.startsWith("/") ||
fileUrlPath(input.declaredPath) !== undefined);
const emittedSourceCandidates =
@@ -175,12 +251,47 @@ const resolvePath = (input: ResolveArtifactPathInput): CandidateResolution => {
: [];
if (isModuleUrlPath && input.moduleKind === "import")
return (
resolveFileCandidates(input, [confined, ...emittedSourceCandidates]) ??
notFoundCandidate()
resolveRuntimeSourceCandidates(
input,
confined,
emittedSourceCandidates,
) ?? notFoundCandidate()
);
return resolveCandidate(input, confined, new Set(), emittedSourceCandidates);
};
const resolveRuntimeSourceCandidates = (
input: ResolveArtifactPathInput,
exactPath: string,
sourceCandidates: readonly TypeScriptEmittedSourceCandidate[],
): CandidateResolution | null => {
const source = input.files.get(input.sourcePath);
const exactCandidate: TypeScriptEmittedSourceCandidate = {
path: exactPath,
kind: "implementation",
};
for (const candidate of [exactCandidate, ...sourceCandidates]) {
const target = input.files.get(candidate.path);
if (
target === undefined ||
(source !== undefined &&
target.container_sha256 !== source.container_sha256)
)
continue;
return candidate.kind === "declaration"
? {
resolvedPath: null,
status: "unavailable",
candidatePath: candidate.path,
limitations: [
"Only a TypeScript declaration fallback was found; it does not establish the requested runtime module.",
],
}
: { resolvedPath: candidate.path, status: "resolved", limitations: [] };
}
return null;
};
const rejectDeclaration = (declared: string): CandidateResolution | null => {
if (declared.length === 0)
return unresolvedCandidate("rejected", ["The declared path is empty."]);
@@ -229,7 +340,10 @@ const contextualCandidate = (
return unresolvedCandidate("external", [
"URL and Node builtin schemes are outside static artifact module resolution.",
]);
if (!declared.startsWith(".") && !declared.startsWith("/"))
if (
!isRelativeJavaScriptModuleSpecifier(declared) &&
!declared.startsWith("/")
)
return bareModuleCandidate(input);
if (input.moduleKind !== "require") {
const decoded = decodeModuleUrlPath(stripQueryAndFragment(declared));
@@ -275,7 +389,10 @@ const contextualCandidate = (
"URL",
);
}
return relativeCandidate(input.sourcePath, declared);
const candidate = relativeCandidate(input.sourcePath, declared);
return context === "module-specifier" && input.moduleKind === "import"
? exactUrlPathCandidate(declared, candidate, "module URL")
: candidate;
};
const decodeModuleUrlPath = (
@@ -310,9 +427,12 @@ const bareModuleCandidate = (
return unresolvedCandidate("external", [
"The bare specifier is a Node builtin or invalid package name.",
]);
if (input.moduleKind === "import" && request.packageName.includes("%"))
if (
input.moduleKind === "import" &&
(request.packageName.startsWith(".") || request.packageName.includes("%"))
)
return unresolvedCandidate("rejected", [
"Percent signs are not admitted in ES module package names.",
"ES module package names cannot start with a dot or contain percent signs.",
]);
const self = packageSelfCandidate(input, request);
if (self !== null) return self;
@@ -735,7 +855,7 @@ const htmlCandidate = (
const exactUrlPathCandidate = (
declared: string,
candidate: string,
kind: "HTML" | "URL",
kind: "HTML" | "URL" | "module URL",
): string | CandidateResolution => {
if (!urlPathIsDirectory(declared)) return candidate;
const confined = confineCandidate(candidate);
@@ -839,12 +959,10 @@ const resolveCandidate = (
input: ResolveArtifactPathInput,
candidate: string,
packageChain: ReadonlySet<string>,
emittedSourceCandidates: readonly string[] = [],
emittedSourceCandidates: readonly TypeScriptEmittedSourceCandidate[] = [],
): CandidateResolution =>
resolveFileCandidates(input, [
...fileCandidates(candidate, input.moduleKind),
...emittedSourceCandidates,
]) ??
resolveFileCandidates(input, fileCandidates(candidate, input.moduleKind)) ??
resolveRuntimeSourceCandidates(input, candidate, emittedSourceCandidates) ??
resolveDirectory(
input,
candidate,
@@ -930,11 +1048,26 @@ const resolveFileCandidates = (
(source === undefined ||
target.container_sha256 === source.container_sha256)
)
return { resolvedPath: path, status: "resolved", limitations: [] };
return isRuntimeDeclarationPath(input, path)
? {
resolvedPath: null,
status: "unavailable",
candidatePath: path,
limitations: [
`The TypeScript declaration ${path} is compile-time evidence and cannot identify a runtime module implementation.`,
],
}
: { resolvedPath: path, status: "resolved", limitations: [] };
}
return null;
};
const isRuntimeDeclarationPath = (
input: ResolveArtifactPathInput,
path: string,
): boolean =>
input.moduleKind !== undefined && /\.d\.(?:ts|mts|cts)$/iu.test(path);
const NODE_MODULE_EXTENSIONS = [".js", ".json", ".node"] as const;
const fileCandidates = (
@@ -18,6 +18,7 @@ import type {
JavaScriptModuleOrigin,
JavaScriptSemanticModuleLink,
} from "../../domain/javascript/javascriptSemanticIr.js";
import { moduleResolutionModeForJavaScriptReference } from "../../domain/javascript/javascriptModuleResolutionMode.js";
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
import {
addAstContainsEdge,
@@ -30,7 +31,7 @@ import {
staticInferenceEvidence,
} from "./JavaScriptArtifactGraphEvidence.js";
import {
moduleKindForJavaScriptReference,
artifactPathResolutionProperties,
moduleResolutionModeProperties,
resolveArtifactPathByContext,
type ArtifactPathResolution,
@@ -54,6 +55,7 @@ interface RelationshipInput {
type ResolvedModuleTarget = {
readonly node: ApplicationNode;
readonly limitations: readonly string[];
readonly resolution?: ArtifactPathResolution;
} & (
| {
readonly path: string;
@@ -81,8 +83,13 @@ export const addJavaScriptSourceModules = (
if (asset === undefined || semantic === null || program === undefined)
continue;
const exports = semantic.ir.moduleLinks
.flatMap(({ exportedName }) =>
exportedName === null ? [] : [exportedName],
.flatMap(({ exportedName, kind }) =>
exportedName === null || kind === "type-import" ? [] : [exportedName],
)
.sort(compareUnicodeCodePoints);
const typeExports = semantic.ir.moduleLinks
.flatMap(({ exportedName, kind }) =>
exportedName === null || kind !== "type-import" ? [] : [exportedName],
)
.sort(compareUnicodeCodePoints);
const retainedExports = [...new Set(exports)];
@@ -105,8 +112,12 @@ export const addJavaScriptSourceModules = (
import_relationships: semantic.ir.moduleLinks.filter(
({ kind }) => kind === "import" || kind === "require",
).length,
type_relationships: semantic.ir.moduleLinks.filter(
({ kind }) => kind === "type-import",
).length,
export_relationships: exports.length,
export_names: retainedExports,
type_export_names: [...new Set(typeExports)],
omitted_export_names: 0,
},
evidence: astObservationEvidence({
@@ -152,6 +163,7 @@ export function* addJavaScriptModuleRelationshipsSteps(
for (const link of semantic.ir.moduleLinks) {
// One module can export thousands of bindings, each with return shapes.
yield;
if (link.kind === "type-import") continue;
const input = { context, file, semantic, source, link };
if (isExportLink(link)) addExportRelationship(input, omissions);
else if (link.specifier !== null)
@@ -277,7 +289,7 @@ const resolveModuleTarget = (
sourcePath: input.file.path,
context: "module-specifier",
files: input.context.filesByPath,
moduleKind: moduleKindForJavaScriptReference(
moduleKind: moduleResolutionModeForJavaScriptReference(
input.file.path,
input.link.kind === "require" || input.link.kind === "commonjs-export"
? "require"
@@ -287,15 +299,12 @@ const resolveModuleTarget = (
});
if (resolution.resolution_status !== "resolved")
return {
node: unresolvedModuleNode(
input,
specifier,
resolution.resolution_status,
),
node: unresolvedModuleNode(input, specifier, resolution),
path: null,
file: null,
status: resolution.resolution_status,
limitations: resolution.limitations,
resolution,
};
const path = resolution.resolved_path;
const file = input.context.filesByPath.get(path);
@@ -312,20 +321,24 @@ const resolveModuleTarget = (
file,
status: "resolved",
limitations: resolution.limitations,
resolution,
};
};
const unresolvedModuleNode = (
input: RelationshipInput,
specifier: string,
status: Exclude<ResolvedModuleTarget["status"], "resolved">,
resolution: Extract<
ArtifactPathResolution,
{ resolution_status: Exclude<ResolvedModuleTarget["status"], "resolved"> }
>,
): ApplicationNode =>
input.context.accumulator.addNode({
kind: "javascript-module",
identity: artifactLocalIdentity(
input.file.sha256,
"unresolved-semantic-module",
`${status}:${specifier}`,
`${resolution.resolution_status}:${specifier}`,
),
observations: [
{
@@ -333,7 +346,8 @@ const unresolvedModuleNode = (
properties: {
semantic_role: "module-reference",
declared_specifier: specifier,
resolution_status: status,
resolution_status: resolution.resolution_status,
...artifactPathResolutionProperties(resolution),
},
evidence: staticInferenceEvidence({
sha256: input.file.sha256,
@@ -354,6 +368,9 @@ const moduleOriginForExport = (
): JavaScriptModuleOrigin | null => {
if (link.specifier !== null)
return {
...(link.resolutionMode === undefined
? {}
: { resolutionMode: link.resolutionMode }),
specifier: link.specifier,
importedPath: link.importedName === null ? [] : [link.importedName],
};
@@ -383,6 +400,7 @@ const relationshipProperties = (
exported_name: link.exportedName,
resolved_path: target?.path ?? null,
resolution_status: target?.status ?? null,
...artifactPathResolutionProperties(target?.resolution),
target_file_kind: target?.file?.kind ?? null,
target_json_status:
target?.file?.kind === "json" || target?.file?.kind === "package-json"
@@ -431,6 +449,7 @@ const moduleFormat = (
const extension = posix.extname(path).toLowerCase();
if (extension === ".mjs" || extension === ".mts") return "esm";
if (extension === ".cjs" || extension === ".cts") return "commonjs";
if (extension === ".ts" || extension === ".tsx") return "unknown";
const esm = ir.moduleLinks.some(({ kind }) =>
["import", "export", "re-export"].includes(kind),
);
@@ -1,3 +1,9 @@
import {
inspectSignalDecodersInputSchema,
signalDecodersSchema,
type InspectSignalDecodersInput,
} from "../../domain/signals/signalDecoders.js";
import type { JsonValue } from "../../domain/jsonValue.js";
import { isAbsolute } from "node:path";
import type {
AnalysisExecution,
@@ -24,6 +30,10 @@ export interface SignalRecordingPort {
input: DecodeSignalRecordingInput,
options?: ExecutionOptions,
): Promise<Result<AnalysisExecution, AnalysisError>>;
inspectDecoders(
input: InspectSignalDecodersInput,
options?: ExecutionOptions,
): Promise<Result<AnalysisExecution, AnalysisError>>;
close(): Promise<void>;
}
@@ -74,14 +84,58 @@ export class SignalRecordingService {
"Decoder returned malformed data or changed the selected recording identity/options.",
),
);
const evidence = createEvidence(execution.subject, execution.provider, {
operation,
parameters: input.data,
result: report.data,
confidence: "observed",
locations: execution.locations,
limitations: execution.limitations,
});
return this.#evidence(operation, input.data, execution, options);
}
async inspectDecoders(
raw: unknown,
options?: ExecutionOptions,
): Promise<Result<Evidence, AnalysisError>> {
const operation = "inspect_signal_decoders";
if (options?.signal?.aborted)
return err(new AnalysisCancelledError(operation));
const input = inspectSignalDecodersInputSchema.safeParse(raw);
if (!input.success)
return err(
analysisInputErrorFromIssues(operation, input.error.issues, raw, {
cause: input.error,
}),
);
const decoded = await this.provider.inspectDecoders(input.data, options);
if (!decoded.ok) return decoded;
const execution = decoded.value;
const report = signalDecodersSchema.safeParse(execution.result);
if (
!report.success ||
JSON.stringify(report.data.selection) !== JSON.stringify(input.data) ||
execution.subject?.path !== report.data.tool.path ||
execution.subject.sha256 !== report.data.tool.sha256
)
return err(
new AnalysisOutputError(
operation,
"Decoder discovery returned malformed data or changed the selected installation/options.",
),
);
return this.#evidence(operation, input.data, execution, options);
}
#evidence(
operation: string,
parameters: Readonly<Record<string, JsonValue>>,
execution: AnalysisExecution,
options?: ExecutionOptions,
): Result<Evidence, AnalysisError> {
const evidence = createEvidence(
execution.subject ?? undefined,
execution.provider,
{
operation,
parameters,
result: execution.result,
confidence: "observed",
locations: execution.locations,
limitations: execution.limitations,
},
);
return options?.signal?.aborted
? err(
new AnalysisCancelledError(operation, {
+4
View File
@@ -57,6 +57,10 @@ export class ArtifactResourceScope {
#draining: Promise<void> | undefined;
#closed = false;
get hasPending(): boolean {
return this.#pending.size > 0;
}
/** Admit a resource operation while the scope remains open. */
async run<T>(operation: () => Promise<T>): Promise<T> {
while (true) {
+8 -4
View File
@@ -8,11 +8,11 @@ import {
type NativeCommandRunner,
} from "../native/CommandRunner.js";
import {
lipoSubtypeBits,
parseLipoArchitectures,
type LipoArchitecture,
} from "../native/parsers/lipo.js";
import type { MachoSlice } from "../domain/apple/dylibResolution.js";
import { machoCpuSubtypeBase } from "../domain/apple/machoCpuIdentity.js";
import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
import { readMachoImage, type ReadAt } from "./apple/MachoLoadCommandReader.js";
import {
@@ -81,7 +81,9 @@ export class MachOSliceArtifactReader implements ArtifactReader {
if (!captured.ok)
throw new ArtifactReaderFailure(
captured.error.reason === "cancelled" ? "cancelled" : "unavailable",
"lipo could not enumerate universal Mach-O slices",
captured.error.detail === null
? "lipo could not enumerate universal Mach-O slices"
: `lipo could not enumerate universal Mach-O slices: ${captured.error.detail}`,
{
cause: captured.error,
...(captured.error.cleanupFailure === null &&
@@ -469,8 +471,10 @@ const lipoSubtypeMatches = (
): boolean => {
if (reported.cpu_subtype_code !== null)
return reported.cpu_subtype_code === observedCpuSubtype;
const bits = lipoSubtypeBits(reported.cpu_subtype);
return bits === null || bits === (observedCpuSubtype & 0x00ffffff);
return (
reported.cpu_subtype_base === null ||
reported.cpu_subtype_base === machoCpuSubtypeBase(observedCpuSubtype)
);
};
/**
+22 -11
View File
@@ -280,9 +280,11 @@ export class SafeOutputTree {
}
/** Sync the owned output tree and prevent further writes through this instance. */
async commit(): Promise<void> {
async commit(signal?: AbortSignal): Promise<void> {
this.#assertWritable();
abortIfNeeded(signal);
await this.#assertOwnedDirectories();
abortIfNeeded(signal);
// Windows has no directory fsync; file contents are already synced in write().
if (this.platform === "win32") {
this.#published = true;
@@ -315,6 +317,7 @@ export class SafeOutputTree {
);
await output.handle.sync();
await parent.handle.sync();
abortIfNeeded(signal);
this.#published = true;
} catch (cause: unknown) {
sync = { kind: "failed", cause };
@@ -336,23 +339,26 @@ export class SafeOutputTree {
closeFailure,
this.#outputRoot,
);
if (sync.kind === "failed")
throw ArtifactReaderFailure.withCleanup(
new ArtifactReaderFailure(
"path",
"Could not durably sync extraction output",
{ cause: sync.cause },
),
cleanup,
);
if (sync.kind === "failed") {
const primary = isCancellationFailure(sync.cause)
? sync.cause
: new ArtifactReaderFailure(
"path",
"Could not durably sync extraction output",
{ cause: sync.cause },
);
throw ArtifactReaderFailure.withCleanup(primary, cleanup);
}
throw ArtifactReaderFailure.withCleanup(closeFailure, cleanup);
}
if (sync.kind === "failed")
if (sync.kind === "failed") {
if (isCancellationFailure(sync.cause)) throw sync.cause;
throw new ArtifactReaderFailure(
"path",
"Could not durably sync extraction output",
{ cause: sync.cause },
);
}
}
/** Remove only this operation's unsealed tree and verify absence. */
@@ -590,6 +596,11 @@ const abortIfNeeded = (signal?: AbortSignal): void => {
);
};
const isCancellationFailure = (
cause: unknown,
): cause is ArtifactReaderFailure =>
cause instanceof ArtifactReaderFailure && cause.reason === "cancelled";
const isAlreadyExists = (cause: unknown): boolean =>
cause instanceof Error && "code" in cause && cause.code === "EEXIST";
+4 -10
View File
@@ -17,6 +17,7 @@ import { DirectoryArtifactReader } from "../DirectoryArtifactReader.js";
import { SafeOutputTree } from "../SafeOutputTree.js";
import { SafeOutputTreeCreationFailure } from "../SafeOutputTreeCreationFailure.js";
import { ZipArtifactReader } from "../ZipArtifactReader.js";
import { isZipPackageFormat } from "../../domain/zipPackageFormat.js";
import { MachOSliceArtifactReader } from "../MachOSliceArtifactReader.js";
import {
artifactExtractionResultSchema,
@@ -248,7 +249,7 @@ const materializeSelection = async ({
compareUnicodeCodePoints(left.relative_path, right.relative_path),
);
completed = createExtractionResult(input, inventory, selected, extracted);
await output.commit();
await output.commit(signal);
return completed;
} catch (cause: unknown) {
const cleanupFailures = await releaseExtractionOwners(
@@ -400,13 +401,6 @@ interface ExtractionInventory {
readonly integrityContradictions: readonly IntegrityContradiction[];
}
const ZIP_FORMATS = ["ipa", "apk", "msix", "appx", "zip"] as const;
const isZipFormat = (
format: BinaryTarget["format"],
): format is (typeof ZIP_FORMATS)[number] =>
(ZIP_FORMATS as readonly string[]).includes(format);
/**
* Refuse a format without an extraction reader before inventory work starts.
* The target kind, not the host, is unsupported by extraction. The error keeps
@@ -422,7 +416,7 @@ const requireExtractableFormat = async (
!directory &&
format !== "asar" &&
!isMachOFormat(format) &&
!isZipFormat(format)
!isZipPackageFormat(format)
)
throw new AnalysisUnsupportedTargetError(
"extract_artifact",
@@ -443,7 +437,7 @@ const createReader = async (
if (await requireExtractableFormat(path, input))
return new DirectoryArtifactReader(path);
if (format === "asar") return new AsarArtifactReader(path);
if (isZipFormat(format)) return new ZipArtifactReader(path, format);
if (isZipPackageFormat(format)) return new ZipArtifactReader(path, format);
return new MachOSliceArtifactReader(path, input.environment);
};
+10 -2
View File
@@ -38,6 +38,13 @@ const afterValidation = async <T>(
}
};
const assertReadActive = (signal: AbortSignal | undefined): void => {
if (signal?.aborted)
throw new ArtifactReaderFailure("cancelled", "Artifact read cancelled", {
cause: signal.reason,
});
};
/** Read one bounded regular file and reject replacement or in-place changes. */
export const readStableArtifact = async (
path: string,
@@ -46,7 +53,7 @@ export const readStableArtifact = async (
fileSystem: StableArtifactFileSystem = FILE_SYSTEM,
resourceScope?: ArtifactResourceScope,
): Promise<{ readonly bytes: Buffer; readonly sha256: string }> => {
signal?.throwIfAborted();
assertReadActive(signal);
const before = await fileSystem.lstat(path);
if (!before.isFile() || before.isSymbolicLink())
throw new ArtifactReaderFailure(
@@ -84,7 +91,7 @@ export const readStableArtifact = async (
autoClose: false,
highWaterMark: 65536,
})) {
signal?.throwIfAborted();
assertReadActive(signal);
const value: unknown = chunk;
if (!Buffer.isBuffer(value))
throw new ArtifactReaderFailure(
@@ -120,6 +127,7 @@ export const readStableArtifact = async (
"integrity",
`Artifact changed during read: ${path}`,
);
assertReadActive(signal);
const bytes = Buffer.concat(chunks, count);
return { bytes, sha256: createHash("sha256").update(bytes).digest("hex") };
} catch (cause: unknown) {
@@ -91,7 +91,7 @@ export const browserScenarioCaptureData = (input: {
network_content: input.scenario.capture.network,
event_families: [...new Set(input.scenario.capture.events)].sort(),
},
duration_ms: Date.now() - input.startedAt,
duration_ms: Math.max(0, Math.round(performance.now() - input.startedAt)),
steps: [...input.steps],
events,
limitations: [
@@ -303,6 +303,7 @@ describe("PlaywrightBrowserScenarioProvider", () => {
});
expect(result.value.limitations).toContain(session.eventGaps[0]);
});
it("returns an initial state and every step beyond the former action ceiling", async () => {
const session = new FakeSession("launch");
const provider = new PlaywrightBrowserScenarioProvider(() =>
@@ -376,6 +377,35 @@ describe("PlaywrightBrowserScenarioProvider", () => {
});
});
describe("PlaywrightBrowserScenarioProvider timing", () => {
it("keeps capture elapsed values valid across a backward wall-clock jump", async () => {
let wallClock = 10_000;
vi.spyOn(Date, "now").mockImplementation(() => wallClock);
const session = new FakeSession("launch");
session.onPerform = () => {
wallClock = 1;
};
const provider = new PlaywrightBrowserScenarioProvider(() =>
Promise.resolve(session),
);
try {
const result = await provider.captureScenario(scenario());
if (!result.ok) throw result.error;
expect(wallClock).toBe(1);
expect(Number.isInteger(result.value.duration_ms)).toBe(true);
expect(result.value.duration_ms).toBeGreaterThanOrEqual(0);
expect(
result.value.steps.every(
({ elapsed_ms }) => Number.isInteger(elapsed_ms) && elapsed_ms >= 0,
),
).toBe(true);
} finally {
vi.restoreAllMocks();
}
});
});
describe("PlaywrightBrowserScenarioProvider comparison coverage", () => {
it("keeps quiet captures with different event families from comparing unchanged", async () => {
const capture = async (events: readonly BrowserScenarioEventFamily[]) => {
@@ -172,7 +172,7 @@ const executeStep = async (input: {
),
});
const actionStartedAt = Date.now();
const actionStartedAt = performance.now();
session.setStep(stepIndex);
const requestCancelled = (): boolean => signal?.aborted === true;
let outcome: BrowserScenarioStepOutcome = {
@@ -200,7 +200,7 @@ const executeStep = async (input: {
stepId: action.step_id,
action: action.action,
...outcome,
elapsedMs: Date.now() - actionStartedAt,
elapsedMs: Math.max(0, Math.round(performance.now() - actionStartedAt)),
beforeUrl,
afterUrl: session.sanitizeUrl(session.currentUrl()),
eventStart,
@@ -233,7 +233,7 @@ const runScenario = async (
): Promise<BrowserScenarioCapture> => {
if (options.signal?.aborted === true)
throw new BrowserObservationError(OPERATION, "cancelled");
const startedAt = Date.now();
const startedAt = performance.now();
const session = await openSession(scenario, {
...(options.signal === undefined ? {} : { signal: options.signal }),
retainCleanup,
@@ -250,7 +250,7 @@ const runScenario = async (
await initialStep({
session,
scenario,
elapsedMs: Date.now() - startedAt,
elapsedMs: Math.max(0, Math.round(performance.now() - startedAt)),
signal: options.signal,
}),
);
+10 -4
View File
@@ -72,7 +72,7 @@ export const runElectronActions = async (
): Promise<ElectronActions> => {
const actions: ElectronActions = [];
for (const action of input.actions) {
const actionStartedAt = Date.now();
const actionStartedAt = performance.now();
const windowIndex = actionWindowIndex(action);
let selectedWindow: Page | undefined;
try {
@@ -107,7 +107,10 @@ export const runElectronActions = async (
target:
selectedWindow === undefined ? null : `window:${String(windowIndex)}`,
status: "completed",
elapsed_ms: Date.now() - actionStartedAt,
elapsed_ms: Math.max(
0,
Math.round(performance.now() - actionStartedAt),
),
error: null,
});
} catch (cause: unknown) {
@@ -118,7 +121,10 @@ export const runElectronActions = async (
target:
selectedWindow === undefined ? null : `window:${String(windowIndex)}`,
status: options.signal?.aborted ? "cancelled" : "failed",
elapsed_ms: Date.now() - actionStartedAt,
elapsed_ms: Math.max(
0,
Math.round(performance.now() - actionStartedAt),
),
error: safeActionErrorMessage(cause),
});
break;
@@ -366,7 +372,7 @@ export const runWithExecutionLimits = async <Value>(
new Promise<never>((_, reject) => {
timer = setTimeout(
() => reject(new BrowserObservationError(OPERATION, "timeout")),
Math.max(1, deadline - Date.now()),
Math.max(1, deadline - performance.now()),
);
}),
);
@@ -86,7 +86,7 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati
if (options.signal?.aborted === true)
throw new BrowserObservationError(OPERATION, "cancelled");
await prepareProcessOwnershipInspection(options.signal);
const startupDeadline = Date.now() + STARTUP_TIMEOUT_MS;
const startupDeadline = performance.now() + STARTUP_TIMEOUT_MS;
paths = {
executable: input.executable_path,
application: input.application_path,
@@ -101,7 +101,7 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati
cwd: paths.root,
env: safeElectronEnvironment(this.environment, runId),
args: ["-r", hookPath, paths.application, ...input.args],
timeout: Math.max(1, startupDeadline - Date.now()),
timeout: Math.max(1, startupDeadline - performance.now()),
});
const leaderPid = application.process().pid;
if (
@@ -128,7 +128,7 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati
const state = await runWithExecutionLimits(
readApplicationState(application),
undefined,
options.signal?.aborted ? Date.now() + 5_000 : undefined,
options.signal?.aborted ? performance.now() + 5_000 : undefined,
);
capture = projectElectronActiveCapture(paths, actions, state);
outcome = options.signal?.aborted
@@ -187,7 +187,7 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati
await runWithExecutionLimits(
application.close(),
undefined,
Date.now() + 5_000,
performance.now() + 5_000,
);
} catch (cause: unknown) {
closeError = cause;
@@ -7,7 +7,10 @@ import {
} from "../domain/analysisErrorCore.js";
describe("withPlaywrightExecutionBoundary", () => {
afterEach(() => vi.useRealTimers());
afterEach(() => {
vi.useRealTimers();
vi.restoreAllMocks();
});
it("rejects work that exceeds its absolute remaining duration", async () => {
vi.useFakeTimers();
@@ -31,4 +34,20 @@ describe("withPlaywrightExecutionBoundary", () => {
controller.abort();
await expect(result).rejects.toBeInstanceOf(AnalysisCancelledError);
});
it("keeps its timeout stable across a wall-clock jump", async () => {
vi.spyOn(Date, "now")
.mockReturnValueOnce(1_000)
.mockReturnValue(10_000_000);
await expect(
withPlaywrightExecutionBoundary(
() =>
new Promise<string>((resolve) =>
setTimeout(() => resolve("done"), 5),
),
1_000,
),
).resolves.toBe("done");
});
});
+2 -2
View File
@@ -19,10 +19,10 @@ export const withPlaywrightExecutionBoundary = <Value>(
let settled = false;
let timer: ReturnType<typeof setTimeout> | undefined;
const deadline =
timeoutMs === undefined ? undefined : Date.now() + timeoutMs;
timeoutMs === undefined ? undefined : performance.now() + timeoutMs;
const scheduleTimeout = (): void => {
if (deadline === undefined) return;
const remaining = deadline - Date.now();
const remaining = deadline - performance.now();
if (remaining <= 0) {
fail(new AnalysisTimeoutError(OPERATION, timeoutMs ?? 0));
return;
+1 -1
View File
@@ -133,7 +133,7 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
const secrets = BrowserScenarioSecrets.resolve(scenario, environment);
if (secrets === undefined)
throw new BrowserObservationError(OPERATION, "secret_unavailable");
const startedAt = Date.now();
const startedAt = performance.now();
const opening = openPlaywrightScenarioBrowser(scenario, environment, {
signal: options.signal,
...(options.retainCleanup === undefined
+3 -3
View File
@@ -86,7 +86,7 @@ export const fetchWebSourceMaps = async (
signal?.addEventListener("abort", abortFromCaller, { once: true });
const operationSignal = operationController.signal;
const deadlineAt =
Date.now() + (host.timeoutMs ?? SOURCE_MAP_FETCH_TIMEOUT_MS);
performance.now() + (host.timeoutMs ?? SOURCE_MAP_FETCH_TIMEOUT_MS);
const budget = {
retainedBytes: 0,
deadlineAt,
@@ -110,7 +110,7 @@ export const fetchWebSourceMaps = async (
}
const expandedBytesBeforeItem = budget.decodedRecords.expandedBytes;
budget.decodedRecords.expandedBytes += reservation;
if (operationSignal.aborted || Date.now() >= deadlineAt) {
if (operationSignal.aborted || performance.now() >= deadlineAt) {
items.push(
emptySourceMapItem(
request,
@@ -789,7 +789,7 @@ const checkOperation = (
): void => {
if (callerSignal?.aborted === true) throw callerSignal.reason;
if (operationSignal?.aborted === true) throw operationSignal.reason;
if (Date.now() >= deadlineAt) throw new SourceMapDeadlineError();
if (performance.now() >= deadlineAt) throw new SourceMapDeadlineError();
};
const approvedUrl = (
+31 -1
View File
@@ -1,6 +1,7 @@
import { lstat, realpath } from "node:fs/promises";
import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js";
import { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js";
import { readStableArtifact } from "../../artifacts/readStableArtifact.js";
import { normalizeArtifactPath } from "../../artifacts/ArtifactPaths.js";
import type { WebScriptArtifactPort } from "../../application/WebScriptArtifactPort.js";
@@ -19,10 +20,36 @@ import {
/** Read one digest-verified captured source under the manifest's current location. */
export class LocalWebScriptArtifacts implements WebScriptArtifactPort {
constructor(readonly operation: WebArtifactReadContext["operation"]) {}
constructor(
readonly operation: WebArtifactReadContext["operation"],
readonly resources = new ArtifactResourceScope(),
) {}
close(): Promise<void> {
return this.resources.close();
}
/** Read the selected source; relocated exports retain their reported original root. */
async load(input: SelectedWebScriptInput, options?: ExecutionOptions) {
try {
return await this.resources.run(() => this.#load(input, options));
} catch (cause: unknown) {
return err(
webArtifactReadError(
cause,
{
operation: this.operation,
field: ["manifest_path"],
targetPath: input.manifest_path,
callerSelected: true,
},
options?.signal,
),
);
}
}
async #load(input: SelectedWebScriptInput, options?: ExecutionOptions) {
let field: readonly (string | number)[] = ["manifest_path"];
let targetPath = input.manifest_path;
let callerSelected = true;
@@ -31,6 +58,7 @@ export class LocalWebScriptArtifacts implements WebScriptArtifactPort {
input.manifest_path,
32 * 1024 * 1024,
options?.signal,
this.resources,
);
const manifest = webScriptExportManifestSchema.parse(manifestBytes.value);
field = ["script_index"];
@@ -65,6 +93,8 @@ export class LocalWebScriptArtifacts implements WebScriptArtifactPort {
sourcePath,
16 * 1024 * 1024,
options?.signal,
undefined,
this.resources,
);
await assertContained(root, sourcePath);
if (
+1 -1
View File
@@ -113,7 +113,7 @@ export const publishWebScripts = async (
options.signal,
);
options.signal?.throwIfAborted();
await tree.commit();
await tree.commit(options.signal);
return result;
} catch (cause: unknown) {
if (cause instanceof SafeOutputTreeCreationFailure) tree = cause.tree;
+34 -2
View File
@@ -1,5 +1,6 @@
import { z } from "zod";
import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js";
import type { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js";
import { readStableArtifact } from "../../artifacts/readStableArtifact.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import {
@@ -9,6 +10,8 @@ import {
} from "../../domain/analysisErrorCore.js";
import { ArtifactOperationError } from "../../domain/artifactOperationError.js";
import { ProviderAdapterError } from "../../domain/providerAdapterError.js";
import { ProviderCleanupError } from "../../domain/providerCleanupError.js";
import { analysisErrorWithCleanupFailure } from "../../domain/analysisErrorCleanup.js";
import { safeParseJson } from "../../domain/safeJson.js";
/** The selected artifact and input field owning a reader failure. */
@@ -42,8 +45,15 @@ export const readWebArtifactJson = async (
path: string,
maximumBytes: number,
signal?: AbortSignal,
resourceScope?: ArtifactResourceScope,
) => {
const data = await readStableArtifact(path, maximumBytes, signal);
const data = await readStableArtifact(
path,
maximumBytes,
signal,
undefined,
resourceScope,
);
const value = safeParseJson(decodeWebArtifact(data.bytes));
if (!value.ok) throw new WebArtifactFormatFailure(value.error);
return { ...data, value: value.value };
@@ -55,8 +65,30 @@ export const webArtifactReadError = (
context: WebArtifactReadContext,
signal?: AbortSignal,
): AnalysisError => {
if (cause instanceof ArtifactReaderFailure && cause.cleanup !== undefined) {
const primary =
signal?.aborted === true
? new AnalysisCancelledError(context.operation, { cause })
: new ArtifactOperationError(
context.operation,
cause.reason,
cause.details,
cause.message,
{ cause },
);
return analysisErrorWithCleanupFailure(
primary,
new ProviderCleanupError(
"captured-web-artifacts",
cause.cleanup.resources,
{ reason: cause.cleanup.reason },
{ cause, operation: context.operation },
),
context.operation,
);
}
if (signal?.aborted === true)
return new AnalysisCancelledError(context.operation);
return new AnalysisCancelledError(context.operation, { cause });
const selection = context.callerSelected
? selectedFileError(cause, context)
: undefined;
@@ -1,6 +1,7 @@
import type { WebSourceLocationArtifactPort } from "../../application/WebSourceLocationPorts.js";
import type { WebScriptArtifactPort } from "../../application/WebScriptArtifactPort.js";
import type { ExecutionOptions } from "../../application/AnalysisProvider.js";
import { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js";
import { readStableArtifact } from "../../artifacts/readStableArtifact.js";
import {
WEB_SOURCE_MAP_LIMITS,
@@ -12,14 +13,64 @@ import { decodeWebArtifact, webArtifactReadError } from "./WebArtifactReads.js";
/** Read one verified capture source and the caller-selected map, without fetching. */
export class LocalWebSourceLocationArtifacts implements WebSourceLocationArtifactPort {
readonly scripts: WebScriptArtifactPort;
readonly #resources: ArtifactResourceScope;
constructor(
readonly scripts: WebScriptArtifactPort = new LocalWebScriptArtifacts(
"trace_web_source_location",
),
) {}
scripts?: WebScriptArtifactPort,
resources = scripts instanceof LocalWebScriptArtifacts
? scripts.resources
: new ArtifactResourceScope(),
) {
this.#resources = resources;
this.scripts =
scripts ??
new LocalWebScriptArtifacts("trace_web_source_location", resources);
}
async close(): Promise<void> {
const closeScript =
this.scripts instanceof LocalWebScriptArtifacts &&
this.scripts.resources === this.#resources
? undefined
: this.scripts.close?.bind(this.scripts);
const results = await Promise.allSettled([
Promise.resolve().then(() => this.#resources.close()),
...(closeScript === undefined
? []
: [Promise.resolve().then(closeScript)]),
]);
const failures = results.flatMap((result) =>
result.status === "rejected" ? [result.reason] : [],
);
if (failures.length > 0)
throw new AggregateError(
failures,
"Web source-location artifact cleanup failed",
);
}
/** Preserve actual map identity separately from the selected deployment URL. */
async load(input: WebSourceLocationInput, options?: ExecutionOptions) {
try {
return await this.#resources.run(() => this.#load(input, options));
} catch (cause: unknown) {
return err(
webArtifactReadError(
cause,
{
operation: "trace_web_source_location",
field: ["source_map", "path"],
targetPath: input.source_map.path,
callerSelected: true,
},
options?.signal,
),
);
}
}
async #load(input: WebSourceLocationInput, options?: ExecutionOptions) {
const loaded = await this.scripts.load(input, options);
if (!loaded.ok) return loaded;
try {
@@ -27,6 +78,8 @@ export class LocalWebSourceLocationArtifacts implements WebSourceLocationArtifac
input.source_map.path,
WEB_SOURCE_MAP_LIMITS.mapBytes,
options?.signal,
undefined,
this.#resources,
);
return ok({
...loaded.value,
+13
View File
@@ -162,6 +162,19 @@ it("rejects identical duplicate core fields before schema validation", () => {
expect(() => decodeHarCapture(text, [])).toThrow("duplicate");
});
it("names the violated HAR schema constraint without restating capture values", () => {
expect(() =>
decodeHarCapture(
JSON.stringify({
log: { creator: { name: "x", version: "1" }, entries: [] },
}),
[],
),
).toThrow(
"HAR schema validation failed: required (must have required property 'version').",
);
});
it("preserves prototype-named producer members through the capture boundary", () => {
const text = JSON.stringify(historicalHar()).replace(
'"cache":{}',
+2 -1
View File
@@ -144,7 +144,8 @@ const projectHar = (
(literal) =>
item.includes(literal) ||
contentBase64.includes(literal) ||
sha256.includes(literal),
sha256.includes(literal) ||
String(bytes.length).includes(literal),
);
binary_fields.push(
hidden
+5 -1
View File
@@ -23,9 +23,13 @@ export const createHarValidator = (): ((value: unknown) => void) => {
allowMitmproxyAbsentPostData(value);
if (!validate(value)) {
const error = validate.errors?.[0];
// AJV messages name the violated constraint, such as the missing
// property, without restating any captured value.
throw new CaptureFormatError(
"format",
`HAR schema validation failed: ${error?.keyword ?? "unknown constraint"}.`,
error === undefined
? "HAR schema validation failed: unknown constraint."
: `HAR schema validation failed: ${error.keyword} (${error.message ?? "no detail"}).`,
error?.instancePath ?? "",
);
}
@@ -17,6 +17,7 @@ import { z } from "zod";
import type { ExecutionOptions } from "../../application/AnalysisProvider.js";
import { readStableArtifact } from "../../artifacts/readStableArtifact.js";
import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js";
import { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import {
AnalysisCapabilityUnavailableError,
@@ -26,6 +27,7 @@ import {
} from "../../domain/analysisErrorCore.js";
import { ProviderAdapterError } from "../../domain/providerAdapterError.js";
import { ProviderCleanupError } from "../../domain/providerCleanupError.js";
import { analysisErrorWithCleanupFailure } from "../../domain/analysisErrorCleanup.js";
import { err, ok, type Result } from "../../domain/result.js";
import {
WEB_NETWORK_CAPTURE_LIMITS,
@@ -60,18 +62,52 @@ const replySchema = z.discriminatedUnion("ok", [
/** One owned snapshot/process/root lifecycle for replaceable historical format decoders. */
export class HistoricalCaptureDecoder {
readonly #resources: ArtifactResourceScope;
constructor(
readonly adapters: readonly HistoricalCaptureFormatAdapter[],
readonly environment: Readonly<NodeJS.ProcessEnv>,
readonly createRuntime: () => Promise<
Pick<PrivateRuntimeRoot, "path" | "close">
> = () => PrivateRuntimeRoot.create({ prefix: "rea-web-capture-" }),
) {}
resources = new ArtifactResourceScope(),
) {
this.#resources = resources;
}
/** Retry retained readers and private roots at service shutdown. */
close(): Promise<void> {
return this.#resources.close();
}
/** Snapshot the selected artifact, decode it offline, and release private credentials and processes. */
async inspect(
input: InspectWebNetworkCaptureInput,
options?: ExecutionOptions,
): Promise<Result<WebNetworkCapture, AnalysisError>> {
try {
return await this.#resources.run(() =>
this.inspectAdmitted(input, options),
);
} catch (cause: unknown) {
const error = historicalCaptureFailure(
input,
cause,
"capture-read",
options,
);
return err(
redactExplicitFailure(
this.withReaderCleanup(error, cause),
input.sensitive_values,
),
);
}
}
private async inspectAdmitted(
input: InspectWebNetworkCaptureInput,
options?: ExecutionOptions,
): Promise<Result<WebNetworkCapture, AnalysisError>> {
let runtime: Pick<PrivateRuntimeRoot, "path" | "close"> | undefined;
let result: Result<WebNetworkCapture, AnalysisError>;
@@ -90,6 +126,8 @@ export class HistoricalCaptureDecoder {
input.capture_path,
WEB_NETWORK_CAPTURE_LIMITS.inputBytes,
options?.signal,
undefined,
this.#resources,
);
phase = "decoder";
runtime = await this.createRuntime();
@@ -139,9 +177,16 @@ export class HistoricalCaptureDecoder {
replyPath,
WEB_NETWORK_CAPTURE_LIMITS.outputBytes,
options?.signal,
undefined,
this.#resources,
);
reply = replySchema.parse(JSON.parse(replyFile.bytes.toString("utf8")));
} catch (cause: unknown) {
if (
cause instanceof ArtifactReaderFailure &&
cause.cleanup !== undefined
)
throw cause;
if (options?.signal?.aborted)
throw new AnalysisCancelledError(OPERATION);
throw new AnalysisOutputError(
@@ -230,17 +275,21 @@ export class HistoricalCaptureDecoder {
result = ok({ ...reply.value, ...common, format: input.format });
}
} catch (cause: unknown) {
const error = historicalCaptureFailure(input, cause, phase, options);
result = err(
redactExplicitFailure(
historicalCaptureFailure(input, cause, phase, options),
this.withReaderCleanup(error, cause),
input.sensitive_values,
),
);
}
if (runtime !== undefined) {
try {
await runtime.close();
} catch (cause: unknown) {
const cleanup = await this.#resources.release({
kind: "private-runtime-root",
resource: runtime.path,
root: runtime,
});
if (cleanup.kind === "failed") {
return err(
redactExplicitFailure(
new ProviderCleanupError(
@@ -251,9 +300,12 @@ export class HistoricalCaptureDecoder {
previous_error: result.ok
? null
: projectAnalysisError(result.error),
reason: cause instanceof Error ? cause.message : String(cause),
reason:
cleanup.cause instanceof Error
? cleanup.cause.message
: String(cleanup.cause),
},
{ operation: OPERATION },
{ operation: OPERATION, cause: cleanup.cause },
),
input.sensitive_values,
),
@@ -264,6 +316,34 @@ export class HistoricalCaptureDecoder {
? err(new AnalysisCancelledError(OPERATION))
: result;
}
private withReaderCleanup(
primary: AnalysisError,
cause: unknown,
): AnalysisError {
if (
!(cause instanceof ArtifactReaderFailure) ||
cause.cleanup === undefined
)
return primary;
if (cause.reason === "unavailable")
return new ProviderCleanupError(
"web-network-capture",
cause.cleanup.resources,
{ reason: cause.cleanup.reason },
{ operation: OPERATION, cause },
);
const cleanup = new ProviderCleanupError(
"web-network-capture",
cause.cleanup.resources,
{
reason: cause.cleanup.reason,
previous_error: projectAnalysisError(primary),
},
{ operation: OPERATION, cause },
);
return analysisErrorWithCleanupFailure(primary, cleanup, OPERATION);
}
}
/** The Node capture decoder runs on this executable; only Node.js enforces its V8 heap flags. */
+56 -4
View File
@@ -4,6 +4,7 @@ import type {
} from "../../application/WebModulePorts.js";
import type { WebScriptArtifactPort } from "../../application/WebScriptArtifactPort.js";
import type { ExecutionOptions } from "../../application/AnalysisProvider.js";
import { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js";
import { jsonObjectSchema } from "../../domain/jsonValue.js";
import { err, ok } from "../../domain/result.js";
import type { WebModuleTraceInput } from "../../domain/webModuleTrace.js";
@@ -15,14 +16,64 @@ import {
/** Add an explicitly selected import map to a verified captured-script source. */
export class LocalWebModuleArtifacts implements WebModuleArtifactPort {
readonly scripts: WebScriptArtifactPort;
readonly #resources: ArtifactResourceScope;
constructor(
readonly scripts: WebScriptArtifactPort = new LocalWebScriptArtifacts(
"trace_web_module_imports",
),
) {}
scripts?: WebScriptArtifactPort,
resources = scripts instanceof LocalWebScriptArtifacts
? scripts.resources
: new ArtifactResourceScope(),
) {
this.#resources = resources;
this.scripts =
scripts ??
new LocalWebScriptArtifacts("trace_web_module_imports", resources);
}
async close(): Promise<void> {
const closeScript =
this.scripts instanceof LocalWebScriptArtifacts &&
this.scripts.resources === this.#resources
? undefined
: this.scripts.close?.bind(this.scripts);
const results = await Promise.allSettled([
Promise.resolve().then(() => this.#resources.close()),
...(closeScript === undefined
? []
: [Promise.resolve().then(closeScript)]),
]);
const failures = results.flatMap((result) =>
result.status === "rejected" ? [result.reason] : [],
);
if (failures.length > 0)
throw new AggregateError(failures, "Web module artifact cleanup failed");
}
/** Read selected source and optional map without website requests. */
async load(input: WebModuleTraceInput, options?: ExecutionOptions) {
try {
return await this.#resources.run(() => this.#load(input, options));
} catch (cause: unknown) {
return err(
webArtifactReadError(
cause,
{
operation: "trace_web_module_imports",
field:
input.import_map === undefined
? ["manifest_path"]
: ["import_map", "path"],
targetPath: input.import_map?.path ?? input.manifest_path,
callerSelected: input.import_map !== undefined,
},
options?.signal,
),
);
}
}
async #load(input: WebModuleTraceInput, options?: ExecutionOptions) {
const loaded = await this.scripts.load(input, options);
if (!loaded.ok) return loaded;
if (input.import_map === undefined)
@@ -33,6 +84,7 @@ export class LocalWebModuleArtifacts implements WebModuleArtifactPort {
selected.path,
4 * 1024 * 1024,
options?.signal,
this.#resources,
);
const importMap: WebModuleArtifacts["importMap"] = {
file: {
+217 -125
View File
@@ -1,3 +1,12 @@
import {
type InspectPacketFieldsInput,
packetFieldsCatalogSchema,
} from "../domain/captures/packetFields.js";
import {
parseFieldDiscovery,
FieldDiscoveryBudgetError,
FIELD_RECORD_BUDGET,
} from "./parseFieldDiscovery.js";
import {
readLinuxResourceLimits,
type LinuxResourceLimits,
@@ -91,17 +100,31 @@ export class TsharkCaptureProvider implements PacketCapturePort {
options?: ExecutionOptions,
): Promise<Result<AnalysisExecution, AnalysisError>> {
return this.#resources.run(
() => this.#inspect(input, options),
() => this.#executeRequest(input, options),
options?.signal,
);
}
inspectFields(
input: InspectPacketFieldsInput,
options?: ExecutionOptions,
): Promise<Result<AnalysisExecution, AnalysisError>> {
return this.#resources.run(
() => this.#executeRequest(input, options),
options?.signal,
"inspect_packet_fields",
);
}
close(): Promise<void> {
return this.#resources.close();
}
async #inspect(
input: InspectPacketCaptureInput,
async #executeRequest(
input: InspectPacketCaptureInput | InspectPacketFieldsInput,
options?: ExecutionOptions,
): Promise<Result<AnalysisExecution, AnalysisError>> {
const operation = "path" in input ? OPERATION : "inspect_packet_fields";
// TShark 4.6.9 emits a 22.3 MB glossary; keep capture output capped at 4 MiB.
const limits =
"path" in input ? LIMITS : { ...LIMITS, output_bytes: 32 * 1024 * 1024 };
let phase:
| "configuration"
| "artifact"
@@ -109,12 +132,12 @@ export class TsharkCaptureProvider implements PacketCapturePort {
| "producer"
| "integrity" = "configuration";
let capturedOutput: AnalysisCapturedOutput | undefined;
let resourcePath = input.path;
let resourcePath = "path" in input ? input.path : "TShark field glossary";
let effectiveLimits: LinuxResourceLimits | undefined;
let root: ArtifactAnalysisRuntime | undefined;
let result: Result<AnalysisExecution, AnalysisError>;
try {
if (options?.signal?.aborted) throw new AnalysisCancelledError(OPERATION);
if (options?.signal?.aborted) throw new AnalysisCancelledError(operation);
await this.#resources.release();
if (this.platform !== "linux")
throw new Error(
@@ -141,7 +164,7 @@ export class TsharkCaptureProvider implements PacketCapturePort {
64 * 1024 * 1024,
options?.signal,
);
effectiveLimits = await this.#resources.readLimits(LIMITS);
effectiveLimits = await this.#resources.readLimits(limits);
const limitArguments = [
`--as=${effectiveLimits.address_space_bytes}:${effectiveLimits.address_space_bytes}`,
`--fsize=${effectiveLimits.file_bytes}:${effectiveLimits.file_bytes}`,
@@ -152,7 +175,7 @@ export class TsharkCaptureProvider implements PacketCapturePort {
];
phase = "workspace";
root = await this.createRuntime();
this.#resources.retain(root, undefined);
this.#resources.retain(root, undefined, operation);
const snapshotPath = join(root.path, "capture");
const configurationPath = join(root.path, "configuration");
await mkdir(configurationPath, { mode: 0o700 });
@@ -170,8 +193,8 @@ export class TsharkCaptureProvider implements PacketCapturePort {
},
},
{
timeoutMs: LIMITS.command_timeout_ms,
diagnosticBytes: LIMITS.output_bytes,
timeoutMs: limits.command_timeout_ms,
diagnosticBytes: limits.output_bytes,
acceptNonZeroExit: true,
},
{
@@ -200,24 +223,24 @@ export class TsharkCaptureProvider implements PacketCapturePort {
);
if (currentLimiter.sha256 !== limiterIdentity.sha256)
throw new AnalysisArtifactChangedError(
OPERATION,
operation,
limiter,
"Configured resource limiter changed during analysis.",
{ capturedOutput },
);
if (current.sha256 !== tool.sha256)
throw new AnalysisArtifactChangedError(
OPERATION,
operation,
command,
"Configured TShark executable changed during analysis.",
{ capturedOutput },
);
phase = previousPhase;
resourcePath = input.path;
resourcePath = "path" in input ? input.path : command;
if (output.exitCode !== 0)
throw new ProviderAdapterError(
TSHARK_PROVIDER_IDENTITY.id,
OPERATION,
operation,
{
diagnostics: {
exit_code: output.exitCode ?? null,
@@ -238,86 +261,15 @@ export class TsharkCaptureProvider implements PacketCapturePort {
throw new Error(
"Expected TShark 4.x; inspect the selected installation.",
);
phase = "artifact";
const selected = await this.#resources.read(
input.path,
LIMITS.input_bytes,
options?.signal,
);
phase = "workspace";
await writeFile(snapshotPath, selected.bytes, {
flag: "wx",
mode: 0o400,
});
phase = "producer";
const args = [
"-n",
"-r",
"capture",
"-T",
"json",
"--no-duplicate-keys",
"--temp-dir=.",
...(input.two_pass ? ["-2"] : []),
...(input.packet_limit === null
? []
: ["-c", String(input.packet_limit)]),
...(input.display_filter === null ? [] : ["-Y", input.display_filter]),
...input.decode_as.flatMap((value) => ["-d", value]),
...[...new Set(["frame.number", ...input.fields])].flatMap((field) => [
"-e",
field,
]),
];
const output = await execute(args);
phase = "integrity";
resourcePath = snapshotPath;
if (
(await this.#resources.read(snapshotPath, LIMITS.input_bytes))
.sha256 !== selected.sha256
)
throw new AnalysisArtifactChangedError(
OPERATION,
snapshotPath,
"TShark changed the private capture copy.",
capturedOutput === undefined ? {} : { capturedOutput },
);
phase = "producer";
resourcePath = input.path;
const matchingPackets = parsePacketOutput(
output.stdout.text,
input.fields,
);
const start = Math.min(input.match_offset, matchingPackets.length);
const end =
input.match_limit === null
? matchingPackets.length
: start + Math.min(input.match_limit, matchingPackets.length - start);
const packets = matchingPackets.slice(start, end);
const report = packetCaptureSchema.parse({
artifact: {
path: input.path,
sha256: selected.sha256,
bytes: selected.bytes.length,
},
selection: input,
packets,
output_window: {
matching_packets: matchingPackets.length,
returned_packets: packets.length,
has_more: end < matchingPackets.length,
},
read_scope:
input.packet_limit === null ? "to-eof" : "up-to-input-packet-limit",
configuration: "private-empty-personal-profile",
diagnostics: output.stderr.text,
const metadata = (args: readonly string[], diagnostics: string) => ({
diagnostics,
tool: {
configured_path: configured,
path: command,
sha256: tool.sha256,
version_banner: version.stdout.text,
version_diagnostics: version.stderr.text,
arguments: args,
arguments: [...args],
},
resource_limiter: {
configured_path: configuredLimiter,
@@ -325,33 +277,136 @@ export class TsharkCaptureProvider implements PacketCapturePort {
sha256: limiterIdentity.sha256,
arguments: limitArguments,
},
limits: { ...LIMITS, ...effectiveLimits },
hardware_acquisition: "not-performed",
limits: { ...limits, ...effectiveLimits },
hardware_acquisition: "not-performed" as const,
});
result = ok(
createAnalysisExecution(report, TSHARK_PROVIDER_IDENTITY, {
subject: {
if (!("path" in input)) {
phase = "producer";
const args = ["-G", "fields"];
const output = await execute(args);
const report = packetFieldsCatalogSchema.parse({
selection: input,
...parseFieldDiscovery(output.stdout.text, input),
...metadata(args, output.stderr.text),
limits: {
...limits,
...effectiveLimits,
record_budget_bytes: FIELD_RECORD_BUDGET,
},
configuration: "private-empty-personal-profile",
});
result = ok(
createAnalysisExecution(report, TSHARK_PROVIDER_IDENTITY, {
subject: { path: command, sha256: tool.sha256, format: "file" },
limitations: [
"Protocol and field registrations describe the selected installation, not traffic observed in a capture. Duplicate abbreviations remain distinct producer records with source line numbers.",
"Each request probes again without caching. The private empty personal profile does not fingerprint or isolate installed libraries, global configuration or plugins; they remain trusted.",
"No capture file, interface or device is opened. Resource ceilings do not constitute an OS sandbox. Glossary output is capped at 32 MiB and selected record representation at 8 MiB before expansion.",
],
}),
);
} else {
phase = "artifact";
const selected = await this.#resources.read(
input.path,
LIMITS.input_bytes,
options?.signal,
);
phase = "workspace";
await writeFile(snapshotPath, selected.bytes, {
flag: "wx",
mode: 0o400,
});
phase = "producer";
const args = [
"-n",
"-r",
"capture",
"-T",
"json",
"--no-duplicate-keys",
"--temp-dir=.",
...(input.two_pass ? ["-2"] : []),
...(input.packet_limit === null
? []
: ["-c", String(input.packet_limit)]),
...(input.display_filter === null
? []
: ["-Y", input.display_filter]),
...input.decode_as.flatMap((value) => ["-d", value]),
...[...new Set(["frame.number", ...input.fields])].flatMap(
(field) => ["-e", field],
),
];
const output = await execute(args);
phase = "integrity";
resourcePath = snapshotPath;
if (
(await this.#resources.read(snapshotPath, LIMITS.input_bytes))
.sha256 !== selected.sha256
)
throw new AnalysisArtifactChangedError(
operation,
snapshotPath,
"TShark changed the private capture copy.",
capturedOutput === undefined ? {} : { capturedOutput },
);
phase = "producer";
resourcePath = "path" in input ? input.path : command;
const matchingPackets = parsePacketOutput(
output.stdout.text,
input.fields,
);
const start = Math.min(input.match_offset, matchingPackets.length);
const end =
input.match_limit === null
? matchingPackets.length
: start +
Math.min(input.match_limit, matchingPackets.length - start);
const packets = matchingPackets.slice(start, end);
const report = packetCaptureSchema.parse({
artifact: {
path: input.path,
sha256: selected.sha256,
format: "file",
bytes: selected.bytes.length,
},
locations: [{ kind: "artifact-path", path: input.path }],
limitations: [
"Fields are TShark dissector observations under the selected filter/decode-as/pass settings, not independently verified protocol semantics. Missing fields are absent from the producer output, not known zero or empty values.",
"Packet limits count input packets before display filtering. Filtered results do not establish whole-capture or device traffic absence. Two-pass reassembly cannot recover bytes missing from the recording or beyond the read limit.",
"Repeated field values remain ordered string arrays; raw packet bytes are included only when explicitly selected as a supported field. Export index/score metadata are omitted. All stderr, including warnings on exit zero, is retained.",
"A private empty personal configuration is used. Installed libraries, global configuration and dissector plugins are trusted and not fully fingerprinted. File-only analysis and disabled name resolution do not constitute an OS sandbox.",
"Linux prlimit enforces soft and hard ceilings before execution, preserving stricter inherited limits: at most 1 GiB address space per process, 64 MiB per file, 30 CPU seconds and no core dumps. Effective limits are reported. Resident input is capped at 32 MiB, each command at 30 wall-clock seconds and combined output at 4 MiB. Output overflow fails. These are per-process/per-file limits, not aggregate memory or filesystem quotas; caller-supplied tools and the limiter are trusted.",
],
}),
);
selection: input,
packets,
output_window: {
matching_packets: matchingPackets.length,
returned_packets: packets.length,
has_more: end < matchingPackets.length,
},
read_scope:
input.packet_limit === null ? "to-eof" : "up-to-input-packet-limit",
configuration: "private-empty-personal-profile",
...metadata(args, output.stderr.text),
});
result = ok(
createAnalysisExecution(report, TSHARK_PROVIDER_IDENTITY, {
subject: {
path: input.path,
sha256: selected.sha256,
format: "file",
},
locations: [{ kind: "artifact-path", path: input.path }],
limitations: [
"Fields are TShark dissector observations under the selected filter/decode-as/pass settings, not independently verified protocol semantics. Missing fields are absent from the producer output, not known zero or empty values.",
"Packet limits count input packets before display filtering. Filtered results do not establish whole-capture or device traffic absence. Two-pass reassembly cannot recover bytes missing from the recording or beyond the read limit.",
"Repeated field values remain ordered string arrays; raw packet bytes are included only when explicitly selected as a supported field. Export index/score metadata are omitted. All stderr, including warnings on exit zero, is retained.",
"A private empty personal configuration is used. Installed libraries, global configuration and dissector plugins are trusted and not fully fingerprinted. File-only analysis and disabled name resolution do not constitute an OS sandbox.",
"Linux prlimit enforces soft and hard ceilings before execution, preserving stricter inherited limits: at most 1 GiB address space per process, 64 MiB per file, 30 CPU seconds and no core dumps. Effective limits are reported. Resident input is capped at 32 MiB, each command at 30 wall-clock seconds and combined output at 4 MiB. Output overflow fails. These are per-process/per-file limits, not aggregate memory or filesystem quotas; caller-supplied tools and the limiter are trusted.",
],
}),
);
}
} catch (cause) {
if (
root !== undefined &&
cause instanceof OwnedCommandFailure &&
cause.cleanupOwner !== undefined
)
this.#resources.retain(root, cause.cleanupOwner);
this.#resources.retain(root, cause.cleanupOwner, operation);
result = err(
this.#failure(
cause,
@@ -360,18 +415,23 @@ export class TsharkCaptureProvider implements PacketCapturePort {
root?.path ?? null,
effectiveLimits,
capturedOutput,
operation,
limits,
),
);
}
return this.#resources.finish(result, (execution) =>
createEvidence(execution.subject ?? undefined, execution.provider, {
operation: OPERATION,
parameters: input,
result: execution.result,
confidence: "observed",
locations: execution.locations,
limitations: execution.limitations,
}),
return this.#resources.finish(
result,
(execution) =>
createEvidence(execution.subject ?? undefined, execution.provider, {
operation,
parameters: input,
result: execution.result,
confidence: "observed",
locations: execution.locations,
limitations: execution.limitations,
}),
operation,
);
}
#failure(
@@ -386,6 +446,11 @@ export class TsharkCaptureProvider implements PacketCapturePort {
workspacePath: string | null,
effectiveLimits: LinuxResourceLimits | undefined,
observedOutput: AnalysisCapturedOutput | undefined,
operation = OPERATION,
limits: {
readonly output_bytes: number;
readonly command_timeout_ms: number;
} = LIMITS,
): AnalysisError {
if (cause instanceof AnalysisError) return cause;
if (cause instanceof ArtifactReaderFailure && cause.cleanup !== undefined)
@@ -393,15 +458,34 @@ export class TsharkCaptureProvider implements PacketCapturePort {
cause.cleanup,
cause,
observedOutput,
operation,
);
if (
(cause instanceof Error && cause.name === "AbortError") ||
(cause instanceof ArtifactReaderFailure && cause.reason === "cancelled")
)
return new AnalysisCancelledError(OPERATION, { cause });
return new AnalysisCancelledError(operation, { cause });
if (cause instanceof FieldDiscoveryBudgetError)
return new AnalysisResourceConstraintError(
operation,
"memory",
cause.message,
{ record_budget_bytes: FIELD_RECORD_BUDGET },
{
cause,
...(observedOutput === undefined
? {}
: { capturedOutput: observedOutput }),
},
);
const reason = cause instanceof Error ? cause.message : String(cause);
if (cause instanceof OwnedCommandFailure)
return this.#resources.commandFailure(cause, LIMITS, effectiveLimits);
return this.#resources.commandFailure(
cause,
limits,
effectiveLimits,
operation,
);
if (phase === "workspace") {
const code =
cause instanceof Error &&
@@ -411,13 +495,13 @@ export class TsharkCaptureProvider implements PacketCapturePort {
: null;
if (code === "ENOSPC" || code === "EDQUOT" || code === "EFBIG")
return new AnalysisResourceConstraintError(
OPERATION,
operation,
"file-size",
`Private analysis workspace failed (${code}): ${reason}`,
{ workspace_path: workspacePath, system_code: code },
{ cause },
);
return new ProviderAdapterError(TSHARK_PROVIDER_IDENTITY.id, OPERATION, {
return new ProviderAdapterError(TSHARK_PROVIDER_IDENTITY.id, operation, {
cause,
userMessage: `Cannot prepare the private analysis workspace: ${reason}`,
diagnostics: {
@@ -440,12 +524,12 @@ export class TsharkCaptureProvider implements PacketCapturePort {
["limit", "integrity", "path"].includes(cause.reason)
)
return new AnalysisArtifactChangedError(
OPERATION,
operation,
path,
`Post-run identity verification failed for ${path}: ${reason}`,
detail,
);
return new ProviderAdapterError(TSHARK_PROVIDER_IDENTITY.id, OPERATION, {
return new ProviderAdapterError(TSHARK_PROVIDER_IDENTITY.id, operation, {
...detail,
diagnostics: { stage: "integrity", resource_path: path, reason },
});
@@ -453,10 +537,13 @@ export class TsharkCaptureProvider implements PacketCapturePort {
if (phase === "configuration")
return new AnalysisCapabilityUnavailableError(
TSHARK_PROVIDER_IDENTITY.id,
OPERATION,
operation,
reason,
{
cause,
...(observedOutput === undefined
? {}
: { capturedOutput: observedOutput }),
userMessage: `${reason} On Linux, configure TShark 4.x through REA_TSHARK_COMMAND and util-linux prlimit through REA_TSHARK_PRLIMIT_COMMAND (default /usr/bin/prlimit).`,
},
);
@@ -466,14 +553,14 @@ export class TsharkCaptureProvider implements PacketCapturePort {
cause.reason === "limit"
)
return new AnalysisResourceConstraintError(
OPERATION,
operation,
"file-size",
reason,
{ input_bytes: LIMITS.input_bytes },
{ cause },
);
if (cause instanceof ArtifactReaderFailure && cause.reason === "integrity")
return new AnalysisArtifactChangedError(OPERATION, path, reason, {
return new AnalysisArtifactChangedError(operation, path, reason, {
cause,
});
if (
@@ -482,13 +569,18 @@ export class TsharkCaptureProvider implements PacketCapturePort {
"code" in cause &&
(cause.code === "EACCES" || cause.code === "EPERM")
)
return new AnalysisAccessDeniedError(OPERATION, path, cause.code, {
return new AnalysisAccessDeniedError(operation, path, cause.code, {
cause,
});
if (phase === "artifact")
return new AnalysisInputError(OPERATION, { cause }, [
return new AnalysisInputError(operation, { cause }, [
{ path: ["path"], reason: "invalid_value", message: reason },
]);
return new AnalysisOutputError(OPERATION, reason, { cause });
return new AnalysisOutputError(operation, reason, {
cause,
...(observedOutput === undefined
? {}
: { capturedOutput: observedOutput }),
});
}
}
+92
View File
@@ -0,0 +1,92 @@
import type {
InspectPacketFieldsInput,
PacketFieldRecord,
} from "../domain/captures/packetFields.js";
export class FieldDiscoveryBudgetError extends Error {}
export const FIELD_RECORD_BUDGET = 8 * 1024 * 1024;
/** TShark -G fields: P/name/abbreviation or F/name/abbreviation/type/parent/base/mask/blurb. */
export const parseFieldDiscovery = (
text: string,
input: InspectPacketFieldsInput,
) => {
const copy = (value: string) =>
Buffer.from(value, "utf16le").toString("utf16le");
const query = input.query.toLowerCase();
const records: PacketFieldRecord[] = [];
let examined = 0,
matching = 0,
sourceLine = 0,
budget = 0;
// Walk the bounded text once; allocate record objects only for the requested page.
for (let start = 0; start < text.length;) {
const end = text.indexOf("\n", start);
let line = text.slice(start, end < 0 ? text.length : end);
start = end < 0 ? text.length : end + 1;
sourceLine++;
if (line.endsWith("\r")) line = line.slice(0, -1);
const columns = line.split("\t", 9);
const [tag, name, abbreviation] = columns;
if (
!(
(tag === "P" && columns.length === 3) ||
(tag === "F" && columns.length === 8)
) ||
name === undefined ||
abbreviation === undefined ||
abbreviation === ""
)
throw new Error(
`Malformed TShark field glossary record at line ${sourceLine}`,
);
if ((input.kind === "protocols") !== (tag === "P")) continue;
examined++;
if (
input.protocol !== null &&
(tag === "P" ? abbreviation : columns[4]) !== input.protocol
)
continue;
if (
![name, abbreviation, ...(tag === "F" ? [columns[7] ?? ""] : [])].some(
(value) => value.toLowerCase().includes(query),
)
)
continue;
matching++;
if (
matching <= input.offset ||
records.length >= (input.limit ?? Number.MAX_SAFE_INTEGER)
)
continue;
// Six bytes per source byte covers worst-case JSON escaping, plus record bookkeeping.
budget += Buffer.byteLength(line, "utf8") * 6 + 512;
if (budget > FIELD_RECORD_BUDGET)
throw new FieldDiscoveryBudgetError(
"Selected glossary records exceed the 8 MiB representation budget; narrow the query or page size",
);
// Detach retained fields from the much larger producer string.
records.push({
source_line: sourceLine,
kind: tag === "P" ? "protocol" : "field",
name: copy(name),
abbreviation: copy(abbreviation),
field_type: tag === "F" ? copy(columns[3] ?? "") : null,
parent_protocol: tag === "F" ? copy(columns[4] ?? "") : null,
display_base: tag === "F" ? copy(columns[5] ?? "") : null,
bitmask: tag === "F" ? copy(columns[6] ?? "") : null,
description: tag === "F" ? copy(columns[7] ?? "") : null,
});
}
if (sourceLine === 0)
throw new Error("TShark returned an empty field glossary");
return {
records,
output_window: {
examined,
matching,
returned: records.length,
has_more: Math.max(0, matching - input.offset) > records.length,
},
};
};
+6 -4
View File
@@ -62,15 +62,17 @@ export const registerApktoolCommands = (
.describe(
"Project strings from res/values-<locale> instead of the default",
),
"no-strings": z
strings: z
.boolean()
.default(false)
.describe("Skip projecting string resources"),
.default(true)
.describe(
"Include string resources; use --no-strings to skip projecting them",
),
}),
run: ({ args, options }) =>
execute(CLI_COMMANDS.decodeAndroidResources, "decode_android_resources", {
path: args.path,
include_strings: !options["no-strings"],
include_strings: options.strings,
...(options.locale === undefined ? {} : { locale: options.locale }),
}),
});
+24 -15
View File
@@ -7,11 +7,12 @@ import {
} from "../composition/binaryDiagnostics.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { Logger } from "pino";
import type { CliInstance } from "./types.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { inspectPeResourcesInputSchema } from "../domain/native/peResources.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
/** Inspect operator-selected files through the same shared workflow as MCP. */
export const registerBinaryDiagnosticsCommands = (
@@ -19,7 +20,6 @@ export const registerBinaryDiagnosticsCommands = (
logger: Logger,
environment: Readonly<NodeJS.ProcessEnv>,
): void => {
const pe = createPeResourcesService();
cli.command(CLI_COMMANDS.inspectPeResources, {
description:
"Inspect PE resource identities, payload ranges, hashes and icon-group relationships",
@@ -29,19 +29,28 @@ export const registerBinaryDiagnosticsCommands = (
maxEntries: inspectPeResourcesInputSchema.shape.max_entries,
}),
run: ({ args, options }) =>
withCommandCancellation((signal) =>
logCliCommand(logger, CLI_COMMANDS.inspectPeResources, async () => {
const result = await pe.inspect(
{
path: resolve(args.path),
max_file_bytes: options.maxFileBytes,
max_entries: options.maxEntries,
},
{ signal },
);
return result.ok ? result.value : projectAnalysisError(result.error);
}),
),
withCommandCancellation(async (signal) => {
const pe = createPeResourcesService();
return await logCliCommand(
logger,
CLI_COMMANDS.inspectPeResources,
async () =>
finishAnalysisCommand(
() =>
pe.inspect(
{
path: resolve(args.path),
max_file_bytes: options.maxFileBytes,
max_entries: options.maxEntries,
},
{ signal },
),
pe,
"rea-pe-resources",
"inspect_pe_resources",
),
);
}),
});
const service = createBinaryLayoutService(environment);
cli.command(CLI_COMMANDS.inspectBinaryLayout, {
+57 -37
View File
@@ -1,8 +1,6 @@
import { inspectPacketFieldsInputSchema } from "../domain/captures/packetFields.js";
import { z } from "incur";
import type { PacketCaptureService } from "../application/captures/PacketCaptureService.js";
import { AnalysisError } from "../domain/analysisErrorBase.js";
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import { resolve } from "node:path";
import { createPacketCaptureService } from "../composition/captures.js";
import {
@@ -12,19 +10,60 @@ import {
} from "../domain/captures/packetCapture.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { Logger } from "pino";
import type { CliInstance } from "./types.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
export const registerCaptureCommands = (
cli: CliInstance,
logger: Logger,
environment: Readonly<NodeJS.ProcessEnv>,
service: Pick<
PacketCaptureService,
"inspect" | "close"
"inspect" | "inspectFields" | "close"
> = createPacketCaptureService(environment),
): void => {
cli.command(CLI_COMMANDS.inspectPacketFields, {
description: "Search installed TShark protocol and field definitions",
options: z
.object({
...inspectPacketFieldsInputSchema.shape,
limit: z
.number()
.int()
.positive()
.max(Number.MAX_SAFE_INTEGER)
.optional()
.describe("Maximum matching records to return; defaults to 50"),
all: z
.boolean()
.default(false)
.describe("Return all remaining matches within resource limits"),
})
.superRefine((options, context) => {
if (options.all && options.limit !== undefined)
context.addIssue({
code: "custom",
path: ["all"],
message: "all and limit are mutually exclusive",
});
}),
run: ({ options: { all, ...options } }) =>
withCommandCancellation((signal) =>
logCliCommand(logger, CLI_COMMANDS.inspectPacketFields, async () =>
finishAnalysisCommand(
() =>
service.inspectFields(
{ ...options, ...(all ? { limit: null } : {}) },
{ signal },
),
service,
"tshark-capture",
"inspect_packet_fields",
),
),
),
});
cli.command(CLI_COMMANDS.inspectPacketCapture, {
description:
"Inspect an existing packet capture with caller-supplied TShark",
@@ -60,39 +99,20 @@ export const registerCaptureCommands = (
run: ({ args, options: { all_matches, ...options } }) =>
withCommandCancellation((signal) =>
logCliCommand(logger, CLI_COMMANDS.inspectPacketCapture, async () => {
const result = await service.inspect(
{
...options,
...(all_matches ? { match_limit: null } : {}),
path: resolve(args.path),
},
{ signal },
return finishAnalysisCommand(
() =>
service.inspect(
{
...options,
...(all_matches ? { match_limit: null } : {}),
path: resolve(args.path),
},
{ signal },
),
service,
"tshark-capture",
"inspect_packet_capture",
);
// A one-shot command must retry retained cleanup before discarding its owner.
try {
await service.close();
} catch (cause) {
const cleanup =
cause instanceof AnalysisError
? cause
: new ProviderCleanupError(
"tshark-capture",
[],
{ reason: String(cause) },
{ cause, operation: "inspect_packet_capture" },
);
if (result.ok) cleanup.retainPartialObservation(result.value);
return projectAnalysisError(
result.ok
? cleanup
: analysisErrorWithCleanupFailure(
result.error,
cleanup,
"inspect_packet_capture",
),
);
}
return result.ok ? result.value : projectAnalysisError(result.error);
}),
),
});
+69
View File
@@ -0,0 +1,69 @@
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
import {
AnalysisError,
type AnalysisPartialObservation,
} from "../domain/analysisErrorBase.js";
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import type { Result } from "../domain/result.js";
/** Finish a one-shot CLI operation without discarding its primary result on close failure. */
export const finishAnalysisCommand = async <
Value extends AnalysisPartialObservation,
>(
run: () => Promise<Result<Value, AnalysisError>>,
service: { close(): Promise<void> },
providerId: string,
canonicalOperation: string,
fallbackResource?: string,
): Promise<Value | ReturnType<typeof projectAnalysisError>> => {
let result: Result<Value, AnalysisError>;
try {
result = await run();
} catch (cause) {
result = {
ok: false,
error:
cause instanceof AnalysisError
? cause
: new ProviderAdapterError(providerId, canonicalOperation, { cause }),
};
}
try {
await service.close();
} catch (cause) {
let cleanup: AnalysisError;
if (cause instanceof AnalysisError) cleanup = cause;
else {
const observation =
cause instanceof ArtifactReaderFailure
? ArtifactReaderFailure.cleanupObservation(
cause,
fallbackResource ?? `${canonicalOperation} resources`,
)
: {
reason: cause instanceof Error ? cause.message : String(cause),
resources: [],
};
cleanup = new ProviderCleanupError(
providerId,
observation.resources,
{ reason: observation.reason },
{ cause, operation: canonicalOperation },
);
}
if (result.ok) cleanup.retainPartialObservation(result.value);
return projectAnalysisError(
result.ok
? cleanup
: analysisErrorWithCleanupFailure(
result.error,
cleanup,
canonicalOperation,
),
);
}
return result.ok ? result.value : projectAnalysisError(result.error);
};
+15 -11
View File
@@ -3,14 +3,13 @@ import { resolve } from "node:path";
import { createGoBinaryService } from "../composition/go.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { Logger } from "pino";
import type { CliInstance } from "./types.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
/** Resolve caller paths and inspect static build metadata through the shared workflow. */
export const registerGoCommands = (cli: CliInstance, logger: Logger): void => {
const service = createGoBinaryService();
cli.command(CLI_COMMANDS.inspectGoBinary, {
description:
"Inspect embedded Go compiler, modules and build settings without target execution",
@@ -18,14 +17,19 @@ export const registerGoCommands = (cli: CliInstance, logger: Logger): void => {
path: z.string().describe("Local ELF, PE or thin Mach-O image"),
}),
run: ({ args }) =>
withCommandCancellation((signal) =>
logCliCommand(logger, CLI_COMMANDS.inspectGoBinary, async () => {
const result = await service.inspect(
{ path: resolve(args.path) },
{ signal },
);
return result.ok ? result.value : projectAnalysisError(result.error);
}),
),
withCommandCancellation(async (signal) => {
const service = createGoBinaryService();
return await logCliCommand(
logger,
CLI_COMMANDS.inspectGoBinary,
async () =>
finishAnalysisCommand(
() => service.inspect({ path: resolve(args.path) }, { signal }),
service,
"go-build-info",
"inspect_go_binary",
),
);
}),
});
};
+59 -37
View File
@@ -1,8 +1,6 @@
import { inspectSignalDecodersInputSchema } from "../domain/signals/signalDecoders.js";
import { z } from "incur";
import type { SignalRecordingService } from "../application/signals/SignalRecordingService.js";
import { AnalysisError } from "../domain/analysisErrorBase.js";
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import { resolve } from "node:path";
import { createSignalRecordingService } from "../composition/signals.js";
import {
@@ -12,19 +10,62 @@ import {
} from "../domain/signals/signalRecording.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { Logger } from "pino";
import type { CliInstance } from "./types.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
export const registerSignalCommands = (
cli: CliInstance,
logger: Logger,
environment: Readonly<NodeJS.ProcessEnv>,
service: Pick<
SignalRecordingService,
"decode" | "close"
"decode" | "inspectDecoders" | "close"
> = createSignalRecordingService(environment),
): void => {
cli.command(CLI_COMMANDS.inspectSignalDecoders, {
description: "Search installed sigrok decoders or selected decoder help",
options: z
.object({
...inspectSignalDecodersInputSchema.shape,
limit: z
.number()
.int()
.positive()
.max(Number.MAX_SAFE_INTEGER)
.optional()
.describe(
"Maximum matching decoders or help lines to return; defaults to 50",
),
all: z
.boolean()
.default(false)
.describe("Return all remaining matches"),
})
.superRefine((options, context) => {
if (options.all && options.limit !== undefined)
context.addIssue({
code: "custom",
path: ["all"],
message: "all and limit are mutually exclusive",
});
}),
run: ({ options: { all, ...options } }) =>
withCommandCancellation((signal) =>
logCliCommand(logger, CLI_COMMANDS.inspectSignalDecoders, async () =>
finishAnalysisCommand(
() =>
service.inspectDecoders(
{ ...options, ...(all ? { limit: null } : {}) },
{ signal },
),
service,
"sigrok-recording",
"inspect_signal_decoders",
),
),
),
});
cli.command(CLI_COMMANDS.decodeSignalRecording, {
description:
"Decode an existing signal recording with caller-supplied sigrok",
@@ -60,39 +101,20 @@ export const registerSignalCommands = (
run: ({ args, options: { all_lines, ...options } }) =>
withCommandCancellation((signal) =>
logCliCommand(logger, CLI_COMMANDS.decodeSignalRecording, async () => {
const result = await service.decode(
{
...options,
...(all_lines ? { line_limit: null } : {}),
path: resolve(args.path),
},
{ signal },
return finishAnalysisCommand(
() =>
service.decode(
{
...options,
...(all_lines ? { line_limit: null } : {}),
path: resolve(args.path),
},
{ signal },
),
service,
"sigrok-recording",
"decode_signal_recording",
);
// A one-shot command must retry retained cleanup before discarding its owner.
try {
await service.close();
} catch (cause) {
const cleanup =
cause instanceof AnalysisError
? cause
: new ProviderCleanupError(
"sigrok-recording",
[],
{ reason: String(cause) },
{ cause, operation: "decode_signal_recording" },
);
if (result.ok) cleanup.retainPartialObservation(result.value);
return projectAnalysisError(
result.ok
? cleanup
: analysisErrorWithCleanupFailure(
result.error,
cleanup,
"decode_signal_recording",
),
);
}
return result.ok ? result.value : projectAnalysisError(result.error);
}),
),
});
+18 -14
View File
@@ -3,10 +3,10 @@ import { WEB_NETWORK_CAPTURE_FORMATS } from "../domain/webNetworkCapture.js";
import { createWebNetworkCaptureService } from "../composition/webNetworkCaptures.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { Logger } from "pino";
import type { CliInstance } from "./types.js";
import { withCommandCancellation } from "./commandCancellation.js";
import { finishAnalysisCommand } from "./finishAnalysisCommand.js";
/** Expose offline historical capture inspection with the same selection contract as MCP. */
export const registerWebNetworkCaptureCommands = (
@@ -42,20 +42,24 @@ export const registerWebNetworkCaptureCommands = (
logger,
CLI_COMMANDS.inspectWebNetworkCapture,
async () => {
const result = await service.inspect(
{
capture_path: args.capturePath,
format: args.format,
sensitive_values: options.sensitiveValue,
...(options.record === undefined
? {}
: { record_ordinals: options.record }),
},
{ signal },
return finishAnalysisCommand(
() =>
service.inspect(
{
capture_path: args.capturePath,
format: args.format,
sensitive_values: options.sensitiveValue,
...(options.record === undefined
? {}
: { record_ordinals: options.record }),
},
{ signal },
),
service,
"web-network-capture",
"inspect_web_network_capture",
"Web network capture resources",
);
return result.ok
? result.value
: projectAnalysisError(result.error);
},
),
),
+2
View File
@@ -1,6 +1,8 @@
/** Canonical names used by every one-shot CLI command registration. */
export const CLI_COMMANDS = Object.freeze({
inspectPacketFields: "inspect-packet-fields",
inspectPacketCapture: "inspect-packet-capture",
inspectSignalDecoders: "inspect-signal-decoders",
decodeSignalRecording: "decode-signal-recording",
inspectWasmArtifact: "inspect-wasm-artifact",
inspectSqliteDatabase: "inspect-sqlite-database",

Some files were not shown because too many files have changed in this diff Show More