Compare commits

..
Author SHA1 Message Date
N0zoM1z0 c0d44fe8f5 fix(hopper): distinguish runtime processes from argument text
Parse procfs argument boundaries rather than searching rendered ps command text. Keep executable identity checks and deferred permission failures, eliminate the extra ps process, and put shared process observations in the serial process-boundary lane.
2026-10-13 06:44:26 +00:00
N0zoM1z0 11c188523a test(hopper): verify Linux demo setup and source-bound string coverage
Integrate the verification corrections from #1942 on current main, retaining their original commit history. Reuse the unchanged document inventory and verified source bytes, retain recognition addresses, and reject incomplete or mismatched string evidence.
2026-10-13 06:21:07 +00:00
Luis Taffelli 794debe565 chore(verification): sync Hopper validation with latest main
Integrate main 653067d7, preserving the tested Hopper corrections.
The incoming changes are JavaScript/Apple implementation and regression
updates; Hopper provider, verifier and dependency bytes are unchanged.

Real Hopper runs remain bound to exact head 97b76aa2 and tested checkout
f379bc38. This final integration has no new native execution claim.
26 focused tests, full static check and docs:check passed on this tree.
2026-10-12 19:33:06 -03:00
Luis Taffelli f379bc3882 fix(verification): integrate Linux Hopper lane with current contracts
Integrate main ae8051302 while retaining the Linux setup, orphan detection,
launcher restoration and explicit string-recognition coverage corrections.
Move verification guidance to its current owner and use the test-scoped
temporary-directory helper.

The checkout matches all 3,175 exported source-file digests of the tested
candidate. Git stores the Windows captureGhidraLaunch.cmd fixture with LF;
its exported checkout uses CRLF per .gitattributes. It is outside Linux
Hopper execution. No native run is attributed to different runtime bytes.
Real Hopper conformance: 9e7de373-4686-4820-ae48-801e536589b7.
Uninterrupted Linux setup: 93584b53-944b-47cf-a6a4-b0d4ff8a4961.
The exact 97b76aa2 run remains separate: ae4de7ff-dd2a-4c73-88dc-6dbf1ad7c537.

21 focused tests, check, docs:check, four fixtures, discovery, package
verification and pack dry-run passed. Linux native decoded omissions,
macOS/other builds and installed npm artifacts with Hopper remain unverified.

Refs #339, #340
2026-10-12 19:31:20 -03:00
Luis Taffelli 97b76aa20a fix(verification): report full literal decoding coverage explicitly 2026-10-11 18:46:16 -03:00
Luis Taffelli 8ea0432ffc fix(verification): distinguish Hopper string recognition from byte fidelity 2026-10-11 18:31:29 -03:00
Luis Taffelli b9d9a474cf fix(verification): report identified Hopper leaks before proc errors 2026-10-11 18:28:41 -03:00
Luis Taffelli b88cdc248a fix(verification): detect Hopper through alternate launcher paths 2026-10-11 17:06:32 -03:00
Luis Taffelli c039dd81d5 fix(verification): retain orphan checks on clean Linux runners 2026-10-11 16:35:44 -03:00
Luis Taffelli 58d23e42b6 fix(verification): align Linux Hopper setup lane with current contracts 2026-10-11 16:06:09 -03:00
283 changed files with 2145 additions and 11167 deletions
@@ -76,9 +76,6 @@ explicit path or endpoint and do not need it.
Facts retain all-recorded-events-before-selection scope. Other qlog layouts Facts retain all-recorded-events-before-selection scope. Other qlog layouts
and PCAP correlation are unsupported. and PCAP correlation are unsupported.
- User-owned Electron runtime already open: `list_electron_targets`. - User-owned Electron runtime already open: `list_electron_targets`.
- Packet-field discovery: `inspect_packet_fields` searches installed protocols
and field names/types before choosing fields for `inspect_packet_capture`.
Query/page the relevant records; registrations do not establish observed traffic.
- Recorded packet captures: `inspect_packet_capture` with selected TShark fields, - Recorded packet captures: `inspect_packet_capture` with selected TShark fields,
display filters and decode-as rules. Requires Linux, util-linux prlimit and `REA_TSHARK_COMMAND`; see display filters and decode-as rules. Requires Linux, util-linux prlimit and `REA_TSHARK_COMMAND`; see
[capture guide](https://github.com/morluto/rea/blob/main/docs/packet-captures.md). [capture guide](https://github.com/morluto/rea/blob/main/docs/packet-captures.md).
@@ -92,9 +89,6 @@ explicit path or endpoint and do not need it.
[packet summaries](https://github.com/morluto/rea/blob/main/docs/pcap-analysis.md). [packet summaries](https://github.com/morluto/rea/blob/main/docs/pcap-analysis.md).
Select an exact DNS question for address pages; page exhaustion does not prove Select an exact DNS question for address pages; page exhaustion does not prove
network completeness. This reads recordings and does not acquire traffic. network completeness. This reads recordings and does not acquire traffic.
- Decoder discovery: `inspect_signal_decoders` searches installed decoder IDs
or selected decoder help, including channels and options. Query and page the
relevant details before choosing settings; this does not scan hardware.
- Recorded logic/signal traces: `decode_signal_recording` with explicit decoder, - Recorded logic/signal traces: `decode_signal_recording` with explicit decoder,
channel and protocol settings. On Linux with util-linux prlimit, supply sigrok-cli 0.7.2 through channel and protocol settings. On Linux with util-linux prlimit, supply sigrok-cli 0.7.2 through
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/signal-recordings.md). `REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/signal-recordings.md).
@@ -83,14 +83,6 @@ names, the native API record, and value-flow summaries. Pseudocode offsets and l
code units; use the returned `next_offset` for continuation. Provider limitations code units; use the returned `next_offset` for continuation. Provider limitations
and unavailable facts remain explicit; the complete parent Evidence is retained. and unavailable facts remain explicit; the complete parent Evidence is retained.
`inspect_macho` on a large framework returns megabytes of imports and exports.
Select `detail: "summary"` for slice identity, UUID and per-collection counts,
then page rows with `view: {kind: "macho", facet: "exports", offset: 0, limit: 64}`
and `normalized_result.parent_evidence_id`. Facets are entrypoints,
architectures, build_metadata, load_commands, dependencies, imports, exports,
segments and provenance. `source_exhaustive: false` means unreturned symbols
may exist.
To see which functions or Objective-C methods a Mach-O actually calls in one To see which functions or Objective-C methods a Mach-O actually calls in one
run, use `observe_native_calls` with explicit breakpoints and a bounded run, use `observe_native_calls` with explicit breakpoints and a bounded
`duration_ms`/`max_events`. It launches a new process under LLDB, so the target `duration_ms`/`max_events`. It launches a new process under LLDB, so the target
+16 -77
View File
@@ -5,25 +5,16 @@ on:
branches: [main] branches: [main]
pull_request: pull_request:
branches: [main, "release/*"] branches: [main, "release/*"]
schedule:
- cron: "23 * * * *"
workflow_dispatch: workflow_dispatch:
workflow_call:
inputs:
source_sha:
description: Exact release source; must match the caller revision.
required: true
type: string
permissions: permissions:
contents: read contents: read
concurrency: concurrency:
# Complete qualification finishes even while main receives more pushes. group: ci-${{ github.event.pull_request.number || github.ref }}
# Scheduled/manual main runs coalesce to one active and the latest pending; # The latest main push verifies the full tree, including earlier changes.
# release qualification has its own exact-revision group. # Publication uses its own release workflow and concurrency group.
group: ci-${{ (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || inputs.source_sha != '' || contains(github.event.pull_request.labels.*.name, 'ci:full')) && 'qualification' || 'feedback' }}-${{ inputs.source_sha || github.event.pull_request.number || github.ref }} cancel-in-progress: true
cancel-in-progress: ${{ github.event_name != 'schedule' && github.event_name != 'workflow_dispatch' && inputs.source_sha == '' && !contains(github.event.pull_request.labels.*.name, 'ci:full') }}
env: env:
HUSKY: "0" HUSKY: "0"
@@ -31,7 +22,7 @@ env:
jobs: jobs:
pr-title: pr-title:
name: Conventional PR title name: Conventional PR title
if: github.event_name == 'pull_request' && inputs.source_sha == '' if: github.event_name == 'pull_request'
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Validate title for Release Please - name: Validate title for Release Please
@@ -48,11 +39,7 @@ jobs:
changes: changes:
name: Classify changes name: Classify changes
runs-on: ubuntu-latest runs-on: ubuntu-latest
permissions:
contents: read
actions: read
outputs: outputs:
tier: ${{ steps.plan.outputs.tier }}
code: ${{ steps.plan.outputs.code }} code: ${{ steps.plan.outputs.code }}
runtime: ${{ steps.plan.outputs.runtime }} runtime: ${{ steps.plan.outputs.runtime }}
package: ${{ steps.plan.outputs.package }} package: ${{ steps.plan.outputs.package }}
@@ -75,34 +62,16 @@ jobs:
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
with: with:
ref: ${{ inputs.source_sha || github.sha }}
fetch-depth: 0 fetch-depth: 0
- uses: actions/setup-node@v4 - uses: actions/setup-node@v4
with: with:
node-version-file: .nvmrc node-version-file: .nvmrc
- name: Bind release qualification to the caller revision
if: inputs.source_sha != ''
env:
SOURCE_SHA: ${{ inputs.source_sha }}
run: |
test "${SOURCE_SHA}" = "${GITHUB_SHA}"
test "${SOURCE_SHA}" = "$(git rev-parse HEAD)"
- name: Find a verified ancestor for push selection
id: baseline
if: github.event_name == 'push' && inputs.source_sha == ''
env:
GH_TOKEN: ${{ github.token }}
HEAD_SHA: ${{ github.sha }}
CI_REPOSITORY: ${{ github.repository }}
CI_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
run: node scripts/ci/baseline.mjs
- name: Select verification lanes - name: Select verification lanes
id: plan id: plan
env: env:
BASE_SHA: ${{ steps.baseline.outputs.base || github.event.pull_request.base.sha }} BASE_SHA: ${{ github.event.pull_request.base.sha }}
BASELINE_REASON: ${{ steps.baseline.outputs.reason }} HEAD_SHA: ${{ github.event.pull_request.head.sha }}
HEAD_SHA: ${{ inputs.source_sha || github.event.pull_request.head.sha || github.sha }} CI_FULL: ${{ contains(github.event.pull_request.labels.*.name, 'ci:full') }}
CI_FULL: ${{ inputs.source_sha != '' || contains(github.event.pull_request.labels.*.name, 'ci:full') }}
run: node scripts/ci/plan.mjs run: node scripts/ci/plan.mjs
static: static:
@@ -160,14 +129,7 @@ jobs:
with: with:
name: rea-test-runtime name: rea-test-runtime
path: . path: .
- name: Run complete portable suite with coverage - run: npm run test:ci:shard:run -- --shard=${{ matrix.shard }}
if: needs.changes.outputs.tier == 'full'
run: npm run test:ci:shard:run -- --shard=${{ matrix.shard }}
env:
REA_CI_TEST_HISTORY: .cache/ci-test-history/test-results.json
- name: Run complete portable suite for fast feedback
if: needs.changes.outputs.tier == 'fast'
run: npm run test:run -- --reporter=default --reporter=blob --shard=${{ matrix.shard }}
env: env:
REA_CI_TEST_HISTORY: .cache/ci-test-history/test-results.json REA_CI_TEST_HISTORY: .cache/ci-test-history/test-results.json
- uses: actions/upload-artifact@v4 - uses: actions/upload-artifact@v4
@@ -198,7 +160,7 @@ jobs:
- name: Reuse portable build outputs across runs - name: Reuse portable build outputs across runs
uses: ./.github/actions/portable-build-cache uses: ./.github/actions/portable-build-cache
with: with:
save: ${{ github.event_name != 'pull_request' && github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }} save: "true"
- run: npm run test:prepare - run: npm run test:prepare
- name: Find successful default-branch timing history - name: Find successful default-branch timing history
id: history id: history
@@ -210,7 +172,7 @@ jobs:
run: | run: |
run_id="$(gh api --method GET \ run_id="$(gh api --method GET \
"repos/${CI_REPOSITORY}/actions/workflows/ci.yml/runs" \ "repos/${CI_REPOSITORY}/actions/workflows/ci.yml/runs" \
-f branch="${CI_DEFAULT_BRANCH}" -f event=schedule -f status=success -f per_page=1 \ -f branch="${CI_DEFAULT_BRANCH}" -f event=push -f status=success -f per_page=1 \
--jq '.workflow_runs[0].id // empty')" --jq '.workflow_runs[0].id // empty')"
echo "run-id=${run_id}" >> "${GITHUB_OUTPUT}" echo "run-id=${run_id}" >> "${GITHUB_OUTPUT}"
- name: Download previous timing report - name: Download previous timing report
@@ -253,7 +215,7 @@ jobs:
uses: ./.github/workflows/sqlite-runtime.yml uses: ./.github/workflows/sqlite-runtime.yml
test: test:
name: Portable tests name: Test (coverage + timing)
needs: [changes, test-shard] needs: [changes, test-shard]
if: ${{ !cancelled() }} if: ${{ !cancelled() }}
runs-on: ubuntu-latest runs-on: ubuntu-latest
@@ -279,7 +241,7 @@ jobs:
merge-multiple: true merge-multiple: true
- name: Merge coverage, JUnit, and timing reports - name: Merge coverage, JUnit, and timing reports
id: merge id: merge
if: needs.changes.outputs.code == 'true' && needs.changes.outputs.tier == 'full' && success() if: needs.changes.outputs.code == 'true' && success()
run: >- run: >-
npm run test:ci:merge -- npm run test:ci:merge --
--coverage.reportsDirectory=coverage --coverage.reportsDirectory=coverage
@@ -291,22 +253,11 @@ jobs:
--reporter=json --reporter=json
--outputFile.junit=test-results.xml --outputFile.junit=test-results.xml
--outputFile.json=test-results.json --outputFile.json=test-results.json
- name: Merge fast-suite JUnit and timing reports
id: merge-fast
if: needs.changes.outputs.code == 'true' && needs.changes.outputs.tier == 'fast' && success()
run: >-
npm exec -- vitest --merge-reports
--reporter=default
--reporter=junit
--reporter=json
--outputFile.junit=test-results.xml
--outputFile.json=test-results.json
- name: Summarize slowest test files - name: Summarize slowest test files
if: needs.changes.outputs.code == 'true' && !cancelled() && hashFiles('test-results.json') != '' if: needs.changes.outputs.code == 'true' && !cancelled() && hashFiles('test-results.json') != ''
env: env:
SHARD_RESULT: ${{ needs.test-shard.result }} SHARD_RESULT: ${{ needs.test-shard.result }}
MERGE_RESULT: ${{ needs.changes.outputs.tier == 'full' && steps.merge.outcome || steps.merge-fast.outcome }} MERGE_RESULT: ${{ steps.merge.outcome }}
CI_VERIFY_TEST_INVENTORY: "true"
run: node scripts/ci/test-report.mjs run: node scripts/ci/test-report.mjs
- uses: actions/upload-artifact@v4 - uses: actions/upload-artifact@v4
if: needs.changes.outputs.code == 'true' && !cancelled() && hashFiles('test-results.json') != '' if: needs.changes.outputs.code == 'true' && !cancelled() && hashFiles('test-results.json') != ''
@@ -319,7 +270,7 @@ jobs:
if-no-files-found: error if-no-files-found: error
retention-days: 30 retention-days: 30
- name: Publish successful timing history - name: Publish successful timing history
if: needs.changes.outputs.code == 'true' && needs.changes.outputs.tier == 'full' && success() && needs.test-shard.result == 'success' if: needs.changes.outputs.code == 'true' && success() && needs.test-shard.result == 'success'
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v4
with: with:
name: test-timings name: test-timings
@@ -430,20 +381,9 @@ jobs:
- run: npm run verify:package - run: npm run verify:package
env: env:
npm_config_cache: ${{ steps.npm-cache.outputs.path }} npm_config_cache: ${{ steps.npm-cache.outputs.path }}
REA_VERIFIED_PACKAGE_DIRECTORY: ${{ needs.changes.outputs.tier == 'full' && matrix.platform == 'linux' && matrix.architecture == 'x64' && '.cache/verified-package' || '' }}
- name: Verify real Node Inspector discovery and observation - name: Verify real Node Inspector discovery and observation
timeout-minutes: 10 timeout-minutes: 10
run: npm run verify:inspector run: npm run verify:inspector
- name: Retain the exact qualified release package
if: needs.changes.outputs.tier == 'full' && matrix.platform == 'linux' && matrix.architecture == 'x64'
uses: actions/upload-artifact@v4
with:
name: verified-release-package
path: .cache/verified-package/
include-hidden-files: true
if-no-files-found: error
overwrite: true
retention-days: 7
apple-artifacts: apple-artifacts:
name: Real Apple artifact CLI/MCP E2E name: Real Apple artifact CLI/MCP E2E
@@ -717,7 +657,7 @@ jobs:
required: required:
name: CI required name: CI required
if: ${{ always() }} if: ${{ !cancelled() }}
needs: needs:
[ [
changes, changes,
@@ -747,7 +687,6 @@ jobs:
website, website,
] ]
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 5
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- uses: actions/setup-node@v4 - uses: actions/setup-node@v4
+4
View File
@@ -12,6 +12,10 @@ on:
permissions: permissions:
contents: read contents: read
concurrency:
group: real-binary-layout-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
env: env:
NODE_OPTIONS: --max-old-space-size=768 NODE_OPTIONS: --max-old-space-size=768
HUSKY: "0" HUSKY: "0"
+4
View File
@@ -4,6 +4,10 @@ on:
workflow_call: workflow_call:
workflow_dispatch: workflow_dispatch:
concurrency:
group: real-browser-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
env: env:
HUSKY: "0" HUSKY: "0"
+4
View File
@@ -7,6 +7,10 @@ on:
permissions: permissions:
contents: read contents: read
concurrency:
group: real-evm-interface-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
env: env:
NODE_OPTIONS: --max-old-space-size=768 NODE_OPTIONS: --max-old-space-size=768
HUSKY: "0" HUSKY: "0"
@@ -7,6 +7,10 @@ on:
permissions: permissions:
contents: read contents: read
concurrency:
group: javascript-recovery-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
env: env:
HUSKY: "0" HUSKY: "0"
NODE_OPTIONS: "--max-old-space-size=768" NODE_OPTIONS: "--max-old-space-size=768"
+1 -1
View File
@@ -51,7 +51,7 @@ jobs:
- run: sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y tshark util-linux python3 - run: sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y tshark util-linux python3
- run: npm ci - run: npm ci
- run: npm run build:cached - run: npm run build:cached
- run: npm run test:focused -- tests/boundary/captures/fieldDiscovery.test.ts src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts - run: npm run test:focused -- src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts
- run: node scripts/verify-packet-capture.mjs - run: node scripts/verify-packet-capture.mjs
env: env:
REA_TSHARK_COMMAND: /usr/bin/tshark REA_TSHARK_COMMAND: /usr/bin/tshark
@@ -7,6 +7,10 @@ on:
permissions: permissions:
contents: read contents: read
concurrency:
group: real-recorded-crash-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
env: env:
NODE_OPTIONS: --max-old-space-size=768 NODE_OPTIONS: --max-old-space-size=768
HUSKY: "0" HUSKY: "0"
+1 -1
View File
@@ -44,7 +44,7 @@ jobs:
- run: sudo apt-get update && sudo apt-get install -y sigrok-cli util-linux python3 - run: sudo apt-get update && sudo apt-get install -y sigrok-cli util-linux python3
- run: npm ci - run: npm ci
- run: npm run build:cached - run: npm run build:cached
- run: npm run test:focused -- tests/boundary/signals/decoderDiscovery.test.ts src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts - run: npm run test:focused -- src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts
- run: node scripts/verify-signal-recording.mjs - run: node scripts/verify-signal-recording.mjs
env: env:
REA_SIGROK_COMMAND: /usr/bin/sigrok-cli REA_SIGROK_COMMAND: /usr/bin/sigrok-cli
@@ -7,6 +7,10 @@ on:
permissions: permissions:
contents: read contents: read
concurrency:
group: real-web-network-captures-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
env: env:
HUSKY: "0" HUSKY: "0"
NODE_OPTIONS: --max-old-space-size=768 NODE_OPTIONS: --max-old-space-size=768
+3
View File
@@ -4,6 +4,9 @@ on:
workflow_dispatch: workflow_dispatch:
permissions: permissions:
contents: read contents: read
concurrency:
group: real-web-runtime-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
env: env:
HUSKY: "0" HUSKY: "0"
jobs: jobs:
@@ -4,6 +4,9 @@ on:
workflow_dispatch: workflow_dispatch:
permissions: permissions:
contents: read contents: read
concurrency:
group: real-web-source-map-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
env: env:
HUSKY: "0" HUSKY: "0"
jobs: jobs:
+10 -25
View File
@@ -290,25 +290,13 @@ jobs:
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true' if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
run: npm run docs:check && git diff --exit-code run: npm run docs:check && git diff --exit-code
qualification: publish:
name: Qualify exact release revision name: Publish to npm
needs: [merged-release, release-please] needs: [merged-release, release-please]
if: >- if: >-
always() && !cancelled() && always() && !cancelled() &&
((needs.merged-release.result == 'success' && needs.merged-release.outputs.release_created == 'true') || ((needs.merged-release.result == 'success' && needs.merged-release.outputs.release_created == 'true') ||
(needs.release-please.result == 'success' && inputs.phase == 'publish' && needs.release-please.outputs.release_created == 'true')) (needs.release-please.result == 'success' && inputs.phase == 'publish' && needs.release-please.outputs.release_created == 'true'))
permissions:
contents: read
actions: read
uses: ./.github/workflows/ci.yml
with:
source_sha: ${{ needs.merged-release.outputs.sha || needs.release-please.outputs.sha }}
publish:
name: Publish to npm
needs: [merged-release, release-please, qualification]
if: >-
always() && !cancelled() && needs.qualification.result == 'success'
runs-on: ubuntu-latest runs-on: ubuntu-latest
permissions: permissions:
contents: read contents: read
@@ -327,20 +315,17 @@ jobs:
- run: npm ci - run: npm ci
- name: Download the exact qualified package - name: Prepare Windows native build toolchain
uses: actions/download-artifact@v4 run: sudo apt-get update && sudo apt-get install --no-install-recommends -y g++-mingw-w64-x86-64
with:
name: verified-release-package
path: .cache/verified-package/
- name: Verify qualified package provenance and digest - name: Build and verify bundled Windows native controls
id: package run: npm run build:windows-native && npm run verify:windows-native:artifact
run: node scripts/ci/verified-package.mjs validate .cache/verified-package
- name: Verify packaged release before publishing
run: npm run verify:package
- name: Publish - name: Publish
shell: bash shell: bash
env:
VERIFIED_PACKAGE_TARBALL: ${{ steps.package.outputs.tarball }}
run: | run: |
version="$(node --input-type=module -e 'import packageJson from "./package.json" with { type: "json" }; process.stdout.write(packageJson.version)')" version="$(node --input-type=module -e 'import packageJson from "./package.json" with { type: "json" }; process.stdout.write(packageJson.version)')"
published_version="$(npm view "rea-agents@${version}" version --json 2>/dev/null || true)" published_version="$(npm view "rea-agents@${version}" version --json 2>/dev/null || true)"
@@ -348,7 +333,7 @@ jobs:
echo "rea-agents@${version} is already published; verifying the existing package." echo "rea-agents@${version} is already published; verifying the existing package."
else else
tag="$(node scripts/release-npm-tag.mjs "${version}")" tag="$(node scripts/release-npm-tag.mjs "${version}")"
npm publish "${VERIFIED_PACKAGE_TARBALL}" --access public --tag "${tag}" npm publish --access public --tag "${tag}"
fi fi
- name: Verify published CLI from npm - name: Verify published CLI from npm
+4
View File
@@ -7,6 +7,10 @@ on:
permissions: permissions:
contents: read contents: read
concurrency:
group: website-check-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs: jobs:
check: check:
name: Website checks name: Website checks
-2
View File
@@ -72,5 +72,3 @@ Thumbs.db
# Complete static website output, including generated language editions. # Complete static website output, including generated language editions.
website/_site/ website/_site/
# Temporary translation output prepared before replacing the complete site.
website/.i18n-*/
+1 -1
View File
@@ -1,5 +1,5 @@
{ {
"threshold": 3.5, "threshold": 3.0,
"reporters": ["consoleFull"], "reporters": ["consoleFull"],
"ignore": [ "ignore": [
"node_modules/**", "node_modules/**",
+1 -1
View File
@@ -93,7 +93,7 @@ def project_record(state, secrets):
safe_bytes, credential_url = transport_value(value, path, transport_url) safe_bytes, credential_url = transport_value(value, path, transport_url)
content_base64 = base64.b64encode(value).decode("ascii") content_base64 = base64.b64encode(value).decode("ascii")
digest = hashlib.sha256(value).hexdigest() digest = hashlib.sha256(value).hexdigest()
hidden = any(secret in value or secret in safe_bytes for secret in byte_secrets) or any(secret in content_base64 or secret in digest or secret in str(len(value)) for secret in secrets) hidden = any(secret in value or secret in safe_bytes for secret in byte_secrets) or any(secret in content_base64 or secret in digest for secret in secrets)
safe_text = None safe_text = None
try: try:
safe_text = safe_bytes.decode("utf-8", errors="strict") safe_text = safe_bytes.decode("utf-8", errors="strict")
+1 -1
View File
@@ -174,7 +174,7 @@ Resolution follows dyld:
- `@rpath` tries each `LC_RPATH` of the loading image first, then those of the - `@rpath` tries each `LC_RPATH` of the loading image first, then those of the
image that loaded it, up to the process root. image that loaded it, up to the process root.
- Dependencies load dependents-first in load-command order, and each image - Dependencies load dependents-first in load-command order, and each image
loads once per process. A later @rpath request whose install name matches an loads once per process. A later request whose install name matches an
already loaded image reuses it. already loaded image reuses it.
- Symlinks such as `Versions/Current` are followed inside the analyzed root. - Symlinks such as `Versions/Current` are followed inside the analyzed root.
An escape through `..` or an absolute link is `escapes-target`. An escape through `..` or an absolute link is `escapes-target`.
-4
View File
@@ -57,10 +57,6 @@ Hopper installation or any file write. Declining or pressing Ctrl-C makes no
changes. Agent configuration writes preserve unrelated entries and comments, changes. Agent configuration writes preserve unrelated entries and comments,
create backups, use atomic replacement, and verify their result. Setup and create backups, use atomic replacement, and verify their result. Setup and
uninstall retain an existing `.rea.backup` rather than replacing the first snapshot. uninstall retain an existing `.rea.backup` rather than replacing the first snapshot.
If that snapshot differs from the current configuration, setup previews and
preserves a verified `.rea.backup.<sha256>.recovery` copy of the current bytes.
Both snapshots remain available; a failed rollback reports the current recovery
copy's path and the restore failure.
Setup also retains valid environment overrides in REA's existing server entry, Setup also retains valid environment overrides in REA's existing server entry,
such as `REA_MCP_INPUT_SCHEMA_PROFILE` and `REA_MCP_MAX_RESPONSE_BYTES`. such as `REA_MCP_INPUT_SCHEMA_PROFILE` and `REA_MCP_MAX_RESPONSE_BYTES`.
Detected provider settings replace only matching environment keys. Detected provider settings replace only matching environment keys.
-9
View File
@@ -104,15 +104,6 @@ static inferences. Dynamic or ambiguous IPC is retained without a pairing edge.
Runtime entities retain passive-CDP authority, while a successful static/runtime Runtime entities retain passive-CDP authority, while a successful static/runtime
correspondence is a separately identified cross-layer inference. correspondence is a separately identified cross-layer inference.
Source module links retain their resolution mode. Static imports in `.ts` and
`.tsx` are `compiler-dependent` when compiler options have not been retained.
REA compares Node import and require resolution: a shared outcome remains
available with a limitation; divergent outcomes remain unavailable with both
conditional branches and their limitations. This models source relationships,
not verified emitted code or execution. A declaration file found only through
an emitted-extension fallback is retained as a candidate, not a runtime target;
an explicitly selected path keeps its original identity.
## Evidence and authority ## Evidence and authority
Every observation and edge carries all of the following: Every observation and edge carries all of the following:
-8
View File
@@ -23,14 +23,6 @@ artifact digest and structural graph ID. Semantic tracing requires the semantic
relation graph and reports when it is unavailable rather than treating missing relation graph and reports when it is unavailable rather than treating missing
data as an empty graph. data as an empty graph.
Explicit TypeScript `import type` and `export type` declarations remain source
facts and do not create runtime import or export relationships. Type export names
are reported separately from value export names. Inline `type` specifiers can
leave an empty import or re-export with side effects under
[`verbatimModuleSyntax`](https://www.typescriptlang.org/tsconfig/verbatimModuleSyntax.html);
REA retains those dependency candidates and reports the unavailable compiler
options rather than assuming the dependency is erased.
The semantic graph wire format stores shared non-location provenance once in The semantic graph wire format stores shared non-location provenance once in
its required `evidence_contexts` table. Each node, relation, fingerprint, and its required `evidence_contexts` table. Each node, relation, fingerprint, and
unknown retains its exact `evidence.location` and names the owning context with unknown retains its exact `evidence.location` and names the owning context with
+2 -15
View File
@@ -147,8 +147,8 @@ after target content was read.
## Retained application Evidence inputs ## Retained application Evidence inputs
`analyze_javascript_application`, `inspect_binary_layout`, `inspect_artifact`, `analyze_javascript_application`, `inspect_binary_layout`, `inspect_artifact`, and
`inspect_wasm_artifact`, and `inspect_macho` accept `detail`: `inspect_wasm_artifact` accept `detail`:
`complete` (default) returns the complete analysis Evidence; `summary` records `complete` (default) returns the complete analysis Evidence; `summary` records
that complete Evidence in the session and returns the derived that complete Evidence in the session and returns the derived
`inspect_analysis_view` summary Evidence instead. Its `inspect_analysis_view` summary Evidence instead. Its
@@ -218,19 +218,6 @@ attempting oversized delivery, and returns the same summary with an exact
`parent_evidence_id` for later export or facet reads. Delivery detail does not `parent_evidence_id` for later export or facet reads. Delivery detail does not
reach WABT or change the original source Evidence ID. reach WABT or change the original source Evidence ID.
Completed `inspect_macho` Evidence supports a summary and
`view: {kind: "macho", facet: "exports", offset: 0, limit: 64}`. Facets are
`entrypoints`, `architectures`, `build_metadata`, `load_commands`,
`dependencies`, `imports`, `exports`, `segments`, and `provenance`; pages return
the original rows. The summary keeps the selected slice, header fields and UUID,
and replaces each collection with its returned count, reported total and
`exhaustive` flag. Each page repeats the paged collection's `source_total` and
`source_exhaustive`; exhausting a non-exhaustive collection does not establish
that unreturned symbols are absent. Large frameworks return megabytes of
symbols (Electron Framework: 2.5 MB of imports and exports, 3.3 KB as a
summary), so select `detail: "summary"` on the first call. Delivery detail
never reaches the native tools or changes the source Evidence ID.
`trace_application_feature`, `trace_javascript_semantics`, `trace_application_feature`, `trace_javascript_semantics`,
`compare_application_versions`, `compare_source_to_bundle`, and `compare_application_versions`, `compare_source_to_bundle`, and
`compare_javascript_export_shapes` accept complete inline application Evidence `compare_javascript_export_shapes` accept complete inline application Evidence
-46
View File
@@ -128,49 +128,3 @@ Each provider admits one inspection at a time, including acquisition and cleanup
An occupied provider returns a retryable capacity error before acquiring another An occupied provider returns a retryable capacity error before acquiring another
file or worker. Closing the provider stops new admission and drains accepted work; file or worker. Closing the provider stops new admission and drains accepted work;
failed cleanup remains owned and can be retried. failed cleanup remains owned and can be retried.
## Discover protocols and field names
`inspect_packet_fields` / `inspect-packet-fields` queries the installed TShark
registration glossary before selecting fields for capture inspection:
```sh
rea inspect-packet-fields --query usb
rea inspect-packet-fields --kind fields --protocol usb --query idVendor
rea inspect-packet-fields --kind fields --protocol udp --all
```
`kind` defaults to `protocols`; `fields` selects field definitions. `protocol`
selects an exact protocol abbreviation (the protocol itself or its child fields).
`query` is a case-insensitive literal substring in name, abbreviation or field
description. `offset` skips matching records; `limit` defaults to 50. MCP
`limit: null` or CLI `--all` requests all remaining matches within the resource
budget. `--all` and `--limit` conflict. The window reports examined records of
the selected kind before filters, matching records, returned records and
`has_more`; it never counts capture packets.
Returned records preserve field type, parent protocol, display base, bitmask,
description and original glossary line number. Empty producer strings remain
empty; protocol records have null field-only properties. Repeated abbreviations
remain separate registrations. A listed field is an installation capability, not
proof that the field appears in a selected recording.
The adapter uses the documented `tshark -G fields` grammar under its existing
private empty profile and Linux process limits. TShark 4.6.9's full glossary was
measured at 22,303,197 bytes; discovery admits at most **32 MiB** combined command
output (capture inspection remains at **4 MiB**). Parsing walks lines once and
constructs objects only for the selected page. Returned strings are copied so
retained pages do not retain the complete glossary string. Before constructing a selected
record it charges six times the source UTF-8 bytes plus 512 bytes against an
**8 MiB** representation budget, covering JSON escaping and record bookkeeping.
If that budget would be exceeded, narrow the query or page size; results are not
silently truncated. This is a conservative admission budget, not measured heap
usage. Parent pipe buffers and producer per-process limits remain distinct.
Tool/limiter identity, version, stderr and selected arguments are retained in
Evidence. No capture, interface or device is opened. Discovery shares capture
analysis's admission slot and cleanup owner. Each call rereads the glossary;
installed plugins, libraries and global configuration remain trusted and are not
fully fingerprinted. There is no cache or snapshot-consistent pagination claim.
The real Linux lane checks public CLI/MCP queries against the direct glossary,
including USB vendor-ID typing, negative queries and paging.
+3 -12
View File
@@ -6,10 +6,7 @@ checks to pass. Merging that PR into main starts publication automatically.
The Release workflow validates the merged release metadata and ancestry before The Release workflow validates the merged release metadata and ancestry before
creating a tag. npm and MCP Registry publication use the exact reviewed merge creating a tag. npm and MCP Registry publication use the exact reviewed merge
SHA returned by Release Please. Before either registry is published, the release SHA returned by Release Please. Later main commits cannot change that source.
run calls full CI qualification at that exact revision, including coverage,
portable providers, and every native package host. Later main commits cannot
change or cancel that proof.
Ordinary main pushes and closed, unmerged PRs cannot publish packages. Check Ordinary main pushes and closed, unmerged PRs cannot publish packages. Check
both publication jobs and the published-package canary; a merged PR or GitHub both publication jobs and the published-package canary; a merged PR or GitHub
tag alone does not establish that publication finished. Retry failed jobs in tag alone does not establish that publication finished. Retry failed jobs in
@@ -199,14 +196,8 @@ both registry publishes.
See [npm's provenance implementation](https://github.com/npm/cli/blob/v11.16.0/workspaces/libnpmpublish/lib/provenance.js) See [npm's provenance implementation](https://github.com/npm/cli/blob/v11.16.0/workspaces/libnpmpublish/lib/provenance.js)
for the use of GitHub's workflow ref and commit SHA. for the use of GitHub's workflow ref and commit SHA.
Full qualification builds the bundled Windows controls and verifies the packaged The workflow builds the bundled Windows controls and verifies the packaged
artifact on its native hosts. The Linux x64 verifier retains the exact tarball artifact before npm publication. It then verifies the published CLI, the
it installed and tested, after its workspace cleanup completes. The npm job
downloads that archive from the same workflow run and checks its source SHA,
run identity/attempt, package and Node versions, host identity, and SHA-256.
It publishes that archive without rebuilding or repacking it. Re-running failed
publisher jobs can reuse a prior successful attempt from the same run.
The workflow then verifies the published CLI, the
capability-scoped MCP catalog, and the isolated package update path before capability-scoped MCP catalog, and the isolated package update path before
publishing MCP Registry metadata. publishing MCP Registry metadata.
-34
View File
@@ -106,37 +106,3 @@ One decode is admitted at a time, including acquisition and cleanup. Excess
requests receive a retryable capacity error before acquiring another file or requests receive a retryable capacity error before acquiring another file or
worker. Shutdown closes admission and drains accepted work; failed cleanup worker. Shutdown closes admission and drains accepted work; failed cleanup
remains owned and retryable. remains owned and retryable.
## Discover decoders before choosing settings
`inspect_signal_decoders` / `inspect-signal-decoders` queries the **installed**
sigrok catalog without opening a recording or scanning devices:
```sh
rea inspect-signal-decoders --query uart
rea inspect-signal-decoders --decoder uart --query baud
rea inspect-signal-decoders --decoder i2c --offset 0 --limit 50
```
Omit `decoder` to search decoder IDs and names. Supply one exact decoder ID to
read its literal help, including required/optional channels, options, annotation
classes and documentation. `query` is a case-insensitive literal substring;
matching help retains original one-based line numbers. Empty query selects all.
`offset` skips matches; `limit` defaults to 50. MCP `limit: null` or CLI `--all`
returns all remaining matches; CLI `--all` and `--limit` conflict.
The window reports examined, matching and returned records and `has_more`.
A filtered empty page is not evidence that the installation has no decoders.
Discovery refuses more than 65,535 line separators before constructing record
objects (65,536-line representation budget), in addition to the 4 MiB output cap.
Exceeding that budget is a resource failure with captured output. Returned text
is copied so a retained page does not keep the complete producer output alive.
Discovery shares decoding's owned process, admission slot, resource limits,
cleanup retries and Evidence delivery. It records the executable and limiter
identities, version banner and warnings. Failed version checks and malformed
producer output preserve diagnostics. Help is producer text, not an independently
validated parameter schema. Each request probes again; it does not cache or claim
stable pages across installation changes. Libraries/decoder modules remain
trusted and are not fully fingerprinted. Catalog output includes a SHA-256 of
its executable, not a digest of the complete installation.
+29 -57
View File
@@ -263,13 +263,12 @@ CI owns coverage; aggregate and domain/contract thresholds are maintained in
Coverage does not replace named boundary or real-provider scenarios. Coverage does not replace named boundary or real-provider scenarios.
CI runs six native Vitest shards without retries. Each shard emits a blob CI runs six native Vitest shards without retries. Each shard emits a blob
report; the merge job produces JUnit and JSON timing reports, plus aggregate report; the merge job produces aggregate coverage plus JUnit and JSON timing
coverage during full qualification. It retains available
reports even when a shard fails, uploads available diagnostics, and writes the reports even when a shard fails, uploads available diagnostics, and writes the
slowest files to the workflow summary. Partial reports are marked diagnostic slowest files to the workflow summary. Partial reports are marked diagnostic
only. Failed, cancelled, or missing selected shards still fail the required gate, only. Failed, cancelled, or missing selected shards still fail the required gate,
and aggregate coverage thresholds remain enforced in full qualification. and aggregate coverage thresholds remain enforced by the merge command.
Only successful complete full-tier merges publish the small `test-timings` artifact, with Only successful complete merges publish the small `test-timings` artifact, with
checkout-relative paths; the build job downloads that artifact for the sequencer checkout-relative paths; the build job downloads that artifact for the sequencer
instead of the coverage/JUnit bundle. Missing history falls back to native instead of the coverage/JUnit bundle. Missing history falls back to native
sharding. Static checks, documentation, build, package verification, and sharding. Static checks, documentation, build, package verification, and
@@ -718,7 +717,7 @@ or the GitHub issue and pull-request templates run formatting and
generated-document validation, including the small onboarding example consistency generated-document validation, including the small onboarding example consistency
suite, without the source-test shards or native package suite, without the source-test shards or native package
lanes. Classification compares the PR head with its merge base, so later lanes. Classification compares the PR head with its merge base, so later
base-branch changes do not expand that scope. Static and portable-test aggregate jobs base-branch changes do not expand that scope. Static and coverage aggregate jobs
remain present and fail if classification fails. Every code change retains the remain present and fail if classification fails. Every code change retains the
complete six-shard Linux suite; individual test files are not selected by complete six-shard Linux suite; individual test files are not selected by
imports or filenames. imports or filenames.
@@ -747,61 +746,38 @@ verification stays manual. Renames account for both paths.
This narrower selection trades PR-time coverage for runner capacity. A change to This narrower selection trades PR-time coverage for runner capacity. A change to
unowned shared runtime code that fails only on arm64, macOS, Windows, the Apple unowned shared runtime code that fails only on arm64, macOS, Windows, the Apple
artifact lane, or a provider lane that does not own the file is not caught on artifact lane, or a provider lane that does not own the file is not caught on
the PR; scheduled full qualification reports it after merge. Ownership globs do the PR; the main-push full baseline reports it after merge. Ownership globs do
not follow imports. Use the `ci:full` label when a change may have native-host not follow imports. Use the `ci:full` label when a change may have native-host
or provider effects outside its owners. or provider effects outside its owners.
Ordinary main pushes use the same owner scopes as PRs. Their diff starts at a Main pushes and manual dispatch retain the full baseline, including all portable
successful ancestor push run of this CI workflow, rather than the preceding provider lanes and the native matrix below. A `ci:full` PR label selects the full
push: changes from failed or cancelled intermediate runs remain in scope. baseline on the next PR run; manual dispatch can request it immediately. Each
Missing, untrusted, or non-ancestor history selects every lane. The lookup is
bounded to the last 100 successful push runs; lookup failures broaden selection.
Every code change still runs all six portable shards, without coverage
instrumentation in this fast tier. This deliberately avoids relying on an
import graph for generated files, filesystem inputs, subprocesses, or bridges.
Full qualification runs hourly at minute 23, on manual dispatch, and for the
exact release revision before publication. It includes aggregate coverage,
every portable provider lane, and all native hosts below. A `ci:full` PR label
selects full qualification on the next PR run; manual dispatch can request it
immediately. Fast-tier broad fallbacks retain all owner lanes and the three-host
package matrix; Intel macOS and coverage remain full-tier obligations. Each
classification job records its selection and broad-fallback reasons in the run classification job records its selection and broad-fallback reasons in the run
summary. Representative Git-diff cases live in summary. Representative Git-diff cases live in
`tests/boundary/filesystem/ciChangeScope.test.ts`. `tests/boundary/filesystem/ciChangeScope.test.ts`.
`CI required` always runs and checks classification and every selected lane, `CI required` checks classification and every selected lane, rejecting failed,
rejecting failed, cancelled, or unexpectedly skipped work. Branch protection cancelled, or unexpectedly skipped work. Existing required check names are
should require this aggregate, `Static checks`, and `Portable tests`. Fast runs preserved; changing branch protection to require the new aggregate is a separate
cancel superseded feedback for the same PR or main branch. Scheduled/manual repository-settings operation. Newer commits cancel superseded CI for the same
qualification has a separate concurrency group: active work finishes and only PR or main branch. Release publication has its own concurrency group and retains
the latest pending run is retained. Release qualification is grouped by its its package verification and public-registry canary. Licensed/self-hosted
exact source SHA and cannot be cancelled by newer main pushes. Provider and
website lanes inherit their caller's scheduling; reusable
workflows have no independent cancellation group. Direct manual lane dispatches
run independently on disposable hosted runners. Publication
requires that qualification to succeed and consumes its exact verified tarball,
checking source SHA, workflow run, run attempt, package/Node version and SHA-256
before npm publication. Earlier attempts from the same workflow run are accepted
when rerunning failed publisher jobs; future attempts and other runs are rejected.
The public-registry canary still verifies the published package. Licensed/self-hosted
provider workflows and the Android smoke test retain their explicit triggers. provider workflows and the Android smoke test retain their explicit triggers.
CI exercises the pinned Node.js runtime on native hosted runners: CI exercises the pinned Node.js runtime on native hosted runners:
| Host | Runner | Baseline checks | | Host | Runner | Baseline checks |
| --------------------- | ------------------ | ------------------------------------------------------------------------------------------- | | --------------------- | ------------------ | ---------------------------------------------------------------------------------------- |
| Linux x64 | `ubuntu-latest` | Installed package and real Node Inspector CLI/MCP | | Linux x64 | `ubuntu-latest` | Installed package and real Node Inspector CLI/MCP |
| Linux arm64 (aarch64) | `ubuntu-24.04-arm` | Installed package and real Node Inspector CLI/MCP | | Linux arm64 (aarch64) | `ubuntu-24.04-arm` | Installed package and real Node Inspector CLI/MCP |
| macOS 15 arm64 | `macos-15` | Installed package and real Node Inspector CLI/MCP | | macOS 15 arm64 | `macos-15` | Installed package and real Node Inspector CLI/MCP |
| macOS 15 x64 | `macos-15-intel` | Installed package and real Node Inspector CLI/MCP (hourly, dispatch, release and `ci:full`) | | macOS 15 x64 | `macos-15-intel` | Installed package and real Node Inspector CLI/MCP (main, dispatch and `ci:full` only) |
| Windows x64 | `windows-latest` | Curated capabilities, native controls, installed package and real Node Inspector CLI/MCP | | Windows x64 | `windows-latest` | Curated capabilities, native controls, installed package and real Node Inspector CLI/MCP |
macOS x64 is the slowest package host. Ordinary PRs and main pushes omit it, macOS x64 is the slowest package host. Pull requests without the `ci:full` label
so darwin x64 installed-package and Inspector regressions surface in full omit it, so darwin x64 installed-package and Inspector regressions surface on
qualification. Schedule admission can be delayed by GitHub; the hourly trigger the main-push full baseline rather than on the PR.
does not guarantee an hourly completed result. Release publication waits for
its own complete result at the exact released revision.
Package lanes assert the actual Node platform/architecture before verification Package lanes assert the actual Node platform/architecture before verification
and record those values with the Node version. The package matrix runs up to and record those values with the Node version. The package matrix runs up to
@@ -827,21 +803,17 @@ The Linux source-test shards download a shared runtime and generated test
metadata snapshot produced by the build job with `npm run test:prepare`. The metadata snapshot produced by the build job with `npm run test:prepare`. The
snapshot includes `dist/`, generated skills, the MCP tool catalog, product snapshot includes `dist/`, generated skills, the MCP tool catalog, product
catalog, and managed-conformance metadata. Each shard still installs the locked catalog, and managed-conformance metadata. Each shard still installs the locked
dependencies and runs its assigned tests; full qualification enables coverage. It no longer dependencies and runs its assigned coverage-enabled tests; it no longer
regenerates the same metadata independently. regenerates the same metadata independently.
The build job also retrieves the merged timing report from the last successful The build job also retrieves the merged timing report from the last successful
default-branch scheduled qualification and includes that same report in each shard's snapshot. default-branch push and includes that same report in each shard's snapshot.
CI assigns discovered files by historical duration, with a deterministic CI assigns discovered files by historical duration, with a deterministic
fallback weight for new files, rather than dividing only by file count. Every fallback weight for new files, rather than dividing only by file count. Every
discovered file still belongs to exactly one shard; worker limits and project discovered file still belongs to exactly one shard; worker limits, project
ordering remain unchanged. The merge job rejects successful reports with missing, ordering, coverage instrumentation, and aggregate thresholds remain unchanged.
unexpected, or duplicate portable files. Coverage instrumentation and aggregate
thresholds remain enforced in full qualification.
Missing or unusable history falls back to Vitest's default assignment. Only Missing or unusable history falls back to Vitest's default assignment. Only
timing data is reused; live provider results are never cached. Turbo outputs are timing data is reused; persistent compile caches remain disabled.
restored by task hash, with trusted default-branch runs as cache writers and PRs
as restore-only consumers. npm caches hold downloads, not `node_modules`.
These native baseline checks complement the Linux source-test shards and the These native baseline checks complement the Linux source-test shards and the
separate Apple-artifact and real-provider lanes. Actual Hopper, Ghidra, IDA, separate Apple-artifact and real-provider lanes. Actual Hopper, Ghidra, IDA,
+2 -6
View File
@@ -153,12 +153,8 @@ the private runtime, isolating Ghidra preferences, logs, and caches.
Process creation assigns a suspended child atomically to an owned Job Object. Process creation assigns a suspended child atomically to an owned Job Object.
Membership and kill-on-owner-close policy are verified before resume. Membership and kill-on-owner-close policy are verified before resume.
Breakaway is disabled. Only the child's default file owner is normalized; Breakaway is disabled. Only the child's default file owner is normalized;
the caller token stays unchanged. Caller priority is preserved, and caller CPU the caller token stays unchanged. Caller CPU affinity and priority are
affinity whenever the caller and the child share a processor group. A child that preserved. Close, cancellation, timeout, startup failure, and provider failure
Windows places in another group keeps that group's default when the caller asked
for no bound of its own; a caller pinned to fewer processors than its group
holds is refused there rather than widened.
Close, cancellation, timeout, startup failure, and provider failure
terminate the job and verify all members have settled before runtime cleanup. terminate the job and verify all members have settled before runtime cleanup.
Cleanup removes admitted objects by handle and unlinks reparse entries without Cleanup removes admitted objects by handle and unlinks reparse entries without
following their targets. Abrupt owner death kills the job but can leave following their targets. Abrupt owner death kills the job but can leave
+4 -16
View File
@@ -69,22 +69,10 @@ without this lane reports Windows native controls as unavailable.
limits only for its JavaScript helpers when running on Bun, which ignores limits only for its JavaScript helpers when running on Bun, which ignores
their V8 heap flags; Node.js launches are unchanged. their V8 heap flags; Node.js launches are unchanged.
- Only the suspended child's default file owner is normalized. Caller token - Only the suspended child's default file owner is normalized. Caller token
privileges and ownership remain unchanged. Caller process priority is privileges and ownership remain unchanged. Caller CPU affinity and process
preserved in the child, and caller CPU affinity whenever both occupy the same priority are preserved in the child. Environment names are sorted and
processor group. Windows chooses a new process's group itself and rejects a deduplicated using Windows ordinal comparison, preserving their native Unicode
mask belonging to another group, so a child placed elsewhere keeps that semantics and caller-selected values.
group's default when the caller asked for no bound of its own. A caller
deliberately pinned to fewer processors than its group holds is a constraint
such a child cannot express, and is refused rather than widened to that
group's default. A mask covering its group's whole active set does not
distinguish a caller Windows placed in that group from one that selected the
group deliberately, so such a child can still be placed elsewhere: its
processor count stays bounded, its group placement is not preserved, and
neither that case nor the refusal above has a verification lane, because both
need a host whose processors span more than one group.
Environment names are sorted and deduplicated using Windows
ordinal comparison, preserving their native Unicode semantics and
caller-selected values.
- Cleanup walks locked directory objects, removes files by handle, and unlinks - Cleanup walks locked directory objects, removes files by handle, and unlinks
reparse entries without traversing their targets. Failed removal retains the reparse entries without traversing their targets. Failed removal retains the
root and ancestor leases for retry; a partially removed runtime accepts only root and ancestor leases for retry; a partially removed runtime accepts only
+1 -29
View File
@@ -68,18 +68,6 @@ static ULONGLONG jobMemoryLimit(napi_env env, napi_value value) {
return limit; return limit;
} }
// An affinity mask is meaningful only inside one processor group, so only a
// process occupying exactly one group has a group a child can inherit. A
// process whose threads span groups, or whose membership cannot be read, has no
// single group.
static bool singleProcessorGroup(HANDLE process, USHORT& group) {
USHORT count = 1;
std::array<USHORT, 1> groups{};
if (!GetProcessGroupAffinity(process, &count, groups.data()) || count != 1) return false;
group = groups[0];
return true;
}
static void pipe(Handle& read, Handle& write, bool parentReads = true) { static void pipe(Handle& read, Handle& write, bool parentReads = true) {
SECURITY_ATTRIBUTES attributes{sizeof(SECURITY_ATTRIBUTES), nullptr, TRUE}; SECURITY_ATTRIBUTES attributes{sizeof(SECURITY_ATTRIBUTES), nullptr, TRUE};
HANDLE rawRead, rawWrite; HANDLE rawRead, rawWrite;
@@ -180,27 +168,11 @@ std::unique_ptr<Process> spawnProcess(const std::wstring& command, const std::ws
TOKEN_OWNER owner{reinterpret_cast<TOKEN_USER*>(tokenUser.data())->User.Sid}; TOKEN_OWNER owner{reinterpret_cast<TOKEN_USER*>(tokenUser.data())->User.Sid};
require(SetTokenInformation(token.get(), TokenOwner, &owner, sizeof(owner)), require(SetTokenInformation(token.get(), TokenOwner, &owner, sizeof(owner)),
"Set suspended process file owner failed", command); "Set suspended process file owner failed", command);
// A mask names processors inside one processor group, and Windows assigns a
// new process to a group of its own choosing, so a mask belonging to another
// group is rejected (ERROR_INVALID_PARAMETER). An unconstrained caller, whose
// mask is its group's whole active set, asks for no bound the child's own
// group cannot give, so such a child keeps that group's default and the spawn
// proceeds. A narrower mask is a deliberate CPU constraint that a child in
// another group cannot express, and silently widening it would hand the
// provider tree the whole group, so that spawn fails naming the constraint.
DWORD_PTR affinity = 0, systemAffinity = 0; DWORD_PTR affinity = 0, systemAffinity = 0;
require(GetProcessAffinityMask(GetCurrentProcess(), &affinity, &systemAffinity), require(GetProcessAffinityMask(GetCurrentProcess(), &affinity, &systemAffinity),
"Read caller process affinity failed", command); "Read caller process affinity failed", command);
USHORT callerGroup = 0, childGroup = 0; if (affinity != 0)
const bool sharedGroup = singleProcessorGroup(GetCurrentProcess(), callerGroup) &&
singleProcessorGroup(result->process.get(), childGroup) &&
callerGroup == childGroup;
if (affinity != 0 && sharedGroup)
require(SetProcessAffinityMask(result->process.get(), affinity), "Preserve caller process affinity failed", command); require(SetProcessAffinityMask(result->process.get(), affinity), "Preserve caller process affinity failed", command);
else
require(affinity == 0 || affinity == systemAffinity,
"Caller CPU affinity cannot bound an owned process in another processor group", command,
ERROR_INVALID_PARAMETER);
const DWORD priority = GetPriorityClass(GetCurrentProcess()); const DWORD priority = GetPriorityClass(GetCurrentProcess());
require(priority != 0 && SetPriorityClass(result->process.get(), priority), require(priority != 0 && SetPriorityClass(result->process.get(), priority),
"Preserve caller process priority failed", command); "Preserve caller process priority failed", command);
-76
View File
@@ -1,76 +0,0 @@
import { execFileSync } from "node:child_process";
import { appendFileSync } from "node:fs";
const repository = process.env.CI_REPOSITORY;
const branch = process.env.CI_DEFAULT_BRANCH;
const head = process.env.HEAD_SHA;
const shaPattern = /^[0-9a-f]{40}$/i;
function ancestor(base) {
try {
execFileSync("git", ["merge-base", "--is-ancestor", base, head], {
stdio: "ignore",
});
return true;
} catch {
return false;
}
}
function write(base, reason) {
const output = { base, reason };
if (process.env.GITHUB_OUTPUT) {
appendFileSync(
process.env.GITHUB_OUTPUT,
`base=${base}\nreason=${reason.replace(/[\r\n]/g, " ")}\n`,
);
}
console.log(JSON.stringify(output));
}
if (!repository || !branch || !head || !shaPattern.test(head)) {
write("", "Repository, default branch, or checked-out head is invalid");
} else {
try {
const response = execFileSync(
"gh",
[
"api",
"--method",
"GET",
`repos/${repository}/actions/workflows/ci.yml/runs`,
"-f",
`branch=${branch}`,
"-f",
"event=push",
"-f",
"status=success",
"-F",
"per_page=100",
],
{ encoding: "utf8", timeout: 15_000, maxBuffer: 4 * 1024 * 1024 },
);
const runs = JSON.parse(response).workflow_runs;
if (!Array.isArray(runs))
throw new Error("GitHub returned no workflow run list");
const base = runs.find(
(run) =>
run?.event === "push" &&
run?.conclusion === "success" &&
run?.head_branch === branch &&
run?.head_repository?.full_name === repository &&
typeof run?.head_sha === "string" &&
shaPattern.test(run.head_sha) &&
run.head_sha.toLowerCase() !== head.toLowerCase() &&
ancestor(run.head_sha),
)?.head_sha;
if (base) write(base, "");
else
write(
"",
"No trusted successful CI push baseline is an ancestor of this head",
);
} catch {
write("", "Could not read a successful CI push baseline from GitHub");
}
}
+82 -107
View File
@@ -108,123 +108,98 @@ function classifyPackage(base, head) {
} }
const event = process.env.GITHUB_EVENT_NAME; const event = process.env.GITHUB_EVENT_NAME;
if (!["pull_request", "push", "schedule", "workflow_dispatch"].includes(event)) let files = [];
throw new Error(`Unsupported CI event: ${event}`); if (
const tier = event === "push" ||
event === "schedule" || event === "schedule" ||
event === "workflow_dispatch" || event === "workflow_dispatch" ||
process.env.CI_FULL === "true" process.env.CI_FULL === "true"
? "full" ) {
: "fast";
let files = [];
if (tier === "full") {
full("Full baseline requested by event or ci:full label"); full("Full baseline requested by event or ci:full label");
} else if (event === "pull_request" || event === "push") { } else if (event === "pull_request") {
const base = git("merge-base", process.env.BASE_SHA, process.env.HEAD_SHA);
const head = process.env.HEAD_SHA; const head = process.env.HEAD_SHA;
let base; // Disable rename detection so both the removed and added paths participate.
if (event === "pull_request") { files = git("diff", "--name-only", "--no-renames", "-z", base, head)
base = git("merge-base", process.env.BASE_SHA, head); .split("\0")
} else { .filter(Boolean);
const candidate = process.env.BASE_SHA; for (const file of files) {
const baselineReason = process.env.BASELINE_REASON; if (
try { /^(README(?:_[^/]+)?\.md|AGENTS\.md|CONTRIBUTING\.md|CHANGELOG\.md|SECURITY\.md|native\/windows\/README\.md|\.github\/pull_request_template\.md)$/.test(
if (!candidate || !/^[0-9a-f]{40}$/i.test(candidate)) file,
throw new Error( ) ||
baselineReason || "No valid successful push baseline was found", file.startsWith("docs/") ||
); file.startsWith(".github/ISSUE_TEMPLATE/")
execFileSync("git", ["merge-base", "--is-ancestor", candidate, head], { ) {
stdio: "ignore", selected.docs = true;
}); selected.website ||= matchesAny(file, SCOPES.website);
base = candidate; continue;
} catch {
full(
baselineReason ||
"No valid successful push baseline ancestor; selecting all lanes",
);
} }
} if (/^(src|tests)\/.*\.(?:test|spec)\.[cm]?[jt]sx?$/.test(file)) {
if (base) {
// Disable rename detection so both the removed and added paths participate.
files = git("diff", "--name-only", "--no-renames", "-z", base, head)
.split("\0")
.filter(Boolean);
for (const file of files) {
if (
/^(README(?:_[^/]+)?\.md|AGENTS\.md|CONTRIBUTING\.md|CHANGELOG\.md|SECURITY\.md|native\/windows\/README\.md|\.github\/pull_request_template\.md)$/.test(
file,
) ||
file.startsWith("docs/") ||
file.startsWith(".github/ISSUE_TEMPLATE/")
) {
selected.docs = true;
selected.website ||= matchesAny(file, SCOPES.website);
continue;
}
if (/^(src|tests)\/.*\.(?:test|spec)\.[cm]?[jt]sx?$/.test(file)) {
selected.code = true;
for (const owner of NATIVE_TEST_OWNERS)
selected[owner] ||= matchesAny(file, SCOPES[owner]);
continue;
}
if (matchesAny(file, MANUAL_VERIFIERS)) {
selected.code = true;
continue;
}
const owners = Object.entries(SCOPES)
.filter(([, patterns]) =>
patterns.some((pattern) => matchesGlob(file, pattern)),
)
.map(([owner]) => owner);
for (const owner of owners) selected[owner] = true;
if (
file.startsWith(".github/workflows/") ||
file === ".github/actionlint.yaml"
) {
selected.workflows = true;
if (GUARDED_WORKFLOWS.has(file)) selected.code = true;
if (file !== ".github/workflows/ci.yml") continue;
}
if (owners.length === 1 && owners[0] === "website") continue;
if (/^src\/(contracts|domain)\//.test(file)) selected.docs = true;
if (file === "package.json") {
classifyPackage(base, head);
continue;
}
selected.code = true; selected.code = true;
if (matchesAny(file, FIXTURE_SOURCES)) { for (const owner of NATIVE_TEST_OWNERS)
selected.fixtures = true; selected[owner] ||= matchesAny(file, SCOPES[owner]);
selected.readiness = true; continue;
} }
if (matchesAny(file, READINESS_SOURCES)) selected.readiness = true; if (matchesAny(file, MANUAL_VERIFIERS)) {
if (file.startsWith("tests/")) continue; selected.code = true;
if ( continue;
/^(src\/process\/|src\/filesystem\/|src\/contracts\/tool|src\/server\/createServer|src\/cli\.ts|scripts\/ci\/)/.test( }
file, const owners = Object.entries(SCOPES)
) .filter(([, patterns]) =>
) { patterns.some((pattern) => matchesGlob(file, pattern)),
full(`Shared boundary: ${file}`); )
} else if (owners.length) { .map(([owner]) => owner);
if (file.startsWith("src/") || file.startsWith("bridge/")) for (const owner of owners) selected[owner] = true;
selected.runtime = true; if (
} else if (/^(src|bridge)\//.test(file)) { file.startsWith(".github/workflows/") ||
// Unowned runtime sources keep the Linux x64 installed-package baseline; file === ".github/actionlint.yaml"
// full qualification retains every native host. ) {
selected.workflows = true;
if (GUARDED_WORKFLOWS.has(file)) selected.code = true;
if (file !== ".github/workflows/ci.yml") continue;
}
if (owners.length === 1 && owners[0] === "website") continue;
if (/^src\/(contracts|domain)\//.test(file)) selected.docs = true;
if (file === "package.json") {
classifyPackage(base, head);
continue;
}
selected.code = true;
if (matchesAny(file, FIXTURE_SOURCES)) {
selected.fixtures = true;
selected.readiness = true;
}
if (matchesAny(file, READINESS_SOURCES)) selected.readiness = true;
if (file.startsWith("tests/")) continue;
if (
/^(src\/process\/|src\/filesystem\/|src\/contracts\/tool|src\/server\/createServer|src\/cli\.ts|scripts\/ci\/)/.test(
file,
)
) {
full(`Shared boundary: ${file}`);
} else if (owners.length) {
if (file.startsWith("src/") || file.startsWith("bridge/"))
selected.runtime = true; selected.runtime = true;
reasons.push(`Unowned runtime source: ${file}`); } else if (/^(src|bridge)\//.test(file)) {
} else if (file.startsWith(".agents/skills/")) { // Unowned runtime sources keep the Linux x64 installed-package baseline;
selected.runtime = true; // ci:full, main pushes and manual dispatch retain every native host.
selected.docs = true; selected.runtime = true;
} else { reasons.push(`Unowned runtime source: ${file}`);
full(`Unclassified input: ${file}`); } else if (file.startsWith(".agents/skills/")) {
} selected.runtime = true;
selected.docs = true;
} else {
full(`Unclassified input: ${file}`);
} }
} }
} } else throw new Error(`Unsupported CI event: ${event}`);
selected.runtime ||= selected.package; selected.runtime ||= selected.package;
const linux = { os: "ubuntu-latest", platform: "linux", architecture: "x64" }; const linux = { os: "ubuntu-latest", platform: "linux", architecture: "x64" };
// macOS x64 is the slowest package host. Fast qualification omits it. // macOS x64 is the slowest package host. Pull requests run it only with ci:full;
// main pushes, schedules and manual dispatch always include it.
const intel = const intel =
tier === "full" event !== "pull_request" || process.env.CI_FULL === "true"
? [{ os: "macos-15-intel", platform: "darwin", architecture: "x64" }] ? [{ os: "macos-15-intel", platform: "darwin", architecture: "x64" }]
: []; : [];
const matrix = { const matrix = {
@@ -237,17 +212,17 @@ const matrix = {
] ]
: [linux], : [linux],
}; };
const plan = { ...selected, tier, matrix, files, reasons }; const plan = { ...selected, matrix, files, reasons };
if (process.env.GITHUB_OUTPUT) if (process.env.GITHUB_OUTPUT)
appendFileSync( appendFileSync(
process.env.GITHUB_OUTPUT, process.env.GITHUB_OUTPUT,
Object.entries({ ...selected, tier, matrix: JSON.stringify(matrix) }) Object.entries({ ...selected, matrix: JSON.stringify(matrix) })
.map(([key, value]) => `${key}=${value}\n`) .map(([key, value]) => `${key}=${value}\n`)
.join(""), .join(""),
); );
if (process.env.GITHUB_STEP_SUMMARY) if (process.env.GITHUB_STEP_SUMMARY)
appendFileSync( appendFileSync(
process.env.GITHUB_STEP_SUMMARY, process.env.GITHUB_STEP_SUMMARY,
`### Selected CI lanes (${tier})\n\n${flags.filter((flag) => selected[flag]).join(", ") || "Formatting only"}\n\n${reasons.join("\n\n")}\n`, `### Selected CI lanes\n\n${flags.filter((flag) => selected[flag]).join(", ") || "Formatting only"}\n\n${reasons.join("\n\n")}\n`,
); );
console.log(JSON.stringify(plan)); console.log(JSON.stringify(plan));
-21
View File
@@ -1,4 +1,3 @@
import { execFileSync } from "node:child_process";
import { appendFile, mkdir, readFile, rm, writeFile } from "node:fs/promises"; import { appendFile, mkdir, readFile, rm, writeFile } from "node:fs/promises";
import { isAbsolute, join, relative, resolve, sep } from "node:path"; import { isAbsolute, join, relative, resolve, sep } from "node:path";
@@ -43,26 +42,6 @@ const passed =
process.env.MERGE_RESULT === "success" && process.env.MERGE_RESULT === "success" &&
report.success && report.success &&
report.numFailedTests === 0; report.numFailedTests === 0;
// Successful shards and merged reports must account for the complete portable
// inventory. Coverage thresholds alone cannot prove that every file ran.
if (passed && process.env.CI_VERIFY_TEST_INVENTORY === "true") {
const expected = execFileSync(
"git",
["ls-files", "-z", "--", "src", "tests"],
{ encoding: "utf8", maxBuffer: 8 * 1024 * 1024 },
)
.split("\0")
.filter((path) => path.endsWith(".test.ts"));
const actual = files.map(({ name }) => name);
const expectedSet = new Set(expected);
const actualSet = new Set(actual);
const missing = expected.filter((path) => !actualSet.has(path));
const unexpected = actual.filter((path) => !expectedSet.has(path));
if (missing.length || unexpected.length || actualSet.size !== actual.length)
throw new Error(
`Incomplete portable test inventory: ${JSON.stringify({ missing, unexpected, duplicateFiles: actual.length - actualSet.size })}`,
);
}
const summary = [ const summary = [
complete ? "## Merged test diagnostics" : "## Partial test diagnostics", complete ? "## Merged test diagnostics" : "## Partial test diagnostics",
"", "",
-235
View File
@@ -1,235 +0,0 @@
import { createHash } from "node:crypto";
import { createReadStream } from "node:fs";
import { execFile } from "node:child_process";
import { copyFile, readFile, writeFile } from "node:fs/promises";
import { join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
const MANIFEST_NAME = "manifest.json";
const TARBALL_NAME = "package.tgz";
const MANIFEST_KEYS = [
"schemaVersion",
"sourceSha",
"runId",
"runAttempt",
"platform",
"architecture",
"packageVersion",
"nodeVersion",
"sha256",
].sort();
const execFileAsync = promisify(execFile);
/** Normalize workflow identity and host facts into the manifest-owned fields. */
export function createVerifiedPackageMetadata({
sourceSha,
runId,
runAttempt,
platform,
architecture,
packageVersion,
nodeVersion,
}) {
if (!/^[a-f0-9]{40}$/u.test(sourceSha))
throw new Error("Source SHA must be a full commit SHA");
return {
sourceSha,
runId: parsePositiveInteger(runId, "GITHUB_RUN_ID"),
runAttempt: parsePositiveInteger(runAttempt, "GITHUB_RUN_ATTEMPT"),
platform,
architecture,
packageVersion,
nodeVersion,
};
}
/** Build the single supported manifest representation for a verified tarball. */
function createVerifiedPackageManifest(metadata) {
const manifest = { schemaVersion: 1, ...metadata };
validateManifestShape(manifest);
return manifest;
}
/** Write a manifest after the caller has completed qualification and cleanup. */
export async function writeVerifiedPackageManifest(directory, metadata) {
const manifest = createVerifiedPackageManifest(metadata);
await writeFile(
join(directory, MANIFEST_NAME),
`${JSON.stringify(manifest, null, 2)}\n`,
{ encoding: "utf8" },
);
return manifest;
}
/** Copy the qualified archive under its fixed name and prepare its manifest. */
export async function retainVerifiedPackage({
sourceTarball,
directory,
metadata,
}) {
const tarball = join(directory, TARBALL_NAME);
await copyFile(sourceTarball, tarball);
const sha256 = await sha256File(tarball);
return {
directory,
metadata: createVerifiedPackageManifest({ ...metadata, sha256 }),
};
}
/** Validate a retained package against the current checkout and workflow attempt. */
async function validateVerifiedPackage({
directory,
sourceSha,
runId,
runAttempt,
packageVersion,
nodeVersion,
}) {
if (!/^[a-f0-9]{40}$/u.test(sourceSha))
throw new Error("Expected source SHA must be a full commit SHA");
if (!Number.isSafeInteger(runId) || runId <= 0)
throw new Error("Expected workflow run ID must be a positive integer");
if (!Number.isSafeInteger(runAttempt) || runAttempt <= 0)
throw new Error("Expected workflow run attempt must be a positive integer");
const manifestPath = join(directory, MANIFEST_NAME);
let manifest;
try {
manifest = JSON.parse(await readFile(manifestPath, "utf8"));
} catch (cause) {
throw new Error(`Could not read verified package manifest: ${cause}`, {
cause,
});
}
validateManifestShape(manifest);
assertEqual(manifest.sourceSha, sourceSha, "source SHA");
assertEqual(manifest.runId, runId, "workflow run ID");
// A successful producer may belong to an earlier attempt when only failed
// publisher jobs are rerun. The workflow still requires same-run qualification.
if (manifest.runAttempt > runAttempt)
throw new Error("Verified package workflow run attempt is from the future");
assertEqual(manifest.platform, "linux", "verifier platform");
assertEqual(manifest.architecture, "x64", "verifier architecture");
assertEqual(manifest.packageVersion, packageVersion, "package version");
assertEqual(manifest.nodeVersion, nodeVersion, "Node.js version");
// The artifact format deliberately fixes the name; no manifest value can
// redirect validation to another path.
const tarball = resolve(directory, TARBALL_NAME);
const digest = await sha256File(tarball);
assertEqual(manifest.sha256, digest, "tarball SHA-256");
return tarball;
}
function validateManifestShape(manifest) {
if (
typeof manifest !== "object" ||
manifest === null ||
Array.isArray(manifest)
)
throw new Error("Verified package manifest must be a JSON object");
const keys = Object.keys(manifest).sort();
if (
keys.length !== MANIFEST_KEYS.length ||
keys.some((key, index) => key !== MANIFEST_KEYS[index])
)
throw new Error("Verified package manifest has missing or unknown fields");
if (manifest.schemaVersion !== 1)
throw new Error("Unsupported verified package manifest schema version");
if (
typeof manifest.sourceSha !== "string" ||
!/^[a-f0-9]{40}$/u.test(manifest.sourceSha)
)
throw new Error("Verified package manifest has an invalid source SHA");
if (!Number.isSafeInteger(manifest.runId) || manifest.runId <= 0)
throw new Error("Verified package manifest has an invalid workflow run ID");
if (!Number.isSafeInteger(manifest.runAttempt) || manifest.runAttempt <= 0)
throw new Error(
"Verified package manifest has an invalid workflow run attempt",
);
for (const key of [
"platform",
"architecture",
"packageVersion",
"nodeVersion",
])
if (typeof manifest[key] !== "string" || manifest[key].length === 0)
throw new Error(`Verified package manifest has an invalid ${key}`);
if (
typeof manifest.sha256 !== "string" ||
!/^[a-f0-9]{64}$/u.test(manifest.sha256)
)
throw new Error("Verified package manifest has an invalid tarball SHA-256");
}
function assertEqual(actual, expected, label) {
if (actual !== expected)
throw new Error(`Verified package ${label} mismatch`);
}
async function sha256File(path) {
const hash = createHash("sha256");
for await (const chunk of createReadStream(path)) hash.update(chunk);
return hash.digest("hex");
}
async function validateFromCli(directory) {
const sourceSha = process.env.GITHUB_SHA;
const runId = parsePositiveInteger(
process.env.GITHUB_RUN_ID,
"GITHUB_RUN_ID",
);
const runAttempt = parsePositiveInteger(
process.env.GITHUB_RUN_ATTEMPT,
"GITHUB_RUN_ATTEMPT",
);
if (sourceSha === undefined || !/^[a-f0-9]{40}$/u.test(sourceSha))
throw new Error("GITHUB_SHA must be a full commit SHA");
const checkoutSha = (
await execFileAsync("git", ["rev-parse", "HEAD"])
).stdout.trim();
assertEqual(checkoutSha, sourceSha, "checked out source SHA");
const packageJson = JSON.parse(await readFile("package.json", "utf8"));
const tarball = await validateVerifiedPackage({
directory,
sourceSha,
runId,
runAttempt,
packageVersion: packageJson.version,
nodeVersion: process.version,
});
if (process.env.GITHUB_OUTPUT === undefined)
throw new Error(
"GITHUB_OUTPUT is required to publish the validated tarball path",
);
await writeFile(process.env.GITHUB_OUTPUT, `tarball=${tarball}\n`, {
encoding: "utf8",
flag: "a",
});
process.stdout.write(`${tarball}\n`);
}
function parsePositiveInteger(value, name) {
if (value === undefined || !/^[1-9]\d*$/u.test(value))
throw new Error(`${name} must be a positive integer`);
const parsed = Number(value);
if (!Number.isSafeInteger(parsed))
throw new Error(`${name} must be a safe positive integer`);
return parsed;
}
const invokedPath = process.argv[1];
if (
invokedPath !== undefined &&
resolve(invokedPath) === fileURLToPath(import.meta.url)
) {
const [command, directory, ...extra] = process.argv.slice(2);
if (command !== "validate" || directory === undefined || extra.length > 0) {
process.stderr.write(
"Usage: node scripts/ci/verified-package.mjs validate <artifact-directory>\n",
);
process.exitCode = 2;
} else {
await validateFromCli(directory);
}
}
@@ -45,20 +45,6 @@ class Generator:
assert "ows-secret" not in json.dumps(projected), projected assert "ows-secret" not in json.dumps(projected), projected
if isinstance(value, bytes): if isinstance(value, bytes):
assert projected["binary_fields"][-1]["state"] == "redacted", projected assert projected["binary_fields"][-1]["state"] == "redacted", projected
body = b"A" * 12345
state = {"response": {"content": body}}
control = adapter.project_record(state, [])
field = control["binary_fields"][0]
assert field["state"] == "retained" and field["bytes"] == len(body), control
for literal in ["12345", "2345"]:
# Only the generated byte count matches this exclusion.
assert literal not in control["reported"]["response"]["content"]
assert literal not in field["content_base64"] and literal not in field["sha256"]
projected = adapter.project_record(state, [literal])
assert projected["reported"]["response"]["content"] is None, projected
assert projected["binary_fields"] == [{"pointer": "/response/content", "representation": "producer-bytes", "state": "redacted", "content_base64": None, "bytes": None, "sha256": None}], projected
assert {"pointer": "/response/content", "reason": "explicit-sensitive-value"} in projected["redactions"], projected
assert literal not in json.dumps(projected), projected
(root / "native-error-classifications.json").write_text(json.dumps({"passed": len(classifications) + 2})) (root / "native-error-classifications.json").write_text(json.dumps({"passed": len(classifications) + 2}))
first = http.HTTPFlow(connection.Client(peername=("127.0.0.1", 1), sockname=("127.0.0.1", 2)), connection.Server(address=("example.test", 80))) first = http.HTTPFlow(connection.Client(peername=("127.0.0.1", 1), sockname=("127.0.0.1", 2)), connection.Server(address=("example.test", 80)))
first.id = "original-producer-id" first.id = "original-producer-id"
+23 -17
View File
@@ -1,8 +1,5 @@
import { execFile } from "node:child_process"; import { readFile, readdir, stat } from "node:fs/promises";
import { promisify } from "node:util"; import { basename } from "node:path";
import { readdir, stat } from "node:fs/promises";
const execFileAsync = promisify(execFile);
/** /**
* Observe leftover demo processes on a dedicated clean Linux runner, including * Observe leftover demo processes on a dedicated clean Linux runner, including
@@ -17,9 +14,6 @@ export const assertCleanLinuxHopperRuntime = async (
: [process.env.HOPPER_LAUNCHER_PATH]), : [process.env.HOPPER_LAUNCHER_PATH]),
], ],
) => { ) => {
const { stdout } = await execFileAsync("ps", ["-ax", "-o", "command="]);
if (/\/opt\/hopper\/bin\/Hopper|\bXvfb\b|hopper-demo-x11\.py/u.test(stdout))
throw new Error("Clean Linux runner retained a Hopper demo process");
const launchers = []; const launchers = [];
for (const path of launcherPaths) { for (const path of launcherPaths) {
try { try {
@@ -28,18 +22,34 @@ export const assertCleanLinuxHopperRuntime = async (
if (error?.code !== "ENOENT") throw error; if (error?.code !== "ENOENT") throw error;
} }
} }
if (launchers.length === 0) return;
// /proc exposes the executed file even when argv[0] names a removed symlink. // /proc exposes the executed file even when argv[0] names a removed symlink.
// Compare file identity, not a display path or a process name. // Compare file identity, not a display path or a process name.
let inspectionError; let inspectionError;
for (const entry of await readdir("/proc", { withFileTypes: true })) { for (const entry of await readdir("/proc", { withFileTypes: true })) {
if (!entry.isDirectory() || !/^\d+$/u.test(entry.name)) continue; if (!entry.isDirectory() || !/^\d+$/u.test(entry.name)) continue;
let executable; let retainedRuntime;
try { try {
executable = await stat(`/proc/${entry.name}/exe`, { bigint: true }); // procfs preserves argument boundaries. A caller merely mentioning a
// runtime in its script text or other arguments is not that runtime.
const arguments_ = (
await readFile(`/proc/${entry.name}/cmdline`, "utf8")
).split("\0");
retainedRuntime =
arguments_[0] === "/opt/hopper/bin/Hopper" ||
basename(arguments_[0] ?? "") === "Xvfb" ||
(basename(arguments_[0] ?? "") === "python3" &&
basename(arguments_[1] ?? "") === "hopper-demo-x11.py");
if (!retainedRuntime && launchers.length > 0) {
const executable = await stat(`/proc/${entry.name}/exe`, {
bigint: true,
});
retainedRuntime = launchers.some(
({ dev, ino }) => dev === executable.dev && ino === executable.ino,
);
}
} catch (error) { } catch (error) {
if (error?.code === "ENOENT" || error?.code === "ESRCH") continue; if (error?.code === "ENOENT" || error?.code === "ESRCH") continue;
// Other users' executables may be unreadable. The setup lane launches // Other users' process facts may be unreadable. The setup lane launches
// providers as the caller; unreadable caller-owned processes fail closed. // providers as the caller; unreadable caller-owned processes fail closed.
if (error?.code === "EACCES") { if (error?.code === "EACCES") {
const owner = await stat(`/proc/${entry.name}`).catch((cause) => { const owner = await stat(`/proc/${entry.name}`).catch((cause) => {
@@ -53,11 +63,7 @@ export const assertCleanLinuxHopperRuntime = async (
inspectionError ??= error; inspectionError ??= error;
continue; continue;
} }
if ( if (retainedRuntime)
launchers.some(
({ dev, ino }) => dev === executable.dev && ino === executable.ino,
)
)
throw new Error("Clean Linux runner retained a Hopper demo process"); throw new Error("Clean Linux runner retained a Hopper demo process");
} }
if (inspectionError !== undefined) throw inspectionError; if (inspectionError !== undefined) throw inspectionError;
+4 -3
View File
@@ -1,3 +1,7 @@
import type { z } from "zod";
import type { analysisStringSchema } from "../../src/domain/hopperValues.js";
type VerifierCall = ( type VerifierCall = (
name: string, name: string,
arguments_?: Record<string, unknown>, arguments_?: Record<string, unknown>,
@@ -25,6 +29,3 @@ export function verifyHopperStringObjects(
}[]; }[];
readonly displayEscapesExcludedFromMatching: true; readonly displayEscapesExcludedFromMatching: true;
}>; }>;
import type { z } from "zod";
import type { analysisStringSchema } from "../../src/domain/hopperValues.js";
+1 -45
View File
@@ -1,8 +1,6 @@
import { mkdir, mkdtemp, rm, unlink } from "node:fs/promises"; import { mkdtemp, rm } from "node:fs/promises";
import { execFile } from "node:child_process";
import { tmpdir } from "node:os"; import { tmpdir } from "node:os";
import { join } from "node:path"; import { join } from "node:path";
import { promisify } from "node:util";
import { TOOL_CONTRACTS } from "../dist/contracts/toolContracts.js"; import { TOOL_CONTRACTS } from "../dist/contracts/toolContracts.js";
import { PRODUCT_IDENTITY } from "../dist/identity.js"; import { PRODUCT_IDENTITY } from "../dist/identity.js";
@@ -23,19 +21,12 @@ import { verifyRepositorySkill } from "./verify-repository-skill.mjs";
import { verifyManaged } from "./verify-package-managed.mjs"; import { verifyManaged } from "./verify-package-managed.mjs";
import { verifyUnknownProvider } from "./verify-package-unknown-provider.mjs"; import { verifyUnknownProvider } from "./verify-package-unknown-provider.mjs";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs"; import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
import {
createVerifiedPackageMetadata,
retainVerifiedPackage,
writeVerifiedPackageManifest,
} from "./ci/verified-package.mjs";
import { import {
measureMcpStartup, measureMcpStartup,
profileMcpModuleLoading, profileMcpModuleLoading,
} from "./lib/mcp-startup-probe.mjs"; } from "./lib/mcp-startup-probe.mjs";
const root = process.cwd(); const root = process.cwd();
const execFileAsync = promisify(execFile);
const retentionDirectory = process.env.REA_VERIFIED_PACKAGE_DIRECTORY;
const verifierRun = createVerifierRun(); const verifierRun = createVerifierRun();
const workspace = await mkdtemp(join(tmpdir(), "rea-package-")); const workspace = await mkdtemp(join(tmpdir(), "rea-package-"));
const evidenceRoot = join(workspace, "evidence"); const evidenceRoot = join(workspace, "evidence");
@@ -46,8 +37,6 @@ let mcpModuleLoading;
let update; let update;
let repositorySkill; let repositorySkill;
let report; let report;
let retainedPackage;
let retentionMetadata;
const runStage = async (name, action) => { const runStage = async (name, action) => {
const started = performance.now(); const started = performance.now();
@@ -60,27 +49,6 @@ const runStage = async (name, action) => {
}; };
try { try {
if (retentionDirectory !== undefined && retentionDirectory !== "") {
const sourceSha = (
await execFileAsync("git", ["rev-parse", "HEAD"], { cwd: root })
).stdout.trim();
retentionMetadata = createVerifiedPackageMetadata({
sourceSha,
runId: process.env.GITHUB_RUN_ID,
runAttempt: process.env.GITHUB_RUN_ATTEMPT,
platform: process.platform,
architecture: process.arch,
packageVersion: PRODUCT_IDENTITY.packageVersion,
nodeVersion: process.version,
});
await mkdir(retentionDirectory, { recursive: true });
await unlink(join(retentionDirectory, "manifest.json")).catch((cause) => {
if (
!(cause instanceof Error && "code" in cause && cause.code === "ENOENT")
)
throw cause;
});
}
repositorySkill = await runStage("repository skill", () => repositorySkill = await runStage("repository skill", () =>
verifyRepositorySkill({ root, workspace }), verifyRepositorySkill({ root, workspace }),
); );
@@ -247,13 +215,6 @@ try {
mcpStartup, mcpStartup,
mcpModuleLoading, mcpModuleLoading,
}; };
if (retentionMetadata !== undefined) {
retainedPackage = await retainVerifiedPackage({
sourceTarball: tarball,
directory: retentionDirectory,
metadata: retentionMetadata,
});
}
} finally { } finally {
await runStage("workspace cleanup", () => await runStage("workspace cleanup", () =>
rm(workspace, { recursive: true, force: true }), rm(workspace, { recursive: true, force: true }),
@@ -262,11 +223,6 @@ try {
const verifierLineage = await runStage("verifier lineage", () => const verifierLineage = await runStage("verifier lineage", () =>
completeVerifierRun(verifierRun), completeVerifierRun(verifierRun),
); );
if (retainedPackage !== undefined)
await writeVerifiedPackageManifest(
retainedPackage.directory,
retainedPackage.metadata,
);
process.stdout.write( process.stdout.write(
`${JSON.stringify({ verifier_run: verifierLineage, ...report })}\n`, `${JSON.stringify({ verifier_run: verifierLineage, ...report })}\n`,
); );
+3 -150
View File
@@ -1,9 +1,8 @@
#!/usr/bin/env node #!/usr/bin/env node
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { estimateTokenCount } from "tokenx";
import { execFile } from "node:child_process"; import { execFile } from "node:child_process";
import { createHash } from "node:crypto"; import { createHash } from "node:crypto";
import { readFile, writeFile, readdir, mkdir, rm } from "node:fs/promises"; import { readFile, writeFile, readdir } from "node:fs/promises";
import { isAbsolute, join } from "node:path"; import { isAbsolute, join } from "node:path";
import { fileURLToPath } from "node:url"; import { fileURLToPath } from "node:url";
import { promisify } from "node:util"; import { promisify } from "node:util";
@@ -92,20 +91,6 @@ const source = pcap(1, [
// LINKTYPE_BLUETOOTH_HCI_H4 = 187; H4 command packet, HCI Reset opcode 0x0c03, no parameters. // LINKTYPE_BLUETOOTH_HCI_H4 = 187; H4 command packet, HCI Reset opcode 0x0c03, no parameters.
const bluetoothSource = pcap(187, [Buffer.from("01030c00", "hex")]); const bluetoothSource = pcap(187, [Buffer.from("01030c00", "hex")]);
let cases = 0; let cases = 0;
const discoveryMeasurements = [];
const inheritedAddressSpace = (await readFile("/proc/self/limits", "utf8"))
.split("\n")
.find((line) => line.startsWith("Max address space "))
?.trim()
.match(/^Max address space\s+(\S+)\s+(\S+)\s+bytes$/u);
assert.ok(inheritedAddressSpace);
const expectedAddressSpace = Math.min(
1073741824,
...inheritedAddressSpace
.slice(1)
.map((value) => (value === "unlimited" ? Infinity : Number(value))),
);
try { try {
await writeFile(path, source); await writeFile(path, source);
await writeFile(bluetooth, bluetoothSource); await writeFile(bluetooth, bluetoothSource);
@@ -136,16 +121,14 @@ try {
.digest("hex"), .digest("hex"),
); );
assert.equal(report.hardware_acquisition, "not-performed"); assert.equal(report.hardware_acquisition, "not-performed");
assert.equal(report.limits.address_space_bytes, expectedAddressSpace); assert.equal(report.limits.address_space_bytes, 1073741824);
assert.equal(report.tool.configured_path, process.env.REA_TSHARK_COMMAND); assert.equal(report.tool.configured_path, process.env.REA_TSHARK_COMMAND);
assert.equal( assert.equal(
report.resource_limiter.configured_path, report.resource_limiter.configured_path,
process.env.REA_TSHARK_PRLIMIT_COMMAND ?? "/usr/bin/prlimit", process.env.REA_TSHARK_PRLIMIT_COMMAND ?? "/usr/bin/prlimit",
); );
assert.ok( assert.ok(
report.resource_limiter.arguments.includes( report.resource_limiter.arguments.includes("--as=1073741824:1073741824"),
`--as=${expectedAddressSpace}:${expectedAddressSpace}`,
),
); );
assert.equal(report.configuration, "private-empty-personal-profile"); assert.equal(report.configuration, "private-empty-personal-profile");
const flags = Object.entries(input) const flags = Object.entries(input)
@@ -330,133 +313,6 @@ try {
true, true,
); );
cases++; cases++;
const discovery = (await client.listTools()).tools.find(
({ name }) => name === "inspect_packet_fields",
);
assert.ok(discovery?.outputSchema);
assert.ok(ajv.validateSchema(discovery.inputSchema));
assert.ok(ajv.validateSchema(discovery.outputSchema));
const nativeProfile = join(root.path, "native-profile");
await mkdir(nativeProfile);
const native = (
await execute(process.env.REA_TSHARK_COMMAND, ["-G", "fields"], {
env: { ...environment, WIRESHARK_CONFIG_DIR: nativeProfile },
timeout: 30000,
maxBuffer: 32 * 1024 * 1024,
})
).stdout;
await rm(nativeProfile, { recursive: true });
const nativeLines = native.split(/\r?\n/);
if (nativeLines.at(-1) === "") nativeLines.pop();
for (const input of [
{ query: "usb" },
{ limit: 1 },
{ offset: 1, limit: 2 },
{ kind: "fields", protocol: "usb", query: "idVendor" },
{ kind: "fields", protocol: "udp", limit: null },
{ kind: "fields", query: "rea_no_such_field" },
]) {
const started = performance.now();
const response = await client.callTool({
name: discovery.name,
arguments: input,
});
assert.notEqual(response.isError, true, JSON.stringify(response));
assert.ok(
ajv.validate(discovery.outputSchema, response.structuredContent),
JSON.stringify(ajv.errors),
);
const evidence = parseEvidence(response.structuredContent);
const report = evidence.normalized_result;
const kind = input.kind ?? "protocols";
const query = (input.query ?? "").toLowerCase();
const expectedMatches = nativeLines.flatMap((line, index) => {
const columns = line.split("\t");
if ((kind === "protocols") !== (columns[0] === "P")) return [];
if (
input.protocol !== undefined &&
(columns[0] === "P" ? columns[2] : columns[4]) !== input.protocol
)
return [];
if (
![
columns[1],
columns[2],
...(columns[0] === "F" ? [columns[7]] : []),
].some((value) => value.toLowerCase().includes(query))
)
return [];
return [{ sourceLine: index + 1, columns }];
});
const offset = input.offset ?? 0;
const limit = input.limit === null ? undefined : (input.limit ?? 50);
const expectedPage = expectedMatches.slice(
offset,
limit === undefined ? undefined : offset + limit,
);
assert.deepEqual(
report.records.map((record) => record.source_line),
expectedPage.map((record) => record.sourceLine),
);
assert.deepEqual(report.output_window, {
examined: nativeLines.filter((line) =>
line.startsWith(kind === "protocols" ? "P\t" : "F\t"),
).length,
matching: expectedMatches.length,
returned: expectedPage.length,
has_more:
Math.max(0, expectedMatches.length - offset) > expectedPage.length,
});
discoveryMeasurements.push({
input,
elapsed_ms: +(performance.now() - started).toFixed(2),
response_bytes: Buffer.byteLength(JSON.stringify(response)),
estimated_tokens_one_tool_request_response: estimateTokenCount(
JSON.stringify(discovery) +
JSON.stringify(input) +
JSON.stringify(response),
),
});
for (const record of report.records) {
const columns = nativeLines[record.source_line - 1].split("\t");
assert.equal(record.name, columns[1]);
assert.equal(record.abbreviation, columns[2]);
assert.equal(record.field_type, columns[0] === "F" ? columns[3] : null);
assert.equal(
record.parent_protocol,
columns[0] === "F" ? columns[4] : null,
);
assert.equal(record.description, columns[0] === "F" ? columns[7] : null);
assert.equal(record.display_base, columns[0] === "F" ? columns[5] : null);
assert.equal(record.bitmask, columns[0] === "F" ? columns[6] : null);
}
if (input.query === "idVendor")
assert.ok(
report.records.some(
(row) =>
row.abbreviation === "usb.idVendor" &&
row.field_type === "FT_UINT16",
),
);
if (input.query === "usb")
assert.ok(report.records.some((row) => row.abbreviation === "usb"));
if (input.query === "rea_no_such_field")
assert.equal(report.output_window.matching, 0);
const flags = Object.entries(input).flatMap(([key, value]) =>
value === null ? ["--all"] : [`--${key}`, String(value)],
);
const cli = JSON.parse(
(
await execute(
process.execPath,
[entrypoint, "inspect-packet-fields", ...flags, "--format", "json"],
{ env: environment, timeout: 60000, maxBuffer: 16 * 1024 * 1024 },
)
).stdout,
);
assert.deepEqual(cli.normalized_result, report);
cases++;
}
await client.close(); await client.close();
assert.deepEqual((await readdir(root.path)).sort(), [ assert.deepEqual((await readdir(root.path)).sort(), [
"bluetooth.pcap", "bluetooth.pcap",
@@ -473,9 +329,6 @@ console.log(
{ {
status: "passed", status: "passed",
cases, cases,
discovery_measurements: discoveryMeasurements,
measurement_scope:
"Warm MCP server, fresh TShark processes; tokenx estimates include one advertised tool/request/response, not billed usage. Model-driven selection, CPU and peak process-family memory are unmeasured; no cache/speedup claim.",
host: `${process.platform}/${process.arch}`, host: `${process.platform}/${process.arch}`,
scope: scope:
"real TShark CLI/MCP; synthetic Ethernet/IPv4/UDP/DNS and Bluetooth HCI PCAP; no hardware acquisition", "real TShark CLI/MCP; synthetic Ethernet/IPv4/UDP/DNS and Bluetooth HCI PCAP; no hardware acquisition",
+2 -104
View File
@@ -1,6 +1,5 @@
#!/usr/bin/env node #!/usr/bin/env node
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { estimateTokenCount } from "tokenx";
import { execFile } from "node:child_process"; import { execFile } from "node:child_process";
import { createHash } from "node:crypto"; import { createHash } from "node:crypto";
import { readFile, writeFile, readdir } from "node:fs/promises"; import { readFile, writeFile, readdir } from "node:fs/promises";
@@ -69,20 +68,6 @@ for (const byte of Buffer.from("REA")) {
lines.push(`#${tick + 1000}`, "1!"); lines.push(`#${tick + 1000}`, "1!");
const source = lines.join("\n") + "\n"; const source = lines.join("\n") + "\n";
let cases = 0; let cases = 0;
const discoveryMeasurements = [];
const inheritedAddressSpace = (await readFile("/proc/self/limits", "utf8"))
.split("\n")
.find((line) => line.startsWith("Max address space "))
?.trim()
.match(/^Max address space\s+(\S+)\s+(\S+)\s+bytes$/u);
assert.ok(inheritedAddressSpace);
const expectedAddressSpace = Math.min(
1073741824,
...inheritedAddressSpace
.slice(1)
.map((value) => (value === "unlimited" ? Infinity : Number(value))),
);
try { try {
await writeFile(path, source); await writeFile(path, source);
await client.connect(transport); await client.connect(transport);
@@ -119,16 +104,14 @@ try {
createHash("sha256").update(source).digest("hex"), createHash("sha256").update(source).digest("hex"),
); );
assert.equal(report.hardware_acquisition, "not-performed"); assert.equal(report.hardware_acquisition, "not-performed");
assert.equal(report.limits.address_space_bytes, expectedAddressSpace); assert.equal(report.limits.address_space_bytes, 1073741824);
assert.equal(report.tool.configured_path, process.env.REA_SIGROK_COMMAND); assert.equal(report.tool.configured_path, process.env.REA_SIGROK_COMMAND);
assert.equal( assert.equal(
report.resource_limiter.configured_path, report.resource_limiter.configured_path,
process.env.REA_SIGROK_PRLIMIT_COMMAND ?? "/usr/bin/prlimit", process.env.REA_SIGROK_PRLIMIT_COMMAND ?? "/usr/bin/prlimit",
); );
assert.ok( assert.ok(
report.resource_limiter.arguments.includes( report.resource_limiter.arguments.includes("--as=1073741824:1073741824"),
`--as=${expectedAddressSpace}:${expectedAddressSpace}`,
),
); );
assert.equal(report.diagnostics, ""); assert.equal(report.diagnostics, "");
const flags = Object.entries(input) const flags = Object.entries(input)
@@ -262,88 +245,6 @@ try {
}); });
assert.equal(invalid.isError, true); assert.equal(invalid.isError, true);
cases++; cases++;
const discoveryTool = (await client.listTools()).tools.find(
({ name }) => name === "inspect_signal_decoders",
);
assert.ok(discoveryTool?.outputSchema);
assert.ok(ajv.validateSchema(discoveryTool.inputSchema));
assert.ok(ajv.validateSchema(discoveryTool.outputSchema));
const nativeHelp = (
await execute(
process.env.REA_SIGROK_COMMAND,
["--dont-scan", "--protocol-decoders=uart", "--show"],
{ env: environment, timeout: 30000, maxBuffer: 4 * 1024 * 1024 },
)
).stdout.split(/\r?\n/);
if (nativeHelp.at(-1) === "") nativeHelp.pop();
for (const input of [
{ query: "uart" },
{ limit: 1 },
{ offset: 1, limit: 2 },
{ decoder: "uart", query: "BaUd" },
{ decoder: "uart", offset: 50, limit: null },
{ query: "rea_no_such_decoder" },
]) {
const started = performance.now();
const response = await client.callTool({
name: discoveryTool.name,
arguments: input,
});
assert.notEqual(response.isError, true, JSON.stringify(response));
assert.ok(
ajv.validate(discoveryTool.outputSchema, response.structuredContent),
JSON.stringify(ajv.errors),
);
const evidence = parseEvidence(response.structuredContent);
const report = evidence.normalized_result;
discoveryMeasurements.push({
input,
elapsed_ms: +(performance.now() - started).toFixed(2),
response_bytes: Buffer.byteLength(JSON.stringify(response)),
estimated_tokens_one_tool_request_response: estimateTokenCount(
JSON.stringify(discoveryTool) +
JSON.stringify(input) +
JSON.stringify(response),
),
});
if (input.decoder) {
const matched = nativeHelp
.map((text, i) => ({ line_number: i + 1, text }))
.filter(({ text }) =>
text.toLowerCase().includes((input.query ?? "").toLowerCase()),
);
assert.deepEqual(
report.help_lines,
matched.slice(
input.offset ?? 0,
input.limit === null
? undefined
: (input.offset ?? 0) + (input.limit ?? 50),
),
);
} else if (input.query === "uart")
assert.ok(report.decoders.some(({ id }) => id === "uart"));
const flags = Object.entries(input).flatMap(([key, value]) =>
value === null ? ["--all"] : [`--${key}`, String(value)],
);
const cli = JSON.parse(
(
await execute(
process.execPath,
[entrypoint, "inspect-signal-decoders", ...flags, "--format", "json"],
{ env: environment, timeout: 60000, maxBuffer: 8 * 1024 * 1024 },
)
).stdout,
);
assert.deepEqual(cli.normalized_result, report);
cases++;
}
const unavailable = await client.callTool({
name: discoveryTool.name,
arguments: { decoder: "rea_no_such_decoder" },
});
assert.equal(unavailable.isError, true);
cases++;
await client.close(); await client.close();
assert.deepEqual((await readdir(root.path)).sort(), [ assert.deepEqual((await readdir(root.path)).sort(), [
"paged.vcd", "paged.vcd",
@@ -359,9 +260,6 @@ console.log(
{ {
status: "passed", status: "passed",
cases, cases,
discovery_measurements: discoveryMeasurements,
measurement_scope:
"Warm MCP server, fresh sigrok processes; tokenx estimates include one advertised tool/request/response, not billed model usage. No model-driven selection benchmark, CPU or peak process-family memory measurement; no cache or speedup claim.",
host: `${process.platform}/${process.arch}`, host: `${process.platform}/${process.arch}`,
scope: scope:
"real sigrok UART VCD through compiled CLI/MCP; no physical acquisition", "real sigrok UART VCD through compiled CLI/MCP; no physical acquisition",
-36
View File
@@ -662,42 +662,6 @@ setInterval(()=>{},1000);\n`,
unicodeNamesPreserved: true, unicodeNamesPreserved: true,
}); });
report.processes.windowsEnvironmentSemantics = true; report.processes.windowsEnvironmentSemantics = true;
// Observed through this process's own id: a separate probe process would
// get its own processor group, so it cannot stand in for the process that
// spawns the child.
const callerAffinity = JSON.parse(
execFileSync(boundaryFixture, ["affinity", String(process.pid)], {
encoding: "utf8",
}),
);
const affinityChild = native.call("process_spawn", [
boundaryFixture,
`${quote(boundaryFixture)} affinity`,
workspace,
environment,
]);
const observedAffinity = await wait(affinityChild);
native.call("process_close", [affinityChild.handle]);
assert.equal(observedAffinity.exitCode, 0);
const childAffinity = JSON.parse(observedAffinity.output);
// Masks cross as hexadecimal strings because a mask can exceed the
// safe-integer range. Sharing one processor group makes the caller's mask
// applicable, so it must be inherited exactly. Windows placing the child in
// another group is only permitted when the caller asked for no bound of its
// own, which its group's whole active set establishes; a narrower mask has
// to fail the spawn rather than widen to that group's default.
const sharedGroup =
callerAffinity.singleProcessorGroup &&
childAffinity.singleProcessorGroup &&
callerAffinity.processorGroup === childAffinity.processorGroup;
if (sharedGroup)
assert.equal(childAffinity.affinityMask, callerAffinity.affinityMask);
else assert.equal(callerAffinity.affinityMask, callerAffinity.systemMask);
report.processes.callerAffinityBoundsOwnedProcessOrRefusesToWiden = true;
report.callerAffinity = {
...callerAffinity,
sharedGroupWithChild: sharedGroup,
};
} }
const ownerFixture = join(workspace, "owner.mjs"); const ownerFixture = join(workspace, "owner.mjs");
const ownerPidFile = join(workspace, "crash-child.pid"); const ownerPidFile = join(workspace, "crash-child.pid");
+2 -20
View File
@@ -84,24 +84,6 @@ describe("parseAdbDevicesOutput", () => {
]); ]);
}); });
it("accepts complete multiword states and rejects state prefixes", () => {
const output = [
"serial-no-permission no permissions usb:1-1",
"serial-bad-device deviceXYZ ",
"serial-bad-offline offline-ish",
"serial-bad-auth unauthorizedXYZ",
].join("\n");
const { devices, unparsedLines } = parseAdbDevicesOutput(output);
expect(devices.map(({ serial, state }) => [serial, state])).toEqual([
["serial-no-permission", "no permissions"],
]);
expect(unparsedLines).toEqual([
"serial-bad-device deviceXYZ ",
"serial-bad-offline offline-ish",
"serial-bad-auth unauthorizedXYZ",
]);
});
it("returns unrecognized lines instead of dropping them", () => { it("returns unrecognized lines instead of dropping them", () => {
const output = [ const output = [
"List of devices attached", "List of devices attached",
@@ -212,10 +194,10 @@ describe("parsePackageListOutput", () => {
it("keeps unrecognized lines for partial coverage reporting", () => { it("keeps unrecognized lines for partial coverage reporting", () => {
const { packages, unparsedLines } = parsePackageListOutput( const { packages, unparsedLines } = parsePackageListOutput(
"package:/data/app/x/base.apk=com.ok\n garbage line \n", "package:/data/app/x/base.apk=com.ok\ngarbage line\n",
); );
expect(packages).toHaveLength(1); expect(packages).toHaveLength(1);
expect(unparsedLines).toEqual([" garbage line "]); expect(unparsedLines).toEqual(["garbage line"]);
}); });
}); });
+25 -14
View File
@@ -10,15 +10,11 @@ import {
adbHostFileName, adbHostFileName,
type AdbHostFileNameReason, type AdbHostFileNameReason,
} from "./AdbHostFileName.js"; } from "./AdbHostFileName.js";
import {
ADB_DEVICE_STATES,
type AdbDeviceState,
} from "../domain/adb/adbDeviceAnalysis.js";
/** One device exactly as `adb devices -l` reports it. */ /** One device exactly as `adb devices -l` reports it. */
export interface AdbDeviceEntry { export interface AdbDeviceEntry {
readonly serial: string; readonly serial: string;
readonly state: AdbDeviceState; readonly state: string;
readonly transport: "usb" | "tcp" | "unknown"; readonly transport: "usb" | "tcp" | "unknown";
readonly product: string | null; readonly product: string | null;
readonly model: string | null; readonly model: string | null;
@@ -28,6 +24,25 @@ export interface AdbDeviceEntry {
readonly kind_basis: "emulator_serial_prefix" | "usb_transport" | "none"; readonly kind_basis: "emulator_serial_prefix" | "usb_transport" | "none";
} }
/**
* Device states adb prints after the serial, longest first so two-word
* states match before their single-word prefixes. A line whose remainder
* starts with none of them is not a device line.
*/
const DEVICE_STATES = [
"no permissions",
"unauthorized",
"authorizing",
"offline",
"recovery",
"sideload",
"connecting",
"bootloader",
"device",
"unknown",
"host",
] as const;
const DEVICE_LINE = /^(\S+)\s+(.*)$/u; const DEVICE_LINE = /^(\S+)\s+(.*)$/u;
const metadataValue = (remainder: string, key: string): string | null => { const metadataValue = (remainder: string, key: string): string | null => {
@@ -58,15 +73,11 @@ export const parseAdbDevicesOutput = (
continue; continue;
} }
const match = DEVICE_LINE.exec(line); const match = DEVICE_LINE.exec(line);
const state = ADB_DEVICE_STATES.find((candidate) => { const state = DEVICE_STATES.find((candidate) =>
const remainder = match?.[2]; match?.[2]?.startsWith(candidate),
if (remainder === undefined || !remainder.startsWith(candidate)) );
return false;
const next = remainder[candidate.length];
return next === undefined || /\s/u.test(next);
});
if (match === null || state === undefined) { if (match === null || state === undefined) {
if (rawLine.trim() !== "") unparsedLines.push(rawLine); if (line.trim() !== "") unparsedLines.push(line);
continue; continue;
} }
const serial = match[1]!; const serial = match[1]!;
@@ -181,7 +192,7 @@ export const parsePackageListOutput = (
if (line === "") continue; if (line === "") continue;
const match = PACKAGE_LINE.exec(line); const match = PACKAGE_LINE.exec(line);
if (match === null) { if (match === null) {
unparsedLines.push(rawLine); unparsedLines.push(line);
continue; continue;
} }
packages.push({ packages.push({
-74
View File
@@ -121,52 +121,6 @@ fi`,
]); ]);
}); });
it("refuses device listings with an invalid state token", async () => {
const malformedLine = "serial deviceXYZ ";
const { binary } = await writeAdbStub(
() => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; exit 0; fi
if [ "$1" = "devices" ]; then
printf 'List of devices attached\\n${malformedLine}\\n'
exit 0
fi`,
);
const result = await providerOver(binary).execute({
operation: "list_adb_devices",
input: {},
});
expect(result.ok).toBe(false);
if (result.ok) return;
expect(result.error._tag).toBe("AnalysisProtocolError");
expect(result.error.message).toContain(malformedLine);
});
it("lists rescue and detached transports alongside online devices", async () => {
const { binary } = await writeAdbStub(
() => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; exit 0; fi
if [ "$1" = "devices" ]; then
printf 'List of devices attached\\n'
printf 'serial-online device transport_id:1\\n'
printf 'serial-rescue rescue transport_id:2\\n'
printf 'serial-detached detached transport_id:3\\n'
exit 0
fi`,
);
const result = await evidenceResult(providerOver(binary), {
operation: "list_adb_devices",
input: {},
});
expect(result.devices).toEqual(
expect.arrayContaining([
expect.objectContaining({ serial: "serial-online", state: "device" }),
expect.objectContaining({ serial: "serial-rescue", state: "rescue" }),
expect.objectContaining({
serial: "serial-detached",
state: "detached",
}),
]),
);
});
it("inspects a device through the getprop whitelist", async () => { it("inspects a device through the getprop whitelist", async () => {
const { binary } = await writeAdbStub( const { binary } = await writeAdbStub(
() => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; exit 0; fi () => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; exit 0; fi
@@ -626,34 +580,6 @@ it("projects package details from dumpsys package output", async () => {
}); });
}); });
// Android 10 and 11 append ": restricted=true" to hard- and soft-restricted
// entries of the requested-permissions list (Settings.dumpPackageLPr).
it("counts requested permissions that the device marks as restricted", async () => {
const { binary } = await writeAdbStub(
shellStub({
"dumpsys:package": `Packages:
Package [com.example] (abcdef):
userId=10234
versionName=1.2.3
requested permissions:
android.permission.INTERNET
android.permission.WRITE_EXTERNAL_STORAGE: restricted=true
android.permission.CAMERA
install permissions:
android.permission.INTERNET: granted=true
`,
}),
);
const result = await evidenceResult(providerOver(binary), {
operation: "inspect_adb_package",
input: { serial: "emulator-5554", package: "com.example" },
});
expect(result).toMatchObject({
requested_permissions_count: 3,
coverage: "complete",
});
});
it("reports an uninstalled package from dumpsys refusal output", async () => { it("reports an uninstalled package from dumpsys refusal output", async () => {
const { binary } = await writeAdbStub( const { binary } = await writeAdbStub(
shellStub({ "dumpsys:package": "Unable to find package: com.absent\n" }), shellStub({ "dumpsys:package": "Unable to find package: com.absent\n" }),
+1 -5
View File
@@ -236,11 +236,7 @@ export const parseDumpsysPackageOutput = (
inPermissions = true; inPermissions = true;
continue; continue;
} }
// Android 10 and 11 mark restricted entries as "NAME: restricted=true". if (inPermissions && /^\s{2,}[A-Za-z0-9._]+$/.test(line)) {
if (
inPermissions &&
/^\s{2,}[A-Za-z0-9._]+(?:: restricted=true)?$/u.test(line)
) {
permissions += 1; permissions += 1;
continue; continue;
} }
+3 -5
View File
@@ -199,11 +199,9 @@ export class JadxMcpTransport implements Transport {
throw new Error(`JADX MCP frame exceeds ${MAX_FRAME_BYTES} bytes`); throw new Error(`JADX MCP frame exceeds ${MAX_FRAME_BYTES} bytes`);
this.#appendFrame(fragment, frameBytes); this.#appendFrame(fragment, frameBytes);
if (newline === -1) return; if (newline === -1) return;
const line = new TextDecoder("utf-8", { const line = this.#frameBuffer
fatal: true, .subarray(0, this.#frameLength)
// Preserve a leading BOM so the JSON-RPC parser can reject it. .toString("utf8");
ignoreBOM: true,
}).decode(this.#frameBuffer.subarray(0, this.#frameLength));
this.#frameBuffer = Buffer.alloc(0); this.#frameBuffer = Buffer.alloc(0);
this.#frameLength = 0; this.#frameLength = 0;
this.onmessage?.(deserializeMessage(line)); this.onmessage?.(deserializeMessage(line));
+2 -2
View File
@@ -397,7 +397,7 @@ const workflowAvailabilityFor = ({
remediation: remediation:
"On Linux x64, set absolute REA_PWNTOOLS_PYTHON to caller-supplied Python with pwntools 4.15.0. No active binary target is required.", "On Linux x64, set absolute REA_PWNTOOLS_PYTHON to caller-supplied Python with pwntools 4.15.0. No active binary target is required.",
}; };
if (name === "inspect_packet_capture" || name === "inspect_packet_fields") if (name === "inspect_packet_capture")
return policy.packetCaptureEnabled === true return policy.packetCaptureEnabled === true
? { reason: "available", remediation: null } ? { reason: "available", remediation: null }
: { : {
@@ -405,7 +405,7 @@ const workflowAvailabilityFor = ({
remediation: remediation:
"On Linux, supply TShark 4.x via absolute REA_TSHARK_COMMAND and util-linux prlimit (REA_TSHARK_PRLIMIT_COMMAND, default /usr/bin/prlimit). No active binary target or hardware is required; the tool is checked at execution.", "On Linux, supply TShark 4.x via absolute REA_TSHARK_COMMAND and util-linux prlimit (REA_TSHARK_PRLIMIT_COMMAND, default /usr/bin/prlimit). No active binary target or hardware is required; the tool is checked at execution.",
}; };
if (name === "decode_signal_recording" || name === "inspect_signal_decoders") if (name === "decode_signal_recording")
return policy.signalRecordingEnabled === true return policy.signalRecordingEnabled === true
? { reason: "available", remediation: null } ? { reason: "available", remediation: null }
: { : {
+2 -6
View File
@@ -78,9 +78,7 @@ export const readJsonObjectFile = (
value === null || value === null ||
Array.isArray(value) Array.isArray(value)
) )
return err( throw new TypeError("JSON document root must be an object");
invalidJson(operation, "JSON document root must be an object"),
);
for (const members of scans) for (const members of scans)
for (const [key, member] of members) { for (const [key, member] of members) {
if (!Object.hasOwn(value, key)) continue; if (!Object.hasOwn(value, key)) continue;
@@ -179,9 +177,7 @@ export const readJsonObjectFile = (
signal === undefined ? {} : { signal }, signal === undefined ? {} : { signal },
); );
if (!rootObject) if (!rootObject)
return err( throw new TypeError("JSON document root must be an object");
invalidJson(operation, "JSON document root must be an object"),
);
const digest = hash.digest("hex"); const digest = hash.digest("hex");
if (previousDigest !== undefined && digest !== previousDigest) if (previousDigest !== undefined && digest !== previousDigest)
throw new RegularFileChangedError(path); throw new RegularFileChangedError(path);
+4 -23
View File
@@ -2,16 +2,12 @@ import { createHash } from "node:crypto";
import { describe, expect, it } from "vitest"; import { describe, expect, it } from "vitest";
import { import { installMacHopper, type MacHopperInstallHost } from "./MacHopper.js";
installMacHopper,
type MacHopperBundleVerification,
type MacHopperInstallHost,
} from "./MacHopper.js";
class FakeMacHopperHost implements MacHopperInstallHost { class FakeMacHopperHost implements MacHopperInstallHost {
readonly archive = new TextEncoder().encode("hopper-dmg"); readonly archive = new TextEncoder().encode("hopper-dmg");
archiveOk = true; archiveOk = true;
verification: MacHopperBundleVerification = "verified"; valid = true;
existing = false; existing = false;
mountSucceeds = true; mountSucceeds = true;
copySucceeds = true; copySucceeds = true;
@@ -19,7 +15,6 @@ class FakeMacHopperHost implements MacHopperInstallHost {
badHash = false; badHash = false;
unmounts = 0; unmounts = 0;
cleanups = 0; cleanups = 0;
installs = 0;
opens = 0; opens = 0;
replaces = false; replaces = false;
@@ -52,7 +47,7 @@ class FakeMacHopperHost implements MacHopperInstallHost {
return Promise.resolve(); return Promise.resolve();
}; };
appBundle = () => Promise.resolve("/mounted/Hopper Disassembler.app"); appBundle = () => Promise.resolve("/mounted/Hopper Disassembler.app");
validBundle = () => Promise.resolve(this.verification); validBundle = () => Promise.resolve(this.valid);
destinationExists = () => Promise.resolve(this.existing); destinationExists = () => Promise.resolve(this.existing);
installBundle = ( installBundle = (
_source: string, _source: string,
@@ -60,7 +55,6 @@ class FakeMacHopperHost implements MacHopperInstallHost {
_stage: string, _stage: string,
replaceExisting: boolean, replaceExisting: boolean,
) => { ) => {
this.installs += 1;
this.replaces = replaceExisting; this.replaces = replaceExisting;
return Promise.resolve(this.copySucceeds); return Promise.resolve(this.copySucceeds);
}; };
@@ -121,7 +115,7 @@ describe("macOS Hopper installation", () => {
it("cleans and detaches after bundle validation fails", async () => { it("cleans and detaches after bundle validation fails", async () => {
const host = new FakeMacHopperHost(); const host = new FakeMacHopperHost();
host.verification = "invalid_signature"; host.valid = false;
await expect(installMacHopper({}, host)).resolves.toEqual({ await expect(installMacHopper({}, host)).resolves.toEqual({
status: "failed", status: "failed",
reason: "bundle", reason: "bundle",
@@ -129,17 +123,4 @@ describe("macOS Hopper installation", () => {
expect(host.unmounts).toBe(1); expect(host.unmounts).toBe(1);
expect(host.cleanups).toBe(1); expect(host.cleanups).toBe(1);
}); });
it("reports a publisher mismatch without copying or launching the bundle", async () => {
const host = new FakeMacHopperHost();
host.verification = "unverified_publisher";
await expect(installMacHopper({}, host)).resolves.toEqual({
status: "failed",
reason: "publisher_verification_failed",
});
expect(host.installs).toBe(0);
expect(host.opens).toBe(0);
expect(host.unmounts).toBe(1);
expect(host.cleanups).toBe(1);
});
}); });
+22 -52
View File
@@ -24,35 +24,7 @@ const execFileAsync = promisify(execFile);
const RELEASES_URL = const RELEASES_URL =
"https://www.hopperapp.com/include/files-api.php?request=releases&public=true"; "https://www.hopperapp.com/include/files-api.php?request=releases&public=true";
const DOWNLOAD_PREFIX = "https://www.hopperapp.com:443/downloader/public/"; const DOWNLOAD_PREFIX = "https://www.hopperapp.com:443/downloader/public/";
const BUNDLE_IDENTIFIER = "com.cryptic-apps.hopper-web-4";
/**
* Apple publisher identity pinned for the official Hopper macOS bundle. The
* team identifier was captured from `Hopper-6.6.0-demo.dmg` at
* `https://www.hopperapp.com:443/downloader/public/Hopper-6.6.0-demo.dmg`
* (34846271 bytes, SHA-1 0be530f038249742590992199c7b042ea20ca955) with
* `codesign -dvvv`, which reported
* `Authority=Developer ID Application: Cryptic Apps (2AMA2753NF)`.
*/
export const HOPPER_PUBLISHER = Object.freeze({
bundleIdentifier: "com.cryptic-apps.hopper-web-4",
teamIdentifier: "2AMA2753NF",
});
/**
* Code requirement enforcing the bundle identifier and signing team together.
* The identifier is read from the sealed CodeDirectory, so it cannot be changed
* by editing `Info.plist` the way the previous string comparison could.
*/
const HOPPER_PUBLISHER_REQUIREMENT =
`= identifier "${HOPPER_PUBLISHER.bundleIdentifier}"` +
` and anchor apple generic` +
` and certificate leaf[subject.OU] = "${HOPPER_PUBLISHER.teamIdentifier}"`;
/** Outcome of reading a mounted bundle's signature and publisher identity. */
export type MacHopperBundleVerification =
| "verified"
| "invalid_signature"
| "unverified_publisher";
/** Stable network and command effects disclosed before macOS Hopper setup. */ /** Stable network and command effects disclosed before macOS Hopper setup. */
export const macHopperInstallDisclosure = { export const macHopperInstallDisclosure = {
@@ -60,7 +32,6 @@ export const macHopperInstallDisclosure = {
commands: [ commands: [
"hdiutil attach -readonly -nobrowse -mountpoint <private-mount> <verified-hopper.dmg>", "hdiutil attach -readonly -nobrowse -mountpoint <private-mount> <verified-hopper.dmg>",
"codesign --verify --deep --strict <mounted-hopper.app>", "codesign --verify --deep --strict <mounted-hopper.app>",
`codesign --verify --strict '-R${HOPPER_PUBLISHER_REQUIREMENT}' <mounted-hopper.app>`,
"ditto <mounted-hopper.app> <user-Applications-stage>", "ditto <mounted-hopper.app> <user-Applications-stage>",
"open <installed-hopper.app>", "open <installed-hopper.app>",
] as const, ] as const,
@@ -81,7 +52,6 @@ export type MacHopperInstallResult =
| "integrity" | "integrity"
| "mount" | "mount"
| "bundle" | "bundle"
| "publisher_verification_failed"
| "destination_exists" | "destination_exists"
| "copy" | "copy"
| "launcher_missing"; | "launcher_missing";
@@ -99,7 +69,7 @@ export interface MacHopperInstallHost {
mount(packagePath: string, mountPath: string): Promise<boolean>; mount(packagePath: string, mountPath: string): Promise<boolean>;
unmount(mountPath: string): Promise<void>; unmount(mountPath: string): Promise<void>;
appBundle(mountPath: string): Promise<string | undefined>; appBundle(mountPath: string): Promise<string | undefined>;
validBundle(appPath: string): Promise<MacHopperBundleVerification>; validBundle(appPath: string): Promise<boolean>;
destinationExists(path: string): Promise<boolean>; destinationExists(path: string): Promise<boolean>;
installBundle( installBundle(
source: string, source: string,
@@ -154,12 +124,8 @@ export const installMacHopper = async (
return { status: "failed", reason: "mount" }; return { status: "failed", reason: "mount" };
mounted = true; mounted = true;
const source = await host.appBundle(mountPath); const source = await host.appBundle(mountPath);
if (source === undefined) return { status: "failed", reason: "bundle" }; if (source === undefined || !(await host.validBundle(source)))
const verification = await host.validBundle(source);
if (verification === "invalid_signature")
return { status: "failed", reason: "bundle" }; return { status: "failed", reason: "bundle" };
if (verification === "unverified_publisher")
return { status: "failed", reason: "publisher_verification_failed" };
const stage = `${destination}.rea-stage`; const stage = `${destination}.rea-stage`;
if ( if (
!(await host.installBundle( !(await host.installBundle(
@@ -256,21 +222,25 @@ export const systemMacHopperInstallHost = (
environment, environment,
)) ))
) )
return "invalid_signature"; return false;
if ( try {
!(await commandSucceeds( const identifier = (
"codesign", await execFileAsync(
[ "/usr/libexec/PlistBuddy",
"--verify", [
"--strict", "-c",
`-R${HOPPER_PUBLISHER_REQUIREMENT}`, "Print :CFBundleIdentifier",
appPath, join(appPath, "Contents/Info.plist"),
], ],
environment, { env: environment },
)) )
) ).stdout.trim();
return "unverified_publisher"; return identifier === BUNDLE_IDENTIFIER;
return "verified"; } catch (cause: unknown) {
// best-effort cleanup: optional bundle probing; unreadable means invalid.
void cause;
return false;
}
}, },
async destinationExists(path) { async destinationExists(path) {
try { try {
+1 -1
View File
@@ -23,7 +23,7 @@ export const createProgressReporter = (
} = {}, } = {},
): ProgressReporter => { ): ProgressReporter => {
const minimumIntervalMs = options.minimumIntervalMs ?? 100; const minimumIntervalMs = options.minimumIntervalMs ?? 100;
const now = options.now ?? (() => performance.now()); const now = options.now ?? Date.now;
let lastCompleted = -1; let lastCompleted = -1;
let lastSentAt = Number.NEGATIVE_INFINITY; let lastSentAt = Number.NEGATIVE_INFINITY;
let sequence = 0; let sequence = 0;
@@ -12,14 +12,12 @@ import {
detectReferenceSourceLanguage, detectReferenceSourceLanguage,
} from "../domain/referenceSourceClassification.js"; } from "../domain/referenceSourceClassification.js";
import type { HistoricalSourceGraphInput } from "../domain/referenceSourceGraph.js"; import type { HistoricalSourceGraphInput } from "../domain/referenceSourceGraph.js";
import { import { parseReferenceSourceImports } from "../domain/referenceSourceImportParsing.js";
parseReferenceSourceImports,
PARSEABLE_REFERENCE_SOURCE_LANGUAGES,
} from "../domain/referenceSourceImportParsing.js";
import type { import type {
ReferenceSourceEntry, ReferenceSourceEntry,
ReferenceSourceRead, ReferenceSourceRead,
} from "../reference/ReferenceSourceReaderTypes.js"; } from "../reference/ReferenceSourceReaderTypes.js";
import { PARSEABLE_REFERENCE_SOURCE_LANGUAGES } from "./ReferenceSourceImportTypes.js";
import { typescriptEmittedSourceCandidates } from "./TypeScriptEmittedSourceCandidates.js"; import { typescriptEmittedSourceCandidates } from "./TypeScriptEmittedSourceCandidates.js";
export interface ParsedReferenceSourceEntries { export interface ParsedReferenceSourceEntries {
@@ -104,11 +102,7 @@ const resolveInternalSpecifier = (
]); ]);
// Retain present runtime files first; substitute only a missing counterpart. // Retain present runtime files first; substitute only a missing counterpart.
// Counterparts cover explicit filenames, not directory/index resolution. // Counterparts cover explicit filenames, not directory/index resolution.
candidates.push( candidates.push(...typescriptEmittedSourceCandidates(fromPath, normalized));
...typescriptEmittedSourceCandidates(fromPath, normalized).map(
({ path }) => path,
),
);
const match = lookupPath.resolveCandidates(candidates); const match = lookupPath.resolveCandidates(candidates);
return match === undefined return match === undefined
? { to: normalized, resolution: "unresolved" } ? { to: normalized, resolution: "unresolved" }
@@ -87,3 +87,6 @@ export const DEFAULT_REFERENCE_SOURCE_IGNORE_PATTERNS = [
".coverage", ".coverage",
"*.log", "*.log",
] as const; ] as const;
export const PARSEABLE_REFERENCE_SOURCE_LANGUAGES: ReadonlySet<string> =
new Set(["JavaScript", "TypeScript", "JSX", "TSX"]);
+25 -40
View File
@@ -19,17 +19,6 @@ export interface RegularFileReadOptions {
readonly resources?: ArtifactResourceScope; readonly resources?: ArtifactResourceScope;
} }
export class InvalidUtf8FileError extends TypeError {
readonly _tag = "InvalidUtf8FileError";
constructor(
readonly path: string,
cause: unknown,
) {
super(`File is not valid UTF-8: ${path}`, { cause });
this.name = this._tag;
}
}
export type RegularFileReadOutcome<Value> = export type RegularFileReadOutcome<Value> =
| { readonly kind: "completed"; readonly value: Value } | { readonly kind: "completed"; readonly value: Value }
| { readonly kind: "failed"; readonly cause: unknown }; | { readonly kind: "failed"; readonly cause: unknown };
@@ -153,34 +142,7 @@ const withRegularFileOwned = async <Value>(
export const readRegularFile = ( export const readRegularFile = (
path: string, path: string,
options: RegularFileReadOptions = {}, options: RegularFileReadOptions = {},
): Promise<Buffer> => readRegularFileContent(path, (bytes) => bytes, options); ): Promise<Buffer> =>
/** Read valid UTF-8 without replacing source bytes or stripping an initial BOM. */
export const readRegularFileText = (
path: string,
options: RegularFileReadOptions = {},
): Promise<string> =>
readRegularFileContent(
path,
(bytes) => {
try {
return new TextDecoder("utf-8", {
fatal: true,
ignoreBOM: true,
}).decode(bytes);
} catch (cause: unknown) {
if (!(cause instanceof TypeError)) throw cause;
throw new InvalidUtf8FileError(path, cause);
}
},
options,
);
const readRegularFileContent = <Value>(
path: string,
decode: (bytes: Buffer) => Value,
options: RegularFileReadOptions,
): Promise<Value> =>
withRegularFile( withRegularFile(
path, path,
async (handle, stats) => { async (handle, stats) => {
@@ -191,7 +153,30 @@ const readRegularFileContent = <Value>(
); );
if (bytes === undefined || bytes.length !== stats.size) if (bytes === undefined || bytes.length !== stats.size)
throw new RegularFileChangedError(path); throw new RegularFileChangedError(path);
return decode(bytes); return bytes;
}, },
options, options,
); );
/** Read one admitted regular file as UTF-8 without waiting on a pipe. */
export const readRegularFileText = async (
path: string,
options: RegularFileReadOptions = {},
): Promise<string> => {
try {
return (await readRegularFile(path, options)).toString("utf8");
} catch (cause: unknown) {
if (
cause instanceof RegularFileCleanupFailure &&
cause.outcome.kind === "completed" &&
Buffer.isBuffer(cause.outcome.value)
)
throw new RegularFileCleanupFailure(
cause.path,
cause.owner,
{ kind: "completed", value: cause.outcome.value.toString("utf8") },
cause.cleanupCause,
);
throw cause;
}
};
-34
View File
@@ -54,40 +54,6 @@ describe("setup workflow", () => {
expect(result.remediation).toContain("could not be updated"); expect(result.remediation).toContain("could not be updated");
}); });
it("reports a failed config rollback with its destination and recovery backup", async () => {
const host = new FakeSetupHost();
host.hopper = "/Applications/Hopper";
host.clients = [{ name: "codex", configPath: "/fixture/codex.json" }];
host.clientResults.set("codex", {
status: "failed",
reason: "rollback",
backupPath: "/fixture/codex.json.rea.backup",
failure: {
kind: "restore",
path: "/fixture/codex.json",
detail: "permission denied",
},
});
const result = await runSetup(
{ ...options(true), clientIds: ["codex"] },
host,
);
expect(result.clients.codex).toEqual({
status: "failed",
reason: "rollback",
backupPath: "/fixture/codex.json.rea.backup",
failure: {
kind: "restore",
path: "/fixture/codex.json",
detail: "permission denied",
},
});
expect(result.remediation).toContain("permission denied");
expect(result.remediation).toContain("/fixture/codex.json.rea.backup");
});
it("explains skill installation recovery", async () => { it("explains skill installation recovery", async () => {
const host = new FakeSetupHost(); const host = new FakeSetupHost();
host.hopper = "/Applications/Hopper"; host.hopper = "/Applications/Hopper";
+15 -160
View File
@@ -19,7 +19,6 @@ import {
} from "./ClientConfigurationDocument.js"; } from "./ClientConfigurationDocument.js";
import { dshDocument, DshCordisDocument } from "./DshCordisPatch.js"; import { dshDocument, DshCordisDocument } from "./DshCordisPatch.js";
import { constants as fsConstants } from "node:fs"; import { constants as fsConstants } from "node:fs";
import { createHash } from "node:crypto";
import { copyFile, mkdir, rm, writeFile } from "node:fs/promises"; import { copyFile, mkdir, rm, writeFile } from "node:fs/promises";
import { dirname } from "node:path"; import { dirname } from "node:path";
import writeFileAtomic from "write-file-atomic"; import writeFileAtomic from "write-file-atomic";
@@ -30,7 +29,6 @@ import { npxRegistrationCommand } from "./ClientRegistrationIdentity.js";
import { MCP_STARTUP_POLICY } from "../mcpStartupPolicy.js"; import { MCP_STARTUP_POLICY } from "../mcpStartupPolicy.js";
import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js"; import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js";
import { import {
readRegularFile,
readRegularFileText, readRegularFileText,
RegularFileCleanupFailure, RegularFileCleanupFailure,
} from "./RegularFileRead.js"; } from "./RegularFileRead.js";
@@ -42,26 +40,6 @@ import type {
} from "./SetupTypes.js"; } from "./SetupTypes.js";
import type { SetupClient } from "./SupportedClients.js"; import type { SetupClient } from "./SupportedClients.js";
export class ClientConfigurationCleanupRollbackFailure extends RegularFileCleanupFailure<string> {
constructor(
failure: RegularFileCleanupFailure<string>,
readonly configurationFailure: Extract<
ClientConfigurationResult,
{ readonly status: "failed"; readonly reason: "rollback" }
>,
) {
super(
failure.path,
failure.owner,
failure.outcome,
new AggregateError(
[failure.cleanupCause, new Error(configurationFailure.failure.detail)],
`Configuration readback cleanup and rollback failed: ${configurationFailure.failure.path}`,
),
);
}
}
/** Configure one supported client's native stdio MCP registration shape. */ /** Configure one supported client's native stdio MCP registration shape. */
export const configureClientConfiguration = ( export const configureClientConfiguration = (
client: SetupClient, client: SetupClient,
@@ -189,23 +167,11 @@ const persistClientConfiguration = async ({
}: ClientConfigurationUpdate): Promise<ClientConfigurationResult> => { }: ClientConfigurationUpdate): Promise<ClientConfigurationResult> => {
const backupPath = const backupPath =
original === undefined ? undefined : `${client.configPath}.rea.backup`; original === undefined ? undefined : `${client.configPath}.rea.backup`;
let recoveryBackupPath: string | undefined; if (
if (backupPath !== undefined && original !== undefined) { backupPath !== undefined &&
const selection = await selectCurrentRecoveryBackup(backupPath, original); !(await preserveConfigBackup(transactionPath, backupPath))
if (!selection.ok) return { status: "failed", reason: "backup" }; )
if ( return { status: "failed", reason: "backup" };
!(await preserveConfigBackup(transactionPath, selection.value.backupPath))
)
return { status: "failed", reason: "backup" };
const verified = await selectCurrentRecoveryBackup(backupPath, original);
if (
!verified.ok ||
verified.value.backupPath !== selection.value.backupPath ||
!verified.value.present
)
return { status: "failed", reason: "backup" };
recoveryBackupPath = verified.value.backupPath;
}
// A native OpenCode V2 entry replaces REA's legacy V1 entry, which would // A native OpenCode V2 entry replaces REA's legacy V1 entry, which would
// otherwise conflict with it. // otherwise conflict with it.
const legacyPath = legacyClientServerPath( const legacyPath = legacyClientServerPath(
@@ -273,39 +239,19 @@ const persistClientConfiguration = async ({
!clientConfigurationValuesEqual(readback.servers[name], entry), !clientConfigurationValuesEqual(readback.servers[name], entry),
) )
) { ) {
return rollbackOrReadbackFailure( await restoreConfig(transactionPath, original);
transactionPath, return { status: "failed", reason: "readback" };
original,
recoveryBackupPath,
);
} }
} catch (cause: unknown) { } catch (cause: unknown) {
if (cause instanceof RegularFileCleanupFailure) { if (cause instanceof RegularFileCleanupFailure) throw cause;
const rollback = await restoreConfig(transactionPath, original);
if (!rollback.ok)
throw new ClientConfigurationCleanupRollbackFailure(cause, {
status: "failed",
reason: "rollback",
...(recoveryBackupPath === undefined
? {}
: { backupPath: recoveryBackupPath }),
failure: rollback.error,
});
throw cause;
}
// Readback failure restores the transaction before reporting. // Readback failure restores the transaction before reporting.
void cause; void cause;
return rollbackOrReadbackFailure( await restoreConfig(transactionPath, original);
transactionPath, return { status: "failed", reason: "readback" };
original,
recoveryBackupPath,
);
} }
return { return {
status: "configured", status: "configured",
...(recoveryBackupPath === undefined ...(backupPath === undefined ? {} : { backupPath }),
? {}
: { backupPath: recoveryBackupPath }),
}; };
}; };
@@ -460,18 +406,9 @@ export const inspectClientConfiguration = async (
"The existing configuration is malformed. Repair it before setup applies any other changes.", "The existing configuration is malformed. Repair it before setup applies any other changes.",
}; };
} }
const backupSelection = await selectCurrentRecoveryBackup(
`${client.configPath}.rea.backup`,
original,
);
if (!backupSelection.ok)
return {
status: "invalid",
remediation: backupSelection.error.detail,
};
return { return {
status: "update", status: "update",
backupPath: backupSelection.value.backupPath, backupPath: `${client.configPath}.rea.backup`,
}; };
}; };
@@ -536,101 +473,19 @@ const preserveConfigBackup = async (
} }
}; };
interface RecoveryBackupSelection {
readonly backupPath: string;
readonly present: boolean;
}
interface RecoveryBackupSelectionFailure {
readonly kind: "unreadable" | "occupied";
readonly path: string;
readonly detail: string;
}
const selectCurrentRecoveryBackup = async (
firstBackupPath: string,
original: string,
): Promise<Result<RecoveryBackupSelection, RecoveryBackupSelectionFailure>> => {
const expected = Buffer.from(original, "utf8");
let firstBackup: Buffer | undefined;
try {
firstBackup = await readRegularFile(firstBackupPath, {
symlinks: "reject",
});
} catch (cause: unknown) {
if (cause instanceof RegularFileCleanupFailure) throw cause;
if (!isMissing(cause))
return err({
kind: "unreadable",
path: firstBackupPath,
detail: `The recovery backup could not be checked: ${errorDetail(cause)}`,
});
}
if (firstBackup === undefined)
return ok({ backupPath: firstBackupPath, present: false });
if (firstBackup.equals(expected))
return ok({ backupPath: firstBackupPath, present: true });
const digest = createHash("sha256").update(original, "utf8").digest("hex");
const recoveryPath = `${firstBackupPath}.${digest}.recovery`;
let recovery: Buffer | undefined;
try {
recovery = await readRegularFile(recoveryPath, { symlinks: "reject" });
} catch (cause: unknown) {
if (cause instanceof RegularFileCleanupFailure) throw cause;
if (!isMissing(cause))
return err({
kind: "unreadable",
path: recoveryPath,
detail: `The recovery backup could not be checked: ${errorDetail(cause)}`,
});
}
if (recovery !== undefined && !recovery.equals(expected))
return err({
kind: "occupied",
path: recoveryPath,
detail: `The recovery path already contains different data: ${recoveryPath}`,
});
return ok({ backupPath: recoveryPath, present: recovery !== undefined });
};
const restoreConfig = async ( const restoreConfig = async (
path: string, path: string,
original: string | undefined, original: string | undefined,
): Promise< ): Promise<void> => {
Result<
void,
{ readonly kind: "restore"; readonly path: string; readonly detail: string }
>
> => {
try { try {
if (original === undefined) await rm(path, { force: true }); if (original === undefined) await rm(path, { force: true });
else await writeFile(path, original, { encoding: "utf8", mode: 0o600 }); else await writeFile(path, original, { encoding: "utf8", mode: 0o600 });
return ok(undefined);
} catch (cause: unknown) { } catch (cause: unknown) {
return err({ kind: "restore", path, detail: errorDetail(cause) }); // The backup remains available for the remediation reported by setup.
void cause;
} }
}; };
const rollbackOrReadbackFailure = async (
targetPath: string,
original: string | undefined,
backupPath: string | undefined,
): Promise<ClientConfigurationResult> => {
const rollback = await restoreConfig(targetPath, original);
return rollback.ok
? { status: "failed", reason: "readback" }
: {
status: "failed",
reason: "rollback",
...(backupPath === undefined ? {} : { backupPath }),
failure: rollback.error,
};
};
const errorDetail = (cause: unknown): string =>
cause instanceof Error ? cause.message : String(cause);
/** Whether REA's entry matches, no legacy entry remains, and no client disable list suppresses it. */ /** Whether REA's entry matches, no legacy entry remains, and no client disable list suppresses it. */
const registrationCurrent = ( const registrationCurrent = (
parsed: ClientConfigurationDocument, parsed: ClientConfigurationDocument,
+7 -11
View File
@@ -6,19 +6,15 @@ import type {
} from "./SetupTypes.js"; } from "./SetupTypes.js";
import type { SetupClient } from "./SupportedClients.js"; import type { SetupClient } from "./SupportedClients.js";
export const failedConfigurationMessage = ( const failedConfigurationMessage = (
result: Extract<ClientConfigurationResult, { readonly status: "failed" }>, reason: "path" | "backup" | "write" | "readback",
): string => { ): string => {
if (result.reason === "path") if (reason === "path")
return "Agent configuration path could not be safely verified. Check its permissions and, if it is a symbolic link, verify that the link resolves to a regular file owned by the current user, then rerun setup."; return "Agent configuration path could not be safely verified. Check its permissions and, if it is a symbolic link, verify that the link resolves to a regular file owned by the current user, then rerun setup.";
if (result.reason === "backup") if (reason === "backup")
return "Agent configuration could not be backed up, so no change was made. Check file permissions, then rerun setup."; return "Agent configuration could not be backed up, so no change was made. Check file permissions, then rerun setup.";
if (result.reason === "write") if (reason === "write")
return "Agent configuration could not be updated. Check file permissions, then rerun setup."; return "Agent configuration could not be updated. Check file permissions, then rerun setup.";
if (result.reason === "rollback")
return result.backupPath === undefined
? `Agent configuration rollback failed at ${result.failure.path}: ${result.failure.detail}; the failed first-run file may remain. Remove or repair it before rerunning setup.`
: `Agent configuration rollback failed at ${result.failure.path}: ${result.failure.detail}. Inspect the preserved backup at ${result.backupPath} and restore the intended configuration, then rerun setup.`;
return "Agent configuration could not be verified after writing. Repair the configuration file or restore its `.rea.backup`, then rerun setup."; return "Agent configuration could not be verified after writing. Repair the configuration file or restore its `.rea.backup`, then rerun setup.";
}; };
@@ -44,12 +40,12 @@ export const configureDetectedClients = async (options: {
); );
options.clients[client.name] = result; options.clients[client.name] = result;
if (result.status === "failed") { if (result.status === "failed") {
failure ??= failedConfigurationMessage(result); failure ??= failedConfigurationMessage(result.reason);
options.onProgress?.({ options.onProgress?.({
actionId, actionId,
label, label,
state: "failed", state: "failed",
detail: failedConfigurationMessage(result), detail: failedConfigurationMessage(result.reason),
}); });
} else if (result.status === "configured") { } else if (result.status === "configured") {
options.appliedActions.push(`configured_${client.name}`); options.appliedActions.push(`configured_${client.name}`);
+1 -7
View File
@@ -33,11 +33,9 @@ import {
} from "./SetupSkill.js"; } from "./SetupSkill.js";
import { import {
clientConfigurationAligned, clientConfigurationAligned,
ClientConfigurationCleanupRollbackFailure,
configureClientConfiguration, configureClientConfiguration,
inspectClientConfiguration, inspectClientConfiguration,
} from "./SetupClientConfiguration.js"; } from "./SetupClientConfiguration.js";
import { failedConfigurationMessage } from "./SetupClients.js";
import { setupInstallFailure } from "./SetupInstallFailure.js"; import { setupInstallFailure } from "./SetupInstallFailure.js";
import { import {
RegularFileCleanupFailure, RegularFileCleanupFailure,
@@ -277,12 +275,8 @@ const runClientConfigurationOperation = async <Value>(
cleanup.outcome.kind === "failed" cleanup.outcome.kind === "failed"
? cleanup.outcome.cause ? cleanup.outcome.cause
: cause.cleanupCause; : cause.cleanupCause;
const rollbackMessage =
cause instanceof ClientConfigurationCleanupRollbackFailure
? ` ${failedConfigurationMessage(cause.configurationFailure)}`
: "";
throw configurationCleanupError( throw configurationCleanupError(
`Could not safely access client configuration at ${cause.path}: ${errorMessage(primary)}.${rollbackMessage}`, `Could not safely access client configuration at ${cause.path}: ${errorMessage(primary)}`,
cause, cause,
cleanup.cleanup, cleanup.cleanup,
); );
@@ -1,23 +0,0 @@
import { describe, expect, it } from "vitest";
import { HOPPER_PUBLISHER } from "./MacHopper.js";
import { setupInstallFailure } from "./SetupInstallFailure.js";
describe("macOS Hopper setup failure projection", () => {
it("projects a publisher mismatch onto its own code and recovery", () => {
expect(setupInstallFailure("publisher_verification_failed")).toEqual({
status: "failed",
code: "publisher_verification_failed",
remediation: `The mounted Hopper application is not signed by the publisher REA pins (${HOPPER_PUBLISHER.bundleIdentifier}, Apple team ${HOPPER_PUBLISHER.teamIdentifier}); nothing was copied or launched. Install Hopper yourself and set HOPPER_LAUNCHER_PATH to its launcher, then rerun rea setup.`,
});
});
it("keeps a broken bundle distinct from a publisher mismatch", () => {
expect(setupInstallFailure("bundle")).toEqual({
status: "failed",
code: "bundle_invalid",
remediation:
"The mounted package did not contain the expected Hopper application bundle. Retry setup; if it repeats, update REA before installing.",
});
});
});
-10
View File
@@ -1,5 +1,3 @@
import { HOPPER_PUBLISHER } from "./MacHopper.js";
export type SetupFailureCode = export type SetupFailureCode =
| "unsupported_host" | "unsupported_host"
| "download_failed" | "download_failed"
@@ -12,7 +10,6 @@ export type SetupFailureCode =
| "destination_exists" | "destination_exists"
| "mount_failed" | "mount_failed"
| "bundle_invalid" | "bundle_invalid"
| "publisher_verification_failed"
| "copy_failed" | "copy_failed"
| "skill_install_failed" | "skill_install_failed"
| "cleanup_failed"; | "cleanup_failed";
@@ -42,7 +39,6 @@ export type HopperInstallFailureReason =
| "destination_exists" | "destination_exists"
| "mount" | "mount"
| "bundle" | "bundle"
| "publisher_verification_failed"
| "copy"; | "copy";
/** Map provider installer failures to stable setup codes and safe recovery. */ /** Map provider installer failures to stable setup codes and safe recovery. */
@@ -119,12 +115,6 @@ export const setupInstallFailure = (
remediation: remediation:
"The mounted package did not contain the expected Hopper application bundle. Retry setup; if it repeats, update REA before installing.", "The mounted package did not contain the expected Hopper application bundle. Retry setup; if it repeats, update REA before installing.",
}; };
case "publisher_verification_failed":
return {
status: "failed",
code: "publisher_verification_failed",
remediation: `The mounted Hopper application is not signed by the publisher REA pins (${HOPPER_PUBLISHER.bundleIdentifier}, Apple team ${HOPPER_PUBLISHER.teamIdentifier}); nothing was copied or launched. Install Hopper yourself and set HOPPER_LAUNCHER_PATH to its launcher, then rerun rea setup.`,
};
case "copy": case "copy":
return { return {
status: "failed", status: "failed",
+1 -1
View File
@@ -320,5 +320,5 @@ const clientConfigurationDetail = (
? " If this plan installs Hopper, REA will add HOPPER_LAUNCHER_PATH using the launcher path reported by that installation." ? " If this plan installs Hopper, REA will add HOPPER_LAUNCHER_PATH using the launcher path reported by that installation."
: " The HOPPER_LAUNCHER_PATH value shown above is the current fallback; if this plan installs Hopper, REA will replace it with the launcher path reported by that installation." : " The HOPPER_LAUNCHER_PATH value shown above is the current fallback; if this plan installs Hopper, REA will replace it with the launcher path reported by that installation."
: ""; : "";
return `Add the REA MCP registration for ${client}; preserve unrelated configuration. The backup path reflects the configuration inspected for this plan; if that file changes before execution, setup will select a recovery path for its current contents. Command: ${command.map((part) => JSON.stringify(part)).join(" ")}.${environmentDetail}${hopperDetail}`; return `Add the REA MCP registration for ${client}; preserve unrelated configuration. Command: ${command.map((part) => JSON.stringify(part)).join(" ")}.${environmentDetail}${hopperDetail}`;
}; };
-10
View File
@@ -26,16 +26,6 @@ export type ClientConfigurationResult =
| { | {
readonly status: "failed"; readonly status: "failed";
readonly reason: "path" | "backup" | "write" | "readback"; readonly reason: "path" | "backup" | "write" | "readback";
}
| {
readonly status: "failed";
readonly reason: "rollback";
readonly backupPath?: string;
readonly failure: {
readonly kind: "restore";
readonly path: string;
readonly detail: string;
};
}; };
/** Read-only state used to block unsafe setup plans before any mutation. */ /** Read-only state used to block unsafe setup plans before any mutation. */
@@ -8,11 +8,6 @@ const EMITTED_SUFFIXES: ReadonlyArray<
[".jsx", [".tsx", ".d.ts"]], [".jsx", [".tsx", ".d.ts"]],
]; ];
export interface TypeScriptEmittedSourceCandidate {
readonly path: string;
readonly kind: "implementation" | "declaration";
}
/** /**
* Return TypeScript source counterparts for an explicit emitted filename. * Return TypeScript source counterparts for an explicit emitted filename.
* Callers retain exact runtime-file precedence and must not use these for * Callers retain exact runtime-file precedence and must not use these for
@@ -21,7 +16,7 @@ export interface TypeScriptEmittedSourceCandidate {
export const typescriptEmittedSourceCandidates = ( export const typescriptEmittedSourceCandidates = (
sourcePath: string, sourcePath: string,
candidate: string, candidate: string,
): readonly TypeScriptEmittedSourceCandidate[] => { ): readonly string[] => {
if (!/\.(?:ts|tsx|mts|cts)$/u.test(sourcePath)) return []; if (!/\.(?:ts|tsx|mts|cts)$/u.test(sourcePath)) return [];
const emitted = EMITTED_SUFFIXES.find(([suffix]) => const emitted = EMITTED_SUFFIXES.find(([suffix]) =>
candidate.endsWith(suffix), candidate.endsWith(suffix),
@@ -30,8 +25,5 @@ export const typescriptEmittedSourceCandidates = (
// Strip the matched suffix by length. posix.extname would miss dotfile // Strip the matched suffix by length. posix.extname would miss dotfile
// names such as ".js", whose whole basename is the emitted suffix itself. // names such as ".js", whose whole basename is the emitted suffix itself.
const stem = candidate.slice(0, candidate.length - emitted[0].length); const stem = candidate.slice(0, candidate.length - emitted[0].length);
return emitted[1].map((suffix) => ({ return emitted[1].map((suffix) => `${stem}${suffix}`);
path: `${stem}${suffix}`,
kind: suffix.startsWith(".d.") ? "declaration" : "implementation",
}));
}; };
+2 -10
View File
@@ -25,11 +25,7 @@ import {
import { isOwnedClientRegistrationCommand } from "./ClientRegistrationIdentity.js"; import { isOwnedClientRegistrationCommand } from "./ClientRegistrationIdentity.js";
import { dshDocument } from "./DshCordisPatch.js"; import { dshDocument } from "./DshCordisPatch.js";
import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js"; import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js";
import { import { readRegularFileText } from "./RegularFileRead.js";
InvalidUtf8FileError,
RegularFileCleanupFailure,
readRegularFileText,
} from "./RegularFileRead.js";
import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js"; import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
import { import {
manualRegistrationRemediation, manualRegistrationRemediation,
@@ -273,17 +269,13 @@ const readClientConfiguration = async (
try { try {
original = await fileSystem.readText(transactionPath); original = await fileSystem.readText(transactionPath);
} catch (cause: unknown) { } catch (cause: unknown) {
const readFailure =
cause instanceof RegularFileCleanupFailure ? cause.cause : cause;
return itemRead( return itemRead(
isMissing(cause) isMissing(cause)
? item(client.name, "skipped", "Configuration does not exist.") ? item(client.name, "skipped", "Configuration does not exist.")
: item( : item(
client.name, client.name,
"failed", "failed",
readFailure instanceof InvalidUtf8FileError "Configuration could not be read. Check file permissions, then rerun uninstall.",
? "Configuration is not valid UTF-8 and was not changed. Repair it, then rerun uninstall."
: "Configuration could not be read. Check file permissions, then rerun uninstall.",
), ),
); );
} }
-1
View File
@@ -22,7 +22,6 @@ export interface WebModuleArtifacts extends SelectedWebScriptArtifacts {
/** Read selected local artifacts without refetching website assets. */ /** Read selected local artifacts without refetching website assets. */
export interface WebModuleArtifactPort { export interface WebModuleArtifactPort {
close?(): Promise<void>;
load( load(
input: WebModuleTraceInput, input: WebModuleTraceInput,
options?: ExecutionOptions, options?: ExecutionOptions,
+1 -8
View File
@@ -1,4 +1,3 @@
import { closeOwnedWebResources } from "./closeOwnedWebResources.js";
import { isAbsolute } from "node:path"; import { isAbsolute } from "node:path";
import { z } from "zod"; import { z } from "zod";
import type { AnalysisError } from "../domain/analysisErrorBase.js"; import type { AnalysisError } from "../domain/analysisErrorBase.js";
@@ -71,13 +70,7 @@ export class WebModuleTraceService {
) {} ) {}
close(): Promise<void> { close(): Promise<void> {
return closeOwnedWebResources( return this.resolver.close?.() ?? Promise.resolve();
[
this.artifacts.close?.bind(this.artifacts),
this.resolver.close?.bind(this.resolver),
],
"Web module trace cleanup failed",
);
} }
async trace( async trace(
-2
View File
@@ -12,6 +12,4 @@ export interface WebNetworkCapturePort {
input: InspectWebNetworkCaptureInput, input: InspectWebNetworkCaptureInput,
options?: ExecutionOptions, options?: ExecutionOptions,
): Promise<Result<WebNetworkCapture, AnalysisError>>; ): Promise<Result<WebNetworkCapture, AnalysisError>>;
/** Retry retained reader/runtime cleanup when the owning service shuts down. */
close?(): Promise<void>;
} }
@@ -27,11 +27,6 @@ const OPERATION = "inspect_web_network_capture";
export class WebNetworkCaptureService { export class WebNetworkCaptureService {
constructor(readonly port: WebNetworkCapturePort) {} constructor(readonly port: WebNetworkCapturePort) {}
/** Release resources retained by the selected capture adapter. */
close(): Promise<void> {
return this.port.close?.() ?? Promise.resolve();
}
/** Return selected complete records inline, without inventing live browser attribution. */ /** Return selected complete records inline, without inventing live browser attribution. */
async inspect( async inspect(
rawInput: unknown, rawInput: unknown,
-1
View File
@@ -8,7 +8,6 @@ import type { ExecutionOptions } from "./AnalysisProvider.js";
/** Read the caller-selected capture export and verify the selected source identity. */ /** Read the caller-selected capture export and verify the selected source identity. */
export interface WebScriptArtifactPort { export interface WebScriptArtifactPort {
close?(): Promise<void>;
load( load(
input: SelectedWebScriptInput, input: SelectedWebScriptInput,
options?: ExecutionOptions, options?: ExecutionOptions,
@@ -20,7 +20,6 @@ export interface WebSourceLocationArtifacts extends SelectedWebScriptArtifacts {
/** Replaceable artifact acquisition boundary for a caller-selected pairing. */ /** Replaceable artifact acquisition boundary for a caller-selected pairing. */
export interface WebSourceLocationArtifactPort { export interface WebSourceLocationArtifactPort {
close?(): Promise<void>;
load( load(
input: WebSourceLocationInput, input: WebSourceLocationInput,
options?: ExecutionOptions, options?: ExecutionOptions,
+1 -9
View File
@@ -1,4 +1,3 @@
import { closeOwnedWebResources } from "./closeOwnedWebResources.js";
import { createHash } from "node:crypto"; import { createHash } from "node:crypto";
import { isAbsolute } from "node:path"; import { isAbsolute } from "node:path";
import { z } from "zod"; import { z } from "zod";
@@ -72,13 +71,7 @@ export class WebSourceLocationService {
) {} ) {}
close(): Promise<void> { close(): Promise<void> {
return closeOwnedWebResources( return this.decoder.close?.() ?? Promise.resolve();
[
this.artifacts.close?.bind(this.artifacts),
this.decoder.close?.bind(this.decoder),
],
"Web source-location cleanup failed",
);
} }
/** Shared CLI/MCP workflow preserves raw declarations and every equal-position match. */ /** Shared CLI/MCP workflow preserves raw declarations and every equal-position match. */
@@ -181,7 +174,6 @@ export class WebSourceLocationService {
); );
} }
} }
const invalidOutput = (message: string): Result<never, AnalysisError> => const invalidOutput = (message: string): Result<never, AnalysisError> =>
err(new AnalysisOutputError(OPERATION, message)); err(new AnalysisOutputError(OPERATION, message));
const invalid = ( const invalid = (
@@ -12,7 +12,6 @@ import type { Result } from "../../domain/result.js";
/** Stable read and PE decoding boundary, independent of an active disassembler session. */ /** Stable read and PE decoding boundary, independent of an active disassembler session. */
export interface PeResourcesPort { export interface PeResourcesPort {
readonly identity: ProviderIdentity; readonly identity: ProviderIdentity;
close?(): Promise<void>;
inspect( inspect(
input: InspectPeResourcesInput, input: InspectPeResourcesInput,
options?: ExecutionOptions, options?: ExecutionOptions,
@@ -21,10 +21,6 @@ import { err, ok, type Result } from "../../domain/result.js";
export class PeResourcesService { export class PeResourcesService {
constructor(readonly provider: PeResourcesPort) {} constructor(readonly provider: PeResourcesPort) {}
close(): Promise<void> {
return this.provider.close?.() ?? Promise.resolve();
}
async inspect( async inspect(
rawInput: unknown, rawInput: unknown,
options?: ExecutionOptions, options?: ExecutionOptions,
@@ -1,9 +1,3 @@
import {
inspectPacketFieldsInputSchema,
packetFieldsCatalogSchema,
type InspectPacketFieldsInput,
} from "../../domain/captures/packetFields.js";
import type { JsonValue } from "../../domain/jsonValue.js";
import { isAbsolute } from "node:path"; import { isAbsolute } from "node:path";
import type { import type {
AnalysisExecution, AnalysisExecution,
@@ -30,10 +24,6 @@ export interface PacketCapturePort {
input: InspectPacketCaptureInput, input: InspectPacketCaptureInput,
options?: ExecutionOptions, options?: ExecutionOptions,
): Promise<Result<AnalysisExecution, AnalysisError>>; ): Promise<Result<AnalysisExecution, AnalysisError>>;
inspectFields(
input: InspectPacketFieldsInput,
options?: ExecutionOptions,
): Promise<Result<AnalysisExecution, AnalysisError>>;
close(): Promise<void>; close(): Promise<void>;
} }
@@ -84,58 +74,14 @@ export class PacketCaptureService {
"Capture analyzer returned malformed data or changed the selected artifact identity/options.", "Capture analyzer returned malformed data or changed the selected artifact identity/options.",
), ),
); );
return this.#evidence(operation, input.data, execution, options); const evidence = createEvidence(execution.subject, execution.provider, {
} operation,
async inspectFields( parameters: input.data,
raw: unknown, result: report.data,
options?: ExecutionOptions, confidence: "observed",
): Promise<Result<Evidence, AnalysisError>> { locations: execution.locations,
const operation = "inspect_packet_fields"; limitations: execution.limitations,
if (options?.signal?.aborted) });
return err(new AnalysisCancelledError(operation));
const input = inspectPacketFieldsInputSchema.safeParse(raw);
if (!input.success)
return err(
analysisInputErrorFromIssues(operation, input.error.issues, raw, {
cause: input.error,
}),
);
const decoded = await this.provider.inspectFields(input.data, options);
if (!decoded.ok) return decoded;
const execution = decoded.value;
const report = packetFieldsCatalogSchema.safeParse(execution.result);
if (
!report.success ||
JSON.stringify(report.data.selection) !== JSON.stringify(input.data) ||
execution.subject?.path !== report.data.tool.path ||
execution.subject.sha256 !== report.data.tool.sha256
)
return err(
new AnalysisOutputError(
operation,
"Field discovery returned malformed data or changed the selected installation/options.",
),
);
return this.#evidence(operation, input.data, execution, options);
}
#evidence(
operation: string,
parameters: Readonly<Record<string, JsonValue>>,
execution: AnalysisExecution,
options?: ExecutionOptions,
): Result<Evidence, AnalysisError> {
const evidence = createEvidence(
execution.subject ?? undefined,
execution.provider,
{
operation,
parameters,
result: execution.result,
confidence: "observed",
locations: execution.locations,
limitations: execution.limitations,
},
);
return options?.signal?.aborted return options?.signal?.aborted
? err( ? err(
new AnalysisCancelledError(operation, { new AnalysisCancelledError(operation, {
-15
View File
@@ -1,15 +0,0 @@
/** Attempt every web workflow cleanup owner and report all failures. */
export const closeOwnedWebResources = async (
close: readonly ((() => Promise<void>) | undefined)[],
message: string,
): Promise<void> => {
const results = await Promise.allSettled(
close.flatMap((owner) =>
owner === undefined ? [] : [Promise.resolve().then(owner)],
),
);
const failures = results.flatMap((result) =>
result.status === "rejected" ? [result.reason] : [],
);
if (failures.length > 0) throw new AggregateError(failures, message);
};
@@ -809,42 +809,6 @@ it("keeps an active script message bucket after its last message is evicted", as
await manager.closeAll(); await manager.closeAll();
}); });
it("retains the newest contiguous messages across repeated budget evictions", async () => {
const { bindings, manager, started } = await startLocalAttachedSession();
expect(started.ok).toBe(true);
if (!started.ok) return;
bindings.device.session.nextPayload = { index: -1 };
const loaded = await manager.loadScript(started.value.sessionId, {
sourceKind: "inline",
source: "send({index: -1});",
});
expect(loaded.ok).toBe(true);
const script = bindings.device.session.scripts[0];
expect(script).toBeDefined();
if (script === undefined) return;
// About 500 of these fit the 16 MiB budget, so 2,000 sends evict and
// compact the queue several times.
const filler = "x".repeat(32_000);
for (let index = 0; index < 2_000; index += 1)
script.message.emit({ type: "send", payload: { index, filler } }, null);
const status = manager.status(started.value.sessionId);
expect(status?.messagesTruncated).toBe(true);
const indexes = (status?.messages ?? []).map(
(message) => (message as { payload: { index: number } }).payload.index,
);
expect(indexes.length).toBeGreaterThan(400);
expect(indexes.at(-1)).toBe(1_999);
expect(indexes).toEqual(
Array.from(
{ length: indexes.length },
(_, offset) => 2_000 - indexes.length + offset,
),
);
await manager.closeAll();
});
it("serializes a final remote release against a new same-endpoint session", async () => { it("serializes a final remote release against a new same-endpoint session", async () => {
const bindings = new FakeDeviceManager(); const bindings = new FakeDeviceManager();
let enteredRemoval: (() => void) | undefined; let enteredRemoval: (() => void) | undefined;
-1
View File
@@ -12,7 +12,6 @@ import type { Result } from "../../domain/result.js";
/** Replaceable offline build-information reader with no target-execution authority. */ /** Replaceable offline build-information reader with no target-execution authority. */
export interface GoBinaryPort { export interface GoBinaryPort {
readonly identity: ProviderIdentity; readonly identity: ProviderIdentity;
close?(): Promise<void>;
inspect( inspect(
input: InspectGoBinaryInput, input: InspectGoBinaryInput,
options?: ExecutionOptions, options?: ExecutionOptions,
-4
View File
@@ -34,10 +34,6 @@ const architectureForSubject = (architecture: string) => {
export class GoBinaryService { export class GoBinaryService {
constructor(readonly provider: GoBinaryPort) {} constructor(readonly provider: GoBinaryPort) {}
close(): Promise<void> {
return this.provider.close?.() ?? Promise.resolve();
}
/** Inspect one explicitly selected file without acquiring a binary session. */ /** Inspect one explicitly selected file without acquiring a binary session. */
async inspect( async inspect(
rawInput: unknown, rawInput: unknown,
@@ -1,6 +1,6 @@
import { createHash } from "node:crypto"; import { createHash } from "node:crypto";
import { moduleResolutionModeForJavaScriptReference } from "../../domain/javascript/javascriptModuleResolutionMode.js"; import { moduleKindForJavaScriptReference } from "./JavaScriptArtifactPathResolution.js";
import { setImmediate } from "node:timers/promises"; import { setImmediate } from "node:timers/promises";
import { resolveJavaScriptSourceMapReferences } from "../../domain/javascript/javascriptSourceMapPaths.js"; import { resolveJavaScriptSourceMapReferences } from "../../domain/javascript/javascriptSourceMapPaths.js";
@@ -384,10 +384,7 @@ function* analyzeArtifactFileSteps<
analysis.parse_status === "complete" || analysis.parse_status === "partial" analysis.parse_status === "complete" || analysis.parse_status === "partial"
? yield* analyzeParsedJavaScriptSemanticsSteps( ? yield* analyzeParsedJavaScriptSemanticsSteps(
parsed, parsed,
moduleResolutionModeForJavaScriptReference( moduleKindForJavaScriptReference(file.path, "static-import"),
file.path,
"static-import",
),
) )
: null; : null;
const projected = const projected =
@@ -66,7 +66,6 @@ export interface AstEdgeInput {
/** Inferred static relationship edge values. */ /** Inferred static relationship edge values. */
export interface InferenceEdgeInput extends AstEdgeInput { export interface InferenceEdgeInput extends AstEdgeInput {
readonly limitations?: readonly string[];
readonly relation: readonly relation:
| "imports" | "imports"
| "loads" | "loads"
@@ -163,9 +162,6 @@ export const addStaticInferenceEdge = (
range: input.range, range: input.range,
operation: "map-static-relationship", operation: "map-static-relationship",
coverage: input.coverage, coverage: input.coverage,
...(input.limitations === undefined
? {}
: { limitations: input.limitations }),
}), }),
}); });
}; };
@@ -351,7 +347,7 @@ export const resolveArtifactPath = (
specifier: string, specifier: string,
sourcePath: string, sourcePath: string,
files: ReadonlyMap<string, JavaScriptArtifactFile>, files: ReadonlyMap<string, JavaScriptArtifactFile>,
moduleKind?: "import" | "require" | "compiler-dependent", moduleKind?: "import" | "require",
): string | null => ): string | null =>
resolveArtifactPathByContext({ resolveArtifactPathByContext({
declaredPath: specifier, declaredPath: specifier,
@@ -12,6 +12,7 @@ import {
javascriptAnalysisCoverage, javascriptAnalysisCoverage,
linkElectronRoleToAsset, linkElectronRoleToAsset,
moduleLookupKey, moduleLookupKey,
resolveArtifactPath,
selfReferenceOmissions, selfReferenceOmissions,
sourceNodeFor, sourceNodeFor,
type JavaScriptArtifactGraphContext, type JavaScriptArtifactGraphContext,
@@ -22,11 +23,9 @@ import {
staticInferenceEvidence, staticInferenceEvidence,
} from "./JavaScriptArtifactGraphEvidence.js"; } from "./JavaScriptArtifactGraphEvidence.js";
import { import {
artifactPathResolutionProperties, moduleKindForJavaScriptReference,
resolveArtifactPathByContext, resolveArtifactPathByContext,
type ArtifactPathResolution,
} from "./JavaScriptArtifactPathResolution.js"; } from "./JavaScriptArtifactPathResolution.js";
import { moduleResolutionModeForJavaScriptReference } from "../../domain/javascript/javascriptModuleResolutionMode.js";
type StaticReference = JavaScriptStaticAnalysis["references"][number]; type StaticReference = JavaScriptStaticAnalysis["references"][number];
type StaticEndpoint = JavaScriptStaticAnalysis["endpoints"][number]; type StaticEndpoint = JavaScriptStaticAnalysis["endpoints"][number];
@@ -42,12 +41,9 @@ interface FindingInput<Value> {
} }
interface ResolvedReference { interface ResolvedReference {
readonly path: string | null; readonly path: string;
readonly file: JavaScriptArtifactFile | null; readonly file: JavaScriptArtifactFile;
readonly node: ApplicationNode; readonly node: ApplicationNode;
readonly limitations: readonly string[];
readonly resolutionStatus?: string;
readonly resolution?: ArtifactPathResolution;
} }
/** Project imports, workers, roles, endpoints, storage, and source-map links. */ /** Project imports, workers, roles, endpoints, storage, and source-map links. */
@@ -94,7 +90,7 @@ const addReference = (
const resolved = const resolved =
input.value.specifier === null input.value.specifier === null
? null ? null
: resolveReference(context, input.file.path, input.value, input.coverage); : resolveReference(context, input.file.path, input.value);
if (input.value.kind === "worker" || input.value.kind === "service-worker") { if (input.value.kind === "worker" || input.value.kind === "service-worker") {
const worker = context.accumulator.addNode({ const worker = context.accumulator.addNode({
kind: input.value.kind === "worker" ? "worker" : "service-worker", kind: input.value.kind === "worker" ? "worker" : "service-worker",
@@ -130,7 +126,7 @@ const addReference = (
relation: "loads", relation: "loads",
properties: { kind: input.value.kind }, properties: { kind: input.value.kind },
}); });
if (resolved !== null && resolved.path !== null) if (resolved !== null)
addStaticInferenceEdge(context, { addStaticInferenceEdge(context, {
source: worker, source: worker,
target: resolved.node, target: resolved.node,
@@ -139,7 +135,6 @@ const addReference = (
coverage: input.coverage, coverage: input.coverage,
relation: "maps_to", relation: "maps_to",
properties: { resolved_path: resolved.path }, properties: { resolved_path: resolved.path },
limitations: resolved.limitations,
}); });
return; return;
} }
@@ -159,25 +154,13 @@ const addReference = (
file: input.file, file: input.file,
range: input.value.location, range: input.value.location,
coverage: input.coverage, coverage: input.coverage,
relation: resolved?.file?.kind === "native-addon" ? "loads" : "imports", relation: resolved?.file.kind === "native-addon" ? "loads" : "imports",
properties: { properties: {
kind: input.value.kind, kind: input.value.kind,
specifier: input.value.specifier, specifier: input.value.specifier,
expression: input.value.expression, expression: input.value.expression,
resolution_mode:
input.value.kind === "static-import" ||
input.value.kind === "dynamic-import" ||
input.value.kind === "require"
? moduleResolutionModeForJavaScriptReference(
input.file.path,
input.value.kind,
)
: null,
resolved_path: resolved?.path ?? null, resolved_path: resolved?.path ?? null,
resolution_status: resolved?.resolutionStatus ?? null,
...artifactPathResolutionProperties(resolved?.resolution),
}, },
...(resolved === null ? {} : { limitations: resolved.limitations }),
}); });
}; };
@@ -390,7 +373,6 @@ const resolveReference = (
context: JavaScriptArtifactGraphContext, context: JavaScriptArtifactGraphContext,
sourcePath: string, sourcePath: string,
reference: StaticReference, reference: StaticReference,
coverage: JavaScriptArtifactGraphCoverage,
): ResolvedReference | null => { ): ResolvedReference | null => {
const specifier = reference.specifier; const specifier = reference.specifier;
if (specifier === null) return null; if (specifier === null) return null;
@@ -410,70 +392,30 @@ const resolveReference = (
const file = context.filesByPath.get(sourcePath); const file = context.filesByPath.get(sourcePath);
return file === undefined return file === undefined
? null ? null
: { : { path: `${sourcePath}#module:${specifier}`, file, node: module };
path: `${sourcePath}#module:${specifier}`,
file,
node: module,
limitations: [],
};
} }
// Worker scripts are URLs relative to the script, not module specifiers. // Worker scripts are URLs relative to the script, not module specifiers.
const resolution = isWorker const path = isWorker
? resolveArtifactPathByContext({ ? resolveArtifactPathByContext({
declaredPath: specifier, declaredPath: specifier,
sourcePath, sourcePath,
context: "url-reference", context: "url-reference",
files: context.filesByPath, files: context.filesByPath,
}) }).resolved_path
: resolveArtifactPathByContext({ : resolveArtifactPath(
declaredPath: specifier, specifier,
sourcePath, sourcePath,
context: "module-specifier", context.filesByPath,
files: context.filesByPath, reference.kind === "require" ||
...("require" === reference.kind || reference.kind === "static-import" ||
"static-import" === reference.kind || reference.kind === "dynamic-import"
"dynamic-import" === reference.kind ? moduleKindForJavaScriptReference(sourcePath, reference.kind)
? { : undefined,
moduleKind: moduleResolutionModeForJavaScriptReference( );
sourcePath, if (path === null) return null;
reference.kind,
),
}
: {}),
});
const path = resolution.resolved_path;
if (path === null) {
if (isWorker) return null;
const source = context.filesByPath.get(sourcePath);
return source === undefined
? null
: {
path: null,
file: null,
node: unresolvedReferenceNode(
context,
source,
reference,
coverage,
resolution,
),
limitations: resolution.limitations,
resolutionStatus: resolution.resolution_status,
resolution,
};
}
const file = context.filesByPath.get(path); const file = context.filesByPath.get(path);
const node = context.fileNodes.get(path); const node = context.fileNodes.get(path);
return file === undefined || node === undefined return file === undefined || node === undefined ? null : { path, file, node };
? null
: {
path,
file,
node,
limitations: resolution.limitations,
resolutionStatus: resolution.resolution_status,
resolution,
};
}; };
const resolveBundlerChunkReference = ( const resolveBundlerChunkReference = (
@@ -511,7 +453,7 @@ const resolveBundlerChunkReference = (
const node = unique[0]; const node = unique[0];
return node === undefined return node === undefined
? null ? null
: { path: `${sourcePath}#chunk:${key}`, file, node, limitations: [] }; : { path: `${sourcePath}#chunk:${key}`, file, node };
}; };
const unresolvedReferenceNode = ( const unresolvedReferenceNode = (
@@ -519,7 +461,6 @@ const unresolvedReferenceNode = (
file: JavaScriptArtifactFile, file: JavaScriptArtifactFile,
reference: StaticReference, reference: StaticReference,
coverage: JavaScriptArtifactGraphCoverage, coverage: JavaScriptArtifactGraphCoverage,
resolution?: ArtifactPathResolution,
): ApplicationNode => ): ApplicationNode =>
context.accumulator.addNode({ context.accumulator.addNode({
kind: "javascript-module", kind: "javascript-module",
@@ -536,10 +477,6 @@ const unresolvedReferenceNode = (
specifier: reference.specifier, specifier: reference.specifier,
expression: reference.expression, expression: reference.expression,
resolution: "unresolved", resolution: "unresolved",
...(resolution === undefined
? {}
: { resolution_status: resolution.resolution_status }),
...artifactPathResolutionProperties(resolution),
}, },
evidence: staticInferenceEvidence({ evidence: staticInferenceEvidence({
sha256: file.sha256, sha256: file.sha256,
@@ -550,7 +487,6 @@ const unresolvedReferenceNode = (
confidence: "low", confidence: "low",
limitations: [ limitations: [
"The static reference could not be resolved within the inventoried artifact.", "The static reference could not be resolved within the inventoried artifact.",
...(resolution?.limitations ?? []),
], ],
}), }),
}, },
@@ -1890,40 +1890,21 @@ describe("TypeScript NodeNext emitted-extension resolution", () => {
).toMatchObject({ resolution_status: "resolved", resolved_path: target }); ).toMatchObject({ resolution_status: "resolved", resolved_path: target });
}, },
); );
});
describe("TypeScript declaration fallback ownership", () => {
const container = "container";
it.each([ it.each([
["./dep.js", "src/dep.d.ts"], ["./dep.js", "src/dep.d.ts"],
["./cfg.mjs", "src/cfg.d.mts"], ["./cfg.mjs", "src/cfg.d.mts"],
["./legacy.cjs", "src/legacy.d.cts"], ["./legacy.cjs", "src/legacy.d.cts"],
] as const)( ] as const)(
"retains %s as compile-time evidence without inventing a runtime target", "resolves %s to a declaration counterpart",
(specifier, target) => { (specifier, target) => {
const files = fileMap([ const files = fileMap([
file("src/main.ts", container), file("src/main.ts", container),
file(target, container, "export const value = 1;"), file(target, container, "export const value = 1;"),
]); ]);
for (const moduleKind of ["import", "require"] as const) {
expect(
resolve({
declaredPath: specifier,
sourcePath: "src/main.ts",
context: "module-specifier",
moduleKind,
files,
}),
).toMatchObject({
resolution_status: "unavailable",
resolved_path: null,
candidate_path: target,
});
}
expect( expect(
resolve({ resolve({
declaredPath: "./" + target.slice(4), declaredPath: specifier,
sourcePath: "src/main.ts", sourcePath: "src/main.ts",
context: "module-specifier", context: "module-specifier",
moduleKind: "import", moduleKind: "import",
@@ -19,13 +19,7 @@ import {
classifyPackageImportTargetString, classifyPackageImportTargetString,
lookupExactPackageImport, lookupExactPackageImport,
} from "../../domain/javascript/javascriptPackageImportAliases.js"; } from "../../domain/javascript/javascriptPackageImportAliases.js";
import { import { typescriptEmittedSourceCandidates } from "../TypeScriptEmittedSourceCandidates.js";
typescriptEmittedSourceCandidates,
type TypeScriptEmittedSourceCandidate,
} from "../TypeScriptEmittedSourceCandidates.js";
import { isRelativeJavaScriptModuleSpecifier } from "../../domain/javascript/javascriptModuleSpecifier.js";
import type { JavaScriptModuleResolutionMode } from "../../domain/javascript/javascriptModuleResolutionMode.js";
import type { JsonValue } from "../../domain/jsonValue.js";
type ArtifactPathResolutionContext = type ArtifactPathResolutionContext =
| "package-entrypoint" | "package-entrypoint"
@@ -44,16 +38,6 @@ interface ArtifactPathResolutionBase {
readonly declared_path: string; readonly declared_path: string;
readonly resolution_context: ArtifactPathResolutionContext; readonly resolution_context: ArtifactPathResolutionContext;
readonly limitations: readonly string[]; readonly limitations: readonly string[];
readonly conditional_resolutions?: readonly ConditionalModuleResolution[];
readonly candidate_path?: string;
}
export interface ConditionalModuleResolution {
readonly mode: "import" | "require";
readonly resolution_status: UnresolvedArtifactPathStatus | "resolved";
readonly resolved_path: string | null;
readonly candidate_path?: string;
readonly limitations: readonly string[];
} }
/** Explicit outcome of artifact-confined path resolution. */ /** Explicit outcome of artifact-confined path resolution. */
@@ -77,13 +61,26 @@ export interface ResolveArtifactPathInput {
readonly files: ReadonlyMap<string, JavaScriptArtifactFile>; readonly files: ReadonlyMap<string, JavaScriptArtifactFile>;
readonly htmlBaseHref?: string | null; readonly htmlBaseHref?: string | null;
/** Import/require semantics; runtime resource paths omit this provenance. */ /** Import/require semantics; runtime resource paths omit this provenance. */
readonly moduleKind?: JavaScriptModuleResolutionMode; readonly moduleKind?: "import" | "require";
} }
/** Loader selected by a source-level JavaScript module reference. */
export const moduleKindForJavaScriptReference = (
sourcePath: string,
referenceKind: "static-import" | "dynamic-import" | "require",
resolutionMode?: "import" | "require",
): "import" | "require" =>
resolutionMode ??
(referenceKind === "require" ||
(referenceKind === "static-import" &&
posix.extname(sourcePath).toLowerCase() === ".cts")
? "require"
: "import");
/** Preserve an explicitly retained loader mode on graph relationships. */ /** Preserve an explicitly retained loader mode on graph relationships. */
export const moduleResolutionModeProperties = ( export const moduleResolutionModeProperties = (
mode: JavaScriptModuleResolutionMode | undefined, mode: "import" | "require" | undefined,
): Readonly<Record<string, JavaScriptModuleResolutionMode>> => ): Readonly<Record<string, "import" | "require">> =>
mode === undefined ? {} : { resolution_mode: mode }; mode === undefined ? {} : { resolution_mode: mode };
const EXTENSIONS = [ const EXTENSIONS = [
@@ -112,116 +109,43 @@ interface PackageFileTarget {
readonly url: URL; readonly url: URL;
} }
type CandidateResolution = { type CandidateResolution =
readonly limitations: readonly string[]; | {
readonly candidatePath?: string; readonly resolvedPath: string;
readonly conditionalResolutions?: readonly ConditionalModuleResolution[]; readonly status: "resolved";
} & ( readonly limitations: readonly string[];
| { readonly resolvedPath: string; readonly status: "resolved" } }
| { | {
readonly resolvedPath: null; readonly resolvedPath: null;
readonly status: UnresolvedArtifactPathStatus; readonly status: UnresolvedArtifactPathStatus;
readonly limitations: readonly string[];
/** A truthy package main failed, so Node must not try ancestor packages. */ /** A truthy package main failed, so Node must not try ancestor packages. */
readonly stopsAncestorLookup?: true; readonly stopsAncestorLookup?: true;
} };
);
/** Resolve one declaration while preserving its original input identity. */ /** Resolve one declaration while preserving its original input identity. */
export const resolveArtifactPathByContext = ( export const resolveArtifactPathByContext = (
input: ResolveArtifactPathInput, input: ResolveArtifactPathInput,
): ArtifactPathResolution => { ): ArtifactPathResolution => {
const resolution = resolvePath(input); const resolution = resolvePath(input);
const facts = {
declared_path: input.declaredPath,
resolution_context: input.context,
limitations: resolution.limitations,
...(resolution.candidatePath === undefined
? {}
: { candidate_path: resolution.candidatePath }),
...(resolution.conditionalResolutions === undefined
? {}
: { conditional_resolutions: resolution.conditionalResolutions }),
};
return resolution.status === "resolved" return resolution.status === "resolved"
? { ? {
...facts, declared_path: input.declaredPath,
resolution_context: input.context,
resolved_path: resolution.resolvedPath, resolved_path: resolution.resolvedPath,
resolution_status: resolution.status, resolution_status: resolution.status,
limitations: resolution.limitations,
} }
: { ...facts, resolved_path: null, resolution_status: resolution.status }; : {
declared_path: input.declaredPath,
resolution_context: input.context,
resolved_path: null,
resolution_status: resolution.status,
limitations: resolution.limitations,
};
}; };
/** Project retained conditional identities without flattening them into display text. */
export const artifactPathResolutionProperties = (
resolution: ArtifactPathResolution | undefined,
): Readonly<Record<string, JsonValue>> => ({
...(resolution?.candidate_path === undefined
? {}
: { candidate_path: resolution.candidate_path }),
...(resolution?.conditional_resolutions === undefined
? {}
: {
conditional_resolutions: resolution.conditional_resolutions.map(
(branch) => ({
mode: branch.mode,
resolution_status: branch.resolution_status,
resolved_path: branch.resolved_path,
limitations: [...branch.limitations],
...(branch.candidate_path === undefined
? {}
: { candidate_path: branch.candidate_path }),
}),
),
}),
});
const conditionalResolution = (
mode: "import" | "require",
result: CandidateResolution,
): ConditionalModuleResolution => ({
mode,
resolution_status: result.status,
resolved_path: result.resolvedPath,
limitations: result.limitations,
...(result.candidatePath === undefined
? {}
: { candidate_path: result.candidatePath }),
});
const resolvePath = (input: ResolveArtifactPathInput): CandidateResolution => { const resolvePath = (input: ResolveArtifactPathInput): CandidateResolution => {
if (input.moduleKind === "compiler-dependent") {
const imported = resolvePath({ ...input, moduleKind: "import" });
const required = resolvePath({ ...input, moduleKind: "require" });
const sameOutcome =
imported.status === required.status &&
imported.resolvedPath === required.resolvedPath &&
imported.candidatePath === required.candidatePath;
const limitation =
"Static TypeScript loader mode is compiler-dependent without retained compiler options. " +
(sameOutcome
? "Node import and require resolution agree on this outcome."
: "Node import and require resolution disagree; neither branch is an established target.");
return sameOutcome
? {
...imported,
limitations: [
...new Set([
...imported.limitations,
...required.limitations,
limitation,
]),
],
}
: {
resolvedPath: null,
status: "unavailable",
limitations: [limitation],
conditionalResolutions: [
conditionalResolution("import", imported),
conditionalResolution("require", required),
],
};
}
// Package import keys are literal lookup identities, not artifact paths. // Package import keys are literal lookup identities, not artifact paths.
// The selected target is admitted separately by resolveExportTarget. // The selected target is admitted separately by resolveExportTarget.
if ( if (
@@ -242,7 +166,7 @@ const resolvePath = (input: ResolveArtifactPathInput): CandidateResolution => {
); );
const isModuleUrlPath = const isModuleUrlPath =
input.context === "module-specifier" && input.context === "module-specifier" &&
(isRelativeJavaScriptModuleSpecifier(input.declaredPath) || (input.declaredPath.startsWith(".") ||
input.declaredPath.startsWith("/") || input.declaredPath.startsWith("/") ||
fileUrlPath(input.declaredPath) !== undefined); fileUrlPath(input.declaredPath) !== undefined);
const emittedSourceCandidates = const emittedSourceCandidates =
@@ -251,47 +175,12 @@ const resolvePath = (input: ResolveArtifactPathInput): CandidateResolution => {
: []; : [];
if (isModuleUrlPath && input.moduleKind === "import") if (isModuleUrlPath && input.moduleKind === "import")
return ( return (
resolveRuntimeSourceCandidates( resolveFileCandidates(input, [confined, ...emittedSourceCandidates]) ??
input, notFoundCandidate()
confined,
emittedSourceCandidates,
) ?? notFoundCandidate()
); );
return resolveCandidate(input, confined, new Set(), emittedSourceCandidates); return resolveCandidate(input, confined, new Set(), emittedSourceCandidates);
}; };
const resolveRuntimeSourceCandidates = (
input: ResolveArtifactPathInput,
exactPath: string,
sourceCandidates: readonly TypeScriptEmittedSourceCandidate[],
): CandidateResolution | null => {
const source = input.files.get(input.sourcePath);
const exactCandidate: TypeScriptEmittedSourceCandidate = {
path: exactPath,
kind: "implementation",
};
for (const candidate of [exactCandidate, ...sourceCandidates]) {
const target = input.files.get(candidate.path);
if (
target === undefined ||
(source !== undefined &&
target.container_sha256 !== source.container_sha256)
)
continue;
return candidate.kind === "declaration"
? {
resolvedPath: null,
status: "unavailable",
candidatePath: candidate.path,
limitations: [
"Only a TypeScript declaration fallback was found; it does not establish the requested runtime module.",
],
}
: { resolvedPath: candidate.path, status: "resolved", limitations: [] };
}
return null;
};
const rejectDeclaration = (declared: string): CandidateResolution | null => { const rejectDeclaration = (declared: string): CandidateResolution | null => {
if (declared.length === 0) if (declared.length === 0)
return unresolvedCandidate("rejected", ["The declared path is empty."]); return unresolvedCandidate("rejected", ["The declared path is empty."]);
@@ -340,10 +229,7 @@ const contextualCandidate = (
return unresolvedCandidate("external", [ return unresolvedCandidate("external", [
"URL and Node builtin schemes are outside static artifact module resolution.", "URL and Node builtin schemes are outside static artifact module resolution.",
]); ]);
if ( if (!declared.startsWith(".") && !declared.startsWith("/"))
!isRelativeJavaScriptModuleSpecifier(declared) &&
!declared.startsWith("/")
)
return bareModuleCandidate(input); return bareModuleCandidate(input);
if (input.moduleKind !== "require") { if (input.moduleKind !== "require") {
const decoded = decodeModuleUrlPath(stripQueryAndFragment(declared)); const decoded = decodeModuleUrlPath(stripQueryAndFragment(declared));
@@ -389,10 +275,7 @@ const contextualCandidate = (
"URL", "URL",
); );
} }
const candidate = relativeCandidate(input.sourcePath, declared); return relativeCandidate(input.sourcePath, declared);
return context === "module-specifier" && input.moduleKind === "import"
? exactUrlPathCandidate(declared, candidate, "module URL")
: candidate;
}; };
const decodeModuleUrlPath = ( const decodeModuleUrlPath = (
@@ -427,12 +310,9 @@ const bareModuleCandidate = (
return unresolvedCandidate("external", [ return unresolvedCandidate("external", [
"The bare specifier is a Node builtin or invalid package name.", "The bare specifier is a Node builtin or invalid package name.",
]); ]);
if ( if (input.moduleKind === "import" && request.packageName.includes("%"))
input.moduleKind === "import" &&
(request.packageName.startsWith(".") || request.packageName.includes("%"))
)
return unresolvedCandidate("rejected", [ return unresolvedCandidate("rejected", [
"ES module package names cannot start with a dot or contain percent signs.", "Percent signs are not admitted in ES module package names.",
]); ]);
const self = packageSelfCandidate(input, request); const self = packageSelfCandidate(input, request);
if (self !== null) return self; if (self !== null) return self;
@@ -855,7 +735,7 @@ const htmlCandidate = (
const exactUrlPathCandidate = ( const exactUrlPathCandidate = (
declared: string, declared: string,
candidate: string, candidate: string,
kind: "HTML" | "URL" | "module URL", kind: "HTML" | "URL",
): string | CandidateResolution => { ): string | CandidateResolution => {
if (!urlPathIsDirectory(declared)) return candidate; if (!urlPathIsDirectory(declared)) return candidate;
const confined = confineCandidate(candidate); const confined = confineCandidate(candidate);
@@ -959,10 +839,12 @@ const resolveCandidate = (
input: ResolveArtifactPathInput, input: ResolveArtifactPathInput,
candidate: string, candidate: string,
packageChain: ReadonlySet<string>, packageChain: ReadonlySet<string>,
emittedSourceCandidates: readonly TypeScriptEmittedSourceCandidate[] = [], emittedSourceCandidates: readonly string[] = [],
): CandidateResolution => ): CandidateResolution =>
resolveFileCandidates(input, fileCandidates(candidate, input.moduleKind)) ?? resolveFileCandidates(input, [
resolveRuntimeSourceCandidates(input, candidate, emittedSourceCandidates) ?? ...fileCandidates(candidate, input.moduleKind),
...emittedSourceCandidates,
]) ??
resolveDirectory( resolveDirectory(
input, input,
candidate, candidate,
@@ -1048,26 +930,11 @@ const resolveFileCandidates = (
(source === undefined || (source === undefined ||
target.container_sha256 === source.container_sha256) target.container_sha256 === source.container_sha256)
) )
return isRuntimeDeclarationPath(input, path) return { resolvedPath: path, status: "resolved", limitations: [] };
? {
resolvedPath: null,
status: "unavailable",
candidatePath: path,
limitations: [
`The TypeScript declaration ${path} is compile-time evidence and cannot identify a runtime module implementation.`,
],
}
: { resolvedPath: path, status: "resolved", limitations: [] };
} }
return null; return null;
}; };
const isRuntimeDeclarationPath = (
input: ResolveArtifactPathInput,
path: string,
): boolean =>
input.moduleKind !== undefined && /\.d\.(?:ts|mts|cts)$/iu.test(path);
const NODE_MODULE_EXTENSIONS = [".js", ".json", ".node"] as const; const NODE_MODULE_EXTENSIONS = [".js", ".json", ".node"] as const;
const fileCandidates = ( const fileCandidates = (
@@ -18,7 +18,6 @@ import type {
JavaScriptModuleOrigin, JavaScriptModuleOrigin,
JavaScriptSemanticModuleLink, JavaScriptSemanticModuleLink,
} from "../../domain/javascript/javascriptSemanticIr.js"; } from "../../domain/javascript/javascriptSemanticIr.js";
import { moduleResolutionModeForJavaScriptReference } from "../../domain/javascript/javascriptModuleResolutionMode.js";
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js"; import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
import { import {
addAstContainsEdge, addAstContainsEdge,
@@ -31,7 +30,7 @@ import {
staticInferenceEvidence, staticInferenceEvidence,
} from "./JavaScriptArtifactGraphEvidence.js"; } from "./JavaScriptArtifactGraphEvidence.js";
import { import {
artifactPathResolutionProperties, moduleKindForJavaScriptReference,
moduleResolutionModeProperties, moduleResolutionModeProperties,
resolveArtifactPathByContext, resolveArtifactPathByContext,
type ArtifactPathResolution, type ArtifactPathResolution,
@@ -55,7 +54,6 @@ interface RelationshipInput {
type ResolvedModuleTarget = { type ResolvedModuleTarget = {
readonly node: ApplicationNode; readonly node: ApplicationNode;
readonly limitations: readonly string[]; readonly limitations: readonly string[];
readonly resolution?: ArtifactPathResolution;
} & ( } & (
| { | {
readonly path: string; readonly path: string;
@@ -83,13 +81,8 @@ export const addJavaScriptSourceModules = (
if (asset === undefined || semantic === null || program === undefined) if (asset === undefined || semantic === null || program === undefined)
continue; continue;
const exports = semantic.ir.moduleLinks const exports = semantic.ir.moduleLinks
.flatMap(({ exportedName, kind }) => .flatMap(({ exportedName }) =>
exportedName === null || kind === "type-import" ? [] : [exportedName], exportedName === null ? [] : [exportedName],
)
.sort(compareUnicodeCodePoints);
const typeExports = semantic.ir.moduleLinks
.flatMap(({ exportedName, kind }) =>
exportedName === null || kind !== "type-import" ? [] : [exportedName],
) )
.sort(compareUnicodeCodePoints); .sort(compareUnicodeCodePoints);
const retainedExports = [...new Set(exports)]; const retainedExports = [...new Set(exports)];
@@ -112,12 +105,8 @@ export const addJavaScriptSourceModules = (
import_relationships: semantic.ir.moduleLinks.filter( import_relationships: semantic.ir.moduleLinks.filter(
({ kind }) => kind === "import" || kind === "require", ({ kind }) => kind === "import" || kind === "require",
).length, ).length,
type_relationships: semantic.ir.moduleLinks.filter(
({ kind }) => kind === "type-import",
).length,
export_relationships: exports.length, export_relationships: exports.length,
export_names: retainedExports, export_names: retainedExports,
type_export_names: [...new Set(typeExports)],
omitted_export_names: 0, omitted_export_names: 0,
}, },
evidence: astObservationEvidence({ evidence: astObservationEvidence({
@@ -163,7 +152,6 @@ export function* addJavaScriptModuleRelationshipsSteps(
for (const link of semantic.ir.moduleLinks) { for (const link of semantic.ir.moduleLinks) {
// One module can export thousands of bindings, each with return shapes. // One module can export thousands of bindings, each with return shapes.
yield; yield;
if (link.kind === "type-import") continue;
const input = { context, file, semantic, source, link }; const input = { context, file, semantic, source, link };
if (isExportLink(link)) addExportRelationship(input, omissions); if (isExportLink(link)) addExportRelationship(input, omissions);
else if (link.specifier !== null) else if (link.specifier !== null)
@@ -289,7 +277,7 @@ const resolveModuleTarget = (
sourcePath: input.file.path, sourcePath: input.file.path,
context: "module-specifier", context: "module-specifier",
files: input.context.filesByPath, files: input.context.filesByPath,
moduleKind: moduleResolutionModeForJavaScriptReference( moduleKind: moduleKindForJavaScriptReference(
input.file.path, input.file.path,
input.link.kind === "require" || input.link.kind === "commonjs-export" input.link.kind === "require" || input.link.kind === "commonjs-export"
? "require" ? "require"
@@ -299,12 +287,15 @@ const resolveModuleTarget = (
}); });
if (resolution.resolution_status !== "resolved") if (resolution.resolution_status !== "resolved")
return { return {
node: unresolvedModuleNode(input, specifier, resolution), node: unresolvedModuleNode(
input,
specifier,
resolution.resolution_status,
),
path: null, path: null,
file: null, file: null,
status: resolution.resolution_status, status: resolution.resolution_status,
limitations: resolution.limitations, limitations: resolution.limitations,
resolution,
}; };
const path = resolution.resolved_path; const path = resolution.resolved_path;
const file = input.context.filesByPath.get(path); const file = input.context.filesByPath.get(path);
@@ -321,24 +312,20 @@ const resolveModuleTarget = (
file, file,
status: "resolved", status: "resolved",
limitations: resolution.limitations, limitations: resolution.limitations,
resolution,
}; };
}; };
const unresolvedModuleNode = ( const unresolvedModuleNode = (
input: RelationshipInput, input: RelationshipInput,
specifier: string, specifier: string,
resolution: Extract< status: Exclude<ResolvedModuleTarget["status"], "resolved">,
ArtifactPathResolution,
{ resolution_status: Exclude<ResolvedModuleTarget["status"], "resolved"> }
>,
): ApplicationNode => ): ApplicationNode =>
input.context.accumulator.addNode({ input.context.accumulator.addNode({
kind: "javascript-module", kind: "javascript-module",
identity: artifactLocalIdentity( identity: artifactLocalIdentity(
input.file.sha256, input.file.sha256,
"unresolved-semantic-module", "unresolved-semantic-module",
`${resolution.resolution_status}:${specifier}`, `${status}:${specifier}`,
), ),
observations: [ observations: [
{ {
@@ -346,8 +333,7 @@ const unresolvedModuleNode = (
properties: { properties: {
semantic_role: "module-reference", semantic_role: "module-reference",
declared_specifier: specifier, declared_specifier: specifier,
resolution_status: resolution.resolution_status, resolution_status: status,
...artifactPathResolutionProperties(resolution),
}, },
evidence: staticInferenceEvidence({ evidence: staticInferenceEvidence({
sha256: input.file.sha256, sha256: input.file.sha256,
@@ -368,9 +354,6 @@ const moduleOriginForExport = (
): JavaScriptModuleOrigin | null => { ): JavaScriptModuleOrigin | null => {
if (link.specifier !== null) if (link.specifier !== null)
return { return {
...(link.resolutionMode === undefined
? {}
: { resolutionMode: link.resolutionMode }),
specifier: link.specifier, specifier: link.specifier,
importedPath: link.importedName === null ? [] : [link.importedName], importedPath: link.importedName === null ? [] : [link.importedName],
}; };
@@ -400,7 +383,6 @@ const relationshipProperties = (
exported_name: link.exportedName, exported_name: link.exportedName,
resolved_path: target?.path ?? null, resolved_path: target?.path ?? null,
resolution_status: target?.status ?? null, resolution_status: target?.status ?? null,
...artifactPathResolutionProperties(target?.resolution),
target_file_kind: target?.file?.kind ?? null, target_file_kind: target?.file?.kind ?? null,
target_json_status: target_json_status:
target?.file?.kind === "json" || target?.file?.kind === "package-json" target?.file?.kind === "json" || target?.file?.kind === "package-json"
@@ -449,7 +431,6 @@ const moduleFormat = (
const extension = posix.extname(path).toLowerCase(); const extension = posix.extname(path).toLowerCase();
if (extension === ".mjs" || extension === ".mts") return "esm"; if (extension === ".mjs" || extension === ".mts") return "esm";
if (extension === ".cjs" || extension === ".cts") return "commonjs"; if (extension === ".cjs" || extension === ".cts") return "commonjs";
if (extension === ".ts" || extension === ".tsx") return "unknown";
const esm = ir.moduleLinks.some(({ kind }) => const esm = ir.moduleLinks.some(({ kind }) =>
["import", "export", "re-export"].includes(kind), ["import", "export", "re-export"].includes(kind),
); );
@@ -1,9 +1,3 @@
import {
inspectSignalDecodersInputSchema,
signalDecodersSchema,
type InspectSignalDecodersInput,
} from "../../domain/signals/signalDecoders.js";
import type { JsonValue } from "../../domain/jsonValue.js";
import { isAbsolute } from "node:path"; import { isAbsolute } from "node:path";
import type { import type {
AnalysisExecution, AnalysisExecution,
@@ -30,10 +24,6 @@ export interface SignalRecordingPort {
input: DecodeSignalRecordingInput, input: DecodeSignalRecordingInput,
options?: ExecutionOptions, options?: ExecutionOptions,
): Promise<Result<AnalysisExecution, AnalysisError>>; ): Promise<Result<AnalysisExecution, AnalysisError>>;
inspectDecoders(
input: InspectSignalDecodersInput,
options?: ExecutionOptions,
): Promise<Result<AnalysisExecution, AnalysisError>>;
close(): Promise<void>; close(): Promise<void>;
} }
@@ -84,58 +74,14 @@ export class SignalRecordingService {
"Decoder returned malformed data or changed the selected recording identity/options.", "Decoder returned malformed data or changed the selected recording identity/options.",
), ),
); );
return this.#evidence(operation, input.data, execution, options); const evidence = createEvidence(execution.subject, execution.provider, {
} operation,
async inspectDecoders( parameters: input.data,
raw: unknown, result: report.data,
options?: ExecutionOptions, confidence: "observed",
): Promise<Result<Evidence, AnalysisError>> { locations: execution.locations,
const operation = "inspect_signal_decoders"; limitations: execution.limitations,
if (options?.signal?.aborted) });
return err(new AnalysisCancelledError(operation));
const input = inspectSignalDecodersInputSchema.safeParse(raw);
if (!input.success)
return err(
analysisInputErrorFromIssues(operation, input.error.issues, raw, {
cause: input.error,
}),
);
const decoded = await this.provider.inspectDecoders(input.data, options);
if (!decoded.ok) return decoded;
const execution = decoded.value;
const report = signalDecodersSchema.safeParse(execution.result);
if (
!report.success ||
JSON.stringify(report.data.selection) !== JSON.stringify(input.data) ||
execution.subject?.path !== report.data.tool.path ||
execution.subject.sha256 !== report.data.tool.sha256
)
return err(
new AnalysisOutputError(
operation,
"Decoder discovery returned malformed data or changed the selected installation/options.",
),
);
return this.#evidence(operation, input.data, execution, options);
}
#evidence(
operation: string,
parameters: Readonly<Record<string, JsonValue>>,
execution: AnalysisExecution,
options?: ExecutionOptions,
): Result<Evidence, AnalysisError> {
const evidence = createEvidence(
execution.subject ?? undefined,
execution.provider,
{
operation,
parameters,
result: execution.result,
confidence: "observed",
locations: execution.locations,
limitations: execution.limitations,
},
);
return options?.signal?.aborted return options?.signal?.aborted
? err( ? err(
new AnalysisCancelledError(operation, { new AnalysisCancelledError(operation, {
-4
View File
@@ -57,10 +57,6 @@ export class ArtifactResourceScope {
#draining: Promise<void> | undefined; #draining: Promise<void> | undefined;
#closed = false; #closed = false;
get hasPending(): boolean {
return this.#pending.size > 0;
}
/** Admit a resource operation while the scope remains open. */ /** Admit a resource operation while the scope remains open. */
async run<T>(operation: () => Promise<T>): Promise<T> { async run<T>(operation: () => Promise<T>): Promise<T> {
while (true) { while (true) {
+4 -8
View File
@@ -8,11 +8,11 @@ import {
type NativeCommandRunner, type NativeCommandRunner,
} from "../native/CommandRunner.js"; } from "../native/CommandRunner.js";
import { import {
lipoSubtypeBits,
parseLipoArchitectures, parseLipoArchitectures,
type LipoArchitecture, type LipoArchitecture,
} from "../native/parsers/lipo.js"; } from "../native/parsers/lipo.js";
import type { MachoSlice } from "../domain/apple/dylibResolution.js"; import type { MachoSlice } from "../domain/apple/dylibResolution.js";
import { machoCpuSubtypeBase } from "../domain/apple/machoCpuIdentity.js";
import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js"; import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
import { readMachoImage, type ReadAt } from "./apple/MachoLoadCommandReader.js"; import { readMachoImage, type ReadAt } from "./apple/MachoLoadCommandReader.js";
import { import {
@@ -81,9 +81,7 @@ export class MachOSliceArtifactReader implements ArtifactReader {
if (!captured.ok) if (!captured.ok)
throw new ArtifactReaderFailure( throw new ArtifactReaderFailure(
captured.error.reason === "cancelled" ? "cancelled" : "unavailable", captured.error.reason === "cancelled" ? "cancelled" : "unavailable",
captured.error.detail === null "lipo could not enumerate universal Mach-O slices",
? "lipo could not enumerate universal Mach-O slices"
: `lipo could not enumerate universal Mach-O slices: ${captured.error.detail}`,
{ {
cause: captured.error, cause: captured.error,
...(captured.error.cleanupFailure === null && ...(captured.error.cleanupFailure === null &&
@@ -471,10 +469,8 @@ const lipoSubtypeMatches = (
): boolean => { ): boolean => {
if (reported.cpu_subtype_code !== null) if (reported.cpu_subtype_code !== null)
return reported.cpu_subtype_code === observedCpuSubtype; return reported.cpu_subtype_code === observedCpuSubtype;
return ( const bits = lipoSubtypeBits(reported.cpu_subtype);
reported.cpu_subtype_base === null || return bits === null || bits === (observedCpuSubtype & 0x00ffffff);
reported.cpu_subtype_base === machoCpuSubtypeBase(observedCpuSubtype)
);
}; };
/** /**
+11 -22
View File
@@ -280,11 +280,9 @@ export class SafeOutputTree {
} }
/** Sync the owned output tree and prevent further writes through this instance. */ /** Sync the owned output tree and prevent further writes through this instance. */
async commit(signal?: AbortSignal): Promise<void> { async commit(): Promise<void> {
this.#assertWritable(); this.#assertWritable();
abortIfNeeded(signal);
await this.#assertOwnedDirectories(); await this.#assertOwnedDirectories();
abortIfNeeded(signal);
// Windows has no directory fsync; file contents are already synced in write(). // Windows has no directory fsync; file contents are already synced in write().
if (this.platform === "win32") { if (this.platform === "win32") {
this.#published = true; this.#published = true;
@@ -317,7 +315,6 @@ export class SafeOutputTree {
); );
await output.handle.sync(); await output.handle.sync();
await parent.handle.sync(); await parent.handle.sync();
abortIfNeeded(signal);
this.#published = true; this.#published = true;
} catch (cause: unknown) { } catch (cause: unknown) {
sync = { kind: "failed", cause }; sync = { kind: "failed", cause };
@@ -339,26 +336,23 @@ export class SafeOutputTree {
closeFailure, closeFailure,
this.#outputRoot, this.#outputRoot,
); );
if (sync.kind === "failed") { if (sync.kind === "failed")
const primary = isCancellationFailure(sync.cause) throw ArtifactReaderFailure.withCleanup(
? sync.cause new ArtifactReaderFailure(
: new ArtifactReaderFailure( "path",
"path", "Could not durably sync extraction output",
"Could not durably sync extraction output", { cause: sync.cause },
{ cause: sync.cause }, ),
); cleanup,
throw ArtifactReaderFailure.withCleanup(primary, cleanup); );
}
throw ArtifactReaderFailure.withCleanup(closeFailure, cleanup); throw ArtifactReaderFailure.withCleanup(closeFailure, cleanup);
} }
if (sync.kind === "failed") { if (sync.kind === "failed")
if (isCancellationFailure(sync.cause)) throw sync.cause;
throw new ArtifactReaderFailure( throw new ArtifactReaderFailure(
"path", "path",
"Could not durably sync extraction output", "Could not durably sync extraction output",
{ cause: sync.cause }, { cause: sync.cause },
); );
}
} }
/** Remove only this operation's unsealed tree and verify absence. */ /** Remove only this operation's unsealed tree and verify absence. */
@@ -596,11 +590,6 @@ const abortIfNeeded = (signal?: AbortSignal): void => {
); );
}; };
const isCancellationFailure = (
cause: unknown,
): cause is ArtifactReaderFailure =>
cause instanceof ArtifactReaderFailure && cause.reason === "cancelled";
const isAlreadyExists = (cause: unknown): boolean => const isAlreadyExists = (cause: unknown): boolean =>
cause instanceof Error && "code" in cause && cause.code === "EEXIST"; cause instanceof Error && "code" in cause && cause.code === "EEXIST";
+10 -4
View File
@@ -17,7 +17,6 @@ import { DirectoryArtifactReader } from "../DirectoryArtifactReader.js";
import { SafeOutputTree } from "../SafeOutputTree.js"; import { SafeOutputTree } from "../SafeOutputTree.js";
import { SafeOutputTreeCreationFailure } from "../SafeOutputTreeCreationFailure.js"; import { SafeOutputTreeCreationFailure } from "../SafeOutputTreeCreationFailure.js";
import { ZipArtifactReader } from "../ZipArtifactReader.js"; import { ZipArtifactReader } from "../ZipArtifactReader.js";
import { isZipPackageFormat } from "../../domain/zipPackageFormat.js";
import { MachOSliceArtifactReader } from "../MachOSliceArtifactReader.js"; import { MachOSliceArtifactReader } from "../MachOSliceArtifactReader.js";
import { import {
artifactExtractionResultSchema, artifactExtractionResultSchema,
@@ -249,7 +248,7 @@ const materializeSelection = async ({
compareUnicodeCodePoints(left.relative_path, right.relative_path), compareUnicodeCodePoints(left.relative_path, right.relative_path),
); );
completed = createExtractionResult(input, inventory, selected, extracted); completed = createExtractionResult(input, inventory, selected, extracted);
await output.commit(signal); await output.commit();
return completed; return completed;
} catch (cause: unknown) { } catch (cause: unknown) {
const cleanupFailures = await releaseExtractionOwners( const cleanupFailures = await releaseExtractionOwners(
@@ -401,6 +400,13 @@ interface ExtractionInventory {
readonly integrityContradictions: readonly IntegrityContradiction[]; readonly integrityContradictions: readonly IntegrityContradiction[];
} }
const ZIP_FORMATS = ["ipa", "apk", "msix", "appx", "zip"] as const;
const isZipFormat = (
format: BinaryTarget["format"],
): format is (typeof ZIP_FORMATS)[number] =>
(ZIP_FORMATS as readonly string[]).includes(format);
/** /**
* Refuse a format without an extraction reader before inventory work starts. * Refuse a format without an extraction reader before inventory work starts.
* The target kind, not the host, is unsupported by extraction. The error keeps * The target kind, not the host, is unsupported by extraction. The error keeps
@@ -416,7 +422,7 @@ const requireExtractableFormat = async (
!directory && !directory &&
format !== "asar" && format !== "asar" &&
!isMachOFormat(format) && !isMachOFormat(format) &&
!isZipPackageFormat(format) !isZipFormat(format)
) )
throw new AnalysisUnsupportedTargetError( throw new AnalysisUnsupportedTargetError(
"extract_artifact", "extract_artifact",
@@ -437,7 +443,7 @@ const createReader = async (
if (await requireExtractableFormat(path, input)) if (await requireExtractableFormat(path, input))
return new DirectoryArtifactReader(path); return new DirectoryArtifactReader(path);
if (format === "asar") return new AsarArtifactReader(path); if (format === "asar") return new AsarArtifactReader(path);
if (isZipPackageFormat(format)) return new ZipArtifactReader(path, format); if (isZipFormat(format)) return new ZipArtifactReader(path, format);
return new MachOSliceArtifactReader(path, input.environment); return new MachOSliceArtifactReader(path, input.environment);
}; };
+2 -10
View File
@@ -38,13 +38,6 @@ const afterValidation = async <T>(
} }
}; };
const assertReadActive = (signal: AbortSignal | undefined): void => {
if (signal?.aborted)
throw new ArtifactReaderFailure("cancelled", "Artifact read cancelled", {
cause: signal.reason,
});
};
/** Read one bounded regular file and reject replacement or in-place changes. */ /** Read one bounded regular file and reject replacement or in-place changes. */
export const readStableArtifact = async ( export const readStableArtifact = async (
path: string, path: string,
@@ -53,7 +46,7 @@ export const readStableArtifact = async (
fileSystem: StableArtifactFileSystem = FILE_SYSTEM, fileSystem: StableArtifactFileSystem = FILE_SYSTEM,
resourceScope?: ArtifactResourceScope, resourceScope?: ArtifactResourceScope,
): Promise<{ readonly bytes: Buffer; readonly sha256: string }> => { ): Promise<{ readonly bytes: Buffer; readonly sha256: string }> => {
assertReadActive(signal); signal?.throwIfAborted();
const before = await fileSystem.lstat(path); const before = await fileSystem.lstat(path);
if (!before.isFile() || before.isSymbolicLink()) if (!before.isFile() || before.isSymbolicLink())
throw new ArtifactReaderFailure( throw new ArtifactReaderFailure(
@@ -91,7 +84,7 @@ export const readStableArtifact = async (
autoClose: false, autoClose: false,
highWaterMark: 65536, highWaterMark: 65536,
})) { })) {
assertReadActive(signal); signal?.throwIfAborted();
const value: unknown = chunk; const value: unknown = chunk;
if (!Buffer.isBuffer(value)) if (!Buffer.isBuffer(value))
throw new ArtifactReaderFailure( throw new ArtifactReaderFailure(
@@ -127,7 +120,6 @@ export const readStableArtifact = async (
"integrity", "integrity",
`Artifact changed during read: ${path}`, `Artifact changed during read: ${path}`,
); );
assertReadActive(signal);
const bytes = Buffer.concat(chunks, count); const bytes = Buffer.concat(chunks, count);
return { bytes, sha256: createHash("sha256").update(bytes).digest("hex") }; return { bytes, sha256: createHash("sha256").update(bytes).digest("hex") };
} catch (cause: unknown) { } catch (cause: unknown) {
@@ -91,7 +91,7 @@ export const browserScenarioCaptureData = (input: {
network_content: input.scenario.capture.network, network_content: input.scenario.capture.network,
event_families: [...new Set(input.scenario.capture.events)].sort(), event_families: [...new Set(input.scenario.capture.events)].sort(),
}, },
duration_ms: Math.max(0, Math.round(performance.now() - input.startedAt)), duration_ms: Date.now() - input.startedAt,
steps: [...input.steps], steps: [...input.steps],
events, events,
limitations: [ limitations: [
@@ -303,7 +303,6 @@ describe("PlaywrightBrowserScenarioProvider", () => {
}); });
expect(result.value.limitations).toContain(session.eventGaps[0]); expect(result.value.limitations).toContain(session.eventGaps[0]);
}); });
it("returns an initial state and every step beyond the former action ceiling", async () => { it("returns an initial state and every step beyond the former action ceiling", async () => {
const session = new FakeSession("launch"); const session = new FakeSession("launch");
const provider = new PlaywrightBrowserScenarioProvider(() => const provider = new PlaywrightBrowserScenarioProvider(() =>
@@ -377,35 +376,6 @@ describe("PlaywrightBrowserScenarioProvider", () => {
}); });
}); });
describe("PlaywrightBrowserScenarioProvider timing", () => {
it("keeps capture elapsed values valid across a backward wall-clock jump", async () => {
let wallClock = 10_000;
vi.spyOn(Date, "now").mockImplementation(() => wallClock);
const session = new FakeSession("launch");
session.onPerform = () => {
wallClock = 1;
};
const provider = new PlaywrightBrowserScenarioProvider(() =>
Promise.resolve(session),
);
try {
const result = await provider.captureScenario(scenario());
if (!result.ok) throw result.error;
expect(wallClock).toBe(1);
expect(Number.isInteger(result.value.duration_ms)).toBe(true);
expect(result.value.duration_ms).toBeGreaterThanOrEqual(0);
expect(
result.value.steps.every(
({ elapsed_ms }) => Number.isInteger(elapsed_ms) && elapsed_ms >= 0,
),
).toBe(true);
} finally {
vi.restoreAllMocks();
}
});
});
describe("PlaywrightBrowserScenarioProvider comparison coverage", () => { describe("PlaywrightBrowserScenarioProvider comparison coverage", () => {
it("keeps quiet captures with different event families from comparing unchanged", async () => { it("keeps quiet captures with different event families from comparing unchanged", async () => {
const capture = async (events: readonly BrowserScenarioEventFamily[]) => { const capture = async (events: readonly BrowserScenarioEventFamily[]) => {
@@ -172,7 +172,7 @@ const executeStep = async (input: {
), ),
}); });
const actionStartedAt = performance.now(); const actionStartedAt = Date.now();
session.setStep(stepIndex); session.setStep(stepIndex);
const requestCancelled = (): boolean => signal?.aborted === true; const requestCancelled = (): boolean => signal?.aborted === true;
let outcome: BrowserScenarioStepOutcome = { let outcome: BrowserScenarioStepOutcome = {
@@ -200,7 +200,7 @@ const executeStep = async (input: {
stepId: action.step_id, stepId: action.step_id,
action: action.action, action: action.action,
...outcome, ...outcome,
elapsedMs: Math.max(0, Math.round(performance.now() - actionStartedAt)), elapsedMs: Date.now() - actionStartedAt,
beforeUrl, beforeUrl,
afterUrl: session.sanitizeUrl(session.currentUrl()), afterUrl: session.sanitizeUrl(session.currentUrl()),
eventStart, eventStart,
@@ -233,7 +233,7 @@ const runScenario = async (
): Promise<BrowserScenarioCapture> => { ): Promise<BrowserScenarioCapture> => {
if (options.signal?.aborted === true) if (options.signal?.aborted === true)
throw new BrowserObservationError(OPERATION, "cancelled"); throw new BrowserObservationError(OPERATION, "cancelled");
const startedAt = performance.now(); const startedAt = Date.now();
const session = await openSession(scenario, { const session = await openSession(scenario, {
...(options.signal === undefined ? {} : { signal: options.signal }), ...(options.signal === undefined ? {} : { signal: options.signal }),
retainCleanup, retainCleanup,
@@ -250,7 +250,7 @@ const runScenario = async (
await initialStep({ await initialStep({
session, session,
scenario, scenario,
elapsedMs: Math.max(0, Math.round(performance.now() - startedAt)), elapsedMs: Date.now() - startedAt,
signal: options.signal, signal: options.signal,
}), }),
); );
+4 -10
View File
@@ -72,7 +72,7 @@ export const runElectronActions = async (
): Promise<ElectronActions> => { ): Promise<ElectronActions> => {
const actions: ElectronActions = []; const actions: ElectronActions = [];
for (const action of input.actions) { for (const action of input.actions) {
const actionStartedAt = performance.now(); const actionStartedAt = Date.now();
const windowIndex = actionWindowIndex(action); const windowIndex = actionWindowIndex(action);
let selectedWindow: Page | undefined; let selectedWindow: Page | undefined;
try { try {
@@ -107,10 +107,7 @@ export const runElectronActions = async (
target: target:
selectedWindow === undefined ? null : `window:${String(windowIndex)}`, selectedWindow === undefined ? null : `window:${String(windowIndex)}`,
status: "completed", status: "completed",
elapsed_ms: Math.max( elapsed_ms: Date.now() - actionStartedAt,
0,
Math.round(performance.now() - actionStartedAt),
),
error: null, error: null,
}); });
} catch (cause: unknown) { } catch (cause: unknown) {
@@ -121,10 +118,7 @@ export const runElectronActions = async (
target: target:
selectedWindow === undefined ? null : `window:${String(windowIndex)}`, selectedWindow === undefined ? null : `window:${String(windowIndex)}`,
status: options.signal?.aborted ? "cancelled" : "failed", status: options.signal?.aborted ? "cancelled" : "failed",
elapsed_ms: Math.max( elapsed_ms: Date.now() - actionStartedAt,
0,
Math.round(performance.now() - actionStartedAt),
),
error: safeActionErrorMessage(cause), error: safeActionErrorMessage(cause),
}); });
break; break;
@@ -372,7 +366,7 @@ export const runWithExecutionLimits = async <Value>(
new Promise<never>((_, reject) => { new Promise<never>((_, reject) => {
timer = setTimeout( timer = setTimeout(
() => reject(new BrowserObservationError(OPERATION, "timeout")), () => reject(new BrowserObservationError(OPERATION, "timeout")),
Math.max(1, deadline - performance.now()), Math.max(1, deadline - Date.now()),
); );
}), }),
); );
@@ -86,7 +86,7 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati
if (options.signal?.aborted === true) if (options.signal?.aborted === true)
throw new BrowserObservationError(OPERATION, "cancelled"); throw new BrowserObservationError(OPERATION, "cancelled");
await prepareProcessOwnershipInspection(options.signal); await prepareProcessOwnershipInspection(options.signal);
const startupDeadline = performance.now() + STARTUP_TIMEOUT_MS; const startupDeadline = Date.now() + STARTUP_TIMEOUT_MS;
paths = { paths = {
executable: input.executable_path, executable: input.executable_path,
application: input.application_path, application: input.application_path,
@@ -101,7 +101,7 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati
cwd: paths.root, cwd: paths.root,
env: safeElectronEnvironment(this.environment, runId), env: safeElectronEnvironment(this.environment, runId),
args: ["-r", hookPath, paths.application, ...input.args], args: ["-r", hookPath, paths.application, ...input.args],
timeout: Math.max(1, startupDeadline - performance.now()), timeout: Math.max(1, startupDeadline - Date.now()),
}); });
const leaderPid = application.process().pid; const leaderPid = application.process().pid;
if ( if (
@@ -128,7 +128,7 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati
const state = await runWithExecutionLimits( const state = await runWithExecutionLimits(
readApplicationState(application), readApplicationState(application),
undefined, undefined,
options.signal?.aborted ? performance.now() + 5_000 : undefined, options.signal?.aborted ? Date.now() + 5_000 : undefined,
); );
capture = projectElectronActiveCapture(paths, actions, state); capture = projectElectronActiveCapture(paths, actions, state);
outcome = options.signal?.aborted outcome = options.signal?.aborted
@@ -187,7 +187,7 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati
await runWithExecutionLimits( await runWithExecutionLimits(
application.close(), application.close(),
undefined, undefined,
performance.now() + 5_000, Date.now() + 5_000,
); );
} catch (cause: unknown) { } catch (cause: unknown) {
closeError = cause; closeError = cause;
@@ -7,10 +7,7 @@ import {
} from "../domain/analysisErrorCore.js"; } from "../domain/analysisErrorCore.js";
describe("withPlaywrightExecutionBoundary", () => { describe("withPlaywrightExecutionBoundary", () => {
afterEach(() => { afterEach(() => vi.useRealTimers());
vi.useRealTimers();
vi.restoreAllMocks();
});
it("rejects work that exceeds its absolute remaining duration", async () => { it("rejects work that exceeds its absolute remaining duration", async () => {
vi.useFakeTimers(); vi.useFakeTimers();
@@ -34,20 +31,4 @@ describe("withPlaywrightExecutionBoundary", () => {
controller.abort(); controller.abort();
await expect(result).rejects.toBeInstanceOf(AnalysisCancelledError); await expect(result).rejects.toBeInstanceOf(AnalysisCancelledError);
}); });
it("keeps its timeout stable across a wall-clock jump", async () => {
vi.spyOn(Date, "now")
.mockReturnValueOnce(1_000)
.mockReturnValue(10_000_000);
await expect(
withPlaywrightExecutionBoundary(
() =>
new Promise<string>((resolve) =>
setTimeout(() => resolve("done"), 5),
),
1_000,
),
).resolves.toBe("done");
});
}); });
+2 -2
View File
@@ -19,10 +19,10 @@ export const withPlaywrightExecutionBoundary = <Value>(
let settled = false; let settled = false;
let timer: ReturnType<typeof setTimeout> | undefined; let timer: ReturnType<typeof setTimeout> | undefined;
const deadline = const deadline =
timeoutMs === undefined ? undefined : performance.now() + timeoutMs; timeoutMs === undefined ? undefined : Date.now() + timeoutMs;
const scheduleTimeout = (): void => { const scheduleTimeout = (): void => {
if (deadline === undefined) return; if (deadline === undefined) return;
const remaining = deadline - performance.now(); const remaining = deadline - Date.now();
if (remaining <= 0) { if (remaining <= 0) {
fail(new AnalysisTimeoutError(OPERATION, timeoutMs ?? 0)); fail(new AnalysisTimeoutError(OPERATION, timeoutMs ?? 0));
return; return;
+1 -1
View File
@@ -133,7 +133,7 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
const secrets = BrowserScenarioSecrets.resolve(scenario, environment); const secrets = BrowserScenarioSecrets.resolve(scenario, environment);
if (secrets === undefined) if (secrets === undefined)
throw new BrowserObservationError(OPERATION, "secret_unavailable"); throw new BrowserObservationError(OPERATION, "secret_unavailable");
const startedAt = performance.now(); const startedAt = Date.now();
const opening = openPlaywrightScenarioBrowser(scenario, environment, { const opening = openPlaywrightScenarioBrowser(scenario, environment, {
signal: options.signal, signal: options.signal,
...(options.retainCleanup === undefined ...(options.retainCleanup === undefined
+3 -3
View File
@@ -86,7 +86,7 @@ export const fetchWebSourceMaps = async (
signal?.addEventListener("abort", abortFromCaller, { once: true }); signal?.addEventListener("abort", abortFromCaller, { once: true });
const operationSignal = operationController.signal; const operationSignal = operationController.signal;
const deadlineAt = const deadlineAt =
performance.now() + (host.timeoutMs ?? SOURCE_MAP_FETCH_TIMEOUT_MS); Date.now() + (host.timeoutMs ?? SOURCE_MAP_FETCH_TIMEOUT_MS);
const budget = { const budget = {
retainedBytes: 0, retainedBytes: 0,
deadlineAt, deadlineAt,
@@ -110,7 +110,7 @@ export const fetchWebSourceMaps = async (
} }
const expandedBytesBeforeItem = budget.decodedRecords.expandedBytes; const expandedBytesBeforeItem = budget.decodedRecords.expandedBytes;
budget.decodedRecords.expandedBytes += reservation; budget.decodedRecords.expandedBytes += reservation;
if (operationSignal.aborted || performance.now() >= deadlineAt) { if (operationSignal.aborted || Date.now() >= deadlineAt) {
items.push( items.push(
emptySourceMapItem( emptySourceMapItem(
request, request,
@@ -789,7 +789,7 @@ const checkOperation = (
): void => { ): void => {
if (callerSignal?.aborted === true) throw callerSignal.reason; if (callerSignal?.aborted === true) throw callerSignal.reason;
if (operationSignal?.aborted === true) throw operationSignal.reason; if (operationSignal?.aborted === true) throw operationSignal.reason;
if (performance.now() >= deadlineAt) throw new SourceMapDeadlineError(); if (Date.now() >= deadlineAt) throw new SourceMapDeadlineError();
}; };
const approvedUrl = ( const approvedUrl = (
+1 -31
View File
@@ -1,7 +1,6 @@
import { lstat, realpath } from "node:fs/promises"; import { lstat, realpath } from "node:fs/promises";
import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path"; import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js"; import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js";
import { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js";
import { readStableArtifact } from "../../artifacts/readStableArtifact.js"; import { readStableArtifact } from "../../artifacts/readStableArtifact.js";
import { normalizeArtifactPath } from "../../artifacts/ArtifactPaths.js"; import { normalizeArtifactPath } from "../../artifacts/ArtifactPaths.js";
import type { WebScriptArtifactPort } from "../../application/WebScriptArtifactPort.js"; import type { WebScriptArtifactPort } from "../../application/WebScriptArtifactPort.js";
@@ -20,36 +19,10 @@ import {
/** Read one digest-verified captured source under the manifest's current location. */ /** Read one digest-verified captured source under the manifest's current location. */
export class LocalWebScriptArtifacts implements WebScriptArtifactPort { export class LocalWebScriptArtifacts implements WebScriptArtifactPort {
constructor( constructor(readonly operation: WebArtifactReadContext["operation"]) {}
readonly operation: WebArtifactReadContext["operation"],
readonly resources = new ArtifactResourceScope(),
) {}
close(): Promise<void> {
return this.resources.close();
}
/** Read the selected source; relocated exports retain their reported original root. */ /** Read the selected source; relocated exports retain their reported original root. */
async load(input: SelectedWebScriptInput, options?: ExecutionOptions) { async load(input: SelectedWebScriptInput, options?: ExecutionOptions) {
try {
return await this.resources.run(() => this.#load(input, options));
} catch (cause: unknown) {
return err(
webArtifactReadError(
cause,
{
operation: this.operation,
field: ["manifest_path"],
targetPath: input.manifest_path,
callerSelected: true,
},
options?.signal,
),
);
}
}
async #load(input: SelectedWebScriptInput, options?: ExecutionOptions) {
let field: readonly (string | number)[] = ["manifest_path"]; let field: readonly (string | number)[] = ["manifest_path"];
let targetPath = input.manifest_path; let targetPath = input.manifest_path;
let callerSelected = true; let callerSelected = true;
@@ -58,7 +31,6 @@ export class LocalWebScriptArtifacts implements WebScriptArtifactPort {
input.manifest_path, input.manifest_path,
32 * 1024 * 1024, 32 * 1024 * 1024,
options?.signal, options?.signal,
this.resources,
); );
const manifest = webScriptExportManifestSchema.parse(manifestBytes.value); const manifest = webScriptExportManifestSchema.parse(manifestBytes.value);
field = ["script_index"]; field = ["script_index"];
@@ -93,8 +65,6 @@ export class LocalWebScriptArtifacts implements WebScriptArtifactPort {
sourcePath, sourcePath,
16 * 1024 * 1024, 16 * 1024 * 1024,
options?.signal, options?.signal,
undefined,
this.resources,
); );
await assertContained(root, sourcePath); await assertContained(root, sourcePath);
if ( if (

Some files were not shown because too many files have changed in this diff Show More