Compare commits
46
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0ac6832aa6 | ||
|
|
83e52559b8 | ||
|
|
824ecf109d | ||
|
|
8b4c3de3b3 | ||
|
|
f998fcb1cc | ||
|
|
f88c99021b | ||
|
|
49e6f00ed0 | ||
|
|
af88184eee | ||
|
|
c1efa6467d | ||
|
|
1e51c732bb | ||
|
|
7956934965 | ||
|
|
db4e360846 | ||
|
|
7bb4ac219c | ||
|
|
3df67c640a | ||
|
|
044aa34ce6 | ||
|
|
31bfff6fdf | ||
|
|
026126eafe | ||
|
|
29cc1b8746 | ||
|
|
ad41afa2b5 | ||
|
|
69a8b3a954 | ||
|
|
4733fc810a | ||
|
|
d21132fc97 | ||
|
|
66c344e289 | ||
|
|
28bd0d990a | ||
|
|
2612e474b6 | ||
|
|
a11fbc0df9 | ||
|
|
639b73097b | ||
|
|
bb074d84cb | ||
|
|
204e2c42a1 | ||
|
|
87edf14940 | ||
|
|
890dfa49f6 | ||
|
|
bc7846db91 | ||
|
|
5485483f7d | ||
|
|
c916850852 | ||
|
|
26038912f6 | ||
|
|
aa7c5b7b5e | ||
|
|
095fad9406 | ||
|
|
8b2f8514c5 | ||
|
|
c57e759ae6 | ||
|
|
7b25a73ac4 | ||
|
|
aa257ea4b1 | ||
|
|
4999e4be2c | ||
|
|
92cc0edd8e | ||
|
|
573598c632 | ||
|
|
8d3cfef753 | ||
|
|
975dff2e25 |
@@ -0,0 +1,127 @@
|
||||
---
|
||||
name: rea-changelog-update
|
||||
description: Prepare or rewrite REA release changelogs and GitHub release notes from pinned Git history, with verified contributor thanks and Release Please synchronization.
|
||||
---
|
||||
|
||||
# REA changelog update
|
||||
|
||||
Use this for release-note preparation and contributor attribution. Read
|
||||
[docs/releasing.md](../../../docs/releasing.md) from the repository root for
|
||||
publication authority and the existing checkpoint checks. This skill prepares
|
||||
notes; it does not authorize merging, tagging, or publishing.
|
||||
|
||||
## Inventory before prose
|
||||
|
||||
Fetch complete history and tags. Select the published baseline tag from
|
||||
`.release-please-manifest.json` on the release PR's **base branch**, and pin that
|
||||
branch's exact SHA as the editorial target. REA tags are `rea-agents-VERSION`.
|
||||
Use the proposed version, including its prerelease suffix; do not change the
|
||||
`always-bump-minor` strategy to imitate OpenClaw's calendar versions.
|
||||
|
||||
```bash
|
||||
node scripts/release-notes.mjs inventory \
|
||||
--repository morluto/rea --base rea-agents-PREVIOUS \
|
||||
--target FULL_BASE_BRANCH_SHA --version VERSION \
|
||||
--manifest /tmp/rea-VERSION.contributions.json
|
||||
node scripts/release-notes.mjs record \
|
||||
--manifest /tmp/rea-VERSION.contributions.json \
|
||||
--output /tmp/rea-VERSION.record.md
|
||||
```
|
||||
|
||||
Read the inventory, not just the existing bot draft. It includes side-branch
|
||||
commits, GitHub-associated PRs whose merge commits are in the range, commit
|
||||
authors/co-authors, and confirmed closing issues with their reporters. GitHub
|
||||
connections are paginated. Network errors stop generation; rerun into a new
|
||||
output file after recovery. Existing output files are never overwritten.
|
||||
|
||||
Unresolved commit identities are retained as names/emails with `user: null`;
|
||||
never guess a handle or thank the merger as a substitute. The report preserves
|
||||
unknowns. This is a reviewed metadata snapshot, not independent proof that a
|
||||
human authored a patch. For unattributed identities, unusual merge strategies,
|
||||
backports, reversions, or references that GitHub did not associate, inspect
|
||||
the source PRs and diffs and report the limitation. The current collector uses
|
||||
confirmed PR-closing relationships for issue-reporter credit; unrelated issue
|
||||
mentions are not automatically credited. Do not claim that merely inventoried
|
||||
work still ships after a revert.
|
||||
|
||||
## Write for users and retain credit
|
||||
|
||||
Rewrite only the newest `## [VERSION]` section in `CHANGELOG.md`, preserving its
|
||||
version/date/link heading. Use single-line Markdown list items under:
|
||||
|
||||
- `### Highlights`: a few release-defining user outcomes; no quota for small releases.
|
||||
- `### Changes`: capabilities and meaningful behavior changes.
|
||||
- `### Fixes`: reliability, correctness, compatibility, and safety fixes.
|
||||
- `### ⚠ BREAKING CHANGES`: retain when applicable, with migrations and original
|
||||
PR/commit references required by the existing checkpoint validator.
|
||||
|
||||
Group related work by analyst workflow or user impact. Explain the resulting
|
||||
behavior; internal cleanup, tests, CI, and routine documentation generally
|
||||
belong only in the contribution record. Read ambiguous PRs/diffs before making
|
||||
claims. Distinguish simulated-provider checks from real-provider verification.
|
||||
|
||||
Each bullet covering a PR carries its link and `Thanks @...` for its verified
|
||||
human author, known co-authors, and confirmed closing-issue reporters. Grouping
|
||||
must preserve **all** those credits in the same bullet. Include the issue
|
||||
links. Direct-commit bullets carry the commit link and known human credit.
|
||||
Exclude GitHub bots and known agent accounts. REA's observed Anthropic and
|
||||
Cursor co-author trailers resolve to `@claude` and `@cursoragent` with GitHub
|
||||
type `User`; the credit policy excludes them while retaining their metadata.
|
||||
Do not copy OpenClaw's maintainer-specific exclusions.
|
||||
Use explicit GitHub links for cross-repository references to avoid ambiguous
|
||||
`#NNN` numbers. Unknown handles stay unknown; they are not fabricated credit.
|
||||
|
||||
Append the generated `### Complete contribution record` verbatim as the last
|
||||
section of this release. It accounts for every discovered in-range PR and
|
||||
direct contribution, including internal work omitted from the prose. Do not
|
||||
hand-edit generated credits to make validation pass. Refresh the inventory
|
||||
when product history changes; a prose-only correction can reuse the snapshot.
|
||||
|
||||
## Finalize with Release Please
|
||||
|
||||
Before changing the live release PR, have authorization for that PR update.
|
||||
Local preparation and validation can proceed independently.
|
||||
|
||||
1. Add `rea:release-notes-finalized` to the selected open release PR **before**
|
||||
editing its branch/body. Main proposal pushes respect this label. This is
|
||||
our workflow guard, not Release Please's `autorelease: snooze` mechanism.
|
||||
A proposal already running must finish before editing. Do not dispatch
|
||||
manual preparation over finalized notes.
|
||||
2. On that release branch, save the reviewed inventory as
|
||||
`docs/releases/VERSION.contributions.json`. This is durable public release
|
||||
provenance, not a temporary planning file. Do not add credentials or local
|
||||
environment dumps. Keep the existing version artifacts aligned.
|
||||
3. Run the existing checkpoint validator and the new checker:
|
||||
|
||||
```bash
|
||||
node scripts/release-notes.mjs check \
|
||||
--manifest docs/releases/VERSION.contributions.json
|
||||
node scripts/release-notes.mjs render \
|
||||
--manifest docs/releases/VERSION.contributions.json \
|
||||
--output /tmp/rea-VERSION.notes.md
|
||||
```
|
||||
|
||||
The checker supports inline PR bodies, including their surrounding metadata,
|
||||
up to GitHub's 65,536-character limit. It fails on larger notes rather than
|
||||
truncating credit; Release Please's overflow-link bodies are not integrated.
|
||||
|
||||
4. Update the release PR body with the rendered section while preserving its
|
||||
Release Please header/footer and component/version structure. Use a body
|
||||
file for `gh pr edit --body-file`; changing only `CHANGELOG.md` does not
|
||||
change Release Please's release body. Push only authorized changes.
|
||||
5. Freshly fetch the PR body to a file and run `check --pr-body FILE`. Review
|
||||
the complete diff and wait for current CI. Publication checks the body
|
||||
again before release creation. Avoid further body edits while publishing.
|
||||
|
||||
The finalization label freezes bot refreshes, not main. New product changes
|
||||
after the inventory target require refreshing the notes before publication.
|
||||
The workflow checks target ancestry and allows only release metadata paths
|
||||
after that target. It also requires the inventory target to equal the reviewed
|
||||
merge's first parent, so intervening main commits require a fresh inventory.
|
||||
Review diffs within the allowed release metadata paths too: the path guard does
|
||||
not prove that package dependency/script edits are version-only changes.
|
||||
To resume automatic proposals, remove the finalization label and let the bot
|
||||
regenerate; expect editorial changes to be replaced.
|
||||
|
||||
Existing releases without a contribution inventory retain their current
|
||||
publication path. Never retag or republish a package to correct release prose.
|
||||
@@ -38,3 +38,11 @@ or absence:
|
||||
Process captures are opt-in behavioral evidence, not a security sandbox. V3
|
||||
captures cannot be upgraded to V4; rerun the original scenario. Distinguish root
|
||||
exit from descendant settlement and require freshness when the task needs it.
|
||||
A timed-out target that must write a final report needs `finalization_ms`: REA
|
||||
sends SIGTERM to the captured root, keeps observing, and sends SIGKILL once
|
||||
only if the target has not exited by the end of the interval.
|
||||
`exit.finalization.signals` lists each attempt with its delivery result; the
|
||||
observed exit signal stays in `exit.signal`, `exit.code` is null for every
|
||||
deadline exit, and the code of a finalizer that exited on its own is in
|
||||
`exit.finalization.exit_code` (null on a signal exit or when no exit was
|
||||
observed).
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
self-hosted-runner:
|
||||
labels:
|
||||
- hopper
|
||||
- ghidra-12-1-2
|
||||
# Hosted label introduced after the pinned actionlint release.
|
||||
- macos-15-intel
|
||||
+145
-50
@@ -5,6 +5,10 @@ on:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main, "release/*"]
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: ci-${{ github.event.pull_request.number || github.ref }}
|
||||
@@ -36,37 +40,39 @@ jobs:
|
||||
name: Classify changes
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
code: ${{ steps.package.outputs.required }}
|
||||
package: ${{ steps.package.outputs.required }}
|
||||
code: ${{ steps.plan.outputs.code }}
|
||||
runtime: ${{ steps.plan.outputs.runtime }}
|
||||
package: ${{ steps.plan.outputs.package }}
|
||||
apple: ${{ steps.plan.outputs.apple }}
|
||||
fixtures: ${{ steps.plan.outputs.fixtures }}
|
||||
readiness: ${{ steps.plan.outputs.readiness }}
|
||||
docs: ${{ steps.plan.outputs.docs }}
|
||||
workflows: ${{ steps.plan.outputs.workflows }}
|
||||
browser: ${{ steps.plan.outputs.browser }}
|
||||
binary_layout: ${{ steps.plan.outputs.binary_layout }}
|
||||
evm: ${{ steps.plan.outputs.evm }}
|
||||
javascript_recovery: ${{ steps.plan.outputs.javascript_recovery }}
|
||||
recorded_crash: ${{ steps.plan.outputs.recorded_crash }}
|
||||
web_network: ${{ steps.plan.outputs.web_network }}
|
||||
web_runtime: ${{ steps.plan.outputs.web_runtime }}
|
||||
web_source_map: ${{ steps.plan.outputs.web_source_map }}
|
||||
nativeaot: ${{ steps.plan.outputs.nativeaot }}
|
||||
website: ${{ steps.plan.outputs.website }}
|
||||
matrix: ${{ steps.plan.outputs.matrix }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
- name: Select package and Windows lanes
|
||||
id: package
|
||||
shell: bash
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Select verification lanes
|
||||
id: plan
|
||||
env:
|
||||
BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: |
|
||||
if [[ "${GITHUB_EVENT_NAME}" != "pull_request" ]]; then
|
||||
echo "required=true" >> "${GITHUB_OUTPUT}"
|
||||
# Compare the PR's changes, excluding unrelated updates on its base.
|
||||
elif git diff --quiet "${BASE_SHA}...${HEAD_SHA}" -- \
|
||||
. \
|
||||
':(exclude)docs/**' \
|
||||
':(top,exclude,glob)README*.md' \
|
||||
':(exclude)AGENTS.md' \
|
||||
':(exclude)CONTRIBUTING.md'; then
|
||||
echo "required=false" >> "${GITHUB_OUTPUT}"
|
||||
else
|
||||
diff_status=$?
|
||||
# A failed comparison must fail classification, not report a scope.
|
||||
if [[ "${diff_status}" -ne 1 ]]; then
|
||||
exit "${diff_status}"
|
||||
fi
|
||||
echo "required=true" >> "${GITHUB_OUTPUT}"
|
||||
fi
|
||||
CI_FULL: ${{ contains(github.event.pull_request.labels.*.name, 'ci:full') }}
|
||||
run: node scripts/ci/plan.mjs
|
||||
|
||||
static:
|
||||
name: Static checks
|
||||
@@ -84,17 +90,17 @@ jobs:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- if: github.event_name == 'push' || needs.changes.outputs.code == 'true'
|
||||
- if: needs.changes.outputs.code == 'true'
|
||||
run: npm run check:ci
|
||||
- if: github.event_name == 'push' || needs.changes.outputs.code == 'true'
|
||||
- if: needs.changes.outputs.code == 'true'
|
||||
run: npm run verify:test-discovery
|
||||
- if: github.event_name == 'pull_request' && needs.changes.outputs.code != 'true'
|
||||
- if: needs.changes.outputs.code != 'true'
|
||||
run: npm run format:check
|
||||
|
||||
test-shard:
|
||||
name: Test shard (${{ matrix.shard }})
|
||||
needs: [changes, build]
|
||||
if: github.event_name == 'push' || needs.changes.outputs.code == 'true'
|
||||
if: needs.changes.outputs.code == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
fail-fast: false
|
||||
@@ -131,7 +137,7 @@ jobs:
|
||||
build:
|
||||
name: Build package
|
||||
needs: changes
|
||||
if: github.event_name == 'push' || needs.changes.outputs.code == 'true'
|
||||
if: needs.changes.outputs.code == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
@@ -224,7 +230,7 @@ jobs:
|
||||
conformance-fixtures:
|
||||
name: Source-built conformance fixtures
|
||||
needs: changes
|
||||
if: github.event_name == 'push' || needs.changes.outputs.code == 'true'
|
||||
if: needs.changes.outputs.fixtures == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
@@ -235,7 +241,7 @@ jobs:
|
||||
windows-native-build:
|
||||
name: Build bundled Windows x64 native controls
|
||||
needs: changes
|
||||
if: github.event_name == 'push' || needs.changes.outputs.package == 'true'
|
||||
if: needs.changes.outputs.runtime == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
@@ -265,24 +271,11 @@ jobs:
|
||||
package-e2e:
|
||||
name: Native package baseline (${{ matrix.platform }} ${{ matrix.architecture }})
|
||||
needs: [changes, windows-native-build]
|
||||
if: github.event_name == 'push' || needs.changes.outputs.package == 'true'
|
||||
if: needs.changes.outputs.runtime == 'true'
|
||||
strategy:
|
||||
fail-fast: false
|
||||
max-parallel: 2
|
||||
matrix:
|
||||
include:
|
||||
- os: ubuntu-latest
|
||||
platform: linux
|
||||
architecture: x64
|
||||
- os: ubuntu-24.04-arm
|
||||
platform: linux
|
||||
architecture: arm64
|
||||
- os: macos-15
|
||||
platform: darwin
|
||||
architecture: arm64
|
||||
- os: macos-15-intel
|
||||
platform: darwin
|
||||
architecture: x64
|
||||
matrix: ${{ fromJSON(needs.changes.outputs.matrix) }}
|
||||
runs-on: ${{ matrix.os }}
|
||||
timeout-minutes: 20
|
||||
env:
|
||||
@@ -317,7 +310,7 @@ jobs:
|
||||
apple-artifacts:
|
||||
name: Real Apple artifact CLI/MCP E2E
|
||||
needs: changes
|
||||
if: github.event_name == 'push' || needs.changes.outputs.code == 'true'
|
||||
if: needs.changes.outputs.apple == 'true'
|
||||
runs-on: macos-15
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
@@ -350,7 +343,7 @@ jobs:
|
||||
reconstruction-readiness:
|
||||
name: Reconstruction readiness
|
||||
needs: changes
|
||||
if: github.event_name == 'push' || needs.changes.outputs.code == 'true'
|
||||
if: needs.changes.outputs.readiness == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
@@ -379,7 +372,7 @@ jobs:
|
||||
windows-curated:
|
||||
name: Windows capability and package lane
|
||||
needs: [changes, windows-native-build]
|
||||
if: github.event_name == 'push' || needs.changes.outputs.package == 'true'
|
||||
if: needs.changes.outputs.package == 'true'
|
||||
runs-on: windows-latest
|
||||
timeout-minutes: 25
|
||||
steps:
|
||||
@@ -476,7 +469,8 @@ jobs:
|
||||
|
||||
docs:
|
||||
name: Docs Generation
|
||||
if: github.event_name == 'pull_request'
|
||||
needs: changes
|
||||
if: needs.changes.outputs.docs == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
@@ -496,3 +490,104 @@ jobs:
|
||||
docs/verification/managed-conformance-*.json
|
||||
skills/reverse-engineer-anything
|
||||
if-no-files-found: error
|
||||
|
||||
workflows:
|
||||
name: Validate workflows
|
||||
needs: changes
|
||||
if: needs.changes.outputs.workflows == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Validate workflow syntax and expressions
|
||||
run: go run github.com/rhysd/actionlint/cmd/actionlint@v1.7.7 -shellcheck= -pyflakes=
|
||||
|
||||
browser:
|
||||
needs: changes
|
||||
if: needs.changes.outputs.browser == 'true'
|
||||
uses: ./.github/workflows/real-browser.yml
|
||||
|
||||
binary_layout:
|
||||
needs: changes
|
||||
if: needs.changes.outputs.binary_layout == 'true'
|
||||
uses: ./.github/workflows/real-binary-layout.yml
|
||||
|
||||
evm:
|
||||
needs: changes
|
||||
if: needs.changes.outputs.evm == 'true'
|
||||
uses: ./.github/workflows/real-evm-interface.yml
|
||||
|
||||
javascript_recovery:
|
||||
needs: changes
|
||||
if: needs.changes.outputs.javascript_recovery == 'true'
|
||||
uses: ./.github/workflows/real-javascript-recovery.yml
|
||||
|
||||
recorded_crash:
|
||||
needs: changes
|
||||
if: needs.changes.outputs.recorded_crash == 'true'
|
||||
uses: ./.github/workflows/real-recorded-crash.yml
|
||||
|
||||
web_network:
|
||||
needs: changes
|
||||
if: needs.changes.outputs.web_network == 'true'
|
||||
uses: ./.github/workflows/real-web-network-captures.yml
|
||||
|
||||
web_runtime:
|
||||
needs: changes
|
||||
if: needs.changes.outputs.web_runtime == 'true'
|
||||
uses: ./.github/workflows/real-web-runtime.yml
|
||||
|
||||
web_source_map:
|
||||
needs: changes
|
||||
if: needs.changes.outputs.web_source_map == 'true'
|
||||
uses: ./.github/workflows/real-web-source-map.yml
|
||||
|
||||
nativeaot:
|
||||
needs: changes
|
||||
if: needs.changes.outputs.nativeaot == 'true'
|
||||
uses: ./.github/workflows/nativeaot-fixtures.yml
|
||||
|
||||
website:
|
||||
needs: changes
|
||||
if: needs.changes.outputs.website == 'true'
|
||||
uses: ./.github/workflows/website-check.yml
|
||||
|
||||
required:
|
||||
name: CI required
|
||||
if: ${{ always() }}
|
||||
needs:
|
||||
[
|
||||
changes,
|
||||
pr-title,
|
||||
static,
|
||||
build,
|
||||
test-shard,
|
||||
test,
|
||||
conformance-fixtures,
|
||||
windows-native-build,
|
||||
package-e2e,
|
||||
apple-artifacts,
|
||||
reconstruction-readiness,
|
||||
windows-curated,
|
||||
docs,
|
||||
workflows,
|
||||
browser,
|
||||
binary_layout,
|
||||
evm,
|
||||
javascript_recovery,
|
||||
recorded_crash,
|
||||
web_network,
|
||||
web_runtime,
|
||||
web_source_map,
|
||||
nativeaot,
|
||||
website,
|
||||
]
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Require every selected lane to pass
|
||||
env:
|
||||
CI_JOBS: ${{ toJSON(needs) }}
|
||||
run: node scripts/ci/required.mjs
|
||||
|
||||
@@ -2,12 +2,7 @@ name: Source-built NativeAOT fixtures
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
pull_request:
|
||||
paths:
|
||||
- "bridge/ghidra/extensions/nativeaot/**"
|
||||
- "scripts/generate-nativeaot-fixtures.mjs"
|
||||
- "tests/fixtures/nativeaot/**"
|
||||
- ".github/workflows/nativeaot-fixtures.yml"
|
||||
workflow_call:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
@@ -7,36 +7,7 @@ on:
|
||||
description: "Also verify the complete 65,281-section result over MCP (slow)"
|
||||
type: boolean
|
||||
default: false
|
||||
pull_request:
|
||||
paths:
|
||||
- ".github/workflows/real-binary-layout.yml"
|
||||
- "bridge/pwntools/**"
|
||||
- "src/native/pwntools/**"
|
||||
- "src/domain/native/binaryLayout*"
|
||||
- "src/application/binaryDiagnostics/**"
|
||||
- "src/contracts/native/binaryDiagnosticsToolContracts.ts"
|
||||
- "src/composition/binaryDiagnostics.ts"
|
||||
- "src/server/registerBinaryDiagnosticsTools.ts"
|
||||
- "src/server/createServer.ts"
|
||||
- "src/cli/binaryDiagnosticsCommands.ts"
|
||||
- "src/cli.ts"
|
||||
- "src/cliCommandNames.ts"
|
||||
- "src/contracts/toolContracts.ts"
|
||||
- "src/contracts/toolEffects.ts"
|
||||
- "src/process/OwnedCommand.ts"
|
||||
- "src/process/ProviderProcess.ts"
|
||||
- "src/process/ProviderDeadline.ts"
|
||||
- "src/process/ProcessOwnership*.ts"
|
||||
- "src/process/PrivateRuntimeRoot.ts"
|
||||
- "src/artifacts/readStableArtifact*.ts"
|
||||
- "src/artifacts/ArtifactReader.ts"
|
||||
- "src/domain/analysisError*.ts"
|
||||
- "scripts/verify-binary-layout.mjs"
|
||||
- "scripts/lib/elf-section-name-fixtures.mjs"
|
||||
- "scripts/lib/large-result-mcp.mjs"
|
||||
- "scripts/fixtures/binary-layout*"
|
||||
- "package.json"
|
||||
- "package-lock.json"
|
||||
workflow_call:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
@@ -1,40 +1,7 @@
|
||||
name: Real browser verification
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- ".github/workflows/real-browser.yml"
|
||||
- "scripts/verify-real-browser.mjs"
|
||||
- "scripts/verify-browser-network.mjs"
|
||||
- "scripts/verify-browser-scripts.mjs"
|
||||
- "scripts/verify-browser-modules.mjs"
|
||||
- "scripts/verify-browser-dom-destinations.mjs"
|
||||
- "scripts/lib/browser-*.mjs"
|
||||
- "scripts/fixtures/browser-network-site.mjs"
|
||||
- "scripts/fixtures/browser-script-site.mjs"
|
||||
- "scripts/fixtures/browser-module-site.mjs"
|
||||
- "scripts/fixtures/browser-verifier-site.mjs"
|
||||
- "scripts/fixtures/page-cdp-proxy.mjs"
|
||||
- "src/browser/**"
|
||||
- "src/domain/browser*.ts"
|
||||
- "src/domain/webScript*.ts"
|
||||
- "src/domain/webModule*.ts"
|
||||
- "src/application/WebModule*.ts"
|
||||
- "src/composition/webModules.ts"
|
||||
- "src/artifacts/readStableArtifact.ts"
|
||||
- "src/artifacts/ArtifactPaths.ts"
|
||||
- "src/application/WebScriptExportService.ts"
|
||||
- "src/contracts/webScriptToolContracts.ts"
|
||||
- "src/server/registerWebScriptTool.ts"
|
||||
- "src/server/registerWebModuleTool.ts"
|
||||
- "src/server/createServer.ts"
|
||||
- "src/cli/webModuleCommands.ts"
|
||||
- "src/cli.ts"
|
||||
- "src/cliCommandNames.ts"
|
||||
- "src/contracts/toolEffects.ts"
|
||||
- "src/cli/webScriptCommands.ts"
|
||||
- "package.json"
|
||||
- "package-lock.json"
|
||||
workflow_call:
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
|
||||
@@ -2,33 +2,7 @@ name: Real offline EVM interface
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
pull_request:
|
||||
paths:
|
||||
- ".github/workflows/real-evm-interface.yml"
|
||||
- "src/evm/**"
|
||||
- "src/domain/evm/**"
|
||||
- "src/application/evm/**"
|
||||
- "src/contracts/evm/**"
|
||||
- "src/composition/evm.ts"
|
||||
- "src/cli/evmCommands.ts"
|
||||
- "src/server/registerEvmTools.ts"
|
||||
- "src/server/createServer.ts"
|
||||
- "src/cli.ts"
|
||||
- "src/cliCommandNames.ts"
|
||||
- "src/contracts/toolContracts.ts"
|
||||
- "src/contracts/toolEffects.ts"
|
||||
- "src/process/OwnedCommand.ts"
|
||||
- "src/process/ProviderProcess.ts"
|
||||
- "src/process/ProcessOwnership*.ts"
|
||||
- "src/process/ProviderDeadline.ts"
|
||||
- "src/process/PrivateRuntimeRoot.ts"
|
||||
- "src/artifacts/readStableArtifact*.ts"
|
||||
- "src/artifacts/ArtifactReader.ts"
|
||||
- "scripts/verify-evm-interface.mjs"
|
||||
- "scripts/fixtures/evm-interface.sol"
|
||||
- "third_party/evmole/**"
|
||||
- "package.json"
|
||||
- "package-lock.json"
|
||||
workflow_call:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
@@ -1,32 +1,7 @@
|
||||
name: Real JavaScript recovery
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- ".github/workflows/real-javascript-recovery.yml"
|
||||
- "src/javascript/recovery/**"
|
||||
- "src/domain/javascript/**"
|
||||
- "src/application/javascript/JavaScriptRecovery*.ts"
|
||||
- "src/composition/javascriptRecovery.ts"
|
||||
- "src/contracts/javascript/javascriptRecoveryToolContracts.ts"
|
||||
- "src/cli/javascriptRecoveryCommands.ts"
|
||||
- "src/cli/javascriptApplicationAnalysis.ts"
|
||||
- "src/server/registerJavaScriptRecoveryTool.ts"
|
||||
- "src/server/createServer.ts"
|
||||
- "src/server/sessionAvailabilityPolicy.ts"
|
||||
- "src/application/CapabilityInventory.ts"
|
||||
- "src/cli.ts"
|
||||
- "src/cliCommandNames.ts"
|
||||
- "src/contracts/toolContracts.ts"
|
||||
- "src/contracts/toolEffects.ts"
|
||||
- "scripts/rea.mjs"
|
||||
- "src/process/**"
|
||||
- "src/artifacts/SafeOutputTree.ts"
|
||||
- "scripts/verify/javascript/recovery.mjs"
|
||||
- "scripts/fixtures/javascript-recovery/**"
|
||||
- "third_party/wakaru"
|
||||
- "package.json"
|
||||
- "package-lock.json"
|
||||
workflow_call:
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
|
||||
@@ -2,34 +2,7 @@ name: Real recorded crash evidence
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
pull_request:
|
||||
paths:
|
||||
- ".github/workflows/real-recorded-crash.yml"
|
||||
- "bridge/pwntools/**"
|
||||
- "bridge/pwndbg/**"
|
||||
- "src/native/pwntools/**"
|
||||
- "src/native/pwndbg/**"
|
||||
- "src/domain/native/recordedCrash*"
|
||||
- "src/application/binaryDiagnostics/**"
|
||||
- "src/contracts/native/binaryDiagnosticsToolContracts.ts"
|
||||
- "src/contracts/toolEffects.ts"
|
||||
- "src/contracts/toolOutputSchemaGroups.ts"
|
||||
- "src/composition/binaryDiagnostics.ts"
|
||||
- "src/server/registerRecordedCrashTools.ts"
|
||||
- "src/server/createServer.ts"
|
||||
- "src/server/sessionAvailabilityPolicy.ts"
|
||||
- "src/cli/binaryDiagnosticsCommands.ts"
|
||||
- "src/cliCommandNames.ts"
|
||||
- "src/process/**"
|
||||
- "src/artifacts/readStableArtifact*.ts"
|
||||
- "src/domain/analysisError*.ts"
|
||||
- "scripts/verify-recorded-crash.mjs"
|
||||
- "scripts/verify/recorded-crash/**"
|
||||
- "scripts/lib/recorded-core-fixtures.py"
|
||||
- "scripts/fixtures/recorded-crash.c"
|
||||
- "third_party/pwndbg/**"
|
||||
- "package.json"
|
||||
- "package-lock.json"
|
||||
workflow_call:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
name: Real Termux browser verification
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: real-termux-${{ github.ref }}
|
||||
cancel-in-progress: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
|
||||
jobs:
|
||||
real-termux:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 45
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Download the pinned Termux app and verify its release checksum
|
||||
shell: bash
|
||||
run: |
|
||||
mkdir -p "$RUNNER_TEMP/rea-termux"
|
||||
cd "$RUNNER_TEMP/rea-termux"
|
||||
release=https://github.com/termux/termux-app/releases/download/v0.118.3
|
||||
apk=termux-app_v0.118.3+github-debug_x86_64.apk
|
||||
sums=termux-app_v0.118.3+github-debug_sha256sums
|
||||
curl --fail --location --retry 3 --output "$apk" "$release/$apk"
|
||||
curl --fail --location --retry 3 --output "$sums" "$release/$sums"
|
||||
awk -v apk="$apk" '$2 == apk || $2 == "*" apk' "$sums" > selected.sha256
|
||||
test "$(wc -l < selected.sha256)" -eq 1
|
||||
sha256sum --check selected.sha256
|
||||
- name: Enable emulator acceleration
|
||||
run: |
|
||||
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
|
||||
sudo udevadm control --reload-rules
|
||||
sudo udevadm trigger --name-match=kvm
|
||||
- name: Build and capture a page inside Termux
|
||||
uses: reactivecircus/android-emulator-runner@4c44018e59b437e86cdfc41da381398f93ed8808 # v2
|
||||
with:
|
||||
api-level: 30
|
||||
arch: x86_64
|
||||
target: google_apis
|
||||
ram-size: 4096M
|
||||
heap-size: 512M
|
||||
disk-size: 8192M
|
||||
disable-animations: true
|
||||
script: bash scripts/verify/termux/emulator.sh
|
||||
- name: Upload Termux and Android diagnostics
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: termux-browser-diagnostics
|
||||
path: ${{ runner.temp }}/rea-termux/diagnostics
|
||||
if-no-files-found: warn
|
||||
@@ -2,38 +2,7 @@ name: Real historical web network captures
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
pull_request:
|
||||
paths:
|
||||
- ".github/workflows/real-web-network-captures.yml"
|
||||
- "bridge/mitmproxy/**"
|
||||
- "src/browser/history/**"
|
||||
- "src/domain/webNetworkCapture*.ts"
|
||||
- "src/domain/analysisError*.ts"
|
||||
- "src/contracts/errorSchemas.ts"
|
||||
- "src/domain/explicitSensitive*.ts"
|
||||
- "src/application/WebNetworkCapture*"
|
||||
- "src/composition/webNetworkCaptures.ts"
|
||||
- "src/cli/webNetworkCaptureCommands.ts"
|
||||
- "src/server/registerWebNetworkCaptureTool.ts"
|
||||
- "src/server/createServer.ts"
|
||||
- "src/server/toolRegistrationOptions.ts"
|
||||
- "src/cli.ts"
|
||||
- "src/cliCommandNames.ts"
|
||||
- "src/contracts/toolContracts.ts"
|
||||
- "src/contracts/toolEffects.ts"
|
||||
- "src/process/OwnedCommand.ts"
|
||||
- "src/process/ProviderProcess.ts"
|
||||
- "src/process/ProviderDeadline.ts"
|
||||
- "src/process/ProcessOwnership*.ts"
|
||||
- "src/process/PrivateRuntimeRoot.ts"
|
||||
- "src/process/capture/ProcessCaptureLifecycle.ts"
|
||||
- "src/artifacts/readStableArtifact*.ts"
|
||||
- "src/artifacts/ArtifactReader.ts"
|
||||
- "src/contracts/webNetworkCaptureToolContracts.ts"
|
||||
- "scripts/verify-web-network-captures.mjs"
|
||||
- "scripts/fixtures/generate-mitmproxy-capture.py"
|
||||
- "package.json"
|
||||
- "package-lock.json"
|
||||
workflow_call:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
@@ -1,34 +1,6 @@
|
||||
name: Real website runtime attribution
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- ".github/workflows/real-web-runtime.yml"
|
||||
- "scripts/verify-browser-runtime.mjs"
|
||||
- "scripts/lib/browser-runtime-*.mjs"
|
||||
- "src/process/ProviderProcess.ts"
|
||||
- "src/process/ProcessOwnership*.ts"
|
||||
- "scripts/fixtures/browser-runtime-site.mjs"
|
||||
- "src/browser/execution/**"
|
||||
- "src/browser/CdpConnection.ts"
|
||||
- "src/browser/CdpEndpoint.ts"
|
||||
- "src/browser/CdpTarget*.ts"
|
||||
- "src/browser/CdpCaptureValues.ts"
|
||||
- "src/browser/CdpAuthorized*.ts"
|
||||
- "src/domain/webRuntime.ts"
|
||||
- "src/domain/webExecution.ts"
|
||||
- "src/domain/webEventListeners.ts"
|
||||
- "src/domain/browserObservationError*.ts"
|
||||
- "src/application/WebRuntime*.ts"
|
||||
- "src/composition/webRuntime.ts"
|
||||
- "src/contracts/webRuntimeToolContracts.ts"
|
||||
- "src/contracts/toolEffects.ts"
|
||||
- "src/server/registerWebRuntimeTools.ts"
|
||||
- "src/server/createServer.ts"
|
||||
- "src/cli/webRuntimeCommands.ts"
|
||||
- "src/cli/commandCancellation.ts"
|
||||
- "src/cli.ts"
|
||||
- "package.json"
|
||||
- "package-lock.json"
|
||||
workflow_call:
|
||||
workflow_dispatch:
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
@@ -1,27 +1,6 @@
|
||||
name: Real website source-map verification
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- ".github/workflows/real-web-source-map.yml"
|
||||
- "scripts/verify-browser-source-maps.mjs"
|
||||
- "scripts/lib/browser-source-map-e2e.mjs"
|
||||
- "scripts/fixtures/browser-source-map-site.mjs"
|
||||
- "src/javascript/sourceMaps/**"
|
||||
- "src/browser/assets/**"
|
||||
- "src/domain/webSource*.ts"
|
||||
- "src/domain/webScript*.ts"
|
||||
- "src/application/WebSourceLocation*.ts"
|
||||
- "src/application/ValidateSelectedWebScript.ts"
|
||||
- "src/composition/webSourceLocations.ts"
|
||||
- "src/server/registerWebSourceLocationTool.ts"
|
||||
- "src/server/createServer.ts"
|
||||
- "src/contracts/webScriptToolContracts.ts"
|
||||
- "src/process/**"
|
||||
- "src/cli/webSourceLocationCommands.ts"
|
||||
- "src/cli.ts"
|
||||
- "src/artifacts/readStableArtifact.ts"
|
||||
- "package.json"
|
||||
- "package-lock.json"
|
||||
workflow_call:
|
||||
workflow_dispatch:
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
@@ -35,7 +35,21 @@ jobs:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
steps:
|
||||
- name: Respect finalized release notes
|
||||
id: editorial
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
GH_REPO: ${{ github.repository }}
|
||||
run: |
|
||||
finalized=$(gh api --paginate "repos/${GH_REPO}/pulls?state=open&base=main&per_page=100" \
|
||||
--jq '.[] | select(.head.repo.full_name == .base.repo.full_name) | select(.head.ref | startswith("release-please--branches--main--")) | select(any(.labels[]; .name == "rea:release-notes-finalized")) | .number')
|
||||
if [[ -n "${finalized}" ]]; then
|
||||
echo "Finalized release notes on PR ${finalized}; automatic proposal refresh is paused." >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "frozen=true" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- uses: googleapis/release-please-action@v4
|
||||
if: steps.editorial.outputs.frozen != 'true'
|
||||
with:
|
||||
token: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
target-branch: main
|
||||
@@ -80,6 +94,16 @@ jobs:
|
||||
node scripts/verify-release-checkpoint.mjs \
|
||||
--phase publish --release-branch main --source-sha "$SOURCE_SHA"
|
||||
|
||||
- name: Validate finalized notes before release creation
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
GH_REPO: ${{ github.repository }}
|
||||
RELEASE_PR: ${{ github.event.pull_request.number }}
|
||||
SOURCE_SHA: ${{ github.event.pull_request.merge_commit_sha }}
|
||||
run: |
|
||||
node scripts/release-notes.mjs publication --repository "$GH_REPO" \
|
||||
--source "$SOURCE_SHA" --pr "$RELEASE_PR"
|
||||
|
||||
- uses: googleapis/release-please-action@v4
|
||||
id: release
|
||||
with:
|
||||
@@ -192,6 +216,16 @@ jobs:
|
||||
--phase "$RELEASE_PHASE" --release-branch "$RELEASE_BRANCH"
|
||||
--source-sha "$SOURCE_SHA"
|
||||
|
||||
- name: Validate manual finalized notes before release creation
|
||||
if: inputs.phase == 'publish'
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
GH_REPO: ${{ github.repository }}
|
||||
SOURCE_SHA: ${{ steps.branch.outputs.sha }}
|
||||
run: |
|
||||
node scripts/release-notes.mjs publication --repository "$GH_REPO" \
|
||||
--source "$SOURCE_SHA"
|
||||
|
||||
- uses: googleapis/release-please-action@v4
|
||||
id: release
|
||||
with:
|
||||
|
||||
@@ -1,12 +1,7 @@
|
||||
name: Website checks
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches: [main, "release/*"]
|
||||
paths:
|
||||
- "website/**"
|
||||
- "scripts/*website*.py"
|
||||
- ".github/workflows/**"
|
||||
workflow_call:
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
|
||||
@@ -148,10 +148,15 @@ Generated fixtures and manifests remain ignored.
|
||||
Use the [release guide](docs/releasing.md). Main pushes automatically refresh a
|
||||
Release Please PR with the next version and changelog. Review its migration
|
||||
notes, wait for the current CI checks to pass, and merge it into main to publish.
|
||||
|
||||
The Release workflow validates and tags that merge; both publishers build its
|
||||
exact SHA. Ordinary main pushes cannot publish packages. An optional manual
|
||||
`release/VERSION` checkpoint retains explicit preparation and publication.
|
||||
|
||||
For grouped release notes with verified contributor thanks, use the repository's
|
||||
`rea-changelog-update` skill in `.agents/skills/rea-changelog-update/`. Its inventory,
|
||||
credit checks, and finalization process are described in the release guide.
|
||||
|
||||
Keep new implementation commits on main for the next release. The workflow
|
||||
owns packaged-artifact verification, npm publication, the published CLI/MCP
|
||||
canary, and MCP Registry publication. Sync manual checkpoint metadata back to
|
||||
|
||||
@@ -0,0 +1,151 @@
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.program.model.address.Address;
|
||||
import ghidra.program.model.address.AddressSpace;
|
||||
import ghidra.program.model.listing.Function;
|
||||
import ghidra.program.model.listing.Program;
|
||||
import ghidra.program.model.mem.MemoryBlock;
|
||||
import ghidra.program.model.symbol.SourceType;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.LinkOption;
|
||||
import java.nio.file.Path;
|
||||
import java.security.MessageDigest;
|
||||
import java.util.ArrayList;
|
||||
import java.util.HexFormat;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
* Apply caller-declared analysis seeds before default auto-analysis.
|
||||
*
|
||||
* The single argument is REA's canonical seed snapshot: one
|
||||
* {@code 0x<hex>\t(function|code|label)[\tname]} line per seed. Code seeds are
|
||||
* applied after functions (highest address first), then labels. The outcome is stored as a
|
||||
* program property that the bridge reports during its handshake.
|
||||
*/
|
||||
public final class ReaGhidraApplySeeds extends GhidraScript {
|
||||
static final String REPORT_OPTION = "REA analysis seeds";
|
||||
|
||||
private record Seed(Address address, String kind, String name) {}
|
||||
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
String[] args = getScriptArgs();
|
||||
if (args.length != 1) {
|
||||
throw new IllegalArgumentException("REA seed script requires exactly one seed file");
|
||||
}
|
||||
Path path = Path.of(args[0]);
|
||||
if (!Files.isRegularFile(path, LinkOption.NOFOLLOW_LINKS)) {
|
||||
throw new IllegalArgumentException("REA seed file is not a regular file");
|
||||
}
|
||||
byte[] bytes = Files.readAllBytes(path);
|
||||
String sha256 = HexFormat.of().formatHex(
|
||||
MessageDigest.getInstance("SHA-256").digest(bytes)
|
||||
);
|
||||
AddressSpace space = currentProgram.getAddressFactory().getDefaultAddressSpace();
|
||||
List<Seed> seeds = new ArrayList<>();
|
||||
for (String line : new String(bytes, StandardCharsets.UTF_8).split("\n")) {
|
||||
if (line.isEmpty()) continue;
|
||||
String[] fields = line.split("\t", -1);
|
||||
if (fields.length < 2 || fields.length > 3 || !fields[0].startsWith("0x")) {
|
||||
throw new IllegalArgumentException("REA seed snapshot is malformed");
|
||||
}
|
||||
Address address;
|
||||
try {
|
||||
address = space.getAddress(Long.parseUnsignedLong(fields[0].substring(2), 16));
|
||||
} catch (ghidra.program.model.address.AddressOutOfBoundsException e) {
|
||||
address = null;
|
||||
}
|
||||
seeds.add(new Seed(address, fields[1], fields.length == 3 ? fields[2] : null));
|
||||
}
|
||||
|
||||
// Highest address first: a function body computed before analysis stops at
|
||||
// an existing function entry, so later creations cannot swallow earlier ones.
|
||||
seeds.sort((left, right) -> {
|
||||
if (left.address() == null) return right.address() == null ? 0 : 1;
|
||||
if (right.address() == null) return -1;
|
||||
return right.address().compareTo(left.address());
|
||||
});
|
||||
int unmapped = 0;
|
||||
int logged = 0;
|
||||
int codeDecoded = 0, codeFailed = 0;
|
||||
int functionCreated = 0, functionExisting = 0, functionFailed = 0;
|
||||
int labelApplied = 0, labelFailed = 0;
|
||||
for (String pass : new String[] { "function", "code", "label" }) {
|
||||
for (Seed seed : seeds) {
|
||||
if (!seed.kind().equals(pass)) continue;
|
||||
monitor.checkCancelled();
|
||||
MemoryBlock block = seed.address() == null
|
||||
? null
|
||||
: currentProgram.getMemory().getBlock(seed.address());
|
||||
if (block == null || (!pass.equals("label") && !block.isInitialized())) {
|
||||
unmapped++;
|
||||
continue;
|
||||
}
|
||||
switch (pass) {
|
||||
case "code" -> {
|
||||
if (decoded(seed.address())) codeDecoded++;
|
||||
else {
|
||||
codeFailed++;
|
||||
if (logged++ < 20) println("REA seed failed: code " + seed.address() + " " + why(seed.address()));
|
||||
}
|
||||
}
|
||||
case "function" -> {
|
||||
Function existing = getFunctionAt(seed.address());
|
||||
if (existing != null) {
|
||||
functionExisting++;
|
||||
if (seed.name() != null) {
|
||||
existing.setName(seed.name(), SourceType.USER_DEFINED);
|
||||
}
|
||||
} else if (decoded(seed.address()) &&
|
||||
createFunction(seed.address(), seed.name()) != null) {
|
||||
functionCreated++;
|
||||
} else {
|
||||
functionFailed++;
|
||||
if (logged++ < 20) println("REA seed failed: function " + seed.address() + " " + why(seed.address()));
|
||||
}
|
||||
}
|
||||
default -> {
|
||||
try {
|
||||
createLabel(seed.address(), seed.name(), true, SourceType.USER_DEFINED);
|
||||
labelApplied++;
|
||||
} catch (Exception e) {
|
||||
labelFailed++;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
String report = "{\"format\":\"rea-ghidra-seeds-v1\",\"sha256\":\"" + sha256 + "\"" +
|
||||
",\"entries\":" + seeds.size() +
|
||||
",\"function_created\":" + functionCreated +
|
||||
",\"function_existing\":" + functionExisting +
|
||||
",\"function_failed\":" + functionFailed +
|
||||
",\"code_decoded\":" + codeDecoded +
|
||||
",\"code_failed\":" + codeFailed +
|
||||
",\"label_applied\":" + labelApplied +
|
||||
",\"label_failed\":" + labelFailed +
|
||||
",\"unmapped\":" + unmapped + "}";
|
||||
currentProgram.getOptions(Program.PROGRAM_INFO).setString(REPORT_OPTION, report);
|
||||
println("REA seeds applied: " + report);
|
||||
}
|
||||
|
||||
private String why(Address address) {
|
||||
if (getDataAt(address) != null) return "defined data " + getDataAt(address).getDataType().getName();
|
||||
var containing = getInstructionContaining(address);
|
||||
if (containing != null && !containing.getAddress().equals(address))
|
||||
return "inside instruction at " + containing.getAddress();
|
||||
Function owner = getFunctionContaining(address);
|
||||
if (owner != null) return "inside function " + owner.getEntryPoint();
|
||||
return getInstructionAt(address) == null ? "undecodable" : "function creation refused";
|
||||
}
|
||||
|
||||
/** Decode one instruction flow at an address, keeping an existing decode. */
|
||||
private boolean decoded(Address address) {
|
||||
if (getInstructionAt(address) != null) return true;
|
||||
if (getDataAt(address) != null || getInstructionContaining(address) != null) {
|
||||
return false;
|
||||
}
|
||||
disassemble(address);
|
||||
return getInstructionAt(address) != null;
|
||||
}
|
||||
}
|
||||
@@ -27,6 +27,7 @@ import java.util.Comparator;
|
||||
import java.util.HashSet;
|
||||
import java.util.IdentityHashMap;
|
||||
import java.util.Iterator;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import java.util.Map;
|
||||
@@ -80,12 +81,14 @@ import ghidra.program.database.mem.FileBytes;
|
||||
import ghidra.program.model.address.SegmentedAddress;
|
||||
import ghidra.program.model.reloc.Relocation;
|
||||
import java.io.ByteArrayOutputStream;
|
||||
import ghidra.program.model.symbol.Namespace;
|
||||
import ghidra.program.model.symbol.RefType;
|
||||
import ghidra.program.model.symbol.Reference;
|
||||
import ghidra.program.model.symbol.ReferenceIterator;
|
||||
import ghidra.program.model.symbol.SourceType;
|
||||
import ghidra.program.model.symbol.Symbol;
|
||||
import ghidra.program.model.symbol.SymbolIterator;
|
||||
import ghidra.program.model.symbol.SymbolTable;
|
||||
import ghidra.program.model.pcode.FunctionPrototype;
|
||||
import ghidra.program.model.pcode.HighFunction;
|
||||
import ghidra.program.model.pcode.HighParam;
|
||||
@@ -375,7 +378,10 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
case "inspect_native_instruction" -> inspectNativeInstruction(request.params);
|
||||
case "resolve_native_call_targets" -> resolveNativeCallTargets(request.params);
|
||||
case "xrefs" -> xrefs(request.params);
|
||||
case "analyze_function" -> analyzeFunction(request.params);
|
||||
case "analyze_function" -> {
|
||||
FunctionAnalysisObservation observation = analyzeFunctionObservation(request.params);
|
||||
yield functionObservation(observation.dossier(), observation.limitations());
|
||||
}
|
||||
case "set_address_name", "set_addresses_names" -> {
|
||||
if (!descriptor.transport.equals("unix-socket"))
|
||||
throw new RequestFailure("method_unavailable", "Windows P0 does not admit database mutation");
|
||||
@@ -432,6 +438,11 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
result.add("target", target);
|
||||
JsonArray extensions = analysisExtensions.reports();
|
||||
if (extensions.size() > 0) result.add("analysis_extensions", extensions);
|
||||
// Written by ReaGhidraApplySeeds before auto-analysis; REA checks it
|
||||
// against the committed profile.
|
||||
String seeds = currentProgram.getOptions(ghidra.program.model.listing.Program.PROGRAM_INFO)
|
||||
.getString("REA analysis seeds", null);
|
||||
if (seeds != null) result.add("analysis_seeds", JsonParser.parseString(seeds));
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -650,10 +661,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
private JsonElement procedurePseudocode(JsonObject params) throws Exception {
|
||||
requireKeys(params, Set.of("document", "procedure"));
|
||||
requireDocument(params);
|
||||
String value = decompiledText(
|
||||
decompile(resolveProcedure(requireString(params, "procedure")))
|
||||
Function function = resolveProcedure(requireString(params, "procedure"));
|
||||
String value = decompiledText(decompile(function));
|
||||
return functionObservation(
|
||||
value == null ? JsonNull.INSTANCE : GSON.toJsonTree(value),
|
||||
noReturnCallLimitations(
|
||||
currentProgram.getListing().getInstructions(function.getBody(), true)
|
||||
)
|
||||
);
|
||||
return value == null ? JsonNull.INSTANCE : GSON.toJsonTree(value);
|
||||
}
|
||||
|
||||
private JsonElement procedureAssembly(JsonObject params) throws Exception {
|
||||
@@ -661,7 +676,10 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
requireDocument(params);
|
||||
Function function = resolveProcedure(requireString(params, "procedure"));
|
||||
InstructionScan scan = scanInstructions(function, Integer.MAX_VALUE);
|
||||
return GSON.toJsonTree(renderAssembly(scan.instructions));
|
||||
return functionObservation(
|
||||
GSON.toJsonTree(renderAssembly(scan.instructions)),
|
||||
noReturnCallLimitations(scan.instructions)
|
||||
);
|
||||
}
|
||||
|
||||
private JsonObject readFunctionInstructions(JsonObject params) throws Exception {
|
||||
@@ -679,11 +697,13 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
limitations.add(
|
||||
"The fast path does not invoke the decompiler or scan whole-program names and strings."
|
||||
);
|
||||
List<String> boundaryLimitations = noReturnCallLimitations(scan.instructions);
|
||||
for (String limitation : boundaryLimitations) limitations.add(limitation);
|
||||
JsonObject result = new JsonObject();
|
||||
result.add("procedure", procedureIdentity(function));
|
||||
result.add("instructions", items);
|
||||
result.add("limitations", limitations);
|
||||
return result;
|
||||
return functionObservation(result, boundaryLimitations);
|
||||
}
|
||||
|
||||
private JsonArray procedureCalls(JsonObject params, boolean callers) throws Exception {
|
||||
@@ -716,7 +736,12 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
result.addProperty("signature", function.getPrototypeString(false, true));
|
||||
result.add("locals", functionLocals(function));
|
||||
result.add("classification", functionClassification(function));
|
||||
return result;
|
||||
return functionObservation(
|
||||
result,
|
||||
noReturnCallLimitations(
|
||||
currentProgram.getListing().getInstructions(function.getBody(), true)
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
private JsonObject procedureReferences(JsonObject params) throws Exception {
|
||||
@@ -1128,6 +1153,10 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
}
|
||||
|
||||
private JsonObject analyzeFunction(JsonObject params) throws Exception {
|
||||
return analyzeFunctionObservation(params).dossier();
|
||||
}
|
||||
|
||||
private FunctionAnalysisObservation analyzeFunctionObservation(JsonObject params) throws Exception {
|
||||
requireKeys(params, Set.of("procedure"));
|
||||
Function function = resolveProcedure(requireString(params, "procedure"));
|
||||
InstructionScan scan = scanInstructions(function, Integer.MAX_VALUE);
|
||||
@@ -1185,12 +1214,16 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
limitations.add(
|
||||
"Synthetic Ghidra entry-point references without actionable memory sources are omitted."
|
||||
);
|
||||
List<String> boundaryLimitations = noReturnCallLimitations(scan.instructions);
|
||||
for (String limitation : boundaryLimitations) {
|
||||
limitations.add(limitation);
|
||||
}
|
||||
if (decompilation != null && decompilation.getErrorMessage() != null &&
|
||||
!decompilation.getErrorMessage().isBlank()) {
|
||||
limitations.add("Ghidra decompiler diagnostic: " + decompilation.getErrorMessage());
|
||||
}
|
||||
result.add("limitations", limitations);
|
||||
return result;
|
||||
return new FunctionAnalysisObservation(result, boundaryLimitations);
|
||||
}
|
||||
|
||||
private JsonObject nativeValueFlow(DecompileResults decompilation) throws Exception {
|
||||
@@ -1434,18 +1467,22 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
boolean commit = false;
|
||||
try {
|
||||
Function function = currentProgram.getFunctionManager().getFunctionAt(address);
|
||||
String expected = name;
|
||||
SymbolTable table = currentProgram.getSymbolTable();
|
||||
Symbol primary = function == null ? table.getPrimarySymbol(address) : function.getSymbol();
|
||||
Namespace namespace = primary == null
|
||||
? currentProgram.getGlobalNamespace()
|
||||
: primary.getParentNamespace();
|
||||
String expected = primary == null ? name : symbolLeafName(primary, name);
|
||||
for (Symbol symbol : table.getLabelOrFunctionSymbols(expected, namespace)) {
|
||||
if (!address.equals(symbol.getAddress())) return false;
|
||||
}
|
||||
if (function != null) {
|
||||
expected = symbolLeafName(function.getSymbol(), name);
|
||||
function.setName(expected, SourceType.USER_DEFINED);
|
||||
} else {
|
||||
var table = currentProgram.getSymbolTable();
|
||||
Symbol primary = table.getPrimarySymbol(address);
|
||||
if (primary != null && !primary.isDynamic()) {
|
||||
expected = symbolLeafName(primary, name);
|
||||
primary.setName(expected, SourceType.USER_DEFINED);
|
||||
} else {
|
||||
Symbol created = table.createLabel(address, name, SourceType.USER_DEFINED);
|
||||
Symbol created = table.createLabel(address, expected, SourceType.USER_DEFINED);
|
||||
created.setPrimary();
|
||||
}
|
||||
}
|
||||
@@ -1473,16 +1510,33 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
requireDocument(params);
|
||||
if (!params.has("names") || !params.get("names").isJsonObject())
|
||||
throw new RequestFailure("invalid_request", "names must be an object of address -> name");
|
||||
Map<Address, List<AddressNameRequest>> requestsByAddress = new LinkedHashMap<>();
|
||||
JsonObject result = new JsonObject();
|
||||
for (Map.Entry<String, JsonElement> entry : params.getAsJsonObject("names").entrySet()) {
|
||||
boolean ok;
|
||||
try {
|
||||
ok = applyAddressName(parseReaAddress(entry.getKey()), entry.getValue().getAsString());
|
||||
Address address = parseReaAddress(entry.getKey());
|
||||
requestsByAddress.computeIfAbsent(address, ignored -> new ArrayList<>())
|
||||
.add(new AddressNameRequest(entry.getKey(), address, entry.getValue().getAsString()));
|
||||
}
|
||||
catch (RequestFailure failure) {
|
||||
ok = false;
|
||||
result.addProperty(entry.getKey(), false);
|
||||
}
|
||||
result.addProperty(entry.getKey(), ok);
|
||||
}
|
||||
for (List<AddressNameRequest> requests : requestsByAddress.values()) {
|
||||
String name = requests.get(0).name;
|
||||
boolean conflictingAliasNames = requests.stream()
|
||||
.anyMatch(request -> !request.name.equals(name));
|
||||
boolean success = false;
|
||||
if (!conflictingAliasNames) {
|
||||
try {
|
||||
success = applyAddressName(requests.get(0).address, name);
|
||||
}
|
||||
catch (RequestFailure failure) {
|
||||
// Keep per-address failures in the existing batch result shape.
|
||||
}
|
||||
}
|
||||
for (AddressNameRequest request : requests)
|
||||
result.addProperty(request.inputAddress, success);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
@@ -2648,6 +2702,63 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return new InstructionScan(List.copyOf(instructions), iterator.hasNext());
|
||||
}
|
||||
|
||||
private JsonObject functionObservation(
|
||||
JsonElement value,
|
||||
List<String> limitations) {
|
||||
JsonObject result = new JsonObject();
|
||||
result.add("value", value);
|
||||
result.add("limitations", GSON.toJsonTree(limitations));
|
||||
return result;
|
||||
}
|
||||
|
||||
private List<String> noReturnCallLimitations(Iterable<Instruction> instructions)
|
||||
throws Exception {
|
||||
List<String> limitations = new ArrayList<>();
|
||||
for (Instruction instruction : instructions) {
|
||||
monitor.checkCancelled();
|
||||
if (!instruction.getFlowType().isCall() || !instruction.getFlowType().isTerminal()) {
|
||||
continue;
|
||||
}
|
||||
Address[] destinations = instruction.getFlows();
|
||||
StringBuilder limitation = new StringBuilder();
|
||||
limitation.append("Ghidra Listing reports a terminal call at ")
|
||||
.append(canonicalAddress(instruction.getAddress()));
|
||||
if (destinations.length == 0) {
|
||||
limitation.append(" with no resolved callee; its no-return basis is unknown.");
|
||||
}
|
||||
else {
|
||||
limitation.append(" to ");
|
||||
for (int index = 0; index < destinations.length; index++) {
|
||||
if (index > 0) limitation.append(", ");
|
||||
Address destination = destinations[index];
|
||||
limitation.append(canonicalAddress(destination));
|
||||
Function target = currentProgram.getFunctionManager().getFunctionAt(destination);
|
||||
if (target == null) {
|
||||
limitation.append(" (callee identity unknown)");
|
||||
}
|
||||
else {
|
||||
limitation.append(" ").append(procedureName(target));
|
||||
limitation.append(" (external=").append(target.isExternal());
|
||||
limitation.append(", thunk=").append(target.isThunk());
|
||||
limitation.append(", hasNoReturn=").append(target.hasNoReturn()).append(")");
|
||||
}
|
||||
}
|
||||
limitation.append("; fallthrough is excluded by Ghidra's flow model.");
|
||||
Function target = destinations.length == 1
|
||||
? currentProgram.getFunctionManager().getFunctionAt(destinations[0])
|
||||
: null;
|
||||
if (target != null && target.hasNoReturn()) {
|
||||
limitation.append(" This records Ghidra's FunctionManager flag, not an independent verification that the callee cannot return.");
|
||||
}
|
||||
else {
|
||||
limitation.append(" The reason for the terminal flow is not established by this observation.");
|
||||
}
|
||||
}
|
||||
limitations.add(limitation.toString());
|
||||
}
|
||||
return limitations;
|
||||
}
|
||||
|
||||
private static String renderAssembly(List<Instruction> instructions) {
|
||||
return String.join("\n", renderAssemblyLines(instructions));
|
||||
}
|
||||
@@ -3476,7 +3587,9 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
.thenComparing(Reference::isPrimary);
|
||||
|
||||
private record InventoryItem(Address address, String value, JsonObject facts) {}
|
||||
private record AddressNameRequest(String inputAddress, Address address, String name) {}
|
||||
private record FunctionEntry(Function function, InventoryItem item) {}
|
||||
private record FunctionAnalysisObservation(JsonObject dossier, List<String> limitations) {}
|
||||
private record InstructionScan(List<Instruction> instructions, boolean truncated) {}
|
||||
private record SessionDescriptor(
|
||||
String transport,
|
||||
|
||||
@@ -3,20 +3,25 @@ import ghidra.app.script.GhidraScript;
|
||||
import ghidra.program.model.address.Address;
|
||||
import ghidra.program.model.listing.*;
|
||||
import ghidra.program.model.symbol.FlowType;
|
||||
import ghidra.program.model.symbol.Reference;
|
||||
import ghidra.program.model.pcode.PcodeOp;
|
||||
import java.util.ArrayList;
|
||||
import java.util.ArrayDeque;
|
||||
import java.util.Deque;
|
||||
import java.util.HashSet;
|
||||
import java.util.List;
|
||||
import java.util.Set;
|
||||
import java.util.TreeSet;
|
||||
|
||||
/**
|
||||
* Undo wrong no-return flags from auto-analysis. A function flagged no-return that contains a
|
||||
* real RET is an ordinary function (stream readers, cxa_guard_acquire, ...). Callers treated the
|
||||
* call as a terminator and the decompiler truncated them. Clear the flag, then re-disassemble only
|
||||
* real RET is an ordinary function (stream readers, cxa_guard_acquire, ...). Known returning
|
||||
* libc/TLS imports have no body to inspect, so clear their flags by exact external ABI name.
|
||||
* Callers treated the call as a terminator and the decompiler truncated them. Clear the flag, then re-disassemble only
|
||||
* the call sites of the functions that were cleared and refit their owning functions in place.
|
||||
*/
|
||||
public final class ReaGhidraNoReturnFix extends GhidraScript {
|
||||
private static final Set<String> RETURNING_IMPORTS = Set.of(
|
||||
"malloc", "__tls_get_addr", "memcpy", "memset", "strlen", "__errno_location");
|
||||
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
// Windows P0 does not admit analysis-database mutation.
|
||||
@@ -24,66 +29,95 @@ public final class ReaGhidraNoReturnFix extends GhidraScript {
|
||||
FunctionManager fm = currentProgram.getFunctionManager();
|
||||
Listing li = currentProgram.getListing();
|
||||
Set<Address> cleared = new HashSet<>();
|
||||
for (Function f : fm.getFunctions(true)) {
|
||||
Deque<Address> pending = new ArrayDeque<>();
|
||||
for (Function f : fm.getExternalFunctions()) {
|
||||
monitor.checkCancelled();
|
||||
if (!f.hasNoReturn() || f.isThunk()) continue;
|
||||
for (Instruction ins : li.getInstructions(f.getBody(), true)) {
|
||||
FlowType ft = ins.getFlowType();
|
||||
boolean decodedReturn = false;
|
||||
for (PcodeOp operation : ins.getPcode()) {
|
||||
if (operation.getOpcode() == PcodeOp.RETURN) decodedReturn = true;
|
||||
}
|
||||
if (decodedReturn && !ft.isJump() && !ft.isCall()) {
|
||||
// Do not apply import-name assumptions to local functions or to arbitrary externals.
|
||||
if (!f.hasNoReturn() || !RETURNING_IMPORTS.contains(externalAbiName(f))) continue;
|
||||
f.setNoReturn(false);
|
||||
cleared.add(f.getEntryPoint());
|
||||
break;
|
||||
enqueueReturning(f, cleared, pending);
|
||||
}
|
||||
}
|
||||
}
|
||||
// A thunk inherits the flag of the function it forwards to.
|
||||
for (Function f : fm.getFunctions(true)) {
|
||||
monitor.checkCancelled();
|
||||
if (!f.isThunk()) continue;
|
||||
Function target = f.getThunkedFunction(true);
|
||||
if (target != null && cleared.contains(target.getEntryPoint())) {
|
||||
cleared.add(f.getEntryPoint());
|
||||
if (clearDecodedReturn(f, li)) {
|
||||
enqueueReturning(f, cleared, pending);
|
||||
}
|
||||
}
|
||||
int redone = 0;
|
||||
TreeSet<Address> owners = new TreeSet<>();
|
||||
if (!cleared.isEmpty()) {
|
||||
List<Address> sites = new ArrayList<>();
|
||||
for (Instruction ins : li.getInstructions(true)) {
|
||||
while (!pending.isEmpty()) {
|
||||
Address target = pending.removeFirst();
|
||||
TreeSet<Address> sites = new TreeSet<>();
|
||||
// References include computed import calls after Ghidra resolves their destination.
|
||||
// Collect before redisassembly mutates the reference table.
|
||||
for (Reference reference : currentProgram.getReferenceManager().getReferencesTo(target)) {
|
||||
monitor.checkCancelled();
|
||||
FlowType ft = ins.getFlowType();
|
||||
if (!(ft.isCall() && ft.isTerminal())) continue;
|
||||
Address[] flows = ins.getFlows();
|
||||
if (flows.length == 0 || !cleared.contains(flows[0])) continue;
|
||||
// Do not redisassemble into an independently identified function.
|
||||
Address next = ins.getMaxAddress().add(1);
|
||||
if (fm.getFunctionAt(next) != null || !currentProgram.getMemory().contains(next)) continue;
|
||||
sites.add(ins.getAddress());
|
||||
sites.add(reference.getFromAddress());
|
||||
}
|
||||
for (Address s : sites) {
|
||||
monitor.checkCancelled();
|
||||
Instruction ins = li.getInstructionAt(s);
|
||||
if (ins == null) continue;
|
||||
FlowType ft = ins.getFlowType();
|
||||
if (!(ft.isCall() && ft.isTerminal())) continue;
|
||||
Address[] flows = ins.getFlows();
|
||||
boolean callsTarget = false;
|
||||
for (Address flow : flows) {
|
||||
if (flow.equals(target)) callsTarget = true;
|
||||
}
|
||||
if (!callsTarget) continue;
|
||||
// Do not redisassemble into an independently identified function.
|
||||
Address next = ins.getMaxAddress().add(1);
|
||||
if (fm.getFunctionAt(next) != null || !currentProgram.getMemory().contains(next)) continue;
|
||||
Function owner = fm.getFunctionContaining(s);
|
||||
if (owner != null && !owner.isThunk()) owners.add(owner.getEntryPoint());
|
||||
clearListing(ins.getMinAddress(), ins.getMaxAddress());
|
||||
disassemble(s);
|
||||
redone++;
|
||||
}
|
||||
for (Address e : owners) {
|
||||
monitor.checkCancelled();
|
||||
Function f = fm.getFunctionAt(e);
|
||||
// Refit the body in place so the name, signature and calling convention survive.
|
||||
if (f != null) CreateFunctionCmd.fixupFunctionBody(currentProgram, f, monitor);
|
||||
if (owner != null && !owner.isThunk()) {
|
||||
CreateFunctionCmd.fixupFunctionBody(currentProgram, owner, monitor);
|
||||
// Auto-analysis may have propagated the false flag to this truncated caller.
|
||||
// Its RET becomes visible only now; repair its callers in the same pass.
|
||||
if (clearDecodedReturn(owner, li)) enqueueReturning(owner, cleared, pending);
|
||||
}
|
||||
}
|
||||
}
|
||||
// No analyzeChanges here: on libpl2.so it adds about 70 s, which pushes startup past the 330 s deadline.
|
||||
}
|
||||
println(
|
||||
"REA no-return fix: cleared=" + cleared.size() + " sites=" + redone + " refit=" + owners.size());
|
||||
}
|
||||
|
||||
private void enqueueReturning(Function function, Set<Address> cleared, Deque<Address> pending) {
|
||||
if (cleared.add(function.getEntryPoint())) pending.addLast(function.getEntryPoint());
|
||||
// A thunk inherits the flag of the function it forwards to, including through other thunks.
|
||||
Address[] thunks = function.getFunctionThunkAddresses(true);
|
||||
if (thunks == null) return;
|
||||
for (Address thunk : thunks) {
|
||||
if (cleared.add(thunk)) pending.addLast(thunk);
|
||||
}
|
||||
}
|
||||
|
||||
private String externalAbiName(Function function) {
|
||||
String name = function.getName();
|
||||
// Mach-O stores C-linkage symbols with one object-format underscore prefix.
|
||||
if (currentProgram.getExecutableFormat().contains("Mach-O") && name.startsWith("_"))
|
||||
return name.substring(1);
|
||||
return name;
|
||||
}
|
||||
|
||||
private boolean clearDecodedReturn(Function function, Listing listing) throws Exception {
|
||||
if (!function.hasNoReturn() || function.isThunk()) return false;
|
||||
for (Instruction instruction : listing.getInstructions(function.getBody(), true)) {
|
||||
monitor.checkCancelled();
|
||||
FlowType flow = instruction.getFlowType();
|
||||
if (flow.isJump() || flow.isCall()) continue;
|
||||
for (PcodeOp operation : instruction.getPcode()) {
|
||||
if (operation.getOpcode() != PcodeOp.RETURN) continue;
|
||||
function.setNoReturn(false);
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -53,8 +53,19 @@ google-chrome \
|
||||
|
||||
REA does not launch, own, or terminate this browser. Use a dedicated profile and stop it yourself when the investigation is complete. Do not expose the debugging port on a non-loopback interface.
|
||||
|
||||
On Linux, a reachable but locked desktop keyring can stall Chrome before its
|
||||
first navigation. For a dedicated investigation profile, add
|
||||
`--password-store=basic` to bypass that keyring. Empty or placeholder frame URLs
|
||||
then describe an unloaded page; REA continues to require an allowed, committed
|
||||
URL. See [the startup investigation](https://github.com/morluto/rea/issues/1384#issuecomment-6093781686).
|
||||
|
||||
## Request boundary
|
||||
|
||||
CDP connections enforce the shared 64 MiB limit per WebSocket message. A larger
|
||||
reply, including a valid reply from Chrome, fails with `payload_limit`. The limit
|
||||
counts message bytes before JSON parsing; it does not bound the memory used by
|
||||
decoded strings or JSON objects.
|
||||
|
||||
Discovery supplies a literal loopback CDP endpoint and returns all eligible
|
||||
page targets unless an exact HTTP(S) origin filter narrows the result. Follow-up
|
||||
inspection requests select one target; their default scope is its current origin. `localhost`, private-LAN
|
||||
|
||||
@@ -156,6 +156,18 @@ Storage cookies are scoped to the page URL at each capture. Unsupported action
|
||||
tags, unknown fields, duplicate step IDs, and undeclared secret references fail
|
||||
validation.
|
||||
|
||||
Cookies are installed before the initial navigation. Local storage entries are
|
||||
seeded once per browser storage partition; session storage entries are seeded
|
||||
once per origin in each top-level browsing context. Seeding occurs before the
|
||||
first document scripts for that area. An origin first reached by a later action
|
||||
receives its seed on that first visit. Application updates, deletions, and
|
||||
empty values then persist across later navigations in the same area; a new
|
||||
scenario starts with its own seeds. Partitioned frames can receive separate
|
||||
local storage seeds even when they share an origin. The pre-script mechanism
|
||||
uses CDP debugger instrumentation. The observer resumes only pauses whose top
|
||||
frame belongs to its private seed script; page breakpoints and `debugger`
|
||||
statements remain under the attached debugger client's control.
|
||||
|
||||
The result is Evidence with an initial state followed by one record per
|
||||
declared action. Each step reports action status, elapsed time, sanitized URLs,
|
||||
event bounds, and independently typed capture states for screenshot, DOM,
|
||||
|
||||
+31
-1
@@ -12,6 +12,9 @@ configuration, startup deadlines, resource controls, and cleanup recovery.
|
||||
extension. Check the [release boundary](installation.md#released-package-and-main)
|
||||
when using npm.
|
||||
|
||||
See [import language and analysis seeds](installation.md#ghidra-import-language-and-analysis-seeds)
|
||||
to override the header-detected language or seed analysis.
|
||||
|
||||
Provider selection and profile identity follow
|
||||
[MCP contracts](mcp-contracts.md#identity-and-discovery). One deep
|
||||
provider stays bound to the target; runtime failure never selects another
|
||||
@@ -29,9 +32,15 @@ not equivalence with another engine's pseudocode or analysis database.
|
||||
| Strings | Only Ghidra-defined string `Data` is observed. Items report charset, byte length, and whether a required null terminator is missing. The API cannot distinguish a present terminator from fixed/Pascal layouts when no terminator is missing, so that state is named `present_or_not_required`. |
|
||||
| Memory | Memory-block end addresses are exclusive. Read/write/execute, initialization, overlay, address space, and image base are direct Ghidra observations. |
|
||||
| Inventory | Procedure, symbol, and string listings plus searches return the complete matching collection in one response; callers do not provide offsets or result-count limits. |
|
||||
| Search | Literal and Java-regex searches scan the complete immutable inventory and return all matching entries inline. Search waits for a reply or caller cancellation; there are no caller-supplied offsets or result-count limits. |
|
||||
| Search | Literal and Java-regex searches return every match in the immutable provider-defined inventory; string search does not cover every text sequence in artifact bytes. Search waits for a reply or caller cancellation; there are no caller-supplied offsets or result-count limits. |
|
||||
| Analysis state | The socket is exposed only after default auto-analysis completes. Established operations wait for their reply or caller cancellation; there is no fixed response-size ceiling. |
|
||||
|
||||
String-search completeness applies only to Ghidra-defined string `Data`. Packed
|
||||
or undefined literals, including Rust strings, may be absent; an empty result
|
||||
does not prove text is absent. Both list and search Evidence carry this
|
||||
limitation. When an address is known, use `read_bytes` and `xrefs` to inspect
|
||||
bytes and references.
|
||||
|
||||
## Admitted function-analysis semantics
|
||||
|
||||
| Concern | Ghidra contract |
|
||||
@@ -46,6 +55,27 @@ not equivalence with another engine's pseudocode or analysis database.
|
||||
| P-code value flow | Function dossiers include up to 3,000 high-p-code operations, at most 64 inputs per operation, and 12,000 def-use edges. Results identify memory reads/writes, branches, and calls by p-code opcode, and mark decompiler-dead operations. Large results report truncation, an omitted-operation lower bound, and known omitted-input/edge counts. The dossier is an intra-function graph; `trace_native_values` composes derived parameter/argument and return/output dependencies across resolved calls. It does not resolve memory aliasing, call side effects, runtime behavior, persistent state, or business meaning. |
|
||||
| Result extent | Function instruction scans and native API boundary observations remain complete. P-code flow is the explicit bounded exception; its count fields distinguish exact retained counts from lower-bound or known-omitted counts. Caller cancellation and provider failures remain distinct. |
|
||||
|
||||
Function extents and instruction completeness refer to Ghidra's analyzed body.
|
||||
An inferred no-return call can terminate that body before subsequent bytes.
|
||||
Pseudocode, assembly, procedure info, instruction windows, and dossiers report
|
||||
observed terminal call sites in Evidence limitations, including resolved callee
|
||||
names, addresses, and FunctionManager flags when available. Instruction windows
|
||||
and dossiers also retain these warnings in their result limitations. An
|
||||
unresolved callee or unestablished terminal-flow basis remains explicit. These
|
||||
observations qualify the analysis; they do not prove that a no-return flag is
|
||||
incorrect or recover excluded code. Analysis profiles version this observation
|
||||
contract so older snapshots cannot replay unqualified results.
|
||||
|
||||
On non-Windows hosts, the ephemeral import repair clears no-return flags on the
|
||||
known returning external ABI names `malloc`, `__tls_get_addr`, `memcpy`, `memset`,
|
||||
`strlen`, and `__errno_location`. This relies on their expected ABI behavior;
|
||||
external implementations are not inspected. The repair re-disassembles affected
|
||||
terminal calls and refits their owning functions, preserving independent function
|
||||
entries. If a recovered caller reveals a decoded return, its propagated no-return
|
||||
flag and affected callers are repaired in the same pass. Unknown imports and
|
||||
genuine no-return imports keep their flags. Analysis profiles version this repair
|
||||
so snapshots produced before it are not reused. Windows P0 does not run it.
|
||||
|
||||
`npm run verify:ghidra` compiles the versioned C oracle into debug and stripped
|
||||
host-native targets (x86-64/AArch64 ELF on Linux or Mach-O on macOS), plus a native
|
||||
DWARF 4 type-layout object. It proves the admitted operations, external functions, resolved thunks, exports, stripped-name
|
||||
|
||||
+38
-9
@@ -68,13 +68,15 @@ selects the npm release, and persistent MCP registrations are pinned to the
|
||||
version that performed setup. Installing newer instructions does not update a
|
||||
running server or its registration.
|
||||
|
||||
The npm release checked on **2026-10-10** was **6.3.0**, published from
|
||||
The npm release checked on **2026-10-11** was **6.3.0**, published from
|
||||
[`74cf9e14`](https://github.com/morluto/rea/commit/74cf9e1401fcd89482fe39c27cc898f5858e9b72),
|
||||
with bundled skill version 34. It does not include automatic Pi or Hermes
|
||||
registration: `setup --client pi` and `setup --client hermes` reject those IDs
|
||||
before writing configuration. Their setup workflows below require a source build
|
||||
or a later release containing those integrations; `@latest` alone cannot select
|
||||
unpublished changes.
|
||||
with bundled skill version 34. It does not include automatic Qwen Code, Pi, or
|
||||
Hermes registration: `setup --client qwen_code`, `setup --client pi`, and
|
||||
`setup --client hermes` reject those IDs before writing configuration. It also
|
||||
does not honor `OPENCODE_CONFIG_DIR` when selecting OpenCode's configuration
|
||||
directory. These workflows below require a source build or a later release
|
||||
containing the relevant changes; `@latest` alone cannot select unpublished
|
||||
changes.
|
||||
|
||||
The earlier release checked on **2026-10-07** was **5.0.0** (133 MCP tools), published
|
||||
from the fixed checkpoint
|
||||
@@ -199,7 +201,10 @@ but unavailable in a project. An extension that registers `/mcp` can replace
|
||||
Pi's built-in MCP support and use its own configuration instead. Run `/reload`
|
||||
in an existing Pi session after changing servers outside the session.
|
||||
|
||||
For Qwen Code, setup registers `rea` in the `mcpServers` table of the
|
||||
For Qwen Code, automatic setup requires a build containing the integration;
|
||||
npm 6.3.0 does not include it. See
|
||||
[released package and main](#released-package-and-main). In a supported build,
|
||||
setup registers `rea` in the `mcpServers` table of the
|
||||
user-level `~/.qwen/settings.json`, or `$QWEN_HOME/settings.json` when configured.
|
||||
Setup and doctor follow Qwen Code's tilde and working-directory-relative home
|
||||
overrides. Qwen Code also discovers personal skills in `~/.agents/skills`,
|
||||
@@ -288,7 +293,10 @@ for its native configuration contract.
|
||||
|
||||
### Other client settings
|
||||
|
||||
For OpenCode, setup writes the V1 `mcp.rea` entry, which OpenCode V1 and V2
|
||||
For OpenCode, npm 6.3.0 does not honor `OPENCODE_CONFIG_DIR`; use a source build
|
||||
or a later release containing the override support when selecting a custom
|
||||
configuration directory. See [released package and main](#released-package-and-main).
|
||||
In a supported build, setup writes the V1 `mcp.rea` entry, which OpenCode V1 and V2
|
||||
both load. If the configuration already uses OpenCode V2's native
|
||||
`mcp.servers` table, setup registers REA there instead and replaces any earlier
|
||||
`mcp.rea` entry from REA. Setup honors `OPENCODE_CONFIG_DIR` and
|
||||
@@ -455,7 +463,7 @@ vendor-defined limits, and a paid license is optional. REA reuses any detected
|
||||
installation and preserves Hopper during uninstall.
|
||||
|
||||
The supported native host baseline is macOS 12+, Ubuntu 24.04+, Fedora 41+,
|
||||
Nobara 44+, 64-bit Arch Linux, or CachyOS. Ghidra and IDA have their own provider-specific
|
||||
Nobara 44+, 64-bit Arch Linux, CachyOS, or Omarchy. Ghidra and IDA have their own provider-specific
|
||||
host requirements; Windows Ghidra uses the [experimental P0 boundary](windows-ghidra-p0.md).
|
||||
|
||||
On macOS, approved setup downloads the official DMG, checks its published size
|
||||
@@ -725,6 +733,27 @@ is no fixed per-operation or response-size ceiling. Unresolved computed calls
|
||||
remain unknown, reference-kind provenance is preserved, and provider-specific
|
||||
pseudocode is never treated as original source or Hopper-equivalent text.
|
||||
|
||||
### Ghidra import language and analysis seeds
|
||||
|
||||
Ghidra detects the language from the executable header. To set it (for example, Borland C++ PE images):
|
||||
|
||||
```bash
|
||||
export REA_GHIDRA_LANGUAGE_ID="x86:LE:32:default"
|
||||
export REA_GHIDRA_COMPILER_SPEC_ID="borlandcpp" # optional; requires a language
|
||||
```
|
||||
|
||||
REA passes `-processor`/`-cspec`, checks both against the installation's `.ldefs`, and records them in the analysis profile. DOS imports reject an override.
|
||||
|
||||
To seed code that auto-analysis cannot reach (computed jumps, pointers in data):
|
||||
|
||||
```bash
|
||||
export REA_GHIDRA_SEED_FILE=/absolute/path/to/seeds.tsv
|
||||
```
|
||||
|
||||
Each line is `address<TAB>kind[<TAB>name]`, where kind is `function`, `code` or `label` (a label needs a name). Addresses are `0x` hex in the default space; `#` lines are comments. A pre-script applies seeds before auto-analysis: functions from the highest address down, then code, then labels. The seed digest is part of the analysis profile, and the session fails if the applied seeds differ. Result limitations report the outcome counts; a seed fails when it overlaps an instruction Ghidra already decoded.
|
||||
|
||||
Label seeds can name mapped uninitialized storage such as `.bss`; function and code seeds require initialized bytes.
|
||||
|
||||
### Ghidra heap and CPU controls
|
||||
|
||||
Set resource controls in the environment that launches the CLI or MCP server,
|
||||
|
||||
@@ -270,6 +270,14 @@ substitute version-dependent Electron defaults for omitted `webPreferences`.
|
||||
Dynamic option objects, bridge keys, API objects, and IPC channel expressions
|
||||
remain unknown and make coverage partial.
|
||||
|
||||
Electron API identity follows proven lexical module aliases. Reassigned bindings,
|
||||
dynamic selections, and visibly overwritten namespace members are unresolved.
|
||||
Direct and destructuring assignment, loop, update, and delete targets are checked
|
||||
through namespace aliases; unrelated member writes do not discard observed API
|
||||
identity. This static check does
|
||||
not evaluate arbitrary call side effects or runtime registration. Missing findings
|
||||
do not establish that an application has no Electron boundary usage.
|
||||
|
||||
IPC pairing is an inference, not an observation. A renderer `invoke` pairs only
|
||||
with one unique `ipcMain.handle`/`handleOnce` candidate on the same exact literal
|
||||
channel; a renderer send pairs only with one unique `ipcMain.on`/`once`
|
||||
|
||||
@@ -167,6 +167,9 @@ Native call tracing and process capture retain available observations in
|
||||
`details.partial_observation` on failure, including when cleanup succeeds.
|
||||
The observation reports its partial coverage; cleanup details describe host
|
||||
state separately from the execution failure.
|
||||
A process capture that fails during a finalization interval keeps every signal
|
||||
attempt and its delivery result in `details.partial_observation`, with
|
||||
`elapsed_ms` and `exit_code` null when no exit was observed.
|
||||
Derived comparisons and reconstruction verification yield before computation
|
||||
and before publication, so cancellation cannot race with successful Evidence.
|
||||
|
||||
|
||||
@@ -181,6 +181,100 @@ run the same owned-process cleanup path. Settlement reports whether the
|
||||
sampled process group quiesced or whether cleanup was needed or unverifiable;
|
||||
sampling cannot prove that every short-lived or detached descendant was seen.
|
||||
|
||||
### Finalization interval
|
||||
|
||||
By default a deadline sends `SIGKILL` at once. Set `finalization_ms` to give the
|
||||
target time to finish: when `timeout_ms` or `idle_timeout_ms` fires, REA sends
|
||||
`SIGTERM`, keeps capturing output and selected files, and sends `SIGKILL` once
|
||||
if the interval ends first. Cancellation is not delayed: it sends `SIGKILL`
|
||||
immediately, also during finalization, and ends the run as cancelled.
|
||||
|
||||
```json
|
||||
{
|
||||
"executable": "node",
|
||||
"arguments": ["./job.mjs"],
|
||||
"filesystem_observation_paths": ["./reports"],
|
||||
"timeout_ms": 30000,
|
||||
"finalization_ms": 1500
|
||||
}
|
||||
```
|
||||
|
||||
REA binds the captured root's first start identity to its ownership token,
|
||||
checking the identity before and after reading that token. A missing or
|
||||
mismatched token, or an identity change during validation, leaves the identity
|
||||
unverified rather than adopting a replacement process. Both finalization
|
||||
signals recheck that retained identity immediately before the call; the check
|
||||
and signal remain separate operating-system operations. REA does not fall back
|
||||
to a bare PID kill for either finalization signal; failed identity or ownership
|
||||
validation is recorded as `unverified` with its reason. Cancellation before any
|
||||
deadline keeps the existing terminal kill. Descendants are not signalled during
|
||||
finalization; owned-process cleanup handles them afterwards.
|
||||
|
||||
`exit.reason` keeps the initiating deadline (`timeout` or `idle_timeout`).
|
||||
`exit.code` is `null` for every deadline exit; the code of an exit observed
|
||||
during finalization is `exit.finalization.exit_code`, and the observed signal
|
||||
stays in `exit.signal`. `exit.finalization` records what was observed and is
|
||||
absent when no finalization was attempted:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": null,
|
||||
"signal": 0,
|
||||
"reason": "timeout",
|
||||
"finalization": {
|
||||
"requested_ms": 1500,
|
||||
"elapsed_ms": 12,
|
||||
"exit_code": 0,
|
||||
"signals": [
|
||||
{ "signal": "SIGTERM", "sent_at_ms": 0, "delivery": "signaled" }
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- `signals` lists each attempt in order: `SIGTERM` first, at most one `SIGKILL`.
|
||||
`sent_at_ms` is the attempt time on a monotonic clock, relative to the start of
|
||||
finalization, also for an attempt that could not be delivered.
|
||||
- `delivery` is `signaled` (the identity matched and the signal call returned),
|
||||
`gone` (no live process matched), `identity-changed` (the PID now belongs to a
|
||||
different process, which was not signalled) or `unverified` (the identity could
|
||||
not be established or the call failed). None of them proves the process
|
||||
exited.
|
||||
An `unverified` attempt may carry a `reason` string naming why: an unavailable
|
||||
identity and its reason, an inspection or signal error, or delivery that did
|
||||
not settle within the bound (1000 ms). An attempt recorded `unverified` after
|
||||
that bound can still complete later; it stays identity-checked, and its late
|
||||
result is not added to the record.
|
||||
- `elapsed_ms` is when REA observed the exit, relative to the start of
|
||||
finalization, or `null` when no exit was observed. The PTY layer can deliver the
|
||||
exit slightly after the operating system ended the process.
|
||||
- `exit_code` is the code the PTY reported when the exit was observed during
|
||||
finalization and was not caused by a signal (`exit.signal` is `0`); a
|
||||
finalizer that exits with `3` records `3`, one that exits cleanly records `0`.
|
||||
It is `null` when no exit was observed or the target ended on a signal, and it
|
||||
is never inferred. A target that exited on its own near the deadline is
|
||||
recorded the same way.
|
||||
- The observed signal stays in `exit.signal`; `exit.code` is `null` for every
|
||||
deadline exit. The record does not attribute an exit to a particular signal,
|
||||
and a `SIGKILL` from a scenario event next to the escalation cannot be told
|
||||
apart from it.
|
||||
- If the `SIGKILL` cannot be delivered, REA stops waiting for an exit, keeps every
|
||||
observation collected so far with the delivery results, and releases the run
|
||||
through owned-process cleanup.
|
||||
- The `SIGKILL` is not sequenced after the `SIGTERM` delivery result: the two
|
||||
deliveries are independent, because cancellation must not wait for a slow
|
||||
identity check.
|
||||
|
||||
`timeout_ms + finalization_ms + settle_ms` must stay at or below
|
||||
`Number.MAX_SAFE_INTEGER` (9007199254740991), the largest total exactly
|
||||
representable as a JavaScript number; the scenario is rejected otherwise. This is
|
||||
a representability limit, not a timer limit. The nominal lifecycle budget is that
|
||||
sum; polling, snapshotting and cleanup lie outside it. Scenario `events` keep
|
||||
being dispatched during finalization and are recorded as usual. `finalization_ms`
|
||||
of `0`, the default, keeps today's behaviour and the committed identity of every
|
||||
existing scenario. `elapsed_ms` and `sent_at_ms` are timing data and take part in
|
||||
exit comparison like `settlement.elapsed_ms`.
|
||||
|
||||
Every capture requires `truncation_details`, with separate accounting for:
|
||||
|
||||
- `raw_terminal`: original UTF-8 PTY chunk bytes and observed/retained chunk
|
||||
|
||||
@@ -15,6 +15,48 @@ the original run after a partial publication.
|
||||
For a release that needs an independently frozen application checkpoint, use
|
||||
the optional manual path below. It retains explicit prepare and publish phases.
|
||||
|
||||
## Contributor-aware release notes
|
||||
|
||||
Use [the rea-changelog-update skill](https://github.com/morluto/rea/blob/main/.agents/skills/rea-changelog-update/SKILL.md)
|
||||
to turn the bot draft into grouped Highlights, Changes, and Fixes with inline
|
||||
`Thanks @...` credit. `npm run release:notes -- inventory` collects the full
|
||||
selected Git range and paginated GitHub PR/author/closing-issue metadata;
|
||||
`record` produces the complete contribution record, `check` validates credits
|
||||
and optional PR-body equality, and `render` writes the verified release section.
|
||||
Run a subcommand without its required options to see its usage. GitHub reads
|
||||
require an authenticated `gh`; record/check/render work offline from the saved
|
||||
inventory and complete local Git history.
|
||||
|
||||
Save a finalized inventory on the release branch at
|
||||
`docs/releases/VERSION.contributions.json`. Review unknown identities and
|
||||
reverted/backported work explicitly; GitHub metadata does not establish the
|
||||
quality of a user-facing claim. Preserve the existing breaking-change section
|
||||
and checkpoint checks. The contribution inventory is a reviewed snapshot, not
|
||||
a signed attestation; the offline checker verifies local range coverage and
|
||||
credit consistency, not GitHub's authorship independently.
|
||||
|
||||
Before editing the live PR, add `rea:release-notes-finalized` to prevent ordinary
|
||||
main-push proposals from overwriting the editorial notes. Wait for any already
|
||||
running proposal to finish. The guard applies to main-push automation; do not
|
||||
run manual checkpoint preparation over finalized notes. Remove the label to
|
||||
resume bot regeneration. The label does not freeze the base branch: new base
|
||||
commits require re-inventorying and re-reviewing the notes.
|
||||
|
||||
Update both the release branch's `CHANGELOG.md` and the PR body, preserving
|
||||
Release Please's surrounding metadata. Before creating a release, the workflow
|
||||
checks an inventory when present, requires its target to equal the reviewed
|
||||
merge's first parent, and compares the freshly fetched PR body to the changelog.
|
||||
It rejects changes outside the release metadata paths and stale credit records
|
||||
before any tag or publication. Review the diffs within allowed metadata paths
|
||||
as well, especially package dependencies and scripts. Existing candidates without inventories retain their current
|
||||
checks. Merging still starts publication; a notes edit does not authorize a
|
||||
merge or release by itself.
|
||||
|
||||
The checker supports inline Release Please PR bodies up to GitHub's 65,536
|
||||
character limit, including the surrounding header/footer. Overflow-link PR
|
||||
bodies need a separate integration and fail explicitly; credits are never
|
||||
silently truncated.
|
||||
|
||||
## 1. Select a source for a manual checkpoint
|
||||
|
||||
Use the next minor version proposed by Release Please, including releases with
|
||||
|
||||
+67
-6
@@ -391,6 +391,12 @@ regressions until a real fixture establishes equivalent coverage.
|
||||
|
||||
`verify:browser` exercises source-map failure isolation and expanded-output limits through the compiled CLI and stdio MCP with real Chrome. It also submits five 2 MiB source-map annotations, checks retained script identities and explicit map omissions, and closes its owned fixture target. Replacement and document-reset behavior has separate producer-boundary coverage.
|
||||
|
||||
The raw Chrome fixture launchers use `--password-store=basic` so Linux startup
|
||||
does not wait for an unlock prompt from a reachable but locked desktop keyring.
|
||||
Such a stall leaves the initial page unloaded with empty or placeholder CDP URLs;
|
||||
it does not establish permission to capture that page. See
|
||||
[the keyring diagnosis](https://github.com/morluto/rea/issues/1384#issuecomment-6093781686).
|
||||
|
||||
`verify:browser` also captures a source-owned noise canvas as a real PNG above
|
||||
8 MiB through the CLI and stdio MCP, with complete byte/digest parity and real PNG
|
||||
decoding. Its SDK client explicitly permits the larger inline JSON response;
|
||||
@@ -447,6 +453,7 @@ prove behavior on the verified Ghidra 12.1.4 and JDK 21 build.
|
||||
| Ghidra lane | Supported runner/target | Additional local tools |
|
||||
| ------------------------------------------ | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
|
||||
| `npm run verify:ghidra` | Linux x64/arm64 ELF or macOS x64/arm64 Mach-O | Host C compiler, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:seeds` | POSIX host; source-owned 32-bit x86 raw fixture; function order and BSS labels | Ghidra 12.1.4 and full JDK 21 |
|
||||
| `npm run verify:ghidra:swift` | macOS x64/arm64 Mach-O | Host Swift compiler, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:switch` | Linux x64 ELF; GCC/Clang optimized and stripped switch fixtures | GCC, Clang, GNU nm/objdump/strip, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:aarch64-jump-table` | Any supported Ghidra host; AArch64 ELF; byte/halfword tables; host ARM64 Mach-O | Clang with AArch64 target support, Ghidra 12.1.4, and full JDK 21 |
|
||||
@@ -765,12 +772,37 @@ Pull requests that change only root `README*.md` files, `docs/`, `AGENTS.md`,
|
||||
or `CONTRIBUTING.md` run formatting and generated-document validation without
|
||||
the source-test shards or native package lanes. Classification compares the PR
|
||||
head with its merge base, so later base-branch changes do not expand that scope.
|
||||
The required static and coverage aggregate jobs remain present and fail if
|
||||
classification fails. Mixed changes, unknown paths and CI configuration changes
|
||||
retain full checks. Main pushes retain the full baseline, including after
|
||||
documentation merges; newer commits cancel superseded CI for the same PR or
|
||||
main branch. Release publication has its own concurrency group and retains its
|
||||
package verification and public-registry canary.
|
||||
Static and coverage aggregate jobs remain present and fail if classification
|
||||
fails. Every code change retains the complete four-shard Linux suite; individual
|
||||
test files are not selected by imports or filenames.
|
||||
|
||||
`scripts/ci/plan.mjs` selects lanes using the ownership table in
|
||||
`scripts/ci/scopes.mjs`. Provider implementation changes run their real-provider
|
||||
verifiers and the Linux x64 installed-package/Inspector baseline. Test-only
|
||||
changes run the Linux suite without native package verification. Website changes
|
||||
run website checks. Provider workflow changes validate the workflow and run that provider's verifier.
|
||||
Manual-provider/Android workflow changes run workflow validation only.
|
||||
Shared process/contract/CLI boundaries, dependencies, packaging inputs, unknown
|
||||
paths, and changes to the planner select the full baseline. Package script-only
|
||||
edits select their owning verifier when its entrypoint is identifiable; unknown
|
||||
or shared script changes select everything. Android build-script edits retain
|
||||
the Linux suite and workflow validation; real Android verification stays manual.
|
||||
Renames account for both paths.
|
||||
|
||||
Main pushes and manual dispatch retain the full baseline, including all portable
|
||||
provider lanes and the native matrix below. A `ci:full` PR label selects the full
|
||||
baseline on the next PR run; manual dispatch can request it immediately. Each
|
||||
classification job records its selection and broad-fallback reasons in the run
|
||||
summary. Representative Git-diff cases live in
|
||||
`tests/boundary/filesystem/ciChangeScope.test.ts`.
|
||||
|
||||
`CI required` checks classification and every selected lane, rejecting failed,
|
||||
cancelled, or unexpectedly skipped work. Existing required check names are
|
||||
preserved; changing branch protection to require the new aggregate is a separate
|
||||
repository-settings operation. Newer commits cancel superseded CI for the same
|
||||
PR or main branch. Release publication has its own concurrency group and retains
|
||||
its package verification and public-registry canary. Licensed/self-hosted
|
||||
provider workflows and the Android smoke test retain their explicit triggers.
|
||||
|
||||
CI exercises the pinned Node.js runtime on native hosted runners:
|
||||
|
||||
@@ -795,6 +827,35 @@ separate Apple-artifact and real-provider lanes. Actual Hopper, Ghidra, IDA,
|
||||
browser and managed-tool claims require their corresponding verification lanes.
|
||||
Runner labels follow the [GitHub hosted-runner reference](https://docs.github.com/en/actions/reference/runners/github-hosted-runners).
|
||||
|
||||
### Termux Android browser smoke test
|
||||
|
||||
The **Real Termux browser verification** workflow is manual-only while its
|
||||
emulator setup is being validated. After the workflow reaches the default
|
||||
branch, select a revision under **Actions → Real Termux browser verification →
|
||||
Run workflow**. It boots an Android 11 (API 30) x86_64 emulator, installs the
|
||||
checksum-verified Termux 0.118.3 APK, and lets the app initialize its bootstrap.
|
||||
The host stages the selected revision using `git archive HEAD`; no workstation
|
||||
build outputs or dependencies are copied into Android.
|
||||
|
||||
`scripts/verify/termux/emulator.sh` sends a `RUN_COMMAND` intent to Termux's app
|
||||
service. Root access is limited to staging files, sending the intent, and reading
|
||||
diagnostics; installation, compilation, Node, and Chromium run as Termux's app
|
||||
UID. The in-app script installs Termux's Android-linked Node distribution,
|
||||
checks its version against `.nvmrc`, pins npm from `packageManager`, runs
|
||||
`npm ci`, and builds with `build:termux`. Termux repository packages, including
|
||||
Chromium, are resolved at run time; their installed versions are logged.
|
||||
|
||||
The verifier leaves Playwright's cache environment overrides unset, requires
|
||||
a healthy public MCP doctor result, and captures URL plus DOM from a loopback
|
||||
HTTP fixture through `capture-browser-scenario`. It checks completed steps,
|
||||
the exact URL, a DOM marker, and reported browser cleanup. This lane covers one
|
||||
Android emulator and Termux combination, not physical ARM devices, Electron,
|
||||
or the complete desktop browser suite. A workflow definition alone does not
|
||||
establish passing Android coverage; inspect its execution receipt.
|
||||
|
||||
The `termux-browser-diagnostics` artifact retains the Termux command log, exit
|
||||
status, and Android logcat on success or failure. The emulator is disposable.
|
||||
|
||||
## Developer commands
|
||||
|
||||
Use source feedback while editing, explicit boundary checks for the changed
|
||||
|
||||
@@ -7,6 +7,8 @@
|
||||
"src/contracts/errorSchemas.ts",
|
||||
"src/evaluation/AgentEvaluationFixtures.ts",
|
||||
"src/application/RealHopperAssertions.ts",
|
||||
"scripts/verify/termux/browser.mjs",
|
||||
"scripts/verify-real-ghidra-noreturn.mjs",
|
||||
"tests/processOwnershipRunner.ts"
|
||||
],
|
||||
"project": ["src/**/*.ts", "tests/**/*.ts", "scripts/**/*.mjs"],
|
||||
|
||||
Generated
+17
@@ -17,6 +17,7 @@
|
||||
"@lydell/node-pty": "1.1.0",
|
||||
"@modelcontextprotocol/client": "2.3.1",
|
||||
"@modelcontextprotocol/server": "2.3.1",
|
||||
"@xmldom/xmldom": "0.9.12",
|
||||
"@xterm/addon-serialize": "0.14.0",
|
||||
"@xterm/headless": "6.0.0",
|
||||
"@zip.js/zip.js": "2.15.0",
|
||||
@@ -64,6 +65,7 @@
|
||||
"oxlint": "1.87.0",
|
||||
"turbo": "2.11.7",
|
||||
"typescript": "7.0.2",
|
||||
"typescript-termux": "npm:typescript@5.9.3",
|
||||
"vitepress": "1.6.4",
|
||||
"vitest": "5.0.3"
|
||||
},
|
||||
@@ -7367,6 +7369,21 @@
|
||||
"@typescript/typescript-win32-x64": "7.0.2"
|
||||
}
|
||||
},
|
||||
"node_modules/typescript-termux": {
|
||||
"name": "typescript",
|
||||
"version": "5.9.3",
|
||||
"resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz",
|
||||
"integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"bin": {
|
||||
"tsc": "bin/tsc",
|
||||
"tsserver": "bin/tsserver"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=14.17"
|
||||
}
|
||||
},
|
||||
"node_modules/unbash": {
|
||||
"version": "4.0.12",
|
||||
"resolved": "https://registry.npmjs.org/unbash/-/unbash-4.0.12.tgz",
|
||||
|
||||
@@ -53,6 +53,7 @@
|
||||
"bridge/ghidra/ReaGhidraBridge.java",
|
||||
"bridge/ghidra/ReaGhidraExtensions.java",
|
||||
"bridge/ghidra/ReaGhidraPrepareCom.java",
|
||||
"bridge/ghidra/ReaGhidraApplySeeds.java",
|
||||
"bridge/ghidra/ReaGhidraNoReturnFix.java",
|
||||
"bridge/android/ReaJadxBridge.java",
|
||||
"bridge/mitmproxy/capture.py",
|
||||
@@ -83,6 +84,7 @@
|
||||
"third_party/pwndbg/LICENSE.md"
|
||||
],
|
||||
"scripts": {
|
||||
"release:notes": "node scripts/release-notes.mjs",
|
||||
"deps:check": "node scripts/check-dependency-install.mjs",
|
||||
"metadata:generate": "node scripts/run-exclusive.mjs artifacts node scripts/generate-package-metadata.mjs",
|
||||
"metadata:check": "node scripts/generate-package-metadata.mjs --check",
|
||||
@@ -104,6 +106,8 @@
|
||||
"precheck:ci": "npm run deps:check",
|
||||
"check:ci": "node scripts/run-exclusive.mjs artifacts turbo run typecheck lint format:check knip metadata:check jscpd scan:todos && npm run verify:architecture-guards",
|
||||
"build:cached": "node scripts/check-dependency-install.mjs && node scripts/run-exclusive.mjs artifacts turbo run artifacts:skills",
|
||||
"build:termux": "node scripts/run-exclusive.mjs artifacts npm run build:termux:run",
|
||||
"build:termux:run": "npm run compile:termux && npm run artifacts:skills",
|
||||
"format": "oxfmt .",
|
||||
"format:check": "oxfmt --check .",
|
||||
"lint": "oxlint && npm run verify:module-boundaries",
|
||||
@@ -153,6 +157,7 @@
|
||||
"verify:apktool": "npm run build:cached && node scripts/verify-apktool.mjs",
|
||||
"verify:adb": "npm run build:cached && node scripts/verify-adb.mjs",
|
||||
"verify:ghidra": "npm run build:cached && node scripts/verify-real-ghidra.mjs",
|
||||
"verify:ghidra:seeds": "npm run build:cached && node scripts/verify/ghidra/seeds.mjs",
|
||||
"verify:ghidra:mips": "npm run build:cached && node scripts/verify-real-ghidra-mips.mjs",
|
||||
"verify:ghidra:swift": "npm run build:cached && node scripts/verify-real-ghidra-swift.mjs",
|
||||
"verify:ida": "npm run build:cached && node scripts/verify-real-ida.mjs",
|
||||
@@ -239,6 +244,7 @@
|
||||
"verify:recorded:crash": "npm run build:cached && node scripts/verify-recorded-crash.mjs",
|
||||
"mcp-catalog:check:built": "node scripts/generate-mcp-tool-catalog.mjs --check",
|
||||
"compile": "node scripts/clean-build-output.mjs && tsc -p tsconfig.build.json",
|
||||
"compile:termux": "node scripts/clean-build-output.mjs && node --max-old-space-size=3072 node_modules/typescript-termux/lib/tsc.js -p tsconfig.build.json",
|
||||
"artifacts:mcp-catalog": "node scripts/generate-mcp-tool-catalog.mjs",
|
||||
"artifacts:skills": "node scripts/generate-skill-metadata.mjs",
|
||||
"artifacts:product-catalog": "node scripts/generate-product-catalog.mjs",
|
||||
@@ -264,6 +270,7 @@
|
||||
"@lydell/node-pty": "1.1.0",
|
||||
"@modelcontextprotocol/client": "2.3.1",
|
||||
"@modelcontextprotocol/server": "2.3.1",
|
||||
"@xmldom/xmldom": "0.9.12",
|
||||
"@xterm/addon-serialize": "0.14.0",
|
||||
"@xterm/headless": "6.0.0",
|
||||
"@zip.js/zip.js": "2.15.0",
|
||||
@@ -312,6 +319,7 @@
|
||||
"oxlint": "1.87.0",
|
||||
"turbo": "2.11.7",
|
||||
"typescript": "7.0.2",
|
||||
"typescript-termux": "npm:typescript@5.9.3",
|
||||
"vitepress": "1.6.4",
|
||||
"vitest": "5.0.3"
|
||||
}
|
||||
|
||||
@@ -78,7 +78,7 @@ const classes = await mkdtemp(join(output, "classes-"));
|
||||
try {
|
||||
const sources = [
|
||||
...(await files(join(source, "src/main/java"), ".java")),
|
||||
...(await files(join(root, "bridge/ghidra/extensions/nativeaot"), ".java")),
|
||||
...(await files(join(root, "bridge/nativeaot"), ".java")),
|
||||
];
|
||||
const jars = await files(join(install, "Ghidra"), ".jar");
|
||||
const args = [
|
||||
|
||||
@@ -0,0 +1,174 @@
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { appendFileSync } from "node:fs";
|
||||
import { matchesGlob } from "node:path";
|
||||
import { SCOPES } from "./scopes.mjs";
|
||||
|
||||
const flags = [
|
||||
"code",
|
||||
"runtime",
|
||||
"package",
|
||||
"apple",
|
||||
"fixtures",
|
||||
"readiness",
|
||||
"docs",
|
||||
"workflows",
|
||||
...Object.keys(SCOPES),
|
||||
];
|
||||
const selected = Object.fromEntries(flags.map((flag) => [flag, false]));
|
||||
const reasons = [];
|
||||
const full = (reason) => {
|
||||
for (const flag of flags) selected[flag] = true;
|
||||
reasons.push(reason);
|
||||
};
|
||||
const git = (...args) =>
|
||||
execFileSync("git", args, { encoding: "utf8" }).trimEnd();
|
||||
function classifyPackage(base, head) {
|
||||
const before = JSON.parse(git("show", `${base}:package.json`));
|
||||
const after = JSON.parse(git("show", `${head}:package.json`));
|
||||
const { scripts: beforeScripts, ...beforeRest } = before;
|
||||
const { scripts: afterScripts, ...afterRest } = after;
|
||||
if (JSON.stringify(beforeRest) !== JSON.stringify(afterRest)) {
|
||||
full("Package metadata or dependencies changed");
|
||||
return;
|
||||
}
|
||||
selected.code = true;
|
||||
const changed = [
|
||||
...new Set([
|
||||
...Object.keys(beforeScripts ?? {}),
|
||||
...Object.keys(afterScripts ?? {}),
|
||||
]),
|
||||
].filter((key) => beforeScripts?.[key] !== afterScripts?.[key]);
|
||||
for (const script of changed) {
|
||||
if (
|
||||
/^(test(?::|$)|check(?::|$)|lint(?::|$)|format(?::|$)|knip$|jscpd$)/.test(
|
||||
script,
|
||||
)
|
||||
)
|
||||
continue;
|
||||
if (
|
||||
["compile:termux", "build:termux", "build:termux:run"].includes(script)
|
||||
) {
|
||||
selected.workflows = true;
|
||||
reasons.push(
|
||||
`Android build script changed; real Android verification remains manual: ${script}`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
// Verify scripts own provider setup as well as execution. Reuse their
|
||||
// existing source-file ownership rather than maintaining a second map.
|
||||
const entrypoint = (command) =>
|
||||
/^(?:npm run build:cached && )?node (scripts\/[\w/-]+\.mjs)(?: [\w./=-]+)*$/.exec(
|
||||
command ?? "",
|
||||
)?.[1];
|
||||
const beforeEntry = entrypoint(beforeScripts?.[script]);
|
||||
const afterEntry = entrypoint(afterScripts?.[script]);
|
||||
const entries = [beforeEntry, afterEntry].filter(Boolean);
|
||||
const simple =
|
||||
(!beforeScripts?.[script] || beforeEntry) &&
|
||||
(!afterScripts?.[script] || afterEntry);
|
||||
const scriptOwners =
|
||||
simple &&
|
||||
entries.every((entry) =>
|
||||
Object.values(SCOPES).some((patterns) => patterns.includes(entry)),
|
||||
)
|
||||
? Object.entries(SCOPES)
|
||||
.filter(([, patterns]) =>
|
||||
entries.some((entry) => patterns.includes(entry)),
|
||||
)
|
||||
.map(([owner]) => owner)
|
||||
: [];
|
||||
if (scriptOwners.length)
|
||||
for (const owner of scriptOwners) selected[owner] = true;
|
||||
else full(`Shared or unknown package script: ${script}`);
|
||||
}
|
||||
}
|
||||
|
||||
const event = process.env.GITHUB_EVENT_NAME;
|
||||
let files = [];
|
||||
if (
|
||||
event === "push" ||
|
||||
event === "schedule" ||
|
||||
event === "workflow_dispatch" ||
|
||||
process.env.CI_FULL === "true"
|
||||
) {
|
||||
full("Full baseline requested by event or ci:full label");
|
||||
} else if (event === "pull_request") {
|
||||
const base = git("merge-base", process.env.BASE_SHA, process.env.HEAD_SHA);
|
||||
const head = process.env.HEAD_SHA;
|
||||
// Disable rename detection so both the removed and added paths participate.
|
||||
files = git("diff", "--name-only", "--no-renames", "-z", base, head)
|
||||
.split("\0")
|
||||
.filter(Boolean);
|
||||
for (const file of files) {
|
||||
if (
|
||||
/^(README(?:_[^/]+)?\.md|AGENTS\.md|CONTRIBUTING\.md)$/.test(file) ||
|
||||
file.startsWith("docs/")
|
||||
) {
|
||||
selected.docs = true;
|
||||
continue;
|
||||
}
|
||||
if (/^(src|tests)\/.*\.(?:test|spec)\.[cm]?[jt]sx?$/.test(file)) {
|
||||
selected.code = true;
|
||||
continue;
|
||||
}
|
||||
const owners = Object.entries(SCOPES)
|
||||
.filter(([, patterns]) =>
|
||||
patterns.some((pattern) => matchesGlob(file, pattern)),
|
||||
)
|
||||
.map(([owner]) => owner);
|
||||
for (const owner of owners) selected[owner] = true;
|
||||
if (
|
||||
file.startsWith(".github/workflows/") ||
|
||||
file === ".github/actionlint.yaml"
|
||||
) {
|
||||
selected.workflows = true;
|
||||
if (file !== ".github/workflows/ci.yml") continue;
|
||||
}
|
||||
if (owners.length === 1 && owners[0] === "website") continue;
|
||||
if (/^src\/(contracts|domain)\//.test(file)) selected.docs = true;
|
||||
if (file === "package.json") {
|
||||
classifyPackage(base, head);
|
||||
continue;
|
||||
}
|
||||
selected.code = true;
|
||||
if (file.startsWith("tests/")) continue;
|
||||
if (
|
||||
/^(src\/process\/|src\/filesystem\/|src\/contracts\/tool|src\/server\/createServer|src\/cli\.ts|scripts\/ci\/)/.test(
|
||||
file,
|
||||
)
|
||||
) {
|
||||
full(`Shared boundary: ${file}`);
|
||||
} else if (owners.length) {
|
||||
if (file.startsWith("src/") || file.startsWith("bridge/"))
|
||||
selected.runtime = true;
|
||||
} else {
|
||||
full(`Unclassified input: ${file}`);
|
||||
}
|
||||
}
|
||||
} else throw new Error(`Unsupported CI event: ${event}`);
|
||||
selected.runtime ||= selected.package;
|
||||
const linux = { os: "ubuntu-latest", platform: "linux", architecture: "x64" };
|
||||
const matrix = {
|
||||
include: selected.package
|
||||
? [
|
||||
linux,
|
||||
{ os: "ubuntu-24.04-arm", platform: "linux", architecture: "arm64" },
|
||||
{ os: "macos-15", platform: "darwin", architecture: "arm64" },
|
||||
{ os: "macos-15-intel", platform: "darwin", architecture: "x64" },
|
||||
]
|
||||
: [linux],
|
||||
};
|
||||
const plan = { ...selected, matrix, files, reasons };
|
||||
if (process.env.GITHUB_OUTPUT)
|
||||
appendFileSync(
|
||||
process.env.GITHUB_OUTPUT,
|
||||
Object.entries({ ...selected, matrix: JSON.stringify(matrix) })
|
||||
.map(([key, value]) => `${key}=${value}\n`)
|
||||
.join(""),
|
||||
);
|
||||
if (process.env.GITHUB_STEP_SUMMARY)
|
||||
appendFileSync(
|
||||
process.env.GITHUB_STEP_SUMMARY,
|
||||
`### Selected CI lanes\n\n${flags.filter((flag) => selected[flag]).join(", ") || "Formatting only"}\n\n${reasons.join("\n\n")}\n`,
|
||||
);
|
||||
console.log(JSON.stringify(plan));
|
||||
@@ -0,0 +1,25 @@
|
||||
const jobs = JSON.parse(process.env.CI_JOBS);
|
||||
const outputs = jobs.changes.outputs;
|
||||
const required = ["changes", "static", "test"];
|
||||
const lanes = {
|
||||
code: ["build", "test-shard"],
|
||||
runtime: ["windows-native-build", "package-e2e"],
|
||||
package: ["windows-curated"],
|
||||
apple: ["apple-artifacts"],
|
||||
fixtures: ["conformance-fixtures"],
|
||||
readiness: ["reconstruction-readiness"],
|
||||
docs: ["docs"],
|
||||
};
|
||||
for (const [scope, value] of Object.entries(outputs)) {
|
||||
if (value === "true") required.push(...(lanes[scope] ?? [scope]));
|
||||
}
|
||||
const failed = [...new Set(required)].filter(
|
||||
(name) => jobs[name]?.result !== "success",
|
||||
);
|
||||
for (const [name, job] of Object.entries(jobs)) {
|
||||
if (["failure", "cancelled"].includes(job.result) && !failed.includes(name))
|
||||
failed.push(name);
|
||||
}
|
||||
if (failed.length)
|
||||
throw new Error(`CI lanes did not pass: ${failed.join(", ")}`);
|
||||
console.log("Every selected CI lane passed");
|
||||
@@ -0,0 +1,227 @@
|
||||
// Ownership of the real-provider and website verification lanes. Shared build
|
||||
// inputs are classified once in plan.mjs rather than repeated in every workflow.
|
||||
export const SCOPES = {
|
||||
browser: [
|
||||
".github/workflows/real-browser.yml",
|
||||
"scripts/verify-real-browser.mjs",
|
||||
"scripts/verify-browser-network.mjs",
|
||||
"scripts/verify-browser-scripts.mjs",
|
||||
"scripts/verify-browser-modules.mjs",
|
||||
"scripts/verify-browser-dom-destinations.mjs",
|
||||
"scripts/lib/browser-*.mjs",
|
||||
"scripts/fixtures/browser-network-site.mjs",
|
||||
"scripts/fixtures/browser-script-site.mjs",
|
||||
"scripts/fixtures/browser-module-site.mjs",
|
||||
"scripts/fixtures/browser-verifier-site.mjs",
|
||||
"scripts/fixtures/page-cdp-proxy.mjs",
|
||||
"src/browser/**",
|
||||
"src/domain/browser*.ts",
|
||||
"src/domain/webScript*.ts",
|
||||
"src/domain/webModule*.ts",
|
||||
"src/application/WebModule*.ts",
|
||||
"src/composition/webModules.ts",
|
||||
"src/artifacts/readStableArtifact.ts",
|
||||
"src/artifacts/ArtifactPaths.ts",
|
||||
"src/application/WebScriptExportService.ts",
|
||||
"src/contracts/webScriptToolContracts.ts",
|
||||
"src/server/registerWebScriptTool.ts",
|
||||
"src/server/registerWebModuleTool.ts",
|
||||
"src/server/createServer.ts",
|
||||
"src/cli/webModuleCommands.ts",
|
||||
"src/cli.ts",
|
||||
"src/cliCommandNames.ts",
|
||||
"src/contracts/toolEffects.ts",
|
||||
"src/cli/webScriptCommands.ts",
|
||||
],
|
||||
binary_layout: [
|
||||
".github/workflows/real-binary-layout.yml",
|
||||
"bridge/pwntools/**",
|
||||
"src/native/pwntools/**",
|
||||
"src/domain/native/binaryLayout*",
|
||||
"src/application/binaryDiagnostics/**",
|
||||
"src/contracts/native/binaryDiagnosticsToolContracts.ts",
|
||||
"src/composition/binaryDiagnostics.ts",
|
||||
"src/server/registerBinaryDiagnosticsTools.ts",
|
||||
"src/server/createServer.ts",
|
||||
"src/cli/binaryDiagnosticsCommands.ts",
|
||||
"src/cli.ts",
|
||||
"src/cliCommandNames.ts",
|
||||
"src/contracts/toolContracts.ts",
|
||||
"src/contracts/toolEffects.ts",
|
||||
"src/process/OwnedCommand.ts",
|
||||
"src/process/ProviderProcess.ts",
|
||||
"src/process/ProviderDeadline.ts",
|
||||
"src/process/ProcessOwnership*.ts",
|
||||
"src/process/PrivateRuntimeRoot.ts",
|
||||
"src/artifacts/readStableArtifact*.ts",
|
||||
"src/artifacts/ArtifactReader.ts",
|
||||
"src/domain/analysisError*.ts",
|
||||
"scripts/verify-binary-layout.mjs",
|
||||
"scripts/lib/elf-section-name-fixtures.mjs",
|
||||
"scripts/lib/large-result-mcp.mjs",
|
||||
"scripts/fixtures/binary-layout*",
|
||||
],
|
||||
evm: [
|
||||
".github/workflows/real-evm-interface.yml",
|
||||
"src/evm/**",
|
||||
"src/domain/evm/**",
|
||||
"src/application/evm/**",
|
||||
"src/contracts/evm/**",
|
||||
"src/composition/evm.ts",
|
||||
"src/cli/evmCommands.ts",
|
||||
"src/server/registerEvmTools.ts",
|
||||
"src/server/createServer.ts",
|
||||
"src/cli.ts",
|
||||
"src/cliCommandNames.ts",
|
||||
"src/contracts/toolContracts.ts",
|
||||
"src/contracts/toolEffects.ts",
|
||||
"src/process/OwnedCommand.ts",
|
||||
"src/process/ProviderProcess.ts",
|
||||
"src/process/ProcessOwnership*.ts",
|
||||
"src/process/ProviderDeadline.ts",
|
||||
"src/process/PrivateRuntimeRoot.ts",
|
||||
"src/artifacts/readStableArtifact*.ts",
|
||||
"src/artifacts/ArtifactReader.ts",
|
||||
"scripts/verify-evm-interface.mjs",
|
||||
"scripts/fixtures/evm-interface.sol",
|
||||
"third_party/evmole/**",
|
||||
],
|
||||
javascript_recovery: [
|
||||
".github/workflows/real-javascript-recovery.yml",
|
||||
"src/javascript/recovery/**",
|
||||
"src/domain/javascript/**",
|
||||
"src/application/javascript/JavaScriptRecovery*.ts",
|
||||
"src/composition/javascriptRecovery.ts",
|
||||
"src/contracts/javascript/javascriptRecoveryToolContracts.ts",
|
||||
"src/cli/javascriptRecoveryCommands.ts",
|
||||
"src/cli/javascriptApplicationAnalysis.ts",
|
||||
"src/server/registerJavaScriptRecoveryTool.ts",
|
||||
"src/server/createServer.ts",
|
||||
"src/server/sessionAvailabilityPolicy.ts",
|
||||
"src/application/CapabilityInventory.ts",
|
||||
"src/cli.ts",
|
||||
"src/cliCommandNames.ts",
|
||||
"src/contracts/toolContracts.ts",
|
||||
"src/contracts/toolEffects.ts",
|
||||
"scripts/rea.mjs",
|
||||
"src/process/**",
|
||||
"src/artifacts/SafeOutputTree.ts",
|
||||
"scripts/verify/javascript/recovery.mjs",
|
||||
"scripts/fixtures/javascript-recovery/**",
|
||||
"third_party/wakaru",
|
||||
],
|
||||
recorded_crash: [
|
||||
".github/workflows/real-recorded-crash.yml",
|
||||
"bridge/pwntools/**",
|
||||
"bridge/pwndbg/**",
|
||||
"src/native/pwntools/**",
|
||||
"src/native/pwndbg/**",
|
||||
"src/domain/native/recordedCrash*",
|
||||
"src/application/binaryDiagnostics/**",
|
||||
"src/contracts/native/binaryDiagnosticsToolContracts.ts",
|
||||
"src/contracts/toolEffects.ts",
|
||||
"src/contracts/toolOutputSchemaGroups.ts",
|
||||
"src/composition/binaryDiagnostics.ts",
|
||||
"src/server/registerRecordedCrashTools.ts",
|
||||
"src/server/createServer.ts",
|
||||
"src/server/sessionAvailabilityPolicy.ts",
|
||||
"src/cli/binaryDiagnosticsCommands.ts",
|
||||
"src/cliCommandNames.ts",
|
||||
"src/process/**",
|
||||
"src/artifacts/readStableArtifact*.ts",
|
||||
"src/domain/analysisError*.ts",
|
||||
"scripts/verify-recorded-crash.mjs",
|
||||
"scripts/verify/recorded-crash/**",
|
||||
"scripts/lib/recorded-core-fixtures.py",
|
||||
"scripts/fixtures/recorded-crash.c",
|
||||
"third_party/pwndbg/**",
|
||||
],
|
||||
web_network: [
|
||||
".github/workflows/real-web-network-captures.yml",
|
||||
"bridge/mitmproxy/**",
|
||||
"src/browser/history/**",
|
||||
"src/domain/webNetworkCapture*.ts",
|
||||
"src/domain/analysisError*.ts",
|
||||
"src/contracts/errorSchemas.ts",
|
||||
"src/domain/explicitSensitive*.ts",
|
||||
"src/application/WebNetworkCapture*",
|
||||
"src/composition/webNetworkCaptures.ts",
|
||||
"src/cli/webNetworkCaptureCommands.ts",
|
||||
"src/server/registerWebNetworkCaptureTool.ts",
|
||||
"src/server/createServer.ts",
|
||||
"src/server/toolRegistrationOptions.ts",
|
||||
"src/cli.ts",
|
||||
"src/cliCommandNames.ts",
|
||||
"src/contracts/toolContracts.ts",
|
||||
"src/contracts/toolEffects.ts",
|
||||
"src/process/OwnedCommand.ts",
|
||||
"src/process/ProviderProcess.ts",
|
||||
"src/process/ProviderDeadline.ts",
|
||||
"src/process/ProcessOwnership*.ts",
|
||||
"src/process/PrivateRuntimeRoot.ts",
|
||||
"src/process/capture/ProcessCaptureLifecycle.ts",
|
||||
"src/artifacts/readStableArtifact*.ts",
|
||||
"src/artifacts/ArtifactReader.ts",
|
||||
"src/contracts/webNetworkCaptureToolContracts.ts",
|
||||
"scripts/verify-web-network-captures.mjs",
|
||||
"scripts/fixtures/generate-mitmproxy-capture.py",
|
||||
],
|
||||
web_runtime: [
|
||||
".github/workflows/real-web-runtime.yml",
|
||||
"scripts/verify-browser-runtime.mjs",
|
||||
"scripts/lib/browser-runtime-*.mjs",
|
||||
"src/process/ProviderProcess.ts",
|
||||
"src/process/ProcessOwnership*.ts",
|
||||
"scripts/fixtures/browser-runtime-site.mjs",
|
||||
"src/browser/execution/**",
|
||||
"src/browser/CdpConnection.ts",
|
||||
"src/browser/CdpEndpoint.ts",
|
||||
"src/browser/CdpTarget*.ts",
|
||||
"src/browser/CdpCaptureValues.ts",
|
||||
"src/browser/CdpAuthorized*.ts",
|
||||
"src/domain/webRuntime.ts",
|
||||
"src/domain/webExecution.ts",
|
||||
"src/domain/webEventListeners.ts",
|
||||
"src/domain/browserObservationError*.ts",
|
||||
"src/application/WebRuntime*.ts",
|
||||
"src/composition/webRuntime.ts",
|
||||
"src/contracts/webRuntimeToolContracts.ts",
|
||||
"src/contracts/toolEffects.ts",
|
||||
"src/server/registerWebRuntimeTools.ts",
|
||||
"src/server/createServer.ts",
|
||||
"src/cli/webRuntimeCommands.ts",
|
||||
"src/cli/commandCancellation.ts",
|
||||
"src/cli.ts",
|
||||
],
|
||||
web_source_map: [
|
||||
".github/workflows/real-web-source-map.yml",
|
||||
"scripts/verify-browser-source-maps.mjs",
|
||||
"scripts/lib/browser-source-map-e2e.mjs",
|
||||
"scripts/fixtures/browser-source-map-site.mjs",
|
||||
"src/javascript/sourceMaps/**",
|
||||
"src/browser/assets/**",
|
||||
"src/domain/webSource*.ts",
|
||||
"src/domain/webScript*.ts",
|
||||
"src/application/WebSourceLocation*.ts",
|
||||
"src/application/ValidateSelectedWebScript.ts",
|
||||
"src/composition/webSourceLocations.ts",
|
||||
"src/server/registerWebSourceLocationTool.ts",
|
||||
"src/server/createServer.ts",
|
||||
"src/contracts/webScriptToolContracts.ts",
|
||||
"src/process/**",
|
||||
"src/cli/webSourceLocationCommands.ts",
|
||||
"src/cli.ts",
|
||||
"src/artifacts/readStableArtifact.ts",
|
||||
],
|
||||
nativeaot: [
|
||||
"bridge/nativeaot/**",
|
||||
"scripts/generate-nativeaot-fixtures.mjs",
|
||||
"tests/fixtures/nativeaot/**",
|
||||
".github/workflows/nativeaot-fixtures.yml",
|
||||
],
|
||||
website: [
|
||||
"website/**",
|
||||
"scripts/*website*.py",
|
||||
".github/workflows/website-check.yml",
|
||||
],
|
||||
};
|
||||
@@ -18,6 +18,7 @@ export async function startRuntimeFixtureBrowser(executable, origin) {
|
||||
command: executable,
|
||||
arguments: [
|
||||
"--headless=new",
|
||||
"--password-store=basic",
|
||||
...(process.env.REA_BROWSER_NO_SANDBOX === "true"
|
||||
? ["--no-sandbox"]
|
||||
: []),
|
||||
|
||||
@@ -0,0 +1,676 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { createServer } from "node:http";
|
||||
import { once } from "node:events";
|
||||
import { setTimeout as delay } from "node:timers/promises";
|
||||
|
||||
import { CdpConnection } from "../../dist/browser/CdpConnection.js";
|
||||
import { createBrowserScenarioProvider } from "../../dist/composition/browserScenario.js";
|
||||
import { browserScenarioSchema } from "../../dist/domain/browserScenario.js";
|
||||
import {
|
||||
runScenarioCli,
|
||||
scenarioProfiles,
|
||||
} from "./browser-scenario-verifier.mjs";
|
||||
|
||||
const scenarioFor = (browser, origin) =>
|
||||
browserScenarioSchema.parse({
|
||||
browser,
|
||||
start_url: { url: `${origin}/storage-main` },
|
||||
actions: [
|
||||
{
|
||||
step_id: "frame-ready",
|
||||
action: "wait_for",
|
||||
locator: { kind: "css", selector: "#frame-report[data-ready='true']" },
|
||||
state: "attached",
|
||||
timeout_ms: 10_000,
|
||||
},
|
||||
{
|
||||
step_id: "top-level-partition",
|
||||
action: "goto",
|
||||
destination: {
|
||||
url: `${origin.replace("127.0.0.1", "localhost")}/storage-top`,
|
||||
},
|
||||
wait_until: "load",
|
||||
},
|
||||
{
|
||||
step_id: "return-to-origin",
|
||||
action: "goto",
|
||||
destination: { url: `${origin}/storage-main` },
|
||||
wait_until: "load",
|
||||
},
|
||||
],
|
||||
storage: {
|
||||
local_storage: [
|
||||
{
|
||||
origin,
|
||||
entries: [
|
||||
{
|
||||
name: "rea_storage_probe",
|
||||
value: { source: "literal", value: "main-initial" },
|
||||
},
|
||||
{
|
||||
name: "rea_storage_remove",
|
||||
value: { source: "literal", value: "main-remove-seed" },
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
origin: origin.replace("127.0.0.1", "localhost"),
|
||||
entries: [
|
||||
{
|
||||
name: "rea_storage_probe",
|
||||
value: { source: "literal", value: "frame-initial" },
|
||||
},
|
||||
{
|
||||
name: "rea_storage_remove",
|
||||
value: { source: "literal", value: "frame-remove-seed" },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
session_storage: [
|
||||
{
|
||||
origin,
|
||||
entries: [
|
||||
{
|
||||
name: "rea_storage_probe",
|
||||
value: { source: "literal", value: "main-session-initial" },
|
||||
},
|
||||
{
|
||||
name: "rea_storage_empty",
|
||||
value: { source: "literal", value: "main-empty-seed" },
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
origin: origin.replace("127.0.0.1", "localhost"),
|
||||
entries: [
|
||||
{
|
||||
name: "rea_storage_probe",
|
||||
value: { source: "literal", value: "frame-session-initial" },
|
||||
},
|
||||
{
|
||||
name: "rea_storage_empty",
|
||||
value: { source: "literal", value: "frame-empty-seed" },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
capture: { after_each_step: ["dom"], at_end: ["dom"] },
|
||||
});
|
||||
|
||||
const storagePage = (kind, port) => {
|
||||
if (kind === "main")
|
||||
return `<!doctype html><html><body><pre id="main-report"></pre><pre id="frame-report"></pre><iframe src="http://localhost:${port}/storage-frame"></iframe><script>
|
||||
const state = { local: localStorage.getItem("rea_storage_probe"), session: sessionStorage.getItem("rea_storage_probe"), removed: localStorage.getItem("rea_storage_remove"), empty: sessionStorage.getItem("rea_storage_empty") };
|
||||
document.querySelector("#main-report").textContent = JSON.stringify(state);
|
||||
localStorage.setItem("rea_storage_probe", "main-updated");
|
||||
sessionStorage.setItem("rea_storage_probe", "main-session-updated");
|
||||
localStorage.removeItem("rea_storage_remove");
|
||||
sessionStorage.setItem("rea_storage_empty", "");
|
||||
addEventListener("message", (event) => {
|
||||
if (event.data?.kind !== "rea-storage-frame") return;
|
||||
const report = document.querySelector("#frame-report");
|
||||
report.textContent = JSON.stringify(event.data.state);
|
||||
report.dataset.ready = "true";
|
||||
});
|
||||
</script></body></html>`;
|
||||
if (kind === "frame")
|
||||
return `<!doctype html><html><body><pre id="frame-state"></pre><script>
|
||||
const state = { local: localStorage.getItem("rea_storage_probe"), session: sessionStorage.getItem("rea_storage_probe"), removed: localStorage.getItem("rea_storage_remove"), empty: sessionStorage.getItem("rea_storage_empty") };
|
||||
document.querySelector("#frame-state").textContent = JSON.stringify(state);
|
||||
localStorage.setItem("rea_storage_probe", "frame-updated");
|
||||
sessionStorage.setItem("rea_storage_probe", "frame-session-updated");
|
||||
localStorage.removeItem("rea_storage_remove");
|
||||
sessionStorage.setItem("rea_storage_empty", "");
|
||||
parent.postMessage({ kind: "rea-storage-frame", state }, "*");
|
||||
</script></body></html>`;
|
||||
if (kind === "debugger")
|
||||
return `<!doctype html><html><body><pre id="debugger-state"></pre><script>
|
||||
window.reaDebuggerBefore = true;
|
||||
debugger;
|
||||
window.reaDebuggerAfter = true;
|
||||
document.querySelector("#debugger-state").textContent = String(window.reaDebuggerAfter);
|
||||
</script></body></html>`;
|
||||
return `<!doctype html><html><body><pre id="top-report"></pre><script>
|
||||
const state = { local: localStorage.getItem("rea_storage_probe"), session: sessionStorage.getItem("rea_storage_probe"), removed: localStorage.getItem("rea_storage_remove"), empty: sessionStorage.getItem("rea_storage_empty") };
|
||||
document.querySelector("#top-report").textContent = JSON.stringify(state);
|
||||
localStorage.setItem("rea_storage_probe", "top-updated");
|
||||
sessionStorage.setItem("rea_storage_probe", "top-session-updated");
|
||||
</script></body></html>`;
|
||||
};
|
||||
|
||||
const startStorageSite = async () => {
|
||||
let port = 0;
|
||||
const server = createServer((request, response) => {
|
||||
const path = new URL(request.url ?? "/", "http://127.0.0.1").pathname;
|
||||
const kind =
|
||||
path === "/storage-main"
|
||||
? "main"
|
||||
: path === "/storage-frame"
|
||||
? "frame"
|
||||
: path === "/storage-debugger"
|
||||
? "debugger"
|
||||
: path === "/storage-top"
|
||||
? "top"
|
||||
: undefined;
|
||||
if (kind === undefined) {
|
||||
response.writeHead(404).end();
|
||||
return;
|
||||
}
|
||||
response.setHeader("content-type", "text/html; charset=utf-8");
|
||||
response.end(storagePage(kind, port));
|
||||
});
|
||||
server.listen(0, "::");
|
||||
await once(server, "listening");
|
||||
port = server.address().port;
|
||||
return {
|
||||
origin: `http://127.0.0.1:${port}`,
|
||||
close: async () => {
|
||||
server.close();
|
||||
await once(server, "close");
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
const domText = (step) => {
|
||||
assert.equal(step.artifacts.dom.state, "captured");
|
||||
return step.artifacts.dom.value.text;
|
||||
};
|
||||
|
||||
const assertContainsState = (text, id, expected) => {
|
||||
const match = new RegExp(`<pre id="${id}"[^>]*>(.*?)</pre>`, "su").exec(text);
|
||||
assert.ok(match, `Captured DOM omitted #${id}: ${text}`);
|
||||
assert.deepEqual(JSON.parse(match[1]), expected);
|
||||
};
|
||||
|
||||
/** Verify one-time local/session storage initialization on a real Chromium. */
|
||||
export async function verifyScenarioStorage({
|
||||
executable,
|
||||
endpoint,
|
||||
targetId,
|
||||
}) {
|
||||
const site = await startStorageSite();
|
||||
try {
|
||||
await verifyLaunchStorage(executable, site.origin);
|
||||
await verifyConnectedStorage(endpoint, targetId, site.origin);
|
||||
return {
|
||||
mocked: false,
|
||||
launch: true,
|
||||
connect: true,
|
||||
oopifInitialization: true,
|
||||
firstInlineScriptSeeded: true,
|
||||
updatesAndDeletesPersistAcrossNavigation: true,
|
||||
newScenarioGetsFreshSeed: true,
|
||||
externalTargetSurvives: true,
|
||||
cancellationCleanup: true,
|
||||
};
|
||||
} finally {
|
||||
await site.close();
|
||||
}
|
||||
}
|
||||
|
||||
const verifyLaunchStorage = async (executable, origin) => {
|
||||
const scenario = scenarioFor(
|
||||
{ mode: "launch", executable_path: executable },
|
||||
origin,
|
||||
);
|
||||
const profilesBefore = await scenarioProfiles();
|
||||
const result = await runScenarioCli(scenario);
|
||||
const capture = result.normalized_result;
|
||||
assert.ok(capture, "Launch-mode storage scenario returned no result");
|
||||
assertLaunchStorageStates(capture);
|
||||
assert.equal(capture.browser.cleanup, "terminated-owned-process");
|
||||
const profilesAfter = await scenarioProfiles();
|
||||
assert.ok([...profilesAfter].every((profile) => profilesBefore.has(profile)));
|
||||
|
||||
const freshScenario = browserScenarioSchema.parse({
|
||||
...scenario,
|
||||
actions: [scenario.actions[0]],
|
||||
});
|
||||
const fresh = await createBrowserScenarioProvider(
|
||||
process.env,
|
||||
).captureScenario(freshScenario);
|
||||
assert.equal(fresh.ok, true);
|
||||
if (!fresh.ok) throw fresh.error;
|
||||
assertContainsState(domText(fresh.value.steps[0]), "main-report", {
|
||||
local: "main-initial",
|
||||
session: "main-session-initial",
|
||||
removed: "main-remove-seed",
|
||||
empty: "main-empty-seed",
|
||||
});
|
||||
};
|
||||
|
||||
const assertLaunchStorageStates = (capture) => {
|
||||
assertContainsState(domText(capture.steps[0]), "main-report", {
|
||||
local: "main-initial",
|
||||
session: "main-session-initial",
|
||||
removed: "main-remove-seed",
|
||||
empty: "main-empty-seed",
|
||||
});
|
||||
assertContainsState(domText(capture.steps[1]), "frame-report", {
|
||||
local: "frame-initial",
|
||||
session: "frame-session-initial",
|
||||
removed: "frame-remove-seed",
|
||||
empty: "frame-empty-seed",
|
||||
});
|
||||
assertContainsState(domText(capture.steps[2]), "top-report", {
|
||||
local: "frame-updated",
|
||||
session: "frame-session-updated",
|
||||
removed: null,
|
||||
empty: "",
|
||||
});
|
||||
assertContainsState(domText(capture.steps[3]), "main-report", {
|
||||
local: "main-updated",
|
||||
session: "main-session-updated",
|
||||
removed: null,
|
||||
empty: "",
|
||||
});
|
||||
assertContainsState(domText(capture.steps[3]), "frame-report", {
|
||||
local: "top-updated",
|
||||
session: "top-session-updated",
|
||||
removed: null,
|
||||
empty: "",
|
||||
});
|
||||
};
|
||||
|
||||
const verifyConnectedStorage = async (endpoint, targetId, origin) => {
|
||||
const scenario = scenarioFor(
|
||||
{ mode: "connect", cdp_endpoint: endpoint, target_id: targetId },
|
||||
origin,
|
||||
);
|
||||
const targets = await (
|
||||
await fetch(`${endpoint}/json/list`, { signal: AbortSignal.timeout(2_000) })
|
||||
).json();
|
||||
const target = targets.find(({ id }) => id === targetId);
|
||||
assert.ok(target, "External browser target is missing");
|
||||
const observer = await CdpConnection.connect(
|
||||
target.webSocketDebuggerUrl,
|
||||
"capture_browser_scenario",
|
||||
);
|
||||
try {
|
||||
await assertStorageKeyPartition(observer, origin);
|
||||
const result = await runScenarioCli(scenario);
|
||||
const capture = result.normalized_result;
|
||||
assert.ok(capture, "Connect-mode storage scenario returned no result");
|
||||
assertContainsState(domText(capture.steps[0]), "main-report", {
|
||||
local: "main-initial",
|
||||
session: "main-session-initial",
|
||||
removed: "main-remove-seed",
|
||||
empty: "main-empty-seed",
|
||||
});
|
||||
assert.equal(capture.browser.cleanup, "disconnected-external");
|
||||
await verifyCallerDebuggerPause(observer, origin, scenario);
|
||||
await verifyCancelledStorage(observer, endpoint, targetId, scenario);
|
||||
} finally {
|
||||
await Promise.race([
|
||||
restoreTarget(observer, target.url),
|
||||
delay(1_000),
|
||||
]).catch(() => undefined);
|
||||
await Promise.race([observer.close(), delay(1_000)]).catch(() => undefined);
|
||||
}
|
||||
};
|
||||
|
||||
const assertStorageKeyPartition = async (connection, origin) => {
|
||||
const attachedSessions = new Set();
|
||||
const unsubscribe = connection.onEvent((event) => {
|
||||
if (
|
||||
event.method === "Target.attachedToTarget" &&
|
||||
typeof event.params === "object" &&
|
||||
event.params !== null &&
|
||||
"sessionId" in event.params &&
|
||||
typeof event.params.sessionId === "string" &&
|
||||
"targetInfo" in event.params &&
|
||||
typeof event.params.targetInfo === "object" &&
|
||||
event.params.targetInfo !== null &&
|
||||
"type" in event.params.targetInfo &&
|
||||
event.params.targetInfo.type === "iframe"
|
||||
)
|
||||
attachedSessions.add(event.params.sessionId);
|
||||
});
|
||||
await connection.send("Page.enable");
|
||||
await connection.send("Target.setAutoAttach", {
|
||||
autoAttach: true,
|
||||
waitForDebuggerOnStart: false,
|
||||
flatten: true,
|
||||
});
|
||||
try {
|
||||
const start = await connection.send("Runtime.evaluate", {
|
||||
expression: "performance.timeOrigin",
|
||||
returnByValue: true,
|
||||
});
|
||||
await connection.send("Page.navigate", {
|
||||
url: `${origin}/storage-main`,
|
||||
});
|
||||
await waitForPagePath(connection, "/storage-main", start.result.value);
|
||||
|
||||
let childSession;
|
||||
let childFrameId;
|
||||
const deadline = Date.now() + 10_000;
|
||||
while (Date.now() < deadline && childFrameId === undefined) {
|
||||
for (const sessionId of attachedSessions) {
|
||||
try {
|
||||
const tree = await connection.send(
|
||||
"Page.getFrameTree",
|
||||
{},
|
||||
sessionId,
|
||||
);
|
||||
const frame = tree.frameTree?.frame;
|
||||
if (
|
||||
typeof frame?.id === "string" &&
|
||||
frame.url ===
|
||||
`${origin.replace("127.0.0.1", "localhost")}/storage-frame`
|
||||
) {
|
||||
childSession = sessionId;
|
||||
childFrameId = frame.id;
|
||||
break;
|
||||
}
|
||||
} catch {
|
||||
// The iframe may detach between target discovery and frame inspection.
|
||||
}
|
||||
}
|
||||
if (childFrameId === undefined) await delay(25);
|
||||
}
|
||||
assert.ok(
|
||||
childSession && childFrameId,
|
||||
"Storage fixture OOPIF was not attached",
|
||||
);
|
||||
const child = await connection.send(
|
||||
"Storage.getStorageKey",
|
||||
{ frameId: childFrameId },
|
||||
childSession,
|
||||
);
|
||||
const rootTree = await connection.send("Page.getFrameTree");
|
||||
const beforeRoot = await connection.send("Storage.getStorageKey", {
|
||||
frameId: rootTree.frameTree.frame.id,
|
||||
});
|
||||
await connection.send("Page.navigate", {
|
||||
url: `${origin.replace("127.0.0.1", "localhost")}/storage-top`,
|
||||
});
|
||||
await waitForPagePath(connection, "/storage-top", start.result.value);
|
||||
const afterTree = await connection.send("Page.getFrameTree");
|
||||
const topLevel = await connection.send("Storage.getStorageKey", {
|
||||
frameId: afterTree.frameTree.frame.id,
|
||||
});
|
||||
assert.notEqual(
|
||||
child.storageKey,
|
||||
topLevel.storageKey,
|
||||
`Chrome did not expose distinct partition keys: child=${JSON.stringify(child)} top=${JSON.stringify(topLevel)} rootBefore=${JSON.stringify(beforeRoot)}`,
|
||||
);
|
||||
} finally {
|
||||
unsubscribe();
|
||||
await connection.send("Target.setAutoAttach", {
|
||||
autoAttach: false,
|
||||
waitForDebuggerOnStart: false,
|
||||
flatten: true,
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
const waitForPagePath = async (connection, path, priorTimeOrigin) => {
|
||||
const deadline = Date.now() + 10_000;
|
||||
while (Date.now() < deadline) {
|
||||
const result = await connection.send("Runtime.evaluate", {
|
||||
expression: `JSON.stringify({ path: location.pathname, time: performance.timeOrigin })`,
|
||||
returnByValue: true,
|
||||
});
|
||||
const current = JSON.parse(result.result.value);
|
||||
if (current.path === path && current.time !== priorTimeOrigin) return;
|
||||
await delay(25);
|
||||
}
|
||||
throw new Error(`External browser did not navigate to ${path}`);
|
||||
};
|
||||
|
||||
const verifyCallerDebuggerPause = async (caller, origin, scenario) => {
|
||||
await caller.send("Runtime.enable");
|
||||
await caller.send("Debugger.enable");
|
||||
const controller = new AbortController();
|
||||
const debuggerUrl = `${origin}/storage-debugger`;
|
||||
const scripts = new Map();
|
||||
const protocolTrace = [];
|
||||
let pauseObserved = false;
|
||||
let resolvePaused;
|
||||
const paused = new Promise((resolve) => {
|
||||
resolvePaused = (callFrameId) => {
|
||||
pauseObserved = true;
|
||||
resolve(callFrameId);
|
||||
};
|
||||
});
|
||||
const unsubscribe = caller.onEvent((event) => {
|
||||
if (event.method === "Debugger.scriptParsed") {
|
||||
const params = event.params;
|
||||
if (
|
||||
typeof params === "object" &&
|
||||
params !== null &&
|
||||
"scriptId" in params &&
|
||||
typeof params.scriptId === "string" &&
|
||||
"url" in params &&
|
||||
typeof params.url === "string"
|
||||
)
|
||||
scripts.set(params.scriptId, params.url);
|
||||
if (
|
||||
typeof params === "object" &&
|
||||
params !== null &&
|
||||
"url" in params &&
|
||||
typeof params.url === "string" &&
|
||||
params.url.includes("storage")
|
||||
)
|
||||
protocolTrace.push({ method: event.method, url: params.url });
|
||||
} else if (event.method === "Debugger.paused") {
|
||||
const params = event.params;
|
||||
const callFrames =
|
||||
typeof params === "object" && params !== null && "callFrames" in params
|
||||
? params.callFrames
|
||||
: undefined;
|
||||
const frame = Array.isArray(callFrames) ? callFrames[0] : undefined;
|
||||
const scriptId =
|
||||
typeof frame === "object" && frame !== null && "location" in frame
|
||||
? frame.location?.scriptId
|
||||
: undefined;
|
||||
protocolTrace.push({
|
||||
method: event.method,
|
||||
url: typeof scriptId === "string" ? scripts.get(scriptId) : undefined,
|
||||
reason:
|
||||
typeof params === "object" && params !== null && "reason" in params
|
||||
? params.reason
|
||||
: undefined,
|
||||
});
|
||||
if (
|
||||
typeof frame === "object" &&
|
||||
frame !== null &&
|
||||
typeof frame.callFrameId === "string" &&
|
||||
typeof scriptId === "string" &&
|
||||
scripts.get(scriptId) === debuggerUrl
|
||||
)
|
||||
resolvePaused(frame.callFrameId);
|
||||
}
|
||||
});
|
||||
const debuggerScenario = browserScenarioSchema.parse({
|
||||
...scenario,
|
||||
start_url: { url: debuggerUrl },
|
||||
actions: [
|
||||
{ step_id: "after-debugger", action: "wait_for_timeout", duration_ms: 1 },
|
||||
],
|
||||
});
|
||||
let settled = false;
|
||||
const pending = createBrowserScenarioProvider(process.env)
|
||||
.captureScenario(debuggerScenario, { signal: controller.signal })
|
||||
.finally(() => {
|
||||
settled = true;
|
||||
unsubscribe();
|
||||
});
|
||||
try {
|
||||
const callFrameId = await Promise.race([
|
||||
paused,
|
||||
delay(10_000).then(() => {
|
||||
throw new Error(
|
||||
`Caller debugger pause was not observed at ${debuggerUrl}; settled=${String(settled)} events=${JSON.stringify(protocolTrace)}`,
|
||||
);
|
||||
}),
|
||||
]);
|
||||
const pausedState = await caller.send("Debugger.evaluateOnCallFrame", {
|
||||
callFrameId,
|
||||
expression: "window.reaDebuggerAfter === true",
|
||||
returnByValue: true,
|
||||
});
|
||||
assert.equal(
|
||||
pausedState.result.value,
|
||||
false,
|
||||
"Caller-owned debugger frame resumed before the caller inspected it",
|
||||
);
|
||||
await delay(100);
|
||||
assert.equal(
|
||||
settled,
|
||||
false,
|
||||
"Storage observer resumed a caller-owned debugger pause",
|
||||
);
|
||||
await caller.send("Debugger.resume");
|
||||
pauseObserved = false;
|
||||
const result = await pending;
|
||||
if (!result.ok) throw result.error;
|
||||
const debuggerState = await caller.send("Runtime.evaluate", {
|
||||
expression: "window.reaDebuggerAfter === true",
|
||||
returnByValue: true,
|
||||
});
|
||||
assert.equal(debuggerState.result.value, true);
|
||||
} catch (cause) {
|
||||
if (!pauseObserved) controller.abort(cause);
|
||||
else await caller.send("Debugger.resume").catch(() => undefined);
|
||||
const completion = await Promise.race([
|
||||
pending.catch((pendingCause) => ({ ok: false, error: pendingCause })),
|
||||
delay(5_000).then(() => undefined),
|
||||
]);
|
||||
if (completion === undefined)
|
||||
throw new Error("Capture remained pending after cancellation", { cause });
|
||||
const completionError =
|
||||
typeof completion === "object" &&
|
||||
completion !== null &&
|
||||
"error" in completion
|
||||
? completion.error
|
||||
: typeof completion === "object" &&
|
||||
completion !== null &&
|
||||
"ok" in completion &&
|
||||
completion.ok === false
|
||||
? completion.error
|
||||
: undefined;
|
||||
const completionDetail =
|
||||
completionError === undefined
|
||||
? "capture did not report an error"
|
||||
: (() => {
|
||||
const causes = [];
|
||||
let current = completionError;
|
||||
for (
|
||||
let depth = 0;
|
||||
depth < 4 && current !== undefined;
|
||||
depth += 1
|
||||
) {
|
||||
causes.push({
|
||||
name: current.name,
|
||||
tag: current._tag,
|
||||
reason: current.reason,
|
||||
message: current.message,
|
||||
});
|
||||
current = current.cause;
|
||||
}
|
||||
return JSON.stringify(causes);
|
||||
})();
|
||||
throw new Error(`${String(cause)}; capture=${completionDetail}`, {
|
||||
cause,
|
||||
});
|
||||
} finally {
|
||||
unsubscribe();
|
||||
await Promise.race([caller.send("Debugger.disable"), delay(1_000)]).catch(
|
||||
() => undefined,
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
const verifyCancelledStorage = async (
|
||||
observer,
|
||||
endpoint,
|
||||
targetId,
|
||||
scenario,
|
||||
) => {
|
||||
const controller = new AbortController();
|
||||
const cancelledScenario = browserScenarioSchema.parse({
|
||||
...scenario,
|
||||
actions: [
|
||||
{
|
||||
step_id: "cancelled-wait",
|
||||
action: "wait_for",
|
||||
locator: { kind: "css", selector: "#never-storage-ready" },
|
||||
state: "visible",
|
||||
timeout_ms: 30_000,
|
||||
},
|
||||
],
|
||||
});
|
||||
const priorDocument = await observer.send("Runtime.evaluate", {
|
||||
expression: "performance.timeOrigin",
|
||||
returnByValue: true,
|
||||
});
|
||||
const pending = createBrowserScenarioProvider(process.env).captureScenario(
|
||||
cancelledScenario,
|
||||
{ signal: controller.signal },
|
||||
);
|
||||
await waitForNewDocument(
|
||||
observer,
|
||||
"/storage-main",
|
||||
priorDocument.result.value,
|
||||
);
|
||||
const state = await observer.send("Runtime.evaluate", {
|
||||
expression:
|
||||
"JSON.parse(document.querySelector('#main-report').textContent)",
|
||||
returnByValue: true,
|
||||
});
|
||||
assert.deepEqual(state.result.value, {
|
||||
local: "main-initial",
|
||||
session: "main-session-initial",
|
||||
removed: "main-remove-seed",
|
||||
empty: "main-empty-seed",
|
||||
});
|
||||
controller.abort(new Error("cancel storage seed scenario"));
|
||||
const result = await pending;
|
||||
assert.ok(
|
||||
!result.ok ||
|
||||
result.value.steps.some(({ status }) => status === "cancelled"),
|
||||
"Cancellation unexpectedly completed every scenario step",
|
||||
);
|
||||
const targets = await (await fetch(`${endpoint}/json/list`)).json();
|
||||
assert.ok(
|
||||
targets.some(({ id }) => id === targetId),
|
||||
"Cancelled connect-mode storage cleanup closed the external target",
|
||||
);
|
||||
};
|
||||
|
||||
const waitForNewDocument = async (connection, path, priorTimeOrigin) => {
|
||||
const deadline = Date.now() + 10_000;
|
||||
while (Date.now() < deadline) {
|
||||
const result = await connection.send("Runtime.evaluate", {
|
||||
expression: `JSON.stringify({ path: location.pathname, time: performance.timeOrigin, state: document.querySelector('#main-report')?.textContent })`,
|
||||
returnByValue: true,
|
||||
});
|
||||
const current = JSON.parse(result.result.value);
|
||||
if (
|
||||
current.path === path &&
|
||||
current.time !== priorTimeOrigin &&
|
||||
current.state !== undefined
|
||||
)
|
||||
return;
|
||||
await new Promise((resolve) => setTimeout(resolve, 25));
|
||||
}
|
||||
throw new Error(`External browser did not navigate to ${path}`);
|
||||
};
|
||||
|
||||
const restoreTarget = async (connection, url) => {
|
||||
await connection.send("Page.navigate", { url });
|
||||
const deadline = Date.now() + 10_000;
|
||||
while (Date.now() < deadline) {
|
||||
const result = await connection.send("Runtime.evaluate", {
|
||||
expression: "location.href",
|
||||
returnByValue: true,
|
||||
});
|
||||
if (result.result.value === url) return;
|
||||
await new Promise((resolve) => setTimeout(resolve, 25));
|
||||
}
|
||||
throw new Error("External browser did not return to its original URL");
|
||||
};
|
||||
@@ -65,7 +65,11 @@ function assertObservationEvents(result) {
|
||||
throw new Error(
|
||||
"Real Chrome attach-window console observation was missing",
|
||||
);
|
||||
if (result.network.websocket_events.length < 1)
|
||||
if (
|
||||
!result.network.websocket_connections.some(
|
||||
(connection) => connection.events.length > 0,
|
||||
)
|
||||
)
|
||||
throw new Error("Real Chrome WebSocket metadata was missing");
|
||||
}
|
||||
|
||||
@@ -77,8 +81,8 @@ function assertObservationPrivacy(result) {
|
||||
result.network.requests.some(
|
||||
(request) => request.body_shapes.status !== "not_approved",
|
||||
) ||
|
||||
result.network.websocket_events.some(
|
||||
(event) => event.payload_shape !== null,
|
||||
result.network.websocket_connections.some((connection) =>
|
||||
connection.events.some((event) => event.payload_shape !== null),
|
||||
)
|
||||
)
|
||||
throw new Error(
|
||||
@@ -246,7 +250,8 @@ export function assertSensitiveShapes(result) {
|
||||
)
|
||||
throw new Error("Real Chrome selected console text was not preserved");
|
||||
if (
|
||||
!result.network.websocket_events.some(
|
||||
!result.network.websocket_connections.some((connection) =>
|
||||
connection.events.some(
|
||||
(event) =>
|
||||
event.payload_shape?.format === "json" &&
|
||||
event.payload_shape.json_shape?.properties.some(
|
||||
@@ -255,6 +260,7 @@ export function assertSensitiveShapes(result) {
|
||||
path[0].kind === "property" &&
|
||||
path[0].name === "token",
|
||||
),
|
||||
),
|
||||
)
|
||||
)
|
||||
throw new Error("Real Chrome WebSocket JSON shape was missing");
|
||||
|
||||
@@ -62,6 +62,36 @@ export async function verifyGhidraAddressBoundaries(client, entry, cli) {
|
||||
await query("read_bytes", { address: padded, length: 1 }),
|
||||
await query("read_bytes", { address: entry, length: 1 }),
|
||||
);
|
||||
const aliasKeys = [entry, padded];
|
||||
assert.deepEqual(
|
||||
await query("set_addresses_names", {
|
||||
names: {
|
||||
[entry]: "rea_same_address_alias",
|
||||
[padded]: "rea_same_address_alias",
|
||||
},
|
||||
}),
|
||||
Object.fromEntries(aliasKeys.map((address) => [address, true])),
|
||||
"Equivalent address spellings with one requested name share one edit",
|
||||
);
|
||||
const nameBeforeAliasedBatch = await query("address_name", {
|
||||
address: entry,
|
||||
});
|
||||
assert.equal(nameBeforeAliasedBatch, "rea_same_address_alias");
|
||||
assert.deepEqual(
|
||||
await query("set_addresses_names", {
|
||||
names: {
|
||||
[entry]: "rea_first_address_alias",
|
||||
[padded]: "rea_second_address_alias",
|
||||
},
|
||||
}),
|
||||
Object.fromEntries(aliasKeys.map((address) => [address, false])),
|
||||
"Conflicting spellings of one Ghidra address must be rejected together",
|
||||
);
|
||||
assert.equal(
|
||||
await query("address_name", { address: entry }),
|
||||
nameBeforeAliasedBatch,
|
||||
"A rejected alias collision must leave the preexisting target name unchanged",
|
||||
);
|
||||
for (const [command, flags, operation] of [
|
||||
["read-bytes", ["--length", "1"], "read_bytes"],
|
||||
[
|
||||
|
||||
@@ -58,6 +58,15 @@ export async function verifyGhidraBoundaries(
|
||||
`${name}: ${JSON.stringify(validate.errors)}`,
|
||||
);
|
||||
assert.deepEqual(reply.structuredContent, JSON.parse(mcpTextValue(reply)));
|
||||
if (name === "list_strings" || name === "search_strings")
|
||||
assert.ok(
|
||||
reply.structuredContent.limitations.some((limitation) =>
|
||||
/Only Ghidra-defined string Data.*empty search does not establish/u.test(
|
||||
limitation,
|
||||
),
|
||||
),
|
||||
`${name} omitted defined-string coverage`,
|
||||
);
|
||||
successfulCalls++;
|
||||
return result;
|
||||
};
|
||||
@@ -378,6 +387,12 @@ export async function verifyGhidraBoundaries(
|
||||
).some((item) => item.address === marker.address),
|
||||
false,
|
||||
);
|
||||
assert.deepEqual(
|
||||
await call("search_strings", {
|
||||
pattern: `REA_GHIDRA_ABSENT_${randomUUID()}`,
|
||||
}),
|
||||
[],
|
||||
);
|
||||
assert.equal(
|
||||
(await call("inspect_native_instruction", { address: strings[0].address }))
|
||||
.status,
|
||||
|
||||
@@ -183,7 +183,9 @@ export async function verifyGhidraLargeResults({
|
||||
assert.equal(dossier.procedure.name, "rea_frame_probe");
|
||||
assert.ok(dossier.comments.some(({ text }) => text === comment));
|
||||
assert.ok(
|
||||
dossierRecord.raw_result.comments.some(({ text }) => text === comment),
|
||||
dossierRecord.raw_result.value.comments.some(
|
||||
({ text }) => text === comment,
|
||||
),
|
||||
);
|
||||
const { stdout } = await exec(
|
||||
process.execPath,
|
||||
|
||||
@@ -62,6 +62,7 @@ export async function verifyGhidraNamespaceAnnotations({
|
||||
assert.ok(symbol, `Missing demangled fixture global ${namespace}::value`);
|
||||
return { namespace, address: symbol.address };
|
||||
});
|
||||
await verifyDuplicateAddressNames(call, procedures, names);
|
||||
const ambiguous = await reject("procedure_address", { procedure: "same" });
|
||||
assert.equal(ambiguous.code, "invalid_request");
|
||||
for (const { address } of entries)
|
||||
@@ -133,7 +134,8 @@ export async function verifyGhidraNamespaceAnnotations({
|
||||
});
|
||||
assert.deepEqual(unchanged.annotations, qualified.annotations);
|
||||
}
|
||||
await verifyAddressNames(call, [entries[0], entries[2], ...globals]);
|
||||
await verifyAddressNames(call, [entries[0], entries[2]], "function");
|
||||
await verifyAddressNames(call, globals, "label");
|
||||
for (const { namespace, address } of entries) {
|
||||
assert.equal(
|
||||
await call("set_address_name", {
|
||||
@@ -177,24 +179,107 @@ export async function verifyGhidraNamespaceAnnotations({
|
||||
}
|
||||
}
|
||||
|
||||
async function verifyAddressNames(call, symbols) {
|
||||
for (const { namespace, address } of symbols) {
|
||||
async function verifyDuplicateAddressNames(call, procedures, names) {
|
||||
const functionEntries = [
|
||||
"rea_duplicate_function_a",
|
||||
"rea_duplicate_function_b",
|
||||
].map((name) => {
|
||||
const procedure = procedures.find(
|
||||
({ value }) => value === name || value === `_${name}`,
|
||||
);
|
||||
assert.ok(procedure, `Missing duplicate-name fixture function ${name}`);
|
||||
return { address: procedure.address, original: procedure.value };
|
||||
});
|
||||
const labelEntries = ["rea_duplicate_label_a", "rea_duplicate_label_b"].map(
|
||||
(name) => {
|
||||
const symbol = names.find(
|
||||
(item) =>
|
||||
(item.value === name || item.value === `_${name}`) &&
|
||||
item.symbol.primary,
|
||||
);
|
||||
assert.ok(symbol, `Missing duplicate-name fixture label ${name}`);
|
||||
return { address: symbol.address, original: symbol.value };
|
||||
},
|
||||
);
|
||||
|
||||
for (const [kind, entries] of [
|
||||
["function", functionEntries],
|
||||
["label", labelEntries],
|
||||
]) {
|
||||
const [owner, duplicate] = entries;
|
||||
const name = `rea_duplicate_${kind}`;
|
||||
assert.equal(
|
||||
await call("set_address_name", { address, name: "address_renamed" }),
|
||||
await call("set_address_name", { address: owner.address, name }),
|
||||
true,
|
||||
);
|
||||
const qualified = `${namespace}::address_qualified`;
|
||||
assert.equal(
|
||||
await call("set_address_name", { address: owner.address, name }),
|
||||
true,
|
||||
"Renaming the current owner to its existing name remains idempotent",
|
||||
);
|
||||
assert.equal(
|
||||
await call("set_address_name", { address: duplicate.address, name }),
|
||||
false,
|
||||
);
|
||||
assert.equal(
|
||||
await call("address_name", { address: duplicate.address }),
|
||||
duplicate.original,
|
||||
"A rejected duplicate leaves the other address unchanged",
|
||||
);
|
||||
|
||||
const batchName = `rea_batch_duplicate_${kind}`;
|
||||
assert.deepEqual(
|
||||
await call("set_addresses_names", {
|
||||
names: {
|
||||
[owner.address]: batchName,
|
||||
[duplicate.address]: batchName,
|
||||
},
|
||||
}),
|
||||
{ [owner.address]: true, [duplicate.address]: false },
|
||||
);
|
||||
assert.equal(
|
||||
await call("address_name", { address: owner.address }),
|
||||
batchName,
|
||||
);
|
||||
assert.equal(
|
||||
await call("address_name", { address: duplicate.address }),
|
||||
duplicate.original,
|
||||
"A failed batch member rolls back only its own address",
|
||||
);
|
||||
const owners = (await call("list_names")).filter(
|
||||
(item) => item.value === batchName && item.symbol.primary,
|
||||
);
|
||||
assert.deepEqual(
|
||||
owners.map(({ address }) => address),
|
||||
[owner.address],
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
async function verifyAddressNames(call, symbols, kind) {
|
||||
for (const { namespace, address } of symbols) {
|
||||
assert.equal(
|
||||
await call("set_address_name", {
|
||||
address,
|
||||
name: `address_renamed_${kind}`,
|
||||
}),
|
||||
true,
|
||||
);
|
||||
const qualified = `${namespace}::address_qualified_${kind}`;
|
||||
assert.equal(
|
||||
await call("set_address_name", { address, name: qualified }),
|
||||
true,
|
||||
);
|
||||
assert.equal(await call("address_name", { address }), qualified);
|
||||
assert.equal(
|
||||
await call("set_address_name", { address, name: "other::literal" }),
|
||||
await call("set_address_name", {
|
||||
address,
|
||||
name: `other::literal_${kind}`,
|
||||
}),
|
||||
true,
|
||||
);
|
||||
const literal = await call("address_name", { address });
|
||||
assert.equal(literal, `${namespace}::other::literal`);
|
||||
assert.equal(literal, `${namespace}::other::literal_${kind}`);
|
||||
assert.equal(
|
||||
await call("set_address_name", { address, name: literal }),
|
||||
true,
|
||||
@@ -235,7 +320,7 @@ async function verifyAddressNames(call, symbols) {
|
||||
assert.deepEqual(
|
||||
await call("set_addresses_names", {
|
||||
names: {
|
||||
[valid.address]: `${valid.namespace}::batch_renamed`,
|
||||
[valid.address]: `${valid.namespace}::batch_renamed_${kind}`,
|
||||
[invalid.address]: `${invalid.namespace}::`,
|
||||
},
|
||||
}),
|
||||
@@ -243,18 +328,18 @@ async function verifyAddressNames(call, symbols) {
|
||||
);
|
||||
assert.equal(
|
||||
await call("address_name", { address: valid.address }),
|
||||
`${valid.namespace}::batch_renamed`,
|
||||
`${valid.namespace}::batch_renamed_${kind}`,
|
||||
);
|
||||
assert.equal(
|
||||
await call("address_name", { address: invalid.address }),
|
||||
`${invalid.namespace}::address_qualified`,
|
||||
`${invalid.namespace}::address_qualified_${kind}`,
|
||||
);
|
||||
const names = await call("list_names");
|
||||
for (const { namespace, address } of symbols) {
|
||||
const expected =
|
||||
address === valid.address
|
||||
? `${namespace}::batch_renamed`
|
||||
: `${namespace}::address_qualified`;
|
||||
? `${namespace}::batch_renamed_${kind}`
|
||||
: `${namespace}::address_qualified_${kind}`;
|
||||
assert.ok(
|
||||
names.some(
|
||||
(item) =>
|
||||
|
||||
@@ -0,0 +1,149 @@
|
||||
import { z } from "zod";
|
||||
import {
|
||||
releaseAuthorSchema as author,
|
||||
releaseIssueSchema as issue,
|
||||
releasePullRequestSchema,
|
||||
} from "./release-notes.mjs";
|
||||
|
||||
const pageInfo = z.object({
|
||||
hasNextPage: z.boolean(),
|
||||
endCursor: z.string().nullable(),
|
||||
});
|
||||
const connection = (node) => z.object({ nodes: z.array(node), pageInfo });
|
||||
const associated = z.object({
|
||||
number: z.number().int().positive(),
|
||||
mergeCommit: z.object({ oid: z.string() }).nullable(),
|
||||
});
|
||||
const pr = releasePullRequestSchema.omit({ issues: true }).extend({
|
||||
closingIssuesReferences: connection(issue),
|
||||
});
|
||||
const PAGE = "pageInfo { hasNextPage endCursor }";
|
||||
const ACTOR = "login __typename";
|
||||
const AUTHORS = `authors(first:100, after:$cursor) { nodes { name email user { ${ACTOR} } } ${PAGE} }`;
|
||||
const ASSOCIATED = `associatedPullRequests(first:100, after:$cursor) { nodes { number mergeCommit { oid } } ${PAGE} }`;
|
||||
const ISSUES = `closingIssuesReferences(first:100, after:$cursor) { nodes { number url author { ${ACTOR} } repository { nameWithOwner } } ${PAGE} }`;
|
||||
|
||||
async function query(run, repository, selection, cursor = null) {
|
||||
const [owner, name] = repository.split("/");
|
||||
const response = JSON.parse(
|
||||
await run(
|
||||
"gh",
|
||||
["api", "graphql", "--input", "-"],
|
||||
JSON.stringify({
|
||||
query: `query($owner:String!,$name:String!,$cursor:String) { repository(owner:$owner,name:$name) { ${selection} } }`,
|
||||
variables: { owner, name, cursor },
|
||||
}),
|
||||
),
|
||||
);
|
||||
if (response.errors?.length)
|
||||
throw new Error(
|
||||
`GitHub inventory query failed: ${JSON.stringify(response.errors)}`,
|
||||
);
|
||||
if (!response.data?.repository)
|
||||
throw new Error(`GitHub repository unavailable: ${repository}`);
|
||||
return response.data.repository;
|
||||
}
|
||||
|
||||
async function remainingPages(run, repository, selection, field) {
|
||||
const initial = field.initial;
|
||||
const nodes = [...initial.nodes];
|
||||
let page = initial;
|
||||
const cursors = new Set();
|
||||
while (page.pageInfo.hasNextPage) {
|
||||
const cursor = page.pageInfo.endCursor;
|
||||
if (!cursor || cursors.has(cursor))
|
||||
throw new Error(`GitHub returned a non-advancing ${field.name} cursor`);
|
||||
cursors.add(cursor);
|
||||
const response = await query(run, repository, selection, cursor);
|
||||
page = field.schema.parse(response.item?.[field.name]);
|
||||
nodes.push(...page.nodes);
|
||||
}
|
||||
return nodes;
|
||||
}
|
||||
|
||||
/** GitHub's commit authors include co-authors; unresolved identities remain explicit. */
|
||||
export async function collectGitHubCredits(run, repository, commits) {
|
||||
const shipped = new Set(commits.map((commit) => commit.sha));
|
||||
const numbers = new Set();
|
||||
const result = [];
|
||||
// Small query batches limit transient GraphQL work, never the history examined.
|
||||
for (let offset = 0; offset < commits.length; offset += 20) {
|
||||
const batch = commits.slice(offset, offset + 20);
|
||||
const selection = batch
|
||||
.map(
|
||||
(commit, index) =>
|
||||
`c${index}: object(oid:${JSON.stringify(commit.sha)}) { ... on Commit { oid ${AUTHORS} ${ASSOCIATED} } }`,
|
||||
)
|
||||
.join("\n");
|
||||
const response = await query(run, repository, selection);
|
||||
for (const [index, commit] of batch.entries()) {
|
||||
const value = z
|
||||
.object({
|
||||
oid: z.literal(commit.sha),
|
||||
authors: connection(author),
|
||||
associatedPullRequests: connection(associated),
|
||||
})
|
||||
.parse(response[`c${index}`]);
|
||||
const select = (fields) =>
|
||||
`item: object(oid:${JSON.stringify(commit.sha)}) { ... on Commit { ${fields} } }`;
|
||||
const authors = await remainingPages(run, repository, select(AUTHORS), {
|
||||
name: "authors",
|
||||
schema: connection(author),
|
||||
initial: value.authors,
|
||||
});
|
||||
const associations = await remainingPages(
|
||||
run,
|
||||
repository,
|
||||
select(ASSOCIATED),
|
||||
{
|
||||
name: "associatedPullRequests",
|
||||
schema: connection(associated),
|
||||
initial: value.associatedPullRequests,
|
||||
},
|
||||
);
|
||||
const pullRequests = [
|
||||
...new Set(
|
||||
associations
|
||||
.filter((item) => shipped.has(item.mergeCommit?.oid))
|
||||
.map((item) => item.number),
|
||||
),
|
||||
];
|
||||
for (const number of pullRequests) numbers.add(number);
|
||||
result.push({ ...commit, authors, pullRequests });
|
||||
}
|
||||
}
|
||||
const pullRequests = [];
|
||||
const sorted = [...numbers].sort((a, b) => a - b);
|
||||
for (let offset = 0; offset < sorted.length; offset += 20) {
|
||||
const batch = sorted.slice(offset, offset + 20);
|
||||
const fields = `number title url author { ${ACTOR} } mergeCommit { oid } ${ISSUES}`;
|
||||
const response = await query(
|
||||
run,
|
||||
repository,
|
||||
batch
|
||||
.map(
|
||||
(number, index) =>
|
||||
`p${index}: pullRequest(number:${number}) { ${fields} }`,
|
||||
)
|
||||
.join("\n"),
|
||||
);
|
||||
for (const [index, number] of batch.entries()) {
|
||||
const value = pr.parse(response[`p${index}`]);
|
||||
if (value.number !== number || !shipped.has(value.mergeCommit.oid))
|
||||
throw new Error(`PR #${number} changed outside the selected history`);
|
||||
const issues = await remainingPages(
|
||||
run,
|
||||
repository,
|
||||
`item: pullRequest(number:${number}) { ${ISSUES} }`,
|
||||
{
|
||||
name: "closingIssuesReferences",
|
||||
schema: connection(issue),
|
||||
initial: value.closingIssuesReferences,
|
||||
},
|
||||
);
|
||||
const { closingIssuesReferences: _connection, ...metadata } = value;
|
||||
pullRequests.push({ ...metadata, issues });
|
||||
}
|
||||
}
|
||||
return { commits: result, pullRequests };
|
||||
}
|
||||
@@ -0,0 +1,118 @@
|
||||
import { valid } from "semver";
|
||||
import { z } from "zod";
|
||||
import {
|
||||
releaseInventorySchema,
|
||||
verifyInventoryHistory,
|
||||
verifyReleaseNotes,
|
||||
} from "./release-notes.mjs";
|
||||
|
||||
const sha = z.string().regex(/^[a-f0-9]{40}$/u);
|
||||
const repository = z.string().regex(/^[\w.-]+\/[\w.-]+$/u);
|
||||
const pullRequest = z.object({
|
||||
number: z.number().int().positive(),
|
||||
body: z.string(),
|
||||
merged_at: z.string().nullable(),
|
||||
merge_commit_sha: sha.nullable(),
|
||||
head: z.object({
|
||||
ref: z.string(),
|
||||
repo: z.object({ full_name: z.string() }).nullable(),
|
||||
}),
|
||||
labels: z.array(z.object({ name: z.string() })),
|
||||
});
|
||||
|
||||
/** Read the reviewed Git snapshot and live PR before Release Please can mutate tags. */
|
||||
export async function verifyNotesPublication(run, options) {
|
||||
const repo = repository.parse(options.repository);
|
||||
const source = sha.parse(
|
||||
(
|
||||
await run("git", [
|
||||
"rev-parse",
|
||||
"--verify",
|
||||
"--end-of-options",
|
||||
`${options.source}^{commit}`,
|
||||
])
|
||||
).trim(),
|
||||
);
|
||||
const read = (path) => run("git", ["show", `${source}:${path}`]);
|
||||
const { version } = z
|
||||
.object({ version: z.string() })
|
||||
.parse(JSON.parse(await read("package.json")));
|
||||
if (valid(version) !== version || version.includes("+"))
|
||||
throw new Error("Invalid reviewed package version");
|
||||
let candidates;
|
||||
if (options.pr !== undefined) {
|
||||
const number = z.coerce.number().int().positive().parse(options.pr);
|
||||
candidates = [
|
||||
pullRequest.parse(
|
||||
JSON.parse(await run("gh", ["api", `repos/${repo}/pulls/${number}`])),
|
||||
),
|
||||
];
|
||||
} else {
|
||||
candidates = z
|
||||
.array(z.array(pullRequest))
|
||||
.parse(
|
||||
JSON.parse(
|
||||
await run("gh", [
|
||||
"api",
|
||||
"--paginate",
|
||||
"--slurp",
|
||||
`repos/${repo}/commits/${source}/pulls?per_page=100`,
|
||||
]),
|
||||
),
|
||||
)
|
||||
.flat();
|
||||
}
|
||||
const matching = candidates.filter(
|
||||
(pr) =>
|
||||
pr.merged_at !== null &&
|
||||
pr.merge_commit_sha === source &&
|
||||
pr.head.repo?.full_name.toLowerCase() === repo.toLowerCase() &&
|
||||
pr.head.ref.startsWith("release-please--branches--"),
|
||||
);
|
||||
if (matching.length !== 1)
|
||||
throw new Error(
|
||||
"Expected exactly one merged Release Please PR for the reviewed source",
|
||||
);
|
||||
const pr = matching[0];
|
||||
const path = `docs/releases/${version}.contributions.json`;
|
||||
const present =
|
||||
(
|
||||
await run("git", ["ls-tree", "--name-only", source, "--", path])
|
||||
).trim() === path;
|
||||
if (!present) {
|
||||
if (pr.labels.some((label) => label.name === "rea:release-notes-finalized"))
|
||||
throw new Error(
|
||||
"Finalized release PR is missing its contribution inventory",
|
||||
);
|
||||
return { status: "legacy", version, pullRequest: pr.number };
|
||||
}
|
||||
const inventory = releaseInventorySchema.parse(JSON.parse(await read(path)));
|
||||
if (
|
||||
inventory.version !== version ||
|
||||
inventory.repository.toLowerCase() !== repo.toLowerCase()
|
||||
)
|
||||
throw new Error(
|
||||
"Contribution inventory differs from the reviewed release version or repository",
|
||||
);
|
||||
const baseline = z
|
||||
.object({ ".": z.string() })
|
||||
.parse(
|
||||
JSON.parse(
|
||||
await run("git", ["show", `${source}^1:.release-please-manifest.json`]),
|
||||
),
|
||||
)["."];
|
||||
if (valid(baseline) !== baseline || baseline.includes("+"))
|
||||
throw new Error("Invalid release baseline version");
|
||||
const baselineSha = (
|
||||
await run("git", ["rev-parse", `refs/tags/rea-agents-${baseline}^{commit}`])
|
||||
).trim();
|
||||
if (inventory.base !== baselineSha)
|
||||
throw new Error(
|
||||
"Contribution inventory does not start at the published baseline tag",
|
||||
);
|
||||
await verifyInventoryHistory(run, inventory, source);
|
||||
return {
|
||||
status: "verified",
|
||||
...verifyReleaseNotes(inventory, await read("CHANGELOG.md"), pr.body),
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,415 @@
|
||||
import { valid } from "semver";
|
||||
import { z } from "zod";
|
||||
|
||||
const sha = z.string().regex(/^[a-f0-9]{40}$/u);
|
||||
const actor = z.object({
|
||||
login: z.string().regex(/^[A-Za-z0-9][A-Za-z0-9-]*(?:\[bot\])?$/u),
|
||||
__typename: z.string(),
|
||||
});
|
||||
const author = z.object({
|
||||
name: z.string(),
|
||||
email: z.string(),
|
||||
user: actor.nullable(),
|
||||
});
|
||||
const commit = z.object({
|
||||
sha,
|
||||
subject: z.string(),
|
||||
body: z.string(),
|
||||
parents: z.array(sha),
|
||||
authors: z.array(author),
|
||||
pullRequests: z.array(z.number().int().positive()),
|
||||
});
|
||||
const issue = z.object({
|
||||
number: z.number().int().positive(),
|
||||
url: z.url(),
|
||||
author: actor.nullable(),
|
||||
repository: z.object({ nameWithOwner: z.string() }),
|
||||
});
|
||||
const pullRequest = z.object({
|
||||
number: z.number().int().positive(),
|
||||
title: z.string(),
|
||||
url: z.url(),
|
||||
author: actor.nullable(),
|
||||
mergeCommit: z.object({ oid: sha }),
|
||||
issues: z.array(issue),
|
||||
});
|
||||
export {
|
||||
author as releaseAuthorSchema,
|
||||
issue as releaseIssueSchema,
|
||||
pullRequest as releasePullRequestSchema,
|
||||
};
|
||||
export const releaseInventorySchema = z.object({
|
||||
schemaVersion: z.literal(1),
|
||||
repository: z.string().regex(/^[\w.-]+\/[\w.-]+$/u),
|
||||
version: z
|
||||
.string()
|
||||
.refine(
|
||||
(value) => valid(value) === value && !value.includes("+"),
|
||||
"Expected exact release SemVer",
|
||||
),
|
||||
base: sha,
|
||||
target: sha,
|
||||
commits: z.array(commit),
|
||||
pullRequests: z.array(pullRequest),
|
||||
});
|
||||
|
||||
export async function readReleaseHistory(run, base, target) {
|
||||
if (
|
||||
(await run("git", ["rev-parse", "--is-shallow-repository"])).trim() !==
|
||||
"false"
|
||||
)
|
||||
throw new Error(
|
||||
"Release inventory requires complete Git history; fetch --unshallow first",
|
||||
);
|
||||
const resolve = async (ref) =>
|
||||
sha.parse(
|
||||
(
|
||||
await run("git", [
|
||||
"rev-parse",
|
||||
"--verify",
|
||||
"--end-of-options",
|
||||
`${ref}^{commit}`,
|
||||
])
|
||||
).trim(),
|
||||
);
|
||||
const baseSha = await resolve(base);
|
||||
const targetSha = await resolve(target);
|
||||
await run("git", ["merge-base", "--is-ancestor", baseSha, targetSha]);
|
||||
const text = await run("git", [
|
||||
"log",
|
||||
"--reverse",
|
||||
"--topo-order",
|
||||
"-z",
|
||||
"--format=%H%x00%s%x00%B%x00%P",
|
||||
`${baseSha}..${targetSha}`,
|
||||
]);
|
||||
const fields = text.split("\0");
|
||||
if (fields.pop() !== "" || fields.length % 4)
|
||||
throw new Error("Malformed Git release history");
|
||||
const commits = [];
|
||||
for (let index = 0; index < fields.length; index += 4) {
|
||||
commits.push({
|
||||
sha: sha.parse(fields[index]),
|
||||
subject: fields[index + 1],
|
||||
body: fields[index + 2],
|
||||
parents: fields[index + 3].split(" ").filter(Boolean),
|
||||
});
|
||||
}
|
||||
return { base: baseSha, target: targetSha, commits };
|
||||
}
|
||||
|
||||
// REA history maps Anthropic and Cursor's agent trailers to ordinary User actors.
|
||||
// Keep those source identities in the inventory while excluding automation credit.
|
||||
const AGENT_ACCOUNTS = new Set(["claude", "cursoragent"]);
|
||||
const human = (value) =>
|
||||
value?.__typename === "User" &&
|
||||
!value.login.endsWith("[bot]") &&
|
||||
!AGENT_ACCOUNTS.has(value.login.toLowerCase())
|
||||
? value.login
|
||||
: undefined;
|
||||
const unique = (values) =>
|
||||
[...new Set(values.filter(Boolean))].sort((a, b) =>
|
||||
a.toLowerCase().localeCompare(b.toLowerCase()),
|
||||
);
|
||||
|
||||
function prCredits(inventory, pr) {
|
||||
return unique([
|
||||
human(pr.author),
|
||||
...pr.issues.map((item) => human(item.author)),
|
||||
...inventory.commits
|
||||
.filter(
|
||||
(item) =>
|
||||
item.parents.length === 1 && item.pullRequests.includes(pr.number),
|
||||
)
|
||||
.flatMap((item) => item.authors.map((identity) => human(identity.user))),
|
||||
]);
|
||||
}
|
||||
|
||||
function thanks(handles) {
|
||||
return handles.length
|
||||
? ` Thanks ${handles.map((handle) => `@${handle}`).join(", ")}.`
|
||||
: "";
|
||||
}
|
||||
|
||||
export function contributionRecord(raw) {
|
||||
const inventory = releaseInventorySchema.parse(raw);
|
||||
const lines = [
|
||||
"### Complete contribution record",
|
||||
"",
|
||||
`History: \`${inventory.base}..${inventory.target}\`; ${inventory.pullRequests.length} merged PRs, ${inventory.commits.length} commits examined.`,
|
||||
"",
|
||||
];
|
||||
for (const pr of inventory.pullRequests) {
|
||||
const issues = pr.issues
|
||||
.map(
|
||||
(item) =>
|
||||
`[${item.repository.nameWithOwner}#${item.number}](${item.url})`,
|
||||
)
|
||||
.join(", ");
|
||||
lines.push(
|
||||
`- **PR [#${pr.number}](${pr.url})**${issues ? ` Related ${issues}.` : ""}${thanks(prCredits(inventory, pr))}`,
|
||||
);
|
||||
}
|
||||
const direct = inventory.commits.filter(
|
||||
(item) => item.pullRequests.length === 0 && item.parents.length <= 1,
|
||||
);
|
||||
if (direct.length) {
|
||||
lines.push("", "#### Direct contributions", "");
|
||||
for (const item of direct)
|
||||
lines.push(
|
||||
`- [${item.sha.slice(0, 7)}](https://github.com/${inventory.repository}/commit/${item.sha})${thanks(unique(item.authors.map((identity) => human(identity.user))))}`,
|
||||
);
|
||||
}
|
||||
return `${lines.join("\n")}\n`;
|
||||
}
|
||||
|
||||
export function selectedReleaseNotes(text, version) {
|
||||
const headings = [...text.matchAll(/^## \[([^\]]+)\].*$/gmu)];
|
||||
if (headings[0]?.[1] !== version)
|
||||
throw new Error(`Notes must start with release [${version}]`);
|
||||
return text
|
||||
.slice(headings[0].index, headings[1]?.index ?? text.length)
|
||||
.trim();
|
||||
}
|
||||
|
||||
// Notes have one Markdown list item per line; fenced examples are never credit sources.
|
||||
function proseBullets(text) {
|
||||
let fence;
|
||||
return text.split("\n").filter((line) => {
|
||||
const marker = /^\s*(`{3,}|~{3,})/u.exec(line)?.[1];
|
||||
if (marker) {
|
||||
if (!fence) fence = marker;
|
||||
else if (marker[0] === fence[0] && marker.length >= fence.length)
|
||||
fence = undefined;
|
||||
return false;
|
||||
}
|
||||
return !fence && /^[-*] /u.test(line);
|
||||
});
|
||||
}
|
||||
|
||||
function bulletPrs(bullet, repository) {
|
||||
// Resolve explicit links first so cross-repository numbers cannot acquire local credit.
|
||||
const local = [];
|
||||
const withoutLinks = bullet.replace(
|
||||
/\[([^\]]*)\]\(https:\/\/github\.com\/([^/]+\/[^/]+)\/(?:pull|issues)\/(\d+)\)/gu,
|
||||
(_match, _label, repo, number) => {
|
||||
if (repo.toLowerCase() === repository.toLowerCase())
|
||||
local.push(Number(number));
|
||||
return "";
|
||||
},
|
||||
);
|
||||
for (const match of withoutLinks.matchAll(/(?<![\w/])#(\d+)\b/gu))
|
||||
local.push(Number(match[1]));
|
||||
return new Set(local);
|
||||
}
|
||||
|
||||
function requiredCredits(inventory, bullet, references) {
|
||||
const required = [];
|
||||
for (const pr of inventory.pullRequests.filter((item) =>
|
||||
references.has(item.number),
|
||||
)) {
|
||||
for (const handle of prCredits(inventory, pr))
|
||||
required.push({ handle, context: `PR #${pr.number}` });
|
||||
}
|
||||
for (const issue of inventory.pullRequests.flatMap((item) => item.issues)) {
|
||||
const mentioned =
|
||||
(issue.repository.nameWithOwner.toLowerCase() ===
|
||||
inventory.repository.toLowerCase() &&
|
||||
references.has(issue.number)) ||
|
||||
bullet.includes(`](${issue.url})`);
|
||||
const handle = human(issue.author);
|
||||
if (mentioned && handle)
|
||||
required.push({
|
||||
handle,
|
||||
context: `issue ${issue.repository.nameWithOwner}#${issue.number}`,
|
||||
});
|
||||
}
|
||||
for (const match of bullet.matchAll(
|
||||
/https:\/\/github\.com\/([^/]+\/[^/]+)\/commit\/([a-f0-9]{7,40})\b/gu,
|
||||
)) {
|
||||
if (match[1].toLowerCase() !== inventory.repository.toLowerCase()) continue;
|
||||
const matches = inventory.commits.filter((item) =>
|
||||
item.sha.startsWith(match[2]),
|
||||
);
|
||||
if (matches.length !== 1)
|
||||
throw new Error(`Unknown or ambiguous release commit ${match[2]}`);
|
||||
if (matches[0].parents.length > 1) continue;
|
||||
for (const identity of matches[0].authors) {
|
||||
const handle = human(identity.user);
|
||||
if (handle) required.push({ handle, context: `commit ${match[2]}` });
|
||||
}
|
||||
}
|
||||
return required;
|
||||
}
|
||||
|
||||
function verifyEditorialCredits(inventory, prose) {
|
||||
const known = new Set(inventory.pullRequests.map((pr) => pr.number));
|
||||
const issues = new Set(
|
||||
inventory.pullRequests.flatMap((pr) =>
|
||||
pr.issues
|
||||
.filter(
|
||||
(item) =>
|
||||
item.repository.nameWithOwner.toLowerCase() ===
|
||||
inventory.repository.toLowerCase(),
|
||||
)
|
||||
.map((item) => item.number),
|
||||
),
|
||||
);
|
||||
const eligible = new Set(
|
||||
unique([
|
||||
...inventory.pullRequests.flatMap((pr) => prCredits(inventory, pr)),
|
||||
...inventory.commits.flatMap((item) =>
|
||||
item.authors.map((identity) => human(identity.user)),
|
||||
),
|
||||
]).map((handle) => handle.toLowerCase()),
|
||||
);
|
||||
const errors = [];
|
||||
for (const bullet of proseBullets(prose)) {
|
||||
const references = bulletPrs(bullet, inventory.repository);
|
||||
const start = bullet.indexOf("Thanks ");
|
||||
const thanked = new Set(
|
||||
[
|
||||
...bullet
|
||||
.slice(start < 0 ? bullet.length : start)
|
||||
.matchAll(/@([A-Za-z0-9-]+(?:\[bot\])?)/gu),
|
||||
].map((match) => match[1].toLowerCase()),
|
||||
);
|
||||
for (const number of references)
|
||||
if (!known.has(number) && !issues.has(number))
|
||||
errors.push(`Unknown release reference #${number}`);
|
||||
for (const { handle, context } of requiredCredits(
|
||||
inventory,
|
||||
bullet,
|
||||
references,
|
||||
)) {
|
||||
if (!thanked.has(handle.toLowerCase()))
|
||||
errors.push(`Missing Thanks @${handle} in bullet for ${context}`);
|
||||
}
|
||||
for (const handle of thanked)
|
||||
if (!eligible.has(handle))
|
||||
errors.push(`Unverified human credit @${handle}`);
|
||||
}
|
||||
return errors;
|
||||
}
|
||||
|
||||
export function verifyReleaseNotes(raw, text, body) {
|
||||
const inventory = releaseInventorySchema.parse(raw);
|
||||
const notes = selectedReleaseNotes(text, inventory.version);
|
||||
// Release Please's inline issue/PR body limit (pull-request-overflow-handler).
|
||||
if (notes.length > 65_536 || (body !== undefined && body.length > 65_536))
|
||||
throw new Error(
|
||||
"Release notes exceed GitHub's inline PR body limit; overflow-link bodies are not supported by this checker",
|
||||
);
|
||||
if (notes.length > 125_000 || Buffer.byteLength(notes) > 125_000)
|
||||
throw new Error(
|
||||
"Release notes exceed GitHub's body budget; shorten prose without dropping credit",
|
||||
);
|
||||
const record = contributionRecord(inventory).trim();
|
||||
const start = notes.indexOf("### Complete contribution record");
|
||||
if (start < 0 || notes.slice(start).trim() !== record)
|
||||
throw new Error(
|
||||
"Contribution record differs from inventory; regenerate it with release-notes record",
|
||||
);
|
||||
const prose = notes.slice(0, start);
|
||||
const errors = verifyEditorialCredits(inventory, prose);
|
||||
for (const heading of ["Highlights", "Changes", "Fixes"])
|
||||
if (!prose.includes(`### ${heading}\n`))
|
||||
errors.push(`Missing ${heading} section`);
|
||||
if (body !== undefined) {
|
||||
const bodyNotes = selectedReleaseNotes(body, inventory.version)
|
||||
.replace(
|
||||
/\n---\nThis PR was generated with \[Release Please\][\s\S]*$/u,
|
||||
"",
|
||||
)
|
||||
.trim();
|
||||
if (bodyNotes !== notes)
|
||||
errors.push("Release PR body differs from CHANGELOG.md");
|
||||
}
|
||||
if (errors.length) throw new Error(errors.join("\n"));
|
||||
return {
|
||||
version: inventory.version,
|
||||
pullRequests: inventory.pullRequests.length,
|
||||
commits: inventory.commits.length,
|
||||
unresolvedAuthors: inventory.commits.flatMap((item) =>
|
||||
item.authors
|
||||
.filter((identity) => identity.user === null)
|
||||
.map((identity) => ({ sha: item.sha, ...identity })),
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
/** Reject stale inventories and product changes made after the editorial target. */
|
||||
export async function verifyInventoryHistory(run, raw, source) {
|
||||
const inventory = releaseInventorySchema.parse(raw);
|
||||
const history = await readReleaseHistory(
|
||||
run,
|
||||
inventory.base,
|
||||
inventory.target,
|
||||
);
|
||||
const original = inventory.commits.map(
|
||||
({ authors: _authors, pullRequests: _prs, ...item }) => item,
|
||||
);
|
||||
if (JSON.stringify(history.commits) !== JSON.stringify(original))
|
||||
throw new Error("Inventory does not match complete local Git history");
|
||||
const shas = new Set(history.commits.map((item) => item.sha));
|
||||
const numbers = new Set();
|
||||
for (const pr of inventory.pullRequests) {
|
||||
if (numbers.has(pr.number) || !shas.has(pr.mergeCommit.oid))
|
||||
throw new Error(`Duplicate or out-of-range PR #${pr.number}`);
|
||||
numbers.add(pr.number);
|
||||
}
|
||||
for (const item of inventory.commits)
|
||||
for (const number of item.pullRequests)
|
||||
if (!numbers.has(number))
|
||||
throw new Error(`Commit refers to missing PR #${number}`);
|
||||
if (source !== undefined) {
|
||||
const resolved = sha.parse(
|
||||
(
|
||||
await run("git", [
|
||||
"rev-parse",
|
||||
"--verify",
|
||||
"--end-of-options",
|
||||
`${source}^{commit}`,
|
||||
])
|
||||
).trim(),
|
||||
);
|
||||
const parent = (await run("git", ["rev-parse", `${resolved}^1`])).trim();
|
||||
if (parent !== inventory.target)
|
||||
throw new Error(
|
||||
"Release base moved after notes target; regenerate inventory for the reviewed merge's first parent",
|
||||
);
|
||||
await run("git", [
|
||||
"merge-base",
|
||||
"--is-ancestor",
|
||||
inventory.target,
|
||||
resolved,
|
||||
]);
|
||||
const allowed = new Set([
|
||||
"CHANGELOG.md",
|
||||
".release-please-manifest.json",
|
||||
"package.json",
|
||||
"package-lock.json",
|
||||
"server.json",
|
||||
"src/generatedPackageMetadata.ts",
|
||||
"docs/installation.md",
|
||||
`docs/releases/${inventory.version}.contributions.json`,
|
||||
]);
|
||||
const paths = (
|
||||
await run("git", [
|
||||
"diff",
|
||||
"--name-only",
|
||||
"--no-renames",
|
||||
"-z",
|
||||
inventory.target,
|
||||
resolved,
|
||||
])
|
||||
)
|
||||
.split("\0")
|
||||
.filter(Boolean);
|
||||
const product = paths.filter((path) => !allowed.has(path));
|
||||
if (product.length)
|
||||
throw new Error(
|
||||
`Product changed after notes target; regenerate inventory: ${product.join(", ")}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
+16
-1
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { access } from "node:fs/promises";
|
||||
import { dirname, resolve } from "node:path";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
// Route production MCP before importing Incur. Incur owns registration helpers
|
||||
@@ -15,6 +15,21 @@ const isMcpMode =
|
||||
args.length === 1 && (args[0] === "--mcp" || args[0] === "mcp");
|
||||
const isMcpDoctorMode = args[0] === "mcp" && args[1] === "doctor";
|
||||
const packageRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
|
||||
|
||||
// Playwright's registry assumes one of its three desktop host platforms when
|
||||
// it chooses the browser cache directory. Termux reports `android`, although
|
||||
// its Node runtime and filesystem are POSIX-compatible. Keep the cache path
|
||||
// explicit so REA can start on Termux. Browser launch still requires a
|
||||
// caller-supplied Android-compatible Chromium executable.
|
||||
if (
|
||||
process.platform === "android" &&
|
||||
process.env.PLAYWRIGHT_BROWSERS_PATH === undefined
|
||||
) {
|
||||
const cacheRoot =
|
||||
process.env.XDG_CACHE_HOME ?? join(process.env.HOME ?? ".", ".cache");
|
||||
process.env.PLAYWRIGHT_BROWSERS_PATH = join(cacheRoot, "ms-playwright");
|
||||
}
|
||||
|
||||
const runtimeFiles = isMcpMode
|
||||
? ["dist/main.js"]
|
||||
: isMcpDoctorMode
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
#!/usr/bin/env node
|
||||
import { execFile } from "node:child_process";
|
||||
import { readFile, writeFile } from "node:fs/promises";
|
||||
import { parseArgs, promisify } from "node:util";
|
||||
import { collectGitHubCredits } from "./lib/release-notes-github.mjs";
|
||||
import { verifyNotesPublication } from "./lib/release-notes-publication.mjs";
|
||||
import {
|
||||
contributionRecord,
|
||||
readReleaseHistory,
|
||||
releaseInventorySchema,
|
||||
selectedReleaseNotes,
|
||||
verifyInventoryHistory,
|
||||
verifyReleaseNotes,
|
||||
} from "./lib/release-notes.mjs";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
async function run(command, args, input) {
|
||||
const pending = execFileAsync(command, args, {
|
||||
encoding: "utf8",
|
||||
maxBuffer: 64 * 1024 * 1024,
|
||||
});
|
||||
pending.child.stdin.end(input);
|
||||
return (await pending).stdout;
|
||||
}
|
||||
|
||||
async function createInventory(values) {
|
||||
if (!values.repository || !values.base || !values.target || !values.version)
|
||||
throw new Error(
|
||||
"inventory requires --repository, --base, --target and --version",
|
||||
);
|
||||
// Parse caller-selected boundaries before making network requests.
|
||||
const history = await readReleaseHistory(run, values.base, values.target);
|
||||
releaseInventorySchema.omit({ commits: true, pullRequests: true }).parse({
|
||||
schemaVersion: 1,
|
||||
repository: values.repository,
|
||||
version: values.version,
|
||||
base: history.base,
|
||||
target: history.target,
|
||||
});
|
||||
const credits = await collectGitHubCredits(
|
||||
run,
|
||||
values.repository,
|
||||
history.commits,
|
||||
);
|
||||
const inventory = releaseInventorySchema.parse({
|
||||
schemaVersion: 1,
|
||||
repository: values.repository,
|
||||
version: values.version,
|
||||
base: history.base,
|
||||
target: history.target,
|
||||
...credits,
|
||||
});
|
||||
await verifyInventoryHistory(run, inventory);
|
||||
await writeFile(values.manifest, `${JSON.stringify(inventory, null, 2)}\n`, {
|
||||
flag: "wx",
|
||||
});
|
||||
process.stdout.write(
|
||||
`Inventoried ${inventory.commits.length} commits and ${inventory.pullRequests.length} merged PRs. Saved ${values.manifest}\n`,
|
||||
);
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const { values, positionals } = parseArgs({
|
||||
allowPositionals: true,
|
||||
options: {
|
||||
repository: { type: "string" },
|
||||
pr: { type: "string" },
|
||||
base: { type: "string" },
|
||||
target: { type: "string" },
|
||||
version: { type: "string" },
|
||||
manifest: { type: "string" },
|
||||
notes: { type: "string", default: "CHANGELOG.md" },
|
||||
"pr-body": { type: "string" },
|
||||
source: { type: "string" },
|
||||
output: { type: "string" },
|
||||
},
|
||||
});
|
||||
const [command] = positionals;
|
||||
if (command === "publication" && positionals.length === 1) {
|
||||
if (!values.repository || !values.source)
|
||||
throw new Error(
|
||||
"publication requires --repository and --source; --pr selects the merged release PR",
|
||||
);
|
||||
process.stdout.write(
|
||||
`${JSON.stringify(await verifyNotesPublication(run, values), null, 2)}\n`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (
|
||||
positionals.length !== 1 ||
|
||||
!["inventory", "record", "check", "render"].includes(command) ||
|
||||
!values.manifest
|
||||
)
|
||||
throw new Error(
|
||||
"Usage: release-notes inventory|record|check|render --manifest FILE [--repository OWNER/REPO --base TAG --target SHA --version VERSION] [--notes FILE --pr-body FILE --source SHA --output FILE]",
|
||||
);
|
||||
if (command === "inventory") {
|
||||
await createInventory(values);
|
||||
return;
|
||||
}
|
||||
const inventory = releaseInventorySchema.parse(
|
||||
JSON.parse(await readFile(values.manifest, "utf8")),
|
||||
);
|
||||
await verifyInventoryHistory(run, inventory, values.source);
|
||||
if (command === "record") {
|
||||
if (values.output)
|
||||
await writeFile(values.output, contributionRecord(inventory), {
|
||||
flag: "wx",
|
||||
});
|
||||
else process.stdout.write(contributionRecord(inventory));
|
||||
return;
|
||||
}
|
||||
const notes = await readFile(values.notes, "utf8");
|
||||
const body = values["pr-body"]
|
||||
? await readFile(values["pr-body"], "utf8")
|
||||
: undefined;
|
||||
const report = verifyReleaseNotes(inventory, notes, body);
|
||||
if (command === "check") {
|
||||
process.stdout.write(`${JSON.stringify(report, null, 2)}\n`);
|
||||
return;
|
||||
}
|
||||
if (!values.output)
|
||||
throw new Error(
|
||||
"render requires --output (an existing file is never overwritten)",
|
||||
);
|
||||
const rendered = selectedReleaseNotes(notes, inventory.version);
|
||||
await writeFile(values.output, `${rendered}\n`, { flag: "wx" });
|
||||
}
|
||||
|
||||
await main().catch((error) => {
|
||||
process.stderr.write(
|
||||
`Release notes failed: ${error instanceof Error ? error.message : String(error)}\n`,
|
||||
);
|
||||
process.exitCode = 1;
|
||||
});
|
||||
@@ -175,6 +175,7 @@ const callerDirectory = join(workspace, "caller cwd with spaces");
|
||||
const scriptCollisions = [
|
||||
"ReaGhidraBridge.java",
|
||||
"ReaGhidraPrepareCom.java",
|
||||
"ReaGhidraApplySeeds.java",
|
||||
"ReaGhidraNoReturnFix.java",
|
||||
];
|
||||
await mkdir(callerDirectory);
|
||||
|
||||
@@ -45,6 +45,7 @@ import { verifyBrowserDomDestinations } from "./lib/browser-dom-destinations-e2e
|
||||
import { verifyBrowserCaptureMetadataBudget } from "./lib/browser-capture-metadata-budget-e2e.mjs";
|
||||
import { artifactCliEvidence, artifactMcpResult } from "./lib/artifact-e2e.mjs";
|
||||
import { verifyScenarioEnvironment } from "./lib/browser-scenario-environment-e2e.mjs";
|
||||
import { verifyScenarioStorage } from "./lib/browser-scenario-storage-e2e.mjs";
|
||||
|
||||
const REAL_BROWSER_STARTUP_TIMEOUT_MS = 60_000;
|
||||
const SCENARIO_SECRET_VALUE = "rea-browser-verifier-secret";
|
||||
@@ -64,6 +65,7 @@ try {
|
||||
executable,
|
||||
[
|
||||
"--headless=new",
|
||||
"--password-store=basic",
|
||||
...(process.env.REA_BROWSER_NO_SANDBOX === "true"
|
||||
? ["--no-sandbox"]
|
||||
: []),
|
||||
@@ -280,6 +282,11 @@ try {
|
||||
target,
|
||||
site.origin,
|
||||
);
|
||||
const scenarioStorage = await verifyScenarioStorage({
|
||||
executable,
|
||||
endpoint,
|
||||
targetId: target,
|
||||
});
|
||||
const launchedScenario = await createBrowserScenarioProvider(
|
||||
process.env,
|
||||
).captureScenario(
|
||||
@@ -354,7 +361,11 @@ try {
|
||||
scripts: observed.value.scripts.items.length,
|
||||
networkRequests: observed.value.network.requests.length,
|
||||
consoleEvents: observed.value.console.events.length,
|
||||
websocketEvents: observed.value.network.websocket_events.length,
|
||||
websocketConnections: observed.value.network.websocket_connections.length,
|
||||
websocketFrames: observed.value.network.websocket_connections.reduce(
|
||||
(total, connection) => total + connection.events.length,
|
||||
0,
|
||||
),
|
||||
bundleScripts: bundle.capture.scripts_analyzed,
|
||||
sourceMaps: bundle.observations.source_maps.processed,
|
||||
bundle_cli_and_stdio_mcp: true,
|
||||
@@ -373,6 +384,7 @@ try {
|
||||
browserScenarioLaunchCleanup: "terminated-owned-process",
|
||||
scenarioFailure,
|
||||
scenarioEnvironment,
|
||||
scenarioStorage,
|
||||
verified: true,
|
||||
};
|
||||
} finally {
|
||||
|
||||
@@ -21,6 +21,10 @@ import {
|
||||
} from "./verify-real-ghidra-assertions.mjs";
|
||||
|
||||
async function functionCall(client, operation, parameters) {
|
||||
return (await functionObservationCall(client, operation, parameters)).value;
|
||||
}
|
||||
|
||||
async function functionObservationCall(client, operation, parameters) {
|
||||
const input = parseGhidraFunctionInput(operation, parameters);
|
||||
if (!input.ok) throw input.error;
|
||||
const called = await client.callTool(operation, input.value);
|
||||
@@ -153,6 +157,57 @@ export async function verifyDebugFunctionOperations(
|
||||
const denseSwitchDossier = await functionCall(client, "analyze_function", {
|
||||
procedure: denseSwitchProcedure.address,
|
||||
});
|
||||
const terminalProcedure = requireProcedure(
|
||||
procedures,
|
||||
"rea_ghidra_inventory_terminal_call",
|
||||
);
|
||||
const terminalObservations = await Promise.all(
|
||||
[
|
||||
"procedure_info",
|
||||
"procedure_assembly",
|
||||
"read_function_instructions",
|
||||
"procedure_pseudo_code",
|
||||
"analyze_function",
|
||||
].map((operation) =>
|
||||
functionObservationCall(client, operation, {
|
||||
...(operation === "analyze_function" ? {} : { document: null }),
|
||||
procedure: terminalProcedure.address,
|
||||
}),
|
||||
),
|
||||
);
|
||||
const terminalLimitations = terminalObservations.map((result) =>
|
||||
result.limitations.find(
|
||||
(limitation) =>
|
||||
/terminal call at 0x[0-9a-f]+/i.test(limitation) &&
|
||||
/ to 0x[0-9a-f]+/i.test(limitation) &&
|
||||
limitation.includes("abort") &&
|
||||
limitation.includes("hasNoReturn=true"),
|
||||
),
|
||||
);
|
||||
if (
|
||||
terminalLimitations.some((limitation) => limitation === undefined) ||
|
||||
terminalLimitations.some(
|
||||
(limitation) =>
|
||||
!limitation.includes("external=") ||
|
||||
!limitation.includes("thunk=") ||
|
||||
!limitation.includes("fallthrough is excluded") ||
|
||||
!limitation.includes("not an independent verification"),
|
||||
)
|
||||
)
|
||||
throw new Error(
|
||||
`Ghidra no-return boundary evidence was missing callee, address, or flag provenance: ${JSON.stringify(terminalObservations.map(({ limitations }) => limitations))}`,
|
||||
);
|
||||
const returningObservation = await functionObservationCall(
|
||||
client,
|
||||
"procedure_info",
|
||||
{ document: null, procedure: leaf.address },
|
||||
);
|
||||
if (
|
||||
returningObservation.limitations.some((limitation) =>
|
||||
limitation.includes("puts"),
|
||||
)
|
||||
)
|
||||
throw new Error("Ghidra marked an ordinary returning call as terminal");
|
||||
// The debug fixture is compiled -O0, and GCC 16 and later lower dense
|
||||
// switches to compare chains at -O0, so this dossier asserts only its
|
||||
// complete shape; the optimized jump-table variant proves dense jump-table
|
||||
@@ -179,6 +234,8 @@ export async function verifyDebugFunctionOperations(
|
||||
instruction_window: {
|
||||
returned: instructionWindow.instructions.length,
|
||||
},
|
||||
terminal_call_limitations: terminalLimitations,
|
||||
returning_call_limitations: returningObservation.limitations,
|
||||
cancellation,
|
||||
timeout,
|
||||
concurrency: concurrent,
|
||||
|
||||
@@ -0,0 +1,227 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { execFile } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdir, mkdtemp, readFile, rm } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
import { parseBinaryTarget } from "../dist/application/BinaryTargetResolver.js";
|
||||
import { parseConfig } from "../dist/config/parseConfig.js";
|
||||
import { GhidraClient } from "../dist/ghidra/GhidraClient.js";
|
||||
import { inspectGhidraInstallation } from "../dist/ghidra/GhidraInstallation.js";
|
||||
import { SUPPORTED_GHIDRA_VERSION } from "../dist/ghidra/GhidraInstallationPolicy.js";
|
||||
import { GhidraProvider } from "../dist/ghidra/GhidraProvider.js";
|
||||
import { silentLogger } from "../dist/logger.js";
|
||||
import { cleanupOwnedProcessGroup } from "../dist/process/ProcessOwnership.js";
|
||||
import {
|
||||
ProviderProcessSupervisor,
|
||||
spawnOwnedProviderProcess,
|
||||
} from "../dist/process/ProviderProcess.js";
|
||||
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
|
||||
|
||||
const exec = promisify(execFile);
|
||||
const run = createVerifierRun();
|
||||
const procedureName = (name) =>
|
||||
process.platform === "darwin" ? `_${name}` : name;
|
||||
assert.ok(
|
||||
process.platform === "linux" || process.platform === "darwin",
|
||||
"This regression lane requires Linux or macOS",
|
||||
);
|
||||
assert.ok(
|
||||
process.arch === "x64" || process.arch === "arm64",
|
||||
"This regression lane requires x86-64 or arm64",
|
||||
);
|
||||
const config = parseConfig(process.env);
|
||||
if (!config.ok) throw config.error;
|
||||
const installation = inspectGhidraInstallation({
|
||||
environment: process.env,
|
||||
installDir: config.value.ghidraInstallDir,
|
||||
javaHome: config.value.ghidraJavaHome,
|
||||
});
|
||||
assert.equal(installation.status, "available", JSON.stringify(installation));
|
||||
assert.equal(installation.providerVersion, SUPPORTED_GHIDRA_VERSION);
|
||||
const fixtures = fileURLToPath(
|
||||
new URL("../tests/conformance/ghidra", import.meta.url),
|
||||
);
|
||||
const bridge = fileURLToPath(new URL("../bridge/ghidra", import.meta.url));
|
||||
const workspace = await mkdtemp(join(tmpdir(), "rea-ghidra-noreturn-"));
|
||||
let cleaned = true;
|
||||
let report;
|
||||
const digest = (bytes) => createHash("sha256").update(bytes).digest("hex");
|
||||
try {
|
||||
const targetPath = join(
|
||||
workspace,
|
||||
process.platform === "darwin" ? "noreturn.dylib" : "noreturn.so",
|
||||
);
|
||||
const targetOptions =
|
||||
process.platform === "darwin"
|
||||
? [
|
||||
"-dynamiclib",
|
||||
"-undefined",
|
||||
"dynamic_lookup",
|
||||
"-U_FORTIFY_SOURCE",
|
||||
"-D_FORTIFY_SOURCE=0",
|
||||
]
|
||||
: ["-shared"];
|
||||
await exec(process.env.REA_CC ?? "cc", [
|
||||
...targetOptions,
|
||||
"-fPIC",
|
||||
"-O0",
|
||||
"-fno-builtin",
|
||||
join(fixtures, "no-return.c"),
|
||||
"-o",
|
||||
targetPath,
|
||||
]);
|
||||
const targetDigest = digest(await readFile(targetPath));
|
||||
const sourceDigests = Object.fromEntries(
|
||||
await Promise.all(
|
||||
["ReaGhidraBridge.java", "ReaGhidraNoReturnFix.java"].map(
|
||||
async (name) => [name, digest(await readFile(join(bridge, name)))],
|
||||
),
|
||||
),
|
||||
);
|
||||
const project = join(workspace, "project");
|
||||
await mkdir(project);
|
||||
const launch = await spawnOwnedProviderProcess({
|
||||
command: installation.analyzeHeadlessPath,
|
||||
arguments: [
|
||||
project,
|
||||
"NoReturn",
|
||||
"-import",
|
||||
targetPath,
|
||||
"-readOnly",
|
||||
"-deleteProject",
|
||||
"-scriptPath",
|
||||
`${fixtures};${bridge}`,
|
||||
"-postScript",
|
||||
join(fixtures, "ReaNoReturnProbe.java"),
|
||||
bridge,
|
||||
],
|
||||
runId: run.run_id,
|
||||
expectedCommand: null,
|
||||
});
|
||||
const supervisor = new ProviderProcessSupervisor({
|
||||
...launch,
|
||||
ownsProcessLifetime: true,
|
||||
cleanup: () => cleanupOwnedProcessGroup(launch.ownership),
|
||||
});
|
||||
let probe;
|
||||
try {
|
||||
assert.ok(
|
||||
await supervisor.waitForExit(120000),
|
||||
"No-return probe timed out",
|
||||
);
|
||||
assert.ok(
|
||||
await supervisor.waitForOutputClose(1000),
|
||||
"No-return probe output did not close",
|
||||
);
|
||||
const snapshot = supervisor.snapshot();
|
||||
const output = `${snapshot.stdout.text}\n${snapshot.stderr.text}`;
|
||||
assert.equal(snapshot.exitCode, 0, output);
|
||||
const reports = [
|
||||
...output.matchAll(/REA_NO_RETURN_PROBE_JSON (\{[^\r\n]*\})/gu),
|
||||
];
|
||||
assert.equal(reports.length, 1, output);
|
||||
probe = JSON.parse(reports[0][1]);
|
||||
assert.equal(probe.status, "passed");
|
||||
assert.equal(probe.ghidra_version, SUPPORTED_GHIDRA_VERSION);
|
||||
assert.equal(probe.target_sha256, targetDigest);
|
||||
for (const [name, expected] of Object.entries(sourceDigests))
|
||||
assert.equal(probe[name], expected);
|
||||
assert.ok(probe.checks.includes("independent_entry_preserved"));
|
||||
assert.ok(probe.checks.includes("propagated_flag_repaired_rea_tls_outer"));
|
||||
assert.ok(probe.checks.includes("recovered_pseudocode"));
|
||||
} finally {
|
||||
const stopped = await supervisor.stop();
|
||||
cleaned = stopped.status === "verified-cleanup";
|
||||
assert.ok(
|
||||
cleaned,
|
||||
`Owned cleanup incomplete; retained ${workspace}: ${JSON.stringify(stopped)}`,
|
||||
);
|
||||
}
|
||||
|
||||
// Exercise the actual Java server, authenticated transport, request queue and Evidence
|
||||
// projection. This read-only TCP lane also runs where AF_UNIX is unavailable; it does
|
||||
// not claim Windows authority or validation of the default Linux Unix-socket transport.
|
||||
const provider = new GhidraProvider(
|
||||
config.value,
|
||||
silentLogger,
|
||||
process.env,
|
||||
undefined,
|
||||
(options) =>
|
||||
new GhidraClient({ ...options, transport: "authenticated-loopback-tcp" }),
|
||||
);
|
||||
const target = await parseBinaryTarget(targetPath);
|
||||
if (!target.ok) throw target.error;
|
||||
const profile = await provider.resolveAnalysisProfile(target.value);
|
||||
if (!profile.ok) throw profile.error;
|
||||
assert.ok(profile.value.profile);
|
||||
assert.equal(
|
||||
profile.value.profile.parameters.no_return_repair,
|
||||
"returning-imports-decoded-return-v2",
|
||||
);
|
||||
const client = provider.createClient(target.value, profile.value.profile);
|
||||
cleaned = false;
|
||||
const observations = [];
|
||||
const failures = [];
|
||||
try {
|
||||
for (const operation of [
|
||||
"procedure_pseudo_code",
|
||||
"procedure_assembly",
|
||||
"procedure_info",
|
||||
"read_function_instructions",
|
||||
"analyze_function",
|
||||
]) {
|
||||
const result = await client.execute(operation, {
|
||||
procedure: procedureName("rea_abort"),
|
||||
});
|
||||
if (!result.ok) throw result.error;
|
||||
const warnings = result.value.limitations.filter((value) =>
|
||||
/terminal call/u.test(value),
|
||||
);
|
||||
assert.ok(
|
||||
warnings.some(
|
||||
(value) => /abort/u.test(value) && /fallthrough/u.test(value),
|
||||
),
|
||||
operation,
|
||||
);
|
||||
observations.push({ operation, warnings });
|
||||
}
|
||||
const healthy = await client.execute("procedure_pseudo_code", {
|
||||
procedure: procedureName("rea_tls"),
|
||||
});
|
||||
if (!healthy.ok) throw healthy.error;
|
||||
assert.ok(
|
||||
!healthy.value.limitations.some((value) => /terminal call/u.test(value)),
|
||||
);
|
||||
} catch (cause) {
|
||||
failures.push(cause);
|
||||
}
|
||||
const closed = await client.close();
|
||||
cleaned = closed.ok;
|
||||
if (!closed.ok) failures.push(closed.error);
|
||||
if (failures.length > 0)
|
||||
throw new AggregateError(
|
||||
failures,
|
||||
"No-return Evidence verification failed",
|
||||
);
|
||||
assert.equal(digest(await readFile(targetPath)), targetDigest);
|
||||
report = {
|
||||
status: "passed",
|
||||
probe,
|
||||
observations,
|
||||
scope:
|
||||
"Real Ghidra ELF or Mach-O analysis with injected flags; real read-only TCP Evidence projection. No target execution; exact Amethyst artifact and Windows are unverified.",
|
||||
};
|
||||
} finally {
|
||||
if (cleaned) await rm(workspace, { recursive: true, force: true });
|
||||
}
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
...report,
|
||||
cleanup: "complete",
|
||||
verifier_run: await completeVerifierRun(run),
|
||||
}),
|
||||
);
|
||||
@@ -322,7 +322,7 @@ async function functionOperation(operation, parameters) {
|
||||
const result = parseGhidraFunctionResult(operation, called.value);
|
||||
if (!result.ok) throw result.error;
|
||||
observed.add(operation);
|
||||
return result.value;
|
||||
return result.value.value;
|
||||
}
|
||||
|
||||
function assertSession(session, targetSha256, profileDigest) {
|
||||
|
||||
@@ -26,22 +26,49 @@ export async function verifyElectronToolContracts({
|
||||
await mkdtemp(join(tmpdir(), "rea-electron-e2e-")),
|
||||
);
|
||||
const main = join(root, "main.cjs");
|
||||
const preload = join(root, "preload.cjs");
|
||||
const ready = join(root, "ready");
|
||||
const entrypoint = join(repositoryRoot, "scripts", "rea.mjs");
|
||||
const runId = randomUUID();
|
||||
await writeFile(
|
||||
main,
|
||||
`const {app,BrowserWindow}=require("electron");
|
||||
app.whenReady().then(()=>{const window=new BrowserWindow();
|
||||
window.loadFile("renderer.html");require("node:fs").writeFileSync(${JSON.stringify(ready)},"ready");});`,
|
||||
`const electron = require("electron");
|
||||
const path = require("node:path");
|
||||
const { app, BrowserWindow, ipcMain } = electron;
|
||||
const Window = BrowserWindow;
|
||||
const mainIpc = ipcMain;
|
||||
app.whenReady().then(() => {
|
||||
const window = new Window({
|
||||
webPreferences: { preload: path.join(__dirname, "preload.cjs") },
|
||||
});
|
||||
window.loadFile("renderer.html");
|
||||
require("node:fs").writeFileSync(${JSON.stringify(ready)}, "ready");
|
||||
});
|
||||
mainIpc.handle("fixture:echo", (_event, value) => ({ value }));`,
|
||||
);
|
||||
await writeFile(
|
||||
join(root, "renderer.html"),
|
||||
'<!doctype html><button id="run">Run</button><script src="renderer.js"></script>',
|
||||
`<!doctype html>
|
||||
<button id="run">Run</button>
|
||||
<output id="result"></output>
|
||||
<script src="renderer.js"></script>`,
|
||||
);
|
||||
await writeFile(
|
||||
preload,
|
||||
`const electron = require("electron");
|
||||
const { contextBridge, ipcRenderer } = electron;
|
||||
const bridge = contextBridge;
|
||||
const rendererIpc = ipcRenderer;
|
||||
bridge.exposeInMainWorld("fixture", {
|
||||
echo: (value) => rendererIpc.invoke("fixture:echo", value),
|
||||
});`,
|
||||
);
|
||||
await writeFile(
|
||||
join(root, "renderer.js"),
|
||||
'document.querySelector("#run").addEventListener("click",()=>console.log("fixture-click"));',
|
||||
`document.querySelector("#run").addEventListener("click", async () => {
|
||||
const result = await window.fixture.echo("renderer");
|
||||
document.querySelector("#result").textContent = result.value;
|
||||
});`,
|
||||
);
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
@@ -135,6 +162,25 @@ window.loadFile("renderer.html");require("node:fs").writeFileSync(${JSON.stringi
|
||||
const staticAnalysis = await call("analyze_javascript_application", {
|
||||
input_path: root,
|
||||
});
|
||||
assert.equal(staticAnalysis.normalized_result.summary.browser_windows, 1);
|
||||
assert.equal(
|
||||
staticAnalysis.normalized_result.summary.preload_entrypoints,
|
||||
1,
|
||||
);
|
||||
assert.equal(
|
||||
staticAnalysis.normalized_result.summary.context_bridge_apis,
|
||||
1,
|
||||
);
|
||||
assert.equal(staticAnalysis.normalized_result.summary.ipc.main_handlers, 1);
|
||||
assert.equal(
|
||||
staticAnalysis.normalized_result.summary.ipc.renderer_transmissions,
|
||||
1,
|
||||
);
|
||||
assert.equal(
|
||||
staticAnalysis.normalized_result.summary.ipc
|
||||
.paired_renderer_transmissions,
|
||||
1,
|
||||
);
|
||||
const reconciled = await call("reconcile_javascript_runtime", {
|
||||
static_layers: [
|
||||
{
|
||||
@@ -169,7 +215,7 @@ window.loadFile("renderer.html");require("node:fs").writeFileSync(${JSON.stringi
|
||||
executable_path: executable,
|
||||
application_path: main,
|
||||
application_root: root,
|
||||
actions: [{ step_id: "settle", kind: "wait", duration_ms: 50 }],
|
||||
actions: [{ step_id: "ipc", kind: "click", selector: "#run" }],
|
||||
};
|
||||
const active = await client.callTool(
|
||||
{ name: "capture_electron_scenario", arguments: input },
|
||||
@@ -192,6 +238,13 @@ window.loadFile("renderer.html");require("node:fs").writeFileSync(${JSON.stringi
|
||||
assert.equal(capture.application.cleanup, "terminated-owned-process");
|
||||
}
|
||||
assert.equal(capture.actions[0].status, "completed");
|
||||
assert.ok(
|
||||
capture.ipc.events.some(
|
||||
({ kind, channel }) =>
|
||||
kind === "main-handler-invocation" && channel === "fixture:echo",
|
||||
),
|
||||
JSON.stringify(capture.ipc),
|
||||
);
|
||||
assert.equal(
|
||||
capture.coverage.status,
|
||||
"partial_attach",
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { isAbsolute, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import {
|
||||
spawnOwnedProviderProcess,
|
||||
ProviderProcessSupervisor,
|
||||
} from "../../../dist/process/ProviderProcess.js";
|
||||
import { cleanupOwnedProcessGroup } from "../../../dist/process/ProcessOwnership.js";
|
||||
import { createVerifierRun } from "../../lib/verifier-run.mjs";
|
||||
|
||||
const installation = process.env.GHIDRA_INSTALL_DIR;
|
||||
assert.ok(
|
||||
installation && isAbsolute(installation),
|
||||
"Set GHIDRA_INSTALL_DIR to a local Ghidra installation",
|
||||
);
|
||||
assert.notEqual(
|
||||
process.platform,
|
||||
"win32",
|
||||
"This direct headless regression requires a POSIX host; Windows uses its controlled provider lane",
|
||||
);
|
||||
const workspace = await mkdtemp(join(tmpdir(), "rea-ghidra-seed-regression-"));
|
||||
const run = createVerifierRun();
|
||||
let supervisor;
|
||||
try {
|
||||
const target = join(workspace, "seed-functions.bin");
|
||||
const seeds = join(workspace, "seeds.tsv");
|
||||
// Falling through from the lower entry reaches the higher entry. Creating
|
||||
// the higher function first is what keeps the two function bodies separate.
|
||||
await writeFile(target, Buffer.from([0x90, 0x90, 0xc3]));
|
||||
await writeFile(
|
||||
seeds,
|
||||
[
|
||||
"0x0\tfunction\tseed_low",
|
||||
"0x100000000\tfunction\tout_of_range",
|
||||
"0x1\tfunction\tseed_high",
|
||||
"0x1000\tlabel\tseed_global",
|
||||
"0x1000\tcode",
|
||||
"",
|
||||
].join("\n"),
|
||||
);
|
||||
const launch = await spawnOwnedProviderProcess({
|
||||
command: join(installation, "support", "analyzeHeadless"),
|
||||
arguments: [
|
||||
workspace,
|
||||
"seed-regression",
|
||||
"-import",
|
||||
target,
|
||||
"-loader",
|
||||
"BinaryLoader",
|
||||
"-processor",
|
||||
"x86:LE:32:default",
|
||||
"-noanalysis",
|
||||
"-readOnly",
|
||||
"-deleteProject",
|
||||
"-scriptPath",
|
||||
[
|
||||
fileURLToPath(new URL("../../../bridge/ghidra", import.meta.url)),
|
||||
fileURLToPath(
|
||||
new URL("../../../tests/conformance/ghidra", import.meta.url),
|
||||
),
|
||||
].join(";"),
|
||||
"-postScript",
|
||||
"ReaSeedRegressionProbe.java",
|
||||
seeds,
|
||||
],
|
||||
runId: run.run_id,
|
||||
expectedCommand: null,
|
||||
});
|
||||
supervisor = new ProviderProcessSupervisor({
|
||||
...launch,
|
||||
ownsProcessLifetime: true,
|
||||
cleanup: () => cleanupOwnedProcessGroup(launch.ownership),
|
||||
});
|
||||
assert.ok(
|
||||
await supervisor.waitForExit(180_000),
|
||||
"Ghidra seed regression exceeded its startup deadline",
|
||||
);
|
||||
const snapshot = supervisor.snapshot();
|
||||
const output = `${snapshot.stdout.text}\n${snapshot.stderr.text}`;
|
||||
assert.equal(snapshot.exitCode, 0, output);
|
||||
// Headless can exit zero after a script fails, so require the probe's marker.
|
||||
assert.ok(output.includes("REA_SEED_REGRESSION_OK "), output);
|
||||
} finally {
|
||||
if (supervisor !== undefined) {
|
||||
const stopped = await supervisor.stop();
|
||||
assert.notEqual(stopped.status, "incomplete", JSON.stringify(stopped));
|
||||
supervisor.dispose();
|
||||
}
|
||||
await rm(workspace, { recursive: true, force: true });
|
||||
}
|
||||
console.log(
|
||||
"Real Ghidra seed regression passed: descending function order, out-of-range seed, BSS label, BSS code rejection, and owned-process cleanup.",
|
||||
);
|
||||
@@ -0,0 +1,86 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import assert from "node:assert/strict";
|
||||
import { execFile } from "node:child_process";
|
||||
import { createServer } from "node:http";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
const execute = promisify(execFile);
|
||||
assert.equal(process.platform, "android", "Run this verifier inside Termux");
|
||||
assert.equal(process.env.PLAYWRIGHT_BROWSERS_PATH, undefined);
|
||||
// Import no browser provider here: the public dispatcher must establish its
|
||||
// Android cache fallback before Playwright initializes in each subprocess.
|
||||
const { stdout: doctorOutput } = await execute(
|
||||
process.execPath,
|
||||
["scripts/rea.mjs", "mcp", "doctor", "--format", "json"],
|
||||
{ timeout: 120_000, maxBuffer: 8 * 1024 * 1024 },
|
||||
);
|
||||
assert.equal(JSON.parse(doctorOutput).healthy, true, doctorOutput);
|
||||
const executable = `${process.env.PREFIX}/bin/chromium`;
|
||||
const { stdout: chromiumVersion } = await execute(executable, ["--version"], {
|
||||
timeout: 30_000,
|
||||
});
|
||||
const marker = "rea-termux-browser-smoke";
|
||||
const server = createServer((_request, response) => {
|
||||
response.writeHead(200, { "Content-Type": "text/html; charset=utf-8" });
|
||||
response.end(`<!doctype html><html><body><h1>${marker}</h1></body></html>`);
|
||||
});
|
||||
await new Promise((resolve, reject) => {
|
||||
server.once("error", reject);
|
||||
server.listen(0, "127.0.0.1", resolve);
|
||||
});
|
||||
try {
|
||||
const url = `http://127.0.0.1:${server.address().port}/smoke`;
|
||||
const scenario = {
|
||||
browser: { mode: "launch", executable_path: executable },
|
||||
start_url: { url },
|
||||
actions: [
|
||||
{
|
||||
step_id: "ready",
|
||||
action: "wait_for",
|
||||
locator: { kind: "css", selector: "h1" },
|
||||
state: "visible",
|
||||
},
|
||||
],
|
||||
capture: { at_end: ["url", "dom"] },
|
||||
};
|
||||
const { stdout } = await execute(
|
||||
process.execPath,
|
||||
[
|
||||
"scripts/rea.mjs",
|
||||
"capture-browser-scenario",
|
||||
JSON.stringify(scenario),
|
||||
"--json",
|
||||
],
|
||||
{ timeout: 180_000, maxBuffer: 16 * 1024 * 1024 },
|
||||
);
|
||||
const evidence = JSON.parse(stdout);
|
||||
assert.equal(evidence.error, undefined, stdout);
|
||||
const capture = evidence.normalized_result;
|
||||
assert.ok(
|
||||
capture.steps.every((step) => step.status === "completed"),
|
||||
stdout,
|
||||
);
|
||||
const artifacts = capture.steps.at(-1).artifacts;
|
||||
assert.equal(artifacts.url.state, "captured", stdout);
|
||||
assert.equal(artifacts.url.value.url, url);
|
||||
assert.equal(artifacts.dom.state, "captured", stdout);
|
||||
assert.ok(artifacts.dom.value.text.includes(`<h1>${marker}</h1>`), stdout);
|
||||
assert.equal(capture.browser.cleanup, "terminated-owned-process");
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
platform: process.platform,
|
||||
architecture: process.arch,
|
||||
node: process.version,
|
||||
chromium: chromiumVersion.trim(),
|
||||
mcp_doctor: "healthy",
|
||||
url_capture: "passed",
|
||||
dom_capture: "passed",
|
||||
cleanup: capture.browser.cleanup,
|
||||
}),
|
||||
);
|
||||
} finally {
|
||||
await new Promise((resolve, reject) =>
|
||||
server.close((cause) => (cause ? reject(cause) : resolve())),
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,67 @@
|
||||
#!/usr/bin/env bash
|
||||
# Host-side orchestration. Root is used only to stage/read files and send the
|
||||
# intent; TermuxService launches all package/build/browser work as the app UID.
|
||||
set -euo pipefail
|
||||
|
||||
staging="${RUNNER_TEMP:?}/rea-termux"
|
||||
app_home=/data/data/com.termux/files/home
|
||||
prefix=/data/data/com.termux/files/usr
|
||||
mkdir -p "$staging/diagnostics"
|
||||
collect_diagnostics() {
|
||||
adb logcat -d > "$staging/diagnostics/logcat.txt" 2>&1 || true
|
||||
adb exec-out cat "$app_home/rea-ci/run.log" > "$staging/diagnostics/termux.log" 2>&1 || true
|
||||
adb exec-out cat "$app_home/rea-ci/status" > "$staging/diagnostics/status.txt" 2>&1 || true
|
||||
}
|
||||
trap collect_diagnostics EXIT
|
||||
|
||||
adb root
|
||||
adb wait-for-device
|
||||
adb install "$staging/termux-app_v0.118.3+github-debug_x86_64.apk"
|
||||
adb shell am start -n com.termux/.app.TermuxActivity
|
||||
# First launch unpacks the APK's bundled bootstrap. Do not replace it with a
|
||||
# root-installed userland: that would miss Termux's own initialization.
|
||||
deadline=$((SECONDS + 180))
|
||||
until adb shell test -x "$prefix/bin/bash"; do
|
||||
if (( SECONDS >= deadline )); then
|
||||
echo 'Termux bootstrap did not finish within 180 seconds' >&2
|
||||
exit 1
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
|
||||
git archive --format=tar HEAD > "$staging/source.tar"
|
||||
adb push "$staging/source.tar" /data/local/tmp/rea-source.tar
|
||||
adb push scripts/verify/termux/in-app.sh /data/local/tmp/rea-in-app.sh
|
||||
adb shell chmod 644 /data/local/tmp/rea-source.tar /data/local/tmp/rea-in-app.sh
|
||||
app_uid=$(adb shell stat -c %u /data/data/com.termux | tr -d '\r')
|
||||
[[ "$app_uid" =~ ^[0-9]+$ && "$app_uid" != 0 ]]
|
||||
adb shell mkdir -p "$app_home/rea-ci" "$app_home/.termux"
|
||||
adb shell "echo allow-external-apps=true > $app_home/.termux/termux.properties"
|
||||
adb shell cp /data/local/tmp/rea-in-app.sh "$app_home/rea-ci/in-app.sh"
|
||||
adb shell cp /data/local/tmp/rea-source.tar "$app_home/rea-ci/source.tar"
|
||||
adb shell chown -R "$app_uid:$app_uid" "$app_home/rea-ci" "$app_home/.termux"
|
||||
adb shell restorecon -RF "$app_home/rea-ci" "$app_home/.termux"
|
||||
# Reload the external-command policy in a fresh app process before sending it.
|
||||
adb shell am force-stop com.termux
|
||||
adb shell am start -n com.termux/.app.TermuxActivity
|
||||
# Root may send this permission-protected intent; the service executes as the
|
||||
# application, rather than running Node/Chromium through adb shell or su.
|
||||
adb shell am startservice \
|
||||
-n com.termux/.app.RunCommandService \
|
||||
-a com.termux.RUN_COMMAND \
|
||||
--es com.termux.RUN_COMMAND_PATH "$prefix/bin/bash" \
|
||||
--esa com.termux.RUN_COMMAND_ARGUMENTS "-l,$app_home/rea-ci/in-app.sh" \
|
||||
--es com.termux.RUN_COMMAND_WORKDIR "$app_home" \
|
||||
--ez com.termux.RUN_COMMAND_BACKGROUND true
|
||||
|
||||
deadline=$((SECONDS + 2100))
|
||||
until adb shell test -f "$app_home/rea-ci/status"; do
|
||||
if (( SECONDS >= deadline )); then
|
||||
echo 'Termux verification exceeded 35 minutes' >&2
|
||||
exit 1
|
||||
fi
|
||||
sleep 5
|
||||
done
|
||||
adb exec-out cat "$app_home/rea-ci/run.log"
|
||||
status=$(adb shell cat "$app_home/rea-ci/status" | tr -d '\r')
|
||||
[[ "$status" == 0 ]]
|
||||
@@ -0,0 +1,33 @@
|
||||
#!/data/data/com.termux/files/usr/bin/bash
|
||||
set -euo pipefail
|
||||
|
||||
report="$HOME/rea-ci"
|
||||
exec > "$report/run.log" 2>&1
|
||||
# Publish the result only after all commands have settled. The host reads it as
|
||||
# root; no shared-storage permission or helper Android application is needed.
|
||||
trap 'result=$?; printf "%s\n" "$result" > "$report/status"' EXIT
|
||||
set -x
|
||||
[[ "$(id -u)" != 0 ]]
|
||||
export DEBIAN_FRONTEND=noninteractive
|
||||
export HUSKY=0
|
||||
pkg update -y
|
||||
pkg upgrade -y
|
||||
pkg install -y nodejs-lts npm git x11-repo termux-exec
|
||||
pkg install -y chromium
|
||||
mkdir -p "$report/source"
|
||||
tar -xf "$report/source.tar" -C "$report/source"
|
||||
cd "$report/source"
|
||||
node -e 'if (process.platform !== "android") throw new Error("Expected Android Node.js")'
|
||||
node -e 'const expected = require("node:fs").readFileSync(".nvmrc", "utf8").trim().replace(/^v/, ""); if (process.versions.node !== expected) throw new Error(`Termux Node ${process.versions.node} differs from .nvmrc ${expected}`)'
|
||||
# Android uses Termux's Bionic-linked Node distribution; the desktop .nvmrc
|
||||
# binary cannot run here. Keep npm at the repository's exact pinned version.
|
||||
npm install --prefix "$report/toolchain" --no-package-lock --ignore-scripts \
|
||||
"$(node -p 'JSON.parse(require("node:fs").readFileSync("package.json", "utf8")).packageManager')"
|
||||
export PATH="$report/toolchain/node_modules/.bin:$PATH"
|
||||
node --version
|
||||
npm --version
|
||||
dpkg-query -W nodejs-lts chromium termux-exec
|
||||
npm ci
|
||||
npm run build:termux
|
||||
unset PLAYWRIGHT_BROWSERS_PATH npm_config_playwright_browsers_path npm_package_config_playwright_browsers_path
|
||||
node scripts/verify/termux/browser.mjs
|
||||
@@ -22,7 +22,7 @@ describe("doctor", () => {
|
||||
),
|
||||
).toEqual({
|
||||
node: `Install ${SUPPORTED_NODE_VERSION_PROSE}.`,
|
||||
host: "REA supports macOS 12+, Ubuntu 24.04+, Fedora 41+, Nobara 44+, 64-bit Arch Linux, CachyOS, and the experimental Windows x64 Ghidra P0 boundary.",
|
||||
host: "REA supports macOS 12+, Ubuntu 24.04+, Fedora 41+, Nobara 44+, 64-bit Arch Linux, CachyOS, Omarchy, and the experimental Windows x64 Ghidra P0 boundary.",
|
||||
hopper: "Run rea setup to install Hopper, or set HOPPER_LAUNCHER_PATH.",
|
||||
target: "Supply a readable local app or program path.",
|
||||
});
|
||||
|
||||
@@ -113,7 +113,7 @@ const hostCheck = async (host: DoctorHost): Promise<DoctorCheck> => {
|
||||
: `${linuxDistribution.id} ${linuxDistribution.versionId ?? "unknown"}`);
|
||||
return check("host", supported, detail, {
|
||||
remediation:
|
||||
"REA supports macOS 12+, Ubuntu 24.04+, Fedora 41+, Nobara 44+, 64-bit Arch Linux, CachyOS, and the experimental Windows x64 Ghidra P0 boundary.",
|
||||
"REA supports macOS 12+, Ubuntu 24.04+, Fedora 41+, Nobara 44+, 64-bit Arch Linux, CachyOS, Omarchy, and the experimental Windows x64 Ghidra P0 boundary.",
|
||||
classification: "unsupported_host",
|
||||
});
|
||||
};
|
||||
|
||||
@@ -60,12 +60,16 @@ describe("Linux Hopper host classification", () => {
|
||||
['ID=nobara\nVERSION_ID="44"\nID_LIKE="rhel centos fedora"\n', "rpm"],
|
||||
["ID=arch\n", "arch"],
|
||||
["ID=cachyos\nID_LIKE=arch\n", "arch"],
|
||||
] as const)("accepts an official Hopper distribution", (document, family) => {
|
||||
['ID=omarchy\nID_LIKE=arch\nVERSION_ID="4.0.4"\n', "arch"],
|
||||
] as const)(
|
||||
"accepts an explicitly supported distribution",
|
||||
(document, family) => {
|
||||
expect(parseLinuxDistribution(document)).toMatchObject({
|
||||
packageFamily: family,
|
||||
supported: true,
|
||||
});
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("detects unresolved or loader-reported shared libraries", () => {
|
||||
expect(
|
||||
@@ -93,6 +97,8 @@ describe("Linux Hopper host classification", () => {
|
||||
'ID=debian\nVERSION_ID="13"\nID_LIKE=debian\n',
|
||||
"ID=manjaro\nID_LIKE=arch\n",
|
||||
"ID=garuda\nID_LIKE=arch\n",
|
||||
"ID=omarchy\n",
|
||||
"ID=omarchy\nID_LIKE=debian\n",
|
||||
])("rejects unsupported vendor/version combinations", (document) => {
|
||||
expect(parseLinuxDistribution(document).supported).toBe(false);
|
||||
});
|
||||
|
||||
@@ -154,6 +154,7 @@ export const parseLinuxDistribution = (text: string): LinuxDistribution => {
|
||||
packageFamily !== undefined &&
|
||||
(id === "arch" ||
|
||||
id === "cachyos" ||
|
||||
(id === "omarchy" && packageFamily === "arch") ||
|
||||
(id === "ubuntu" && versionAtLeast(versionId, 24)) ||
|
||||
(id === "fedora" && versionAtLeast(versionId, 41)) ||
|
||||
(id === "nobara" && versionAtLeast(versionId, 44)));
|
||||
|
||||
@@ -267,7 +267,7 @@ describe("setup scoped readiness", () => {
|
||||
expect(result.status).toBe("needs_human");
|
||||
expect(host.hopperInstalls).toBe(0);
|
||||
expect(result.remediation).toBe(
|
||||
"Automated Hopper setup supports Ubuntu 24.04+, Fedora 41+, Nobara 44+, 64-bit Arch Linux, and CachyOS; configure an existing supported provider instead.",
|
||||
"Automated Hopper setup supports Ubuntu 24.04+, Fedora 41+, Nobara 44+, 64-bit Arch Linux, CachyOS, and Omarchy; configure an existing supported provider instead.",
|
||||
);
|
||||
});
|
||||
|
||||
|
||||
@@ -90,7 +90,7 @@ export const hostRemediation = async (
|
||||
: undefined;
|
||||
}
|
||||
if ((await host.linuxDistribution())?.supported === true) return undefined;
|
||||
return "Automated Hopper setup supports Ubuntu 24.04+, Fedora 41+, Nobara 44+, 64-bit Arch Linux, and CachyOS; configure an existing supported provider instead.";
|
||||
return "Automated Hopper setup supports Ubuntu 24.04+, Fedora 41+, Nobara 44+, 64-bit Arch Linux, CachyOS, and Omarchy; configure an existing supported provider instead.";
|
||||
};
|
||||
|
||||
/** Production setup effects for Hopper, agent configuration, and the canonical skill directory. */
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { ELECTRON_IDENTITY_LIMITATION } from "../../domain/javascript/javascriptElectronMemberWrites.js";
|
||||
import type { ArtifactInventorySnapshot } from "../../domain/artifactInventorySnapshot.js";
|
||||
import {
|
||||
createJavaScriptApplicationGraph,
|
||||
@@ -255,6 +256,11 @@ const graphLimitations = (
|
||||
"CommonJS and ESM binding relationships were recovered from inert syntax and resolved only within the inventoried artifact container.",
|
||||
"Webpack/Rspack factories were recovered from AST literals; REA did not invoke push handlers or bundle bootstrap code.",
|
||||
"Static imports, entrypoints, workers, endpoints, and storage relationships do not prove runtime execution.",
|
||||
...(context.analysis.files.some(({ javascript }) =>
|
||||
javascript?.limitations.includes(ELECTRON_IDENTITY_LIMITATION),
|
||||
)
|
||||
? [ELECTRON_IDENTITY_LIMITATION]
|
||||
: []),
|
||||
...(electronFindings === 0
|
||||
? []
|
||||
: [
|
||||
|
||||
@@ -1015,6 +1015,7 @@ describe("contextual JavaScript module identity", () => {
|
||||
it("retains CommonJS and ESM file-base identity during inert extraction", () => {
|
||||
const analysis = analyzeJavaScriptStaticSource(
|
||||
`
|
||||
import { BrowserWindow } from "electron";
|
||||
new BrowserWindow({ webPreferences: {
|
||||
preload: path.resolve("ignored", __dirname, "preload.cjs")
|
||||
}});
|
||||
|
||||
@@ -229,6 +229,211 @@ describe("native value parameter-use node membership", () => {
|
||||
});
|
||||
});
|
||||
|
||||
const nativeFlowDossier = (
|
||||
base: ReturnType<typeof functionDossierSchema.parse>,
|
||||
facts: {
|
||||
address: string;
|
||||
name: string;
|
||||
operations: ReturnType<typeof node>[];
|
||||
parameters: { ordinal: number; name: string; data_type: string }[];
|
||||
},
|
||||
) => {
|
||||
const { address, name, operations, parameters } = facts;
|
||||
return jsonValueSchema.parse({
|
||||
...base,
|
||||
procedure: {
|
||||
address,
|
||||
name,
|
||||
classification: null,
|
||||
body: {
|
||||
available: false,
|
||||
reason: "The provider did not report complete function body ranges.",
|
||||
},
|
||||
signature: null,
|
||||
locals: [],
|
||||
},
|
||||
native_value_flow: {
|
||||
available: true,
|
||||
provenance: "ghidra-high-pcode",
|
||||
operations,
|
||||
def_use: [],
|
||||
effects: [],
|
||||
parameters,
|
||||
parameter_uses: [],
|
||||
truncated: false,
|
||||
omitted_operations_lower_bound: 0,
|
||||
known_omitted_inputs: 0,
|
||||
known_omitted_edges: 0,
|
||||
limitations: [],
|
||||
},
|
||||
});
|
||||
};
|
||||
|
||||
const callBindingAnalysis = (
|
||||
callerAddress: string,
|
||||
calleeAddress: string,
|
||||
caller: ReturnType<typeof jsonValueSchema.parse>,
|
||||
callee: ReturnType<typeof jsonValueSchema.parse>,
|
||||
): AnalysisOperationPort => ({
|
||||
execute: async (operation, parameters) => {
|
||||
if (operation === "resolve_native_call_targets")
|
||||
return ok(
|
||||
createAnalysisExecution(
|
||||
{
|
||||
call_site: String(parameters.address),
|
||||
procedure: calleeAddress,
|
||||
status: "direct",
|
||||
mechanism: "direct",
|
||||
targets: [
|
||||
{
|
||||
address: calleeAddress,
|
||||
procedure: calleeAddress,
|
||||
status: "direct",
|
||||
basis: "provider-reference",
|
||||
references: [],
|
||||
},
|
||||
],
|
||||
limitations: [],
|
||||
},
|
||||
provider,
|
||||
{ subject },
|
||||
),
|
||||
);
|
||||
if (operation !== "analyze_function")
|
||||
throw new Error(`Unexpected native trace operation: ${operation}`);
|
||||
if (
|
||||
parameters.procedure !== callerAddress &&
|
||||
parameters.procedure !== calleeAddress
|
||||
)
|
||||
throw new Error(`Unexpected procedure: ${String(parameters.procedure)}`);
|
||||
return ok(
|
||||
createAnalysisExecution(
|
||||
parameters.procedure === callerAddress ? caller : callee,
|
||||
provider,
|
||||
{ subject },
|
||||
),
|
||||
);
|
||||
},
|
||||
});
|
||||
|
||||
describe("native value call-binding fan-out", () => {
|
||||
it("preserves binding order and edge-budget behavior for repeated callees", async () => {
|
||||
const callCount = 128;
|
||||
const callerAddress = "0x1000";
|
||||
const calleeAddress = "0x2000";
|
||||
const base = functionDossierSchema.parse(ghidraFunctionDossier());
|
||||
const caller = nativeFlowDossier(base, {
|
||||
address: callerAddress,
|
||||
name: "caller",
|
||||
operations: Array.from({ length: callCount }, (_, index) => ({
|
||||
...node(
|
||||
`call-${index}`,
|
||||
`0x${(0x1000 + index * 4).toString(16)}`,
|
||||
"CALL",
|
||||
),
|
||||
sequence: index,
|
||||
inputs: [scalar, scalar],
|
||||
})),
|
||||
parameters: [],
|
||||
});
|
||||
const callee = nativeFlowDossier(base, {
|
||||
address: calleeAddress,
|
||||
name: "callee",
|
||||
operations: [node("return", calleeAddress, "RETURN")],
|
||||
parameters: [{ ordinal: 0, name: "value", data_type: "/int" }],
|
||||
});
|
||||
const fanoutAnalysis = callBindingAnalysis(
|
||||
callerAddress,
|
||||
calleeAddress,
|
||||
caller,
|
||||
callee,
|
||||
);
|
||||
const expectedEdges = [
|
||||
...Array.from({ length: callCount }, (_, index) => ({
|
||||
source: `${callerAddress}/call-${index}`,
|
||||
target: calleeAddress,
|
||||
input_index: null,
|
||||
kind: "call",
|
||||
})),
|
||||
...Array.from({ length: callCount }, (_, index) => [
|
||||
{
|
||||
source: `${callerAddress}/call-${index}`,
|
||||
target: `${calleeAddress}/parameter:0`,
|
||||
input_index: 1,
|
||||
kind: "argument-binding",
|
||||
},
|
||||
{
|
||||
source: `${calleeAddress}/return`,
|
||||
target: `${callerAddress}/call-${index}`,
|
||||
input_index: 1,
|
||||
kind: "return-binding",
|
||||
},
|
||||
]).flat(),
|
||||
];
|
||||
const request = {
|
||||
procedure: callerAddress,
|
||||
max_functions: 2,
|
||||
max_call_sites: callCount,
|
||||
max_nodes: callCount + 2,
|
||||
max_edges: callCount * 3,
|
||||
limit: callCount + 2,
|
||||
};
|
||||
|
||||
const complete = await traceNativeValues(fanoutAnalysis, request);
|
||||
if (!complete.ok) throw complete.error;
|
||||
const completeResult = nativeValueTraceSchema.parse(complete.value);
|
||||
const edgeShape = (edges: typeof completeResult.edges) =>
|
||||
edges.map(({ source, target, input_index, kind }) => ({
|
||||
source,
|
||||
target,
|
||||
input_index,
|
||||
kind,
|
||||
}));
|
||||
expect(completeResult).toMatchObject({
|
||||
total_nodes: callCount + 2,
|
||||
total_edges: callCount * 3,
|
||||
truncated: false,
|
||||
unknowns: [],
|
||||
});
|
||||
expect(edgeShape(completeResult.edges)).toEqual(expectedEdges);
|
||||
|
||||
const capped = await traceNativeValues(fanoutAnalysis, {
|
||||
...request,
|
||||
max_edges: callCount * 3 - 1,
|
||||
});
|
||||
if (!capped.ok) throw capped.error;
|
||||
const cappedResult = nativeValueTraceSchema.parse(capped.value);
|
||||
expect(cappedResult).toMatchObject({
|
||||
total_nodes: callCount + 2,
|
||||
total_edges: callCount * 3 - 1,
|
||||
truncated: true,
|
||||
});
|
||||
expect(edgeShape(cappedResult.edges)).toEqual(
|
||||
expectedEdges.slice(0, callCount * 3 - 1),
|
||||
);
|
||||
|
||||
const nodeCapped = await traceNativeValues(fanoutAnalysis, {
|
||||
...request,
|
||||
max_nodes: callCount + 1,
|
||||
});
|
||||
if (!nodeCapped.ok) throw nodeCapped.error;
|
||||
const nodeCappedResult = nativeValueTraceSchema.parse(nodeCapped.value);
|
||||
expect(nodeCappedResult).toMatchObject({
|
||||
total_nodes: callCount + 1,
|
||||
total_edges: callCount * 2,
|
||||
truncated: true,
|
||||
unknowns: Array.from({ length: callCount }, () => ({
|
||||
procedure: callerAddress,
|
||||
address: null,
|
||||
reason: `Parameter bindings for ${calleeAddress} are unavailable or omitted`,
|
||||
})),
|
||||
});
|
||||
expect(edgeShape(nodeCappedResult.edges)).toEqual(
|
||||
expectedEdges.filter(({ kind }) => kind !== "argument-binding"),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("bounded native value dependency composition", () => {
|
||||
it("reports work truncation and stable pagination without traversing ambiguous calls", async () => {
|
||||
const bounded = await traceNativeValues(analysis, {
|
||||
|
||||
@@ -305,10 +305,24 @@ export const traceNativeValues = async (
|
||||
"No seed function could be analyzed",
|
||||
),
|
||||
);
|
||||
const parametersByProcedure = new Map<string, typeof nodes>();
|
||||
const returnsByProcedure = new Map<string, typeof nodes>();
|
||||
for (const node of nodes) {
|
||||
if (node.kind === "parameter") {
|
||||
const parameters = parametersByProcedure.get(node.procedure) ?? [];
|
||||
parameters.push(node);
|
||||
parametersByProcedure.set(node.procedure, parameters);
|
||||
} else if (
|
||||
node.operation?.opcode === "RETURN" &&
|
||||
node.operation.inputs.length > 1
|
||||
) {
|
||||
const returns = returnsByProcedure.get(node.procedure) ?? [];
|
||||
returns.push(node);
|
||||
returnsByProcedure.set(node.procedure, returns);
|
||||
}
|
||||
}
|
||||
for (const binding of callBindings) {
|
||||
const parameters = nodes.filter(
|
||||
(node) => node.procedure === binding.callee && node.kind === "parameter",
|
||||
);
|
||||
const parameters = parametersByProcedure.get(binding.callee) ?? [];
|
||||
if (parameters.length === 0)
|
||||
unknowns.push({
|
||||
procedure: binding.caller,
|
||||
@@ -339,12 +353,7 @@ export const traceNativeValues = async (
|
||||
});
|
||||
}
|
||||
if (binding.output)
|
||||
for (const node of nodes.filter(
|
||||
(node) =>
|
||||
node.procedure === binding.callee &&
|
||||
node.operation?.opcode === "RETURN" &&
|
||||
node.operation.inputs.length > 1,
|
||||
)) {
|
||||
for (const node of returnsByProcedure.get(binding.callee) ?? []) {
|
||||
if (edges.length >= input.max_edges) {
|
||||
truncated = true;
|
||||
break;
|
||||
|
||||
@@ -54,6 +54,7 @@ export class AsarArtifactReader implements ArtifactReader {
|
||||
>();
|
||||
#snapshot: { readonly path: string; readonly sha256: string } | undefined;
|
||||
#snapshotRoot: string | undefined;
|
||||
#snapshotOperation: Promise<void> = Promise.resolve();
|
||||
|
||||
constructor(
|
||||
private readonly path: string,
|
||||
@@ -234,14 +235,59 @@ export class AsarArtifactReader implements ArtifactReader {
|
||||
expectedSha256?: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<void> {
|
||||
await this.#withSnapshotOperation(async () => {
|
||||
abortIfNeeded(signal);
|
||||
if (this.#snapshot === undefined && this.#snapshotRoot !== undefined)
|
||||
await this.close();
|
||||
if (this.#snapshot === undefined) {
|
||||
try {
|
||||
this.#snapshotRoot = await mkdtemp(
|
||||
join(tmpdir(), "rea-asar-snapshot-"),
|
||||
await this.#closeSnapshot();
|
||||
if (this.#snapshot === undefined) await this.#captureSnapshot(signal);
|
||||
if (
|
||||
expectedSha256 !== undefined &&
|
||||
this.#snapshot?.sha256 !== expectedSha256
|
||||
)
|
||||
throw new ArtifactReaderFailure(
|
||||
"integrity",
|
||||
`ASAR container changed before interpretation: ${this.path}`,
|
||||
);
|
||||
}, signal);
|
||||
}
|
||||
|
||||
/** Read the same captured container that supplies headers and packed members. */
|
||||
async openContainer(signal?: AbortSignal): Promise<ReadStream> {
|
||||
await this.prepareContainer(undefined, signal);
|
||||
return createReadStream(
|
||||
this.#snapshotPath(),
|
||||
signal === undefined ? {} : { signal },
|
||||
);
|
||||
}
|
||||
|
||||
async close(): Promise<void> {
|
||||
await this.#withSnapshotOperation(async () => {
|
||||
let cleanupFailure: ArtifactReaderFailure | undefined;
|
||||
for (const [handle, { path }] of this.#unpackedHandles) {
|
||||
try {
|
||||
await this.#closeUnpackedHandle(handle);
|
||||
} catch (cause: unknown) {
|
||||
cleanupFailure = ArtifactReaderFailure.withCleanup(
|
||||
cleanupFailure ?? cause,
|
||||
ArtifactReaderFailure.cleanupObservation(cause, path),
|
||||
);
|
||||
}
|
||||
}
|
||||
try {
|
||||
await this.#closeSnapshot();
|
||||
} catch (cause: unknown) {
|
||||
cleanupFailure = ArtifactReaderFailure.withCleanup(
|
||||
cleanupFailure ?? cause,
|
||||
ArtifactReaderFailure.cleanupObservation(cause, this.path),
|
||||
);
|
||||
}
|
||||
if (cleanupFailure !== undefined) throw cleanupFailure;
|
||||
});
|
||||
}
|
||||
|
||||
async #captureSnapshot(signal?: AbortSignal): Promise<void> {
|
||||
try {
|
||||
this.#snapshotRoot = await mkdtemp(join(tmpdir(), "rea-asar-snapshot-"));
|
||||
const snapshotPath = join(this.#snapshotRoot, "container.asar");
|
||||
const hash = createHash("sha256");
|
||||
const handle = await open(
|
||||
@@ -265,67 +311,36 @@ export class AsarArtifactReader implements ArtifactReader {
|
||||
} catch (cause: unknown) {
|
||||
const primary =
|
||||
signal?.aborted === true
|
||||
? new ArtifactReaderFailure(
|
||||
"cancelled",
|
||||
"ASAR operation cancelled",
|
||||
{ cause },
|
||||
)
|
||||
? new ArtifactReaderFailure("cancelled", "ASAR operation cancelled", {
|
||||
cause,
|
||||
})
|
||||
: asarFailure(this.path, "snapshot", cause);
|
||||
try {
|
||||
await this.close();
|
||||
await this.#closeSnapshot();
|
||||
} catch (cleanupCause: unknown) {
|
||||
throw ArtifactReaderFailure.withCleanup(
|
||||
primary,
|
||||
ArtifactReaderFailure.cleanupObservation(cleanupCause, this.path),
|
||||
ArtifactReaderFailure.cleanupObservation(
|
||||
cleanupCause,
|
||||
this.#snapshotRoot ?? this.path,
|
||||
),
|
||||
);
|
||||
}
|
||||
throw primary;
|
||||
}
|
||||
}
|
||||
if (
|
||||
expectedSha256 !== undefined &&
|
||||
this.#snapshot.sha256 !== expectedSha256
|
||||
)
|
||||
throw new ArtifactReaderFailure(
|
||||
"integrity",
|
||||
`ASAR container changed before interpretation: ${this.path}`,
|
||||
);
|
||||
}
|
||||
|
||||
/** Read the same captured container that supplies headers and packed members. */
|
||||
async openContainer(signal?: AbortSignal): Promise<ReadStream> {
|
||||
await this.prepareContainer(undefined, signal);
|
||||
return createReadStream(
|
||||
this.#snapshotPath(),
|
||||
signal === undefined ? {} : { signal },
|
||||
);
|
||||
}
|
||||
|
||||
async close(): Promise<void> {
|
||||
let cleanupFailure: ArtifactReaderFailure | undefined;
|
||||
for (const [handle, { path }] of this.#unpackedHandles) {
|
||||
try {
|
||||
await this.#closeUnpackedHandle(handle);
|
||||
} catch (cause: unknown) {
|
||||
cleanupFailure = ArtifactReaderFailure.withCleanup(
|
||||
cleanupFailure ?? cause,
|
||||
ArtifactReaderFailure.cleanupObservation(cause, path),
|
||||
);
|
||||
}
|
||||
}
|
||||
async #closeSnapshot(): Promise<void> {
|
||||
const root = this.#snapshotRoot;
|
||||
if (root !== undefined) {
|
||||
try {
|
||||
await this.removeSnapshot(root);
|
||||
if (this.#snapshotRoot === root) this.#snapshotRoot = undefined;
|
||||
} catch (cause: unknown) {
|
||||
cleanupFailure = ArtifactReaderFailure.withCleanup(
|
||||
cleanupFailure ?? cause,
|
||||
{
|
||||
throw ArtifactReaderFailure.withCleanup(cause, {
|
||||
reason: `Could not remove ASAR snapshot: ${root}`,
|
||||
resources: [root],
|
||||
},
|
||||
);
|
||||
});
|
||||
}
|
||||
}
|
||||
if (root === undefined || this.#snapshotRoot === undefined) {
|
||||
@@ -334,7 +349,29 @@ export class AsarArtifactReader implements ArtifactReader {
|
||||
this.#resetArchiveState();
|
||||
this.#entries.clear();
|
||||
}
|
||||
if (cleanupFailure !== undefined) throw cleanupFailure;
|
||||
}
|
||||
|
||||
async #withSnapshotOperation<Value>(
|
||||
operation: () => Promise<Value>,
|
||||
signal?: AbortSignal,
|
||||
): Promise<Value> {
|
||||
const previous = this.#snapshotOperation;
|
||||
let release!: () => void;
|
||||
this.#snapshotOperation = new Promise<void>((resolve) => {
|
||||
release = resolve;
|
||||
});
|
||||
try {
|
||||
await waitForOperation(previous, signal);
|
||||
} catch (cause: unknown) {
|
||||
void previous.then(release, release);
|
||||
throw cause;
|
||||
}
|
||||
try {
|
||||
abortIfNeeded(signal);
|
||||
return await operation();
|
||||
} finally {
|
||||
release();
|
||||
}
|
||||
}
|
||||
|
||||
#snapshotPath(): string {
|
||||
@@ -511,6 +548,35 @@ const abortIfNeeded = (signal?: AbortSignal): void => {
|
||||
throw new ArtifactReaderFailure("cancelled", "ASAR operation cancelled");
|
||||
};
|
||||
|
||||
const waitForOperation = async (
|
||||
operation: Promise<void>,
|
||||
signal?: AbortSignal,
|
||||
): Promise<void> => {
|
||||
if (signal === undefined) return operation;
|
||||
abortIfNeeded(signal);
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
const removeAbortListener = (): void =>
|
||||
signal.removeEventListener("abort", onAbort);
|
||||
const onAbort = (): void => {
|
||||
removeAbortListener();
|
||||
reject(
|
||||
new ArtifactReaderFailure("cancelled", "ASAR operation cancelled"),
|
||||
);
|
||||
};
|
||||
signal.addEventListener("abort", onAbort, { once: true });
|
||||
operation.then(
|
||||
() => {
|
||||
removeAbortListener();
|
||||
resolve();
|
||||
},
|
||||
(cause: unknown) => {
|
||||
removeAbortListener();
|
||||
reject(cause);
|
||||
},
|
||||
);
|
||||
});
|
||||
};
|
||||
|
||||
const toArtifactPath = (listed: string): string => {
|
||||
const portable = sep === "\\" ? listed.replaceAll("\\", "/") : listed;
|
||||
return portable.replace(/^\/+|\/+$/gu, "");
|
||||
|
||||
@@ -191,7 +191,7 @@ export class SafeOutputTree {
|
||||
"format",
|
||||
`Invalid expected extraction size for ${relativePath}`,
|
||||
);
|
||||
const path = normalizeArtifactPath(relativePath);
|
||||
const path = normalizeDestinationPath(relativePath, this.platform);
|
||||
this.#registry.add(path, "file");
|
||||
const lineage = await this.#prepareParent(path);
|
||||
const parent = lineage.at(-1);
|
||||
@@ -569,6 +569,19 @@ const writeAll = async (
|
||||
}
|
||||
};
|
||||
|
||||
const normalizeDestinationPath = (
|
||||
relativePath: string,
|
||||
platform: NodeJS.Platform,
|
||||
): string => {
|
||||
const path = normalizeArtifactPath(relativePath);
|
||||
if (platform === "win32" && path.includes(":"))
|
||||
throw new ArtifactReaderFailure(
|
||||
"path",
|
||||
`Windows destination cannot materialize artifact path as a regular file: ${path}; ':' denotes alternate data stream syntax.`,
|
||||
);
|
||||
return path;
|
||||
};
|
||||
|
||||
const abortIfNeeded = (signal?: AbortSignal): void => {
|
||||
if (signal?.aborted === true)
|
||||
throw new ArtifactReaderFailure(
|
||||
|
||||
@@ -9,7 +9,7 @@ import {
|
||||
} from "../../domain/propertyListKeys.js";
|
||||
import { decodeNibArchive, type NibArchiveDocument } from "./NibArchive.js";
|
||||
import {
|
||||
estimatePropertyListDecodeBytes,
|
||||
preflightPropertyListDecode,
|
||||
InterfaceBuilderDecodeBudget,
|
||||
InterfaceBuilderDecodeBudgetExceeded,
|
||||
} from "./InterfaceBuilderDecodeBudget.js";
|
||||
@@ -47,7 +47,7 @@ export const analyzeInterfaceBuilderBundle = async (input: {
|
||||
const omittedArchives: Array<{ path: string; reason: string }> = [];
|
||||
let unreportedOmittedArchives = 0;
|
||||
const incompleteHierarchies = new Map<string, number>();
|
||||
const prototypeKeyOmissions = new Map<string, number>();
|
||||
const prototypeKeyOmissions = new Map<string, number | null>();
|
||||
let omitted = 0;
|
||||
let attempted = 0;
|
||||
let aggregateInputBytes = 0;
|
||||
@@ -93,6 +93,7 @@ export const analyzeInterfaceBuilderBundle = async (input: {
|
||||
bytes.subarray(0, 10).toString("ascii") === "NIBArchive";
|
||||
let nib: ReturnType<typeof projectNibArchive> | null = null;
|
||||
let xmlText: string | undefined;
|
||||
let binaryPrototypeKeyObserved = false;
|
||||
if (isNibArchive) {
|
||||
const documentBudget = new InterfaceBuilderDecodeBudget(
|
||||
decodeBudget.remainingBytes,
|
||||
@@ -109,23 +110,25 @@ export const analyzeInterfaceBuilderBundle = async (input: {
|
||||
} else {
|
||||
if (bytes.subarray(0, 8).toString("ascii") !== "bplist00")
|
||||
xmlText = decodeXmlPlistText(bytes);
|
||||
const decodedReservation = estimatePropertyListDecodeBytes(
|
||||
const decodedReservation = preflightPropertyListDecode(
|
||||
bytes,
|
||||
decodeBudget.remainingBytes,
|
||||
xmlText,
|
||||
);
|
||||
decodeBudget.reserve(
|
||||
decodedReservation,
|
||||
decodedReservation.estimatedBytes,
|
||||
"plist representation exceeds the aggregate Interface Builder decode budget",
|
||||
);
|
||||
binaryPrototypeKeyObserved =
|
||||
decodedReservation.binaryPrototypeKeyObserved;
|
||||
}
|
||||
const { value: raw, omittedPrototypeKeys } =
|
||||
nib === null
|
||||
? decodePlist(bytes, xmlText)
|
||||
? decodePlist(bytes, xmlText, binaryPrototypeKeyObserved)
|
||||
: { value: nib.raw, omittedPrototypeKeys: 0 };
|
||||
if (nib !== null && nib.omitted > 0)
|
||||
incompleteHierarchies.set(entry.path, nib.omitted);
|
||||
if (omittedPrototypeKeys > 0)
|
||||
if (omittedPrototypeKeys !== 0)
|
||||
prototypeKeyOmissions.set(entry.path, omittedPrototypeKeys);
|
||||
const documentHash = createHash("sha256").update(bytes).digest("hex");
|
||||
documents.push({
|
||||
@@ -226,7 +229,7 @@ const finalizeHierarchyCoverage = (
|
||||
result: ReturnType<typeof buildInterfaceBuilderAnalysis>,
|
||||
counts: {
|
||||
incompleteHierarchies: ReadonlyMap<string, number>;
|
||||
prototypeKeyOmissions: ReadonlyMap<string, number>;
|
||||
prototypeKeyOmissions: ReadonlyMap<string, number | null>;
|
||||
invalid: readonly string[];
|
||||
omittedArchives: readonly { path: string; reason: string }[];
|
||||
unreportedOmittedArchives: number;
|
||||
@@ -641,11 +644,18 @@ const readEntry = async (
|
||||
/** Project decoded plist data and dates while retaining omitted-key coverage. */
|
||||
const decodePlist = (
|
||||
bytes: Buffer,
|
||||
xmlText?: string,
|
||||
): { readonly value: JsonValue; readonly omittedPrototypeKeys: number } => {
|
||||
const { value, omittedPrototypeKeys } =
|
||||
bytes.subarray(0, 8).toString("ascii") === "bplist00"
|
||||
? { value: parseBinary(bytes), omittedPrototypeKeys: 0 }
|
||||
xmlText: string | undefined,
|
||||
binaryPrototypeKeyObserved: boolean,
|
||||
): {
|
||||
readonly value: JsonValue;
|
||||
readonly omittedPrototypeKeys: number | null;
|
||||
} => {
|
||||
const binary = bytes.subarray(0, 8).toString("ascii") === "bplist00";
|
||||
const { value, omittedPrototypeKeys } = binary
|
||||
? {
|
||||
value: parseBinary(bytes),
|
||||
omittedPrototypeKeys: binaryPrototypeKeyObserved ? null : 0,
|
||||
}
|
||||
: parseXmlPropertyList(xmlText ?? decodeXmlPlistText(bytes));
|
||||
return { value: projectPlistValue(value).value, omittedPrototypeKeys };
|
||||
};
|
||||
|
||||
@@ -89,21 +89,24 @@ export class InterfaceBuilderDecodeBudget {
|
||||
}
|
||||
|
||||
/** Preflight the representation created by the pinned plist decoder. */
|
||||
export const estimatePropertyListDecodeBytes = (
|
||||
export const preflightPropertyListDecode = (
|
||||
bytes: Buffer,
|
||||
maxBytes: number,
|
||||
xmlText?: string,
|
||||
): number => {
|
||||
const estimate =
|
||||
): PlistDecodePreflight => {
|
||||
const expansion =
|
||||
bytes.subarray(0, 8).toString("ascii") === "bplist00"
|
||||
? estimateBinaryPlistExpansion(bytes, maxBytes)
|
||||
: estimateXmlPlistExpansion(bytes, maxBytes, xmlText);
|
||||
if (estimate > maxBytes)
|
||||
: {
|
||||
estimatedBytes: estimateXmlPlistExpansion(bytes, maxBytes, xmlText),
|
||||
binaryPrototypeKeyObserved: false,
|
||||
};
|
||||
if (expansion.estimatedBytes > maxBytes)
|
||||
throw new InterfaceBuilderDecodeBudgetExceeded(
|
||||
"aggregate_decode_budget_exhausted",
|
||||
"plist representation exceeds the aggregate Interface Builder decode budget",
|
||||
);
|
||||
return estimate;
|
||||
return expansion;
|
||||
};
|
||||
|
||||
const estimateXmlPlistExpansion = (
|
||||
@@ -173,6 +176,14 @@ export interface BinaryPlistExpansionOptions {
|
||||
readonly bounds?: "strict" | "decoder";
|
||||
}
|
||||
|
||||
/** Preflight size and binary-key observations; XML keys are inspected by decode. */
|
||||
export interface PlistDecodePreflight {
|
||||
readonly estimatedBytes: number;
|
||||
readonly binaryPrototypeKeyObserved: boolean;
|
||||
}
|
||||
|
||||
const PROTOTYPE_KEY = "__proto__";
|
||||
|
||||
/**
|
||||
* Count every expanded binary-plist reference before calling plist.parseBinary,
|
||||
* which recurses on each reference and copies shared containers. Malformed
|
||||
@@ -186,7 +197,7 @@ export const estimateBinaryPlistExpansion = (
|
||||
fail = interfaceBuilderBudgetFailure,
|
||||
bounds = "strict",
|
||||
}: BinaryPlistExpansionOptions = {},
|
||||
): number => {
|
||||
): PlistDecodePreflight => {
|
||||
if (bytes.length < 40)
|
||||
throw new TypeError("binary plist trailer is truncated");
|
||||
const trailer = bytes.length - 32;
|
||||
@@ -218,13 +229,17 @@ export const estimateBinaryPlistExpansion = (
|
||||
)
|
||||
throw new TypeError("binary plist trailer or object table is invalid");
|
||||
let estimate = bytes.length * 2 + objectCount * 16;
|
||||
let binaryPrototypeKeyObserved = false;
|
||||
if (estimate > maxBytes)
|
||||
throw fail("expansion", `binary plist object table exceeds ${budget}`);
|
||||
|
||||
const active = new Set<number>();
|
||||
const pending: Array<{ object: number; depth: number; exit: boolean }> = [
|
||||
{ object: topObject, depth: 0, exit: false },
|
||||
];
|
||||
const pending: Array<{
|
||||
object: number;
|
||||
depth: number;
|
||||
exit: boolean;
|
||||
dictionaryKey: boolean;
|
||||
}> = [{ object: topObject, depth: 0, exit: false, dictionaryKey: false }];
|
||||
let pendingObjectCount = 1;
|
||||
while (pending.length > 0) {
|
||||
const entry = pending.pop();
|
||||
@@ -269,6 +284,26 @@ export const estimateBinaryPlistExpansion = (
|
||||
const byteLength = type === 6 ? size * 2 : size;
|
||||
if (cursor + byteLength > objectsEnd)
|
||||
throw new TypeError("binary plist scalar object is truncated");
|
||||
if (
|
||||
!binaryPrototypeKeyObserved &&
|
||||
entry.dictionaryKey &&
|
||||
(type === 5 || type === 6) &&
|
||||
size === PROTOTYPE_KEY.length
|
||||
) {
|
||||
let matches = true;
|
||||
for (let index = 0; index < PROTOTYPE_KEY.length; index += 1) {
|
||||
const codeUnit =
|
||||
type === 5
|
||||
? (bytes[cursor + index] ?? 0)
|
||||
: ((bytes[cursor + index * 2] ?? 0) << 8) |
|
||||
(bytes[cursor + index * 2 + 1] ?? 0);
|
||||
if (codeUnit !== PROTOTYPE_KEY.charCodeAt(index)) {
|
||||
matches = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (matches) binaryPrototypeKeyObserved = true;
|
||||
}
|
||||
estimate += type === 4 ? size + 4 * Math.ceil(size / 3) : byteLength * 2;
|
||||
if (estimate > maxBytes)
|
||||
throw fail(
|
||||
@@ -289,7 +324,12 @@ export const estimateBinaryPlistExpansion = (
|
||||
`binary plist reference expansion exceeds ${budget}`,
|
||||
);
|
||||
active.add(entry.object);
|
||||
pending.push({ object: entry.object, depth: entry.depth, exit: true });
|
||||
pending.push({
|
||||
object: entry.object,
|
||||
depth: entry.depth,
|
||||
exit: true,
|
||||
dictionaryKey: false,
|
||||
});
|
||||
for (let index = childCount - 1; index >= 0; index -= 1) {
|
||||
const reference = readInteger(
|
||||
cursor + index * referenceSize,
|
||||
@@ -297,9 +337,14 @@ export const estimateBinaryPlistExpansion = (
|
||||
);
|
||||
if (reference < 0 || reference >= objectCount)
|
||||
throw new TypeError("binary plist reference is invalid");
|
||||
pending.push({ object: reference, depth: entry.depth + 1, exit: false });
|
||||
pending.push({
|
||||
object: reference,
|
||||
depth: entry.depth + 1,
|
||||
exit: false,
|
||||
dictionaryKey: type === 13 && index < size,
|
||||
});
|
||||
}
|
||||
pendingObjectCount += childCount;
|
||||
}
|
||||
return estimate;
|
||||
return { estimatedBytes: estimate, binaryPrototypeKeyObserved };
|
||||
};
|
||||
|
||||
@@ -65,6 +65,40 @@ describe("inert keyed archive decoding", () => {
|
||||
"1 dictionary entry keyed __proto__ was omitted because REA results cannot represent that key.",
|
||||
);
|
||||
});
|
||||
it("reports a binary archive dictionary keyed __proto__ as omitted", () => {
|
||||
const placeholder = "proto_key";
|
||||
const encoded = Buffer.from(
|
||||
buildBinary({
|
||||
...archive,
|
||||
[placeholder]: "hidden",
|
||||
$objects: ["$null", "value"],
|
||||
}),
|
||||
);
|
||||
const start = encoded.indexOf(placeholder);
|
||||
expect(start).toBeGreaterThanOrEqual(0);
|
||||
expect(encoded.indexOf(placeholder, start + placeholder.length)).toBe(-1);
|
||||
Buffer.from("__proto__").copy(encoded, start);
|
||||
|
||||
const graph = decodeKeyedArchiveBytes(encoded, { offset: 0, limit: 2 });
|
||||
expect(graph.limitations).toContain(
|
||||
"Dictionary entries keyed __proto__ were omitted because REA results cannot represent that key.",
|
||||
);
|
||||
});
|
||||
it("does not report a binary __proto__ value as an omitted key", () => {
|
||||
const graph = decodeKeyedArchiveBytes(
|
||||
Buffer.from(
|
||||
buildBinary({
|
||||
...archive,
|
||||
note: "__proto__",
|
||||
$objects: ["$null", "value"],
|
||||
}),
|
||||
),
|
||||
{ offset: 0, limit: 2 },
|
||||
);
|
||||
expect(graph.limitations).not.toContain(
|
||||
"Dictionary entries keyed __proto__ were omitted because REA results cannot represent that key.",
|
||||
);
|
||||
});
|
||||
it("rejects missing roots, malformed plist, and non-keyed archives", () => {
|
||||
expect(() =>
|
||||
decodeKeyedArchiveBytes(Buffer.from("bplist00bad"), {
|
||||
@@ -133,6 +167,20 @@ const plistArray = (references: readonly number[]): Buffer => {
|
||||
};
|
||||
const plistString = (value: string): Buffer =>
|
||||
Buffer.concat([Buffer.from([0x50 | value.length]), Buffer.from(value)]);
|
||||
const plistUtf16String = (value: string): Buffer => {
|
||||
const bytes = Buffer.alloc(1 + value.length * 2);
|
||||
bytes[0] = 0x60 | value.length;
|
||||
for (let index = 0; index < value.length; index += 1)
|
||||
bytes.writeUInt16BE(value.charCodeAt(index), 1 + index * 2);
|
||||
return bytes;
|
||||
};
|
||||
const plistExtendedUtf16String = (value: string): Buffer => {
|
||||
const bytes = Buffer.alloc(3 + value.length * 2);
|
||||
bytes.set([0x6f, 0x10, value.length]);
|
||||
for (let index = 0; index < value.length; index += 1)
|
||||
bytes.writeUInt16BE(value.charCodeAt(index), 3 + index * 2);
|
||||
return bytes;
|
||||
};
|
||||
// {"$archiver": "NSKeyedArchiver", "$top": {"root": UID 1}, "$objects": object 8}
|
||||
const keyedRoot = (payload: readonly Buffer[]): Buffer =>
|
||||
binaryPlist([
|
||||
@@ -187,4 +235,91 @@ describe("binary plist reference preflight", () => {
|
||||
[2, "$null"],
|
||||
]);
|
||||
});
|
||||
|
||||
it("recognizes a shared key object used by separate dictionaries", () => {
|
||||
const graph = decodeKeyedArchiveBytes(
|
||||
keyedRoot([
|
||||
plistArray([9, 12]),
|
||||
Buffer.from([0xd1, 0, 10, 0, 11]),
|
||||
plistString("__proto__"),
|
||||
plistString("first"),
|
||||
Buffer.from([0xd1, 0, 10, 0, 13]),
|
||||
plistString("second"),
|
||||
]),
|
||||
selection,
|
||||
);
|
||||
expect(graph.limitations).toContain(
|
||||
"Dictionary entries keyed __proto__ were omitted because REA results cannot represent that key.",
|
||||
);
|
||||
});
|
||||
|
||||
it("reports binary omission presence without claiming a count", () => {
|
||||
const placeholder = "proto_key";
|
||||
const nested = Buffer.from(
|
||||
buildBinary({
|
||||
...archive,
|
||||
[placeholder]: { [placeholder]: "hidden", visible: true },
|
||||
$objects: ["$null", "value"],
|
||||
}),
|
||||
);
|
||||
let index = nested.indexOf(placeholder);
|
||||
while (index !== -1) {
|
||||
Buffer.from("__proto__").copy(nested, index);
|
||||
index = nested.indexOf(placeholder, index + placeholder.length);
|
||||
}
|
||||
const nestedGraph = decodeKeyedArchiveBytes(nested, selection);
|
||||
expect(nestedGraph.limitations).toContain(
|
||||
"Dictionary entries keyed __proto__ were omitted because REA results cannot represent that key.",
|
||||
);
|
||||
|
||||
const shadowedValue = keyedRoot([
|
||||
plistArray([9]),
|
||||
Buffer.from([0xd2, 0, 10, 0, 10, 0, 11, 0, 12]),
|
||||
plistString("shadow"),
|
||||
Buffer.from([0xd1, 0, 13, 0, 14]),
|
||||
plistString("replacement"),
|
||||
plistString(placeholder),
|
||||
plistString("hidden"),
|
||||
]);
|
||||
const keyOffset = shadowedValue.indexOf(Buffer.from(placeholder));
|
||||
expect(keyOffset).toBeGreaterThanOrEqual(0);
|
||||
Buffer.from("__proto__").copy(shadowedValue, keyOffset);
|
||||
const shadowedGraph = decodeKeyedArchiveBytes(shadowedValue, selection);
|
||||
expect(shadowedGraph.limitations).toContain(
|
||||
"Dictionary entries keyed __proto__ were omitted because REA results cannot represent that key.",
|
||||
);
|
||||
expect(shadowedGraph.limitations).not.toContain(
|
||||
"1 dictionary entry keyed __proto__ was omitted because REA results cannot represent that key.",
|
||||
);
|
||||
});
|
||||
|
||||
it("detects UTF-16 binary dictionary keys", () => {
|
||||
const graph = decodeKeyedArchiveBytes(
|
||||
keyedRoot([
|
||||
plistArray([9]),
|
||||
Buffer.from([0xd1, 0, 10, 0, 11]),
|
||||
plistUtf16String("__proto__"),
|
||||
plistString("hidden"),
|
||||
]),
|
||||
selection,
|
||||
);
|
||||
expect(graph.limitations).toContain(
|
||||
"Dictionary entries keyed __proto__ were omitted because REA results cannot represent that key.",
|
||||
);
|
||||
});
|
||||
|
||||
it("detects extended-length UTF-16 binary dictionary keys", () => {
|
||||
const graph = decodeKeyedArchiveBytes(
|
||||
keyedRoot([
|
||||
plistArray([9]),
|
||||
Buffer.from([0xd1, 0, 10, 0, 11]),
|
||||
plistExtendedUtf16String("__proto__"),
|
||||
plistString("hidden"),
|
||||
]),
|
||||
selection,
|
||||
);
|
||||
expect(graph.limitations).toContain(
|
||||
"Dictionary entries keyed __proto__ were omitted because REA results cannot represent that key.",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -6,7 +6,6 @@ import {
|
||||
AnalysisInputError,
|
||||
AnalysisUnsupportedTargetError,
|
||||
} from "../../domain/analysisErrorCore.js";
|
||||
import type { JsonValue } from "../../domain/jsonValue.js";
|
||||
import { projectPlistValue } from "../../domain/apple/plistValue.js";
|
||||
import { createPlistNumberProjection } from "../../domain/apple/plistNumbers.js";
|
||||
import {
|
||||
@@ -51,8 +50,8 @@ export const decodeKeyedArchiveBytes = (
|
||||
// plist.parseBinary recurses on every object reference and copies shared
|
||||
// containers, so a few hundred bytes can exhaust memory or the stack.
|
||||
// RangeError reports a resource limit; a reference cycle is malformed.
|
||||
if (binary)
|
||||
estimateBinaryPlistExpansion(bytes, MAX_DECODE_BYTES, {
|
||||
const binaryExpansion = binary
|
||||
? estimateBinaryPlistExpansion(bytes, MAX_DECODE_BYTES, {
|
||||
budget: "the keyed archive decode budget",
|
||||
fail: (kind, message) =>
|
||||
kind === "cycle"
|
||||
@@ -61,9 +60,16 @@ export const decodeKeyedArchiveBytes = (
|
||||
)
|
||||
: new RangeError(message),
|
||||
bounds: "decoder",
|
||||
});
|
||||
const parsed = binary
|
||||
? { value: parseBinary(bytes), omittedPrototypeKeys: 0 }
|
||||
})
|
||||
: undefined;
|
||||
const parsed =
|
||||
binaryExpansion !== undefined
|
||||
? {
|
||||
value: parseBinary(bytes),
|
||||
omittedPrototypeKeys: binaryExpansion.binaryPrototypeKeyObserved
|
||||
? null
|
||||
: 0,
|
||||
}
|
||||
: parseXmlPropertyList(xmlText ?? "");
|
||||
const { value: plistValue, unknownRealCount } = projectPlistValue(
|
||||
parsed.value,
|
||||
|
||||
@@ -19,6 +19,7 @@ const OPERATION = "capture_browser_scenario" as const;
|
||||
|
||||
const browserScenarioLimitations = (
|
||||
session: BrowserScenarioSessionPort,
|
||||
scenario: BrowserScenario,
|
||||
): string[] => [
|
||||
"Event sequence records provider receipt order; simultaneous browser causality is not inferred.",
|
||||
"Network content is retained only when selected; response bytes are browser-decoded, not compressed wire bytes.",
|
||||
@@ -26,6 +27,12 @@ const browserScenarioLimitations = (
|
||||
"Network content reads settle within 5 seconds each and never wait for unfinished responses or refetch them.",
|
||||
"Playwright scenario events do not expose request initiator stacks; receipt order does not prove causality.",
|
||||
"Storage values are hashed only after declared-secret redaction.",
|
||||
...(scenario.storage.local_storage.length === 0 &&
|
||||
scenario.storage.session_storage.length === 0
|
||||
? []
|
||||
: [
|
||||
"Non-cookie storage seeds use CDP pre-script instrumentation; only pauses in the scenario's private seed script are resumed by this observer.",
|
||||
]),
|
||||
...(session.eventLimitations?.() ?? []),
|
||||
...(session.mode === "connect"
|
||||
? [
|
||||
@@ -65,7 +72,7 @@ export const browserScenarioCaptureData = (input: {
|
||||
steps: [...input.steps],
|
||||
events: input.session.events(),
|
||||
limitations: [
|
||||
...browserScenarioLimitations(input.session),
|
||||
...browserScenarioLimitations(input.session, input.scenario),
|
||||
...(input.limitations ?? []),
|
||||
],
|
||||
});
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import type { WebPageInspection } from "../domain/browserObservationSchemas.js";
|
||||
import { sanitizeBrowserUrl } from "../domain/browserObservation.js";
|
||||
import { inferJsonShape } from "../domain/jsonShape.js";
|
||||
import { safeResponseMetadata } from "./CdpSafeMetadata.js";
|
||||
import {
|
||||
@@ -487,23 +488,20 @@ export const handleWebSocketFrame = (
|
||||
direction: "sent" | "received",
|
||||
): void => {
|
||||
const requestId = cdpStringValue(params.requestId);
|
||||
if (
|
||||
requestId === undefined ||
|
||||
(!state.network.has(requestId) && !state.allowedWebSockets.has(requestId))
|
||||
)
|
||||
return;
|
||||
if (requestId === undefined) return;
|
||||
const connection = state.webSocketConnectionsById.get(requestId);
|
||||
if (connection === undefined) return;
|
||||
const response = recordValue(params.response);
|
||||
const payload = cdpStringValue(response?.payloadData) ?? "";
|
||||
const opcode = Math.max(0, Math.trunc(numberValue(response?.opcode) ?? 0));
|
||||
const decoded = opcode === 1 ? undefined : decodeBase64(payload);
|
||||
if (opcode !== 1 && decoded === undefined)
|
||||
state.completeness.exclude("websocket_frames", "invalid_protocol_value");
|
||||
state.websockets.push({
|
||||
request_id: requestId,
|
||||
connection.events.push({
|
||||
direction,
|
||||
opcode,
|
||||
payload_bytes:
|
||||
opcode === 1 ? Buffer.byteLength(payload) : (decoded?.byteLength ?? 0),
|
||||
opcode === 1 ? Buffer.byteLength(payload) : (decoded?.byteLength ?? null),
|
||||
payload_shape: captureWebSocketShape(state, payload, opcode),
|
||||
});
|
||||
};
|
||||
@@ -512,7 +510,7 @@ const captureWebSocketShape = (
|
||||
state: CdpCaptureEventsState,
|
||||
payload: string,
|
||||
opcode: number,
|
||||
): WebPageInspection["network"]["websocket_events"][number]["payload_shape"] => {
|
||||
): WebPageInspection["network"]["websocket_connections"][number]["events"][number]["payload_shape"] => {
|
||||
if (!state.input.include_websocket_shapes) return null;
|
||||
if (opcode !== 1) return { format: "binary", json_shape: null };
|
||||
const shape = inferJsonShape(payload);
|
||||
@@ -542,9 +540,10 @@ export const handleWebSocketCreated = (
|
||||
);
|
||||
return;
|
||||
}
|
||||
const sourceUrl = sanitizeBrowserUrl(rawUrl).url;
|
||||
let parsed: URL;
|
||||
try {
|
||||
parsed = new URL(rawUrl);
|
||||
parsed = new URL(sourceUrl);
|
||||
} catch (cause: unknown) {
|
||||
// Unparseable URLs are recorded as unsupported, not as failures.
|
||||
void cause;
|
||||
@@ -561,7 +560,18 @@ export const handleWebSocketCreated = (
|
||||
state.completeness.exclude("websocket_connections", "disallowed_origin");
|
||||
return;
|
||||
}
|
||||
state.allowedWebSockets.add(requestId);
|
||||
if (state.webSocketConnectionsById.has(requestId)) {
|
||||
state.completeness.exclude(
|
||||
"websocket_connections",
|
||||
"invalid_protocol_value",
|
||||
);
|
||||
return;
|
||||
}
|
||||
state.webSocketConnectionsById.set(requestId, {
|
||||
request_id: requestId,
|
||||
url: sourceUrl,
|
||||
events: [],
|
||||
});
|
||||
};
|
||||
|
||||
export const handleFrameNavigated = (
|
||||
|
||||
@@ -16,9 +16,11 @@ export interface CdpCaptureEventsState {
|
||||
readonly network: Map<string, NetworkState>;
|
||||
readonly networkRequestTimestamps: Map<string, number>;
|
||||
readonly malformedRedirectRequestIds: Set<string>;
|
||||
readonly allowedWebSockets: Set<string>;
|
||||
readonly webSocketConnectionsById: Map<
|
||||
string,
|
||||
WebPageInspection["network"]["websocket_connections"][number]
|
||||
>;
|
||||
console: WebPageInspection["console"]["events"];
|
||||
websockets: WebPageInspection["network"]["websocket_events"];
|
||||
responseMetadata: WebPageInspection["metadata"]["responses"];
|
||||
agentHints: WebPageInspection["metadata"]["agent_hints"];
|
||||
readonly completeness: CdpCaptureCompleteness;
|
||||
|
||||
@@ -22,9 +22,11 @@ export class CdpCaptureEvents implements CdpCaptureEventsState {
|
||||
readonly network = new Map<string, NetworkState>();
|
||||
readonly networkRequestTimestamps = new Map<string, number>();
|
||||
readonly malformedRedirectRequestIds = new Set<string>();
|
||||
readonly allowedWebSockets = new Set<string>();
|
||||
readonly webSocketConnectionsById = new Map<
|
||||
string,
|
||||
WebPageInspection["network"]["websocket_connections"][number]
|
||||
>();
|
||||
console: WebPageInspection["console"]["events"] = [];
|
||||
websockets: WebPageInspection["network"]["websocket_events"] = [];
|
||||
responseMetadata: WebPageInspection["metadata"]["responses"] = [];
|
||||
agentHints: WebPageInspection["metadata"]["agent_hints"] = [];
|
||||
readonly completeness = new CdpCaptureCompleteness([
|
||||
@@ -63,9 +65,8 @@ export class CdpCaptureEvents implements CdpCaptureEventsState {
|
||||
this.network.clear();
|
||||
this.networkRequestTimestamps.clear();
|
||||
this.malformedRedirectRequestIds.clear();
|
||||
this.allowedWebSockets.clear();
|
||||
this.webSocketConnectionsById.clear();
|
||||
this.console.length = 0;
|
||||
this.websockets.length = 0;
|
||||
this.responseMetadata.length = 0;
|
||||
this.agentHints.length = 0;
|
||||
this.completeness.reset();
|
||||
|
||||
@@ -6,6 +6,11 @@ import { BrowserObservationError } from "../domain/browserObservationError.js";
|
||||
import { CdpCommandRejection } from "./CdpCommandRejection.js";
|
||||
import { type BrowserObservationOperation } from "../domain/browserObservationErrors.js";
|
||||
import { safeParseJson } from "../domain/safeJson.js";
|
||||
import { WEB_RUNTIME_LIMITS } from "../domain/webRuntime.js";
|
||||
|
||||
// ws currently stores maxPayload as a signed 32-bit integer. Larger values can
|
||||
// wrap to a negative number, which disables its limit check.
|
||||
const MAX_WEBSOCKET_PAYLOAD_BYTES = 0x7fff_ffff;
|
||||
|
||||
export interface CdpEvent {
|
||||
readonly method: string;
|
||||
@@ -61,13 +66,25 @@ export class CdpConnection {
|
||||
limits?: { readonly maxPayloadBytes: number },
|
||||
): Promise<CdpConnection> {
|
||||
if (signal?.aborted === true) throw new AnalysisCancelledError(operation);
|
||||
const maxPayloadBytes =
|
||||
limits === undefined
|
||||
? WEB_RUNTIME_LIMITS.protocolBytes
|
||||
: limits.maxPayloadBytes;
|
||||
if (
|
||||
!Number.isSafeInteger(maxPayloadBytes) ||
|
||||
maxPayloadBytes <= 0 ||
|
||||
maxPayloadBytes > MAX_WEBSOCKET_PAYLOAD_BYTES
|
||||
)
|
||||
throw new RangeError(
|
||||
`CDP maxPayloadBytes must be a positive safe integer no greater than ${String(MAX_WEBSOCKET_PAYLOAD_BYTES)}.`,
|
||||
);
|
||||
const socket = new WebSocket(url, {
|
||||
handshakeTimeout: 0,
|
||||
maxPayload: limits?.maxPayloadBytes ?? 0,
|
||||
maxPayload: maxPayloadBytes,
|
||||
perMessageDeflate: false,
|
||||
});
|
||||
await waitForOpen(socket, operation, signal);
|
||||
return new CdpConnection(socket, operation, limits?.maxPayloadBytes ?? 0);
|
||||
return new CdpConnection(socket, operation, maxPayloadBytes);
|
||||
}
|
||||
|
||||
/** Subscribe to validated CDP event envelopes. */
|
||||
|
||||
@@ -300,7 +300,7 @@ const normalizedInspection = (
|
||||
resources: [...captured.resources],
|
||||
network: {
|
||||
requests: [...state.events.network.values()],
|
||||
websocket_events: [...state.events.websockets],
|
||||
websocket_connections: [...state.events.webSocketConnectionsById.values()],
|
||||
coverage_started_at: state.startedAt,
|
||||
prior_activity_available: false,
|
||||
},
|
||||
|
||||
@@ -8,7 +8,12 @@ import {
|
||||
PlaywrightScenarioBrowserCleanupOwner,
|
||||
} from "./PlaywrightScenarioBrowser.js";
|
||||
|
||||
afterEach(() => vi.unstubAllEnvs());
|
||||
const hostPlatform = process.platform;
|
||||
|
||||
afterEach(() => {
|
||||
vi.unstubAllEnvs();
|
||||
Object.defineProperty(process, "platform", { value: hostPlatform });
|
||||
});
|
||||
|
||||
it("locks one attached browser for spellings that discover the same endpoint", () => {
|
||||
expect(attachedScenarioLeaseKey("http://127.0.0.1:9222", "page-1")).toBe(
|
||||
@@ -73,6 +78,43 @@ it("passes the selected environment to the actual browser launch boundary", asyn
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps Termux execution support when launching Chromium on Android", async () => {
|
||||
Object.defineProperty(process, "platform", { value: "android" });
|
||||
vi.stubEnv("PREFIX", "/data/data/com.termux/files/usr");
|
||||
const selected = { LANG: "selected-language", PREFIX: process.env.PREFIX };
|
||||
let launchOptions: Parameters<
|
||||
typeof import("playwright-core").chromium.launchPersistentContext
|
||||
>[1];
|
||||
const scenario = browserScenarioSchema.parse({
|
||||
browser: { mode: "launch", executable_path: process.execPath },
|
||||
start_url: { url: "https://example.test" },
|
||||
actions: [
|
||||
{ step_id: "settle", action: "wait_for_timeout", duration_ms: 1 },
|
||||
],
|
||||
});
|
||||
|
||||
await expect(
|
||||
openPlaywrightScenarioBrowser(scenario, selected, {
|
||||
launcher: {
|
||||
connectOverCDP: async () => {
|
||||
throw new Error("Unexpected connection");
|
||||
},
|
||||
launchPersistentContext: async (_profile, options) => {
|
||||
launchOptions = options;
|
||||
throw new Error("launch boundary reached");
|
||||
},
|
||||
},
|
||||
}),
|
||||
).rejects.toThrow("launch boundary reached");
|
||||
|
||||
expect(launchOptions?.args).toEqual(["--no-sandbox", "--single-process"]);
|
||||
expect(launchOptions?.env).toEqual({
|
||||
LANG: "selected-language",
|
||||
LD_PRELOAD: "/data/data/com.termux/files/usr/lib/libtermux-exec.so",
|
||||
TERMUX_EXEC__PROC_SELF_EXE: process.execPath,
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps attached-target ownership until cancelled cleanup actually settles", async () => {
|
||||
const controller = new AbortController();
|
||||
const closeStarted = deferred<void>();
|
||||
|
||||
@@ -49,6 +49,10 @@ export class PlaywrightScenarioBrowserCleanupOwner {
|
||||
this.#profileRemoved = resources.removeProfile === undefined;
|
||||
}
|
||||
|
||||
get browserClosed(): boolean {
|
||||
return this.#browserClosed;
|
||||
}
|
||||
|
||||
close(
|
||||
finishEvents?: () => Promise<void>,
|
||||
signal?: AbortSignal,
|
||||
@@ -273,13 +277,26 @@ const allowedBrowserEnvironment = (
|
||||
"XAUTHORITY",
|
||||
"XDG_RUNTIME_DIR",
|
||||
];
|
||||
return Object.fromEntries(
|
||||
const selected = Object.fromEntries(
|
||||
allowed
|
||||
.map((name) => [name, environment[name]] as const)
|
||||
.filter(
|
||||
(entry): entry is readonly [string, string] => entry[1] !== undefined,
|
||||
),
|
||||
);
|
||||
if (process.platform !== "android") return selected;
|
||||
|
||||
// Android rejects execve() for binaries below Termux's application data
|
||||
// directory. Termux supplies an exec interceptor that rewrites those calls
|
||||
// through the system linker and removes itself for the launched program.
|
||||
// Playwright must retain it for its Chromium child process.
|
||||
const prefix = environment.PREFIX;
|
||||
if (prefix === undefined) return selected;
|
||||
return {
|
||||
...selected,
|
||||
LD_PRELOAD: `${prefix}/lib/libtermux-exec.so`,
|
||||
TERMUX_EXEC__PROC_SELF_EXE: process.execPath,
|
||||
};
|
||||
};
|
||||
const findConnectedPage = async (
|
||||
browser: Pick<Browser, "contexts">,
|
||||
@@ -495,6 +512,13 @@ export const openPlaywrightScenarioBrowser = async (
|
||||
handleSIGINT: false,
|
||||
handleSIGTERM: false,
|
||||
timeout: 0,
|
||||
// Termux's Chromium cannot create its renderer process through Android's
|
||||
// executable boundary. Single-process mode keeps the browser and page in
|
||||
// the Termux-launched process so scenario navigation remains available.
|
||||
args:
|
||||
process.platform === "android"
|
||||
? ["--no-sandbox", "--single-process"]
|
||||
: [],
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
const cleanup = () =>
|
||||
|
||||
@@ -0,0 +1,171 @@
|
||||
import { EventEmitter } from "node:events";
|
||||
|
||||
import { beforeEach, expect, it, vi } from "vitest";
|
||||
|
||||
const mocks = vi.hoisted(() => ({
|
||||
openBrowser: vi.fn(),
|
||||
storage: undefined as
|
||||
| {
|
||||
initialize: ReturnType<typeof vi.fn>;
|
||||
settle: ReturnType<typeof vi.fn>;
|
||||
close: ReturnType<typeof vi.fn>;
|
||||
}
|
||||
| undefined,
|
||||
finishEvents: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock("./PlaywrightScenarioBrowser.js", () => ({
|
||||
openPlaywrightScenarioBrowser: mocks.openBrowser,
|
||||
failBrowserScenarioOperation: async (
|
||||
cleanup: () => Promise<void>,
|
||||
cause: unknown,
|
||||
) => {
|
||||
try {
|
||||
await cleanup();
|
||||
} catch (cleanupCause: unknown) {
|
||||
throw new AggregateError([cause, cleanupCause]);
|
||||
}
|
||||
throw cause;
|
||||
},
|
||||
}));
|
||||
|
||||
vi.mock("./PlaywrightScenarioStorage.js", () => ({
|
||||
PlaywrightScenarioStorage: {
|
||||
create: vi.fn(async () => mocks.storage),
|
||||
},
|
||||
}));
|
||||
|
||||
vi.mock("./PlaywrightScenarioEvents.js", () => ({
|
||||
PlaywrightScenarioEvents: class {
|
||||
finish() {
|
||||
mocks.finishEvents();
|
||||
}
|
||||
|
||||
result() {
|
||||
return { events: [] };
|
||||
}
|
||||
|
||||
limitations() {
|
||||
return [];
|
||||
}
|
||||
|
||||
setStep() {}
|
||||
|
||||
nextSequence() {
|
||||
return 1;
|
||||
}
|
||||
|
||||
lastSequence() {
|
||||
return 0;
|
||||
}
|
||||
},
|
||||
}));
|
||||
|
||||
import { browserScenarioSchema } from "../domain/browserScenario.js";
|
||||
import { PlaywrightScenarioSession } from "./PlaywrightScenarioSession.js";
|
||||
|
||||
const scenario = browserScenarioSchema.parse({
|
||||
browser: { mode: "launch", executable_path: process.execPath },
|
||||
start_url: { url: "https://storage.example.test/" },
|
||||
actions: [{ step_id: "wait", action: "wait_for_timeout", duration_ms: 1 }],
|
||||
storage: {
|
||||
local_storage: [
|
||||
{
|
||||
origin: "https://storage.example.test",
|
||||
entries: [{ name: "seed", value: { source: "literal", value: "v" } }],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const fixture = () => {
|
||||
const page = Object.assign(new EventEmitter(), {
|
||||
url: () => "about:blank",
|
||||
goto: vi.fn(async () => undefined),
|
||||
context: () => context,
|
||||
});
|
||||
const context = Object.assign(new EventEmitter(), {
|
||||
setDefaultTimeout: vi.fn(),
|
||||
setDefaultNavigationTimeout: vi.fn(),
|
||||
addCookies: vi.fn(async () => undefined),
|
||||
browser: () => browser,
|
||||
});
|
||||
const browser = Object.assign(new EventEmitter(), {
|
||||
version: () => "Chrome 154",
|
||||
});
|
||||
let browserClosed = false;
|
||||
const close = vi.fn(async (finishEvents?: () => Promise<void>) => {
|
||||
await finishEvents?.();
|
||||
browserClosed = true;
|
||||
});
|
||||
const cleanup = {
|
||||
close,
|
||||
get browserClosed() {
|
||||
return browserClosed;
|
||||
},
|
||||
};
|
||||
mocks.openBrowser.mockResolvedValue({
|
||||
page,
|
||||
context,
|
||||
browser,
|
||||
cleanup,
|
||||
profilePath: "/tmp/profile",
|
||||
});
|
||||
mocks.storage = {
|
||||
initialize: vi.fn(async () => undefined),
|
||||
settle: vi.fn(async () => undefined),
|
||||
close: vi.fn(async () => undefined),
|
||||
};
|
||||
return { cleanup, storage: mocks.storage };
|
||||
};
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
});
|
||||
|
||||
it("uses browser disconnect as terminal storage release after detach failure", async () => {
|
||||
const { cleanup, storage } = fixture();
|
||||
const detachFailure = new Error("CDP detach rejected");
|
||||
storage.close.mockRejectedValueOnce(detachFailure);
|
||||
const session = await PlaywrightScenarioSession.open(scenario, {}, {});
|
||||
|
||||
await expect(session.close()).resolves.toBe("terminated-owned-process");
|
||||
expect(storage.close).toHaveBeenCalledTimes(1);
|
||||
expect(cleanup.close).toHaveBeenCalledTimes(1);
|
||||
expect(mocks.finishEvents).toHaveBeenCalledTimes(1);
|
||||
await expect(session.close()).resolves.toBe("terminated-owned-process");
|
||||
expect(storage.close).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("preserves both cleanup failures and retries retained owners", async () => {
|
||||
const { cleanup, storage } = fixture();
|
||||
const detachFailure = new Error("CDP detach rejected");
|
||||
const transportFailure = new Error("Playwright transport close failed");
|
||||
storage.close
|
||||
.mockRejectedValueOnce(detachFailure)
|
||||
.mockResolvedValueOnce(undefined);
|
||||
cleanup.close
|
||||
.mockRejectedValueOnce(transportFailure)
|
||||
.mockImplementationOnce(async (finishEvents?: () => Promise<void>) => {
|
||||
await finishEvents?.();
|
||||
});
|
||||
const session = await PlaywrightScenarioSession.open(scenario, {}, {});
|
||||
|
||||
let failure: unknown;
|
||||
try {
|
||||
await session.close();
|
||||
} catch (cause: unknown) {
|
||||
failure = cause;
|
||||
}
|
||||
expect(failure).toBeInstanceOf(AggregateError);
|
||||
expect((failure as AggregateError).errors).toEqual([
|
||||
detachFailure,
|
||||
transportFailure,
|
||||
]);
|
||||
expect(storage.close).toHaveBeenCalledTimes(1);
|
||||
expect(cleanup.close).toHaveBeenCalledTimes(1);
|
||||
|
||||
await expect(session.close()).resolves.toBe("terminated-owned-process");
|
||||
expect(storage.close).toHaveBeenCalledTimes(2);
|
||||
expect(cleanup.close).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
@@ -13,6 +13,7 @@ import {
|
||||
import { performPlaywrightScenarioAction } from "./PlaywrightScenarioActions.js";
|
||||
import { capturePlaywrightStepArtifacts } from "./PlaywrightScenarioArtifacts.js";
|
||||
import { PlaywrightScenarioEvents } from "./PlaywrightScenarioEvents.js";
|
||||
import { PlaywrightScenarioStorage } from "./PlaywrightScenarioStorage.js";
|
||||
import { withPlaywrightExecutionBoundary } from "./PlaywrightExecutionBoundary.js";
|
||||
import {
|
||||
browserScenarioCaptureData,
|
||||
@@ -21,21 +22,19 @@ import {
|
||||
|
||||
const OPERATION = "capture_browser_scenario" as const;
|
||||
|
||||
const storageSeeds = (
|
||||
blocks: BrowserScenario["storage"]["local_storage"],
|
||||
secrets: BrowserScenarioSecrets,
|
||||
): Record<string, string[][]> => {
|
||||
const seeds: Record<string, string[][]> = {};
|
||||
for (const { origin, entries } of blocks)
|
||||
(seeds[origin] ??= []).push(
|
||||
...entries.map(({ name, value }) => [name, secrets.value(value)]),
|
||||
);
|
||||
return seeds;
|
||||
const throwIfScenarioCancelled = (signal: AbortSignal | undefined): void => {
|
||||
if (signal?.aborted === true)
|
||||
throw new BrowserObservationError(OPERATION, "cancelled");
|
||||
};
|
||||
|
||||
const installStorageSeeds = async (
|
||||
const throwCleanupFailures = (failures: readonly unknown[]): void => {
|
||||
if (failures.length === 1) throw failures[0];
|
||||
if (failures.length > 1)
|
||||
throw new AggregateError(failures, "Browser scenario cleanup failed");
|
||||
};
|
||||
|
||||
const installCookies = async (
|
||||
context: BrowserContext,
|
||||
page: Page,
|
||||
scenario: BrowserScenario,
|
||||
secrets: BrowserScenarioSecrets,
|
||||
): Promise<void> => {
|
||||
@@ -49,19 +48,6 @@ const installStorageSeeds = async (
|
||||
sameSite: cookie.same_site,
|
||||
})),
|
||||
);
|
||||
const storage = {
|
||||
local: storageSeeds(scenario.storage.local_storage, secrets),
|
||||
session: storageSeeds(scenario.storage.session_storage, secrets),
|
||||
};
|
||||
const payload = JSON.stringify(storage).replaceAll("<", "\\u003c");
|
||||
await page.addInitScript(`(() => {
|
||||
const seeds = ${payload};
|
||||
const local = seeds.local[window.location.origin] ?? [];
|
||||
const session = seeds.session[window.location.origin] ?? [];
|
||||
for (const [name, value] of local) window.localStorage.setItem(name, value);
|
||||
for (const [name, value] of session)
|
||||
window.sessionStorage.setItem(name, value);
|
||||
})()`);
|
||||
};
|
||||
|
||||
const blockAttachedServiceWorkers = async (page: Page): Promise<void> => {
|
||||
@@ -93,7 +79,7 @@ const initializePage = async (
|
||||
context.setDefaultNavigationTimeout(0);
|
||||
if (scenario.browser.mode === "connect")
|
||||
await blockAttachedServiceWorkers(page);
|
||||
await installStorageSeeds(context, page, scenario, secrets);
|
||||
await installCookies(context, scenario, secrets);
|
||||
};
|
||||
|
||||
export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
@@ -104,6 +90,9 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
readonly initialUrl: string;
|
||||
private readonly secrets: BrowserScenarioSecrets;
|
||||
private readonly eventCapture: PlaywrightScenarioEvents;
|
||||
private readonly storage: PlaywrightScenarioStorage | undefined;
|
||||
private readonly signal: AbortSignal | undefined;
|
||||
#storageReleased = false;
|
||||
|
||||
private constructor(
|
||||
private readonly opened: OpenedScenarioBrowser,
|
||||
@@ -111,6 +100,8 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
readonly mode: BrowserScenario["browser"]["mode"];
|
||||
readonly secrets: BrowserScenarioSecrets;
|
||||
readonly eventCapture: PlaywrightScenarioEvents;
|
||||
readonly storage: PlaywrightScenarioStorage | undefined;
|
||||
readonly signal: AbortSignal | undefined;
|
||||
},
|
||||
) {
|
||||
this.mode = options.mode;
|
||||
@@ -118,6 +109,8 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
options.mode === "launch" ? "provider-owned" : "external";
|
||||
this.secrets = options.secrets;
|
||||
this.eventCapture = options.eventCapture;
|
||||
this.storage = options.storage;
|
||||
this.signal = options.signal;
|
||||
this.version = opened.browser.version();
|
||||
this.initialUrl = opened.page.url();
|
||||
}
|
||||
@@ -130,8 +123,7 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
readonly retainCleanup?: (close: () => Promise<unknown>) => void;
|
||||
},
|
||||
): Promise<PlaywrightScenarioSession> {
|
||||
if (options.signal?.aborted === true)
|
||||
throw new BrowserObservationError(OPERATION, "cancelled");
|
||||
throwIfScenarioCancelled(options.signal);
|
||||
const secrets = BrowserScenarioSecrets.resolve(scenario, environment);
|
||||
if (secrets === undefined)
|
||||
throw new BrowserObservationError(OPERATION, "secret_unavailable");
|
||||
@@ -143,7 +135,6 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
: { retainCleanup: options.retainCleanup }),
|
||||
});
|
||||
let opened: OpenedScenarioBrowser;
|
||||
let events: PlaywrightScenarioEvents | undefined;
|
||||
try {
|
||||
opened = await withPlaywrightExecutionBoundary(
|
||||
() => opening,
|
||||
@@ -165,42 +156,12 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
});
|
||||
throw cause;
|
||||
}
|
||||
let session: PlaywrightScenarioSession;
|
||||
try {
|
||||
await withPlaywrightExecutionBoundary(
|
||||
() => initializePage(opened.context, opened.page, scenario, secrets),
|
||||
undefined,
|
||||
options.signal,
|
||||
);
|
||||
events = new PlaywrightScenarioEvents({
|
||||
page: opened.page,
|
||||
context: opened.context,
|
||||
ownsContext: scenario.browser.mode === "launch",
|
||||
enabled: new Set(scenario.capture.events),
|
||||
const session = await this.initializeSession(
|
||||
opened,
|
||||
scenario,
|
||||
secrets,
|
||||
network: scenario.capture.network,
|
||||
...(options.signal === undefined ? {} : { signal: options.signal }),
|
||||
});
|
||||
session = new PlaywrightScenarioSession(opened, {
|
||||
mode: scenario.browser.mode,
|
||||
secrets,
|
||||
eventCapture: events,
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
const cleanup = () =>
|
||||
opened.cleanup.close(
|
||||
events === undefined
|
||||
? undefined
|
||||
: () => events?.finish() ?? Promise.resolve(),
|
||||
options.signal,
|
||||
options,
|
||||
);
|
||||
try {
|
||||
return await failBrowserScenarioOperation(cleanup, cause);
|
||||
} catch (failure: unknown) {
|
||||
if (failure !== cause) options.retainCleanup?.(cleanup);
|
||||
throw failure;
|
||||
}
|
||||
}
|
||||
try {
|
||||
await withPlaywrightExecutionBoundary(
|
||||
() =>
|
||||
@@ -211,6 +172,11 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
undefined,
|
||||
options.signal,
|
||||
);
|
||||
await withPlaywrightExecutionBoundary(
|
||||
() => session.storage?.settle() ?? Promise.resolve(),
|
||||
undefined,
|
||||
options.signal,
|
||||
);
|
||||
return session;
|
||||
} catch (cause: unknown) {
|
||||
let cleanup:
|
||||
@@ -219,7 +185,7 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
| "incomplete";
|
||||
let cleanupFailure: { readonly cause: unknown } | undefined;
|
||||
try {
|
||||
cleanup = await session.close();
|
||||
cleanup = await session.close(true);
|
||||
} catch (failure: unknown) {
|
||||
cleanup = "incomplete";
|
||||
cleanupFailure = { cause: failure };
|
||||
@@ -245,6 +211,91 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
}
|
||||
}
|
||||
|
||||
private static async initializeSession(
|
||||
opened: OpenedScenarioBrowser,
|
||||
scenario: BrowserScenario,
|
||||
secrets: BrowserScenarioSecrets,
|
||||
options: {
|
||||
readonly signal?: AbortSignal;
|
||||
readonly retainCleanup?: (close: () => Promise<unknown>) => void;
|
||||
},
|
||||
): Promise<PlaywrightScenarioSession> {
|
||||
let events: PlaywrightScenarioEvents | undefined;
|
||||
let storage: PlaywrightScenarioStorage | undefined;
|
||||
try {
|
||||
await withPlaywrightExecutionBoundary(
|
||||
() => initializePage(opened.context, opened.page, scenario, secrets),
|
||||
undefined,
|
||||
options.signal,
|
||||
);
|
||||
storage = await PlaywrightScenarioStorage.create(
|
||||
opened,
|
||||
scenario,
|
||||
secrets,
|
||||
{
|
||||
...(options.signal === undefined ? {} : { signal: options.signal }),
|
||||
...(options.retainCleanup === undefined
|
||||
? {}
|
||||
: { retainCleanup: options.retainCleanup }),
|
||||
},
|
||||
);
|
||||
await storage?.initialize(opened.page, opened.context);
|
||||
throwIfScenarioCancelled(options.signal);
|
||||
events = new PlaywrightScenarioEvents({
|
||||
page: opened.page,
|
||||
context: opened.context,
|
||||
ownsContext: scenario.browser.mode === "launch",
|
||||
enabled: new Set(scenario.capture.events),
|
||||
secrets,
|
||||
network: scenario.capture.network,
|
||||
...(options.signal === undefined ? {} : { signal: options.signal }),
|
||||
});
|
||||
return new PlaywrightScenarioSession(opened, {
|
||||
mode: scenario.browser.mode,
|
||||
secrets,
|
||||
eventCapture: events,
|
||||
storage,
|
||||
signal: options.signal,
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
let storageReleased = false;
|
||||
const cleanup = async () => {
|
||||
let storageFailure: { readonly cause: unknown } | undefined;
|
||||
let browserFailure: { readonly cause: unknown } | undefined;
|
||||
if (!storageReleased) {
|
||||
try {
|
||||
await storage?.close(options.signal?.aborted === true);
|
||||
} catch (failure: unknown) {
|
||||
storageFailure = { cause: failure };
|
||||
}
|
||||
}
|
||||
try {
|
||||
await opened.cleanup.close(
|
||||
events === undefined
|
||||
? undefined
|
||||
: () => events?.finish() ?? Promise.resolve(),
|
||||
options.signal,
|
||||
);
|
||||
} catch (failure: unknown) {
|
||||
browserFailure = { cause: failure };
|
||||
}
|
||||
if (opened.cleanup.browserClosed) storageReleased = true;
|
||||
throwCleanupFailures([
|
||||
...(storageReleased || storageFailure === undefined
|
||||
? []
|
||||
: [storageFailure.cause]),
|
||||
...(browserFailure === undefined ? [] : [browserFailure.cause]),
|
||||
]);
|
||||
};
|
||||
try {
|
||||
return await failBrowserScenarioOperation(cleanup, cause);
|
||||
} catch (failure: unknown) {
|
||||
if (failure !== cause) options.retainCleanup?.(cleanup);
|
||||
throw failure;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
currentUrl(): string {
|
||||
return this.opened.page.url();
|
||||
}
|
||||
@@ -288,6 +339,11 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
"timeout_ms" in action ? action.timeout_ms : undefined,
|
||||
signal,
|
||||
);
|
||||
await withPlaywrightExecutionBoundary(
|
||||
() => this.storage?.settle() ?? Promise.resolve(),
|
||||
undefined,
|
||||
signal,
|
||||
);
|
||||
}
|
||||
|
||||
async capture(
|
||||
@@ -296,6 +352,11 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
>,
|
||||
signal?: AbortSignal,
|
||||
) {
|
||||
await withPlaywrightExecutionBoundary(
|
||||
() => this.storage?.settle() ?? Promise.resolve(),
|
||||
undefined,
|
||||
signal,
|
||||
);
|
||||
return withPlaywrightExecutionBoundary(
|
||||
() =>
|
||||
capturePlaywrightStepArtifacts({
|
||||
@@ -309,8 +370,28 @@ export class PlaywrightScenarioSession implements BrowserScenarioSessionPort {
|
||||
);
|
||||
}
|
||||
|
||||
async close() {
|
||||
async close(stopLoading = this.signal?.aborted === true) {
|
||||
let storageFailure: { readonly cause: unknown } | undefined;
|
||||
let browserFailure: { readonly cause: unknown } | undefined;
|
||||
if (!this.#storageReleased) {
|
||||
try {
|
||||
await this.storage?.close(stopLoading);
|
||||
} catch (failure: unknown) {
|
||||
storageFailure = { cause: failure };
|
||||
}
|
||||
}
|
||||
try {
|
||||
await this.opened.cleanup.close(() => this.eventCapture.finish());
|
||||
} catch (failure: unknown) {
|
||||
browserFailure = { cause: failure };
|
||||
}
|
||||
if (this.opened.cleanup.browserClosed) this.#storageReleased = true;
|
||||
throwCleanupFailures([
|
||||
...(this.#storageReleased || storageFailure === undefined
|
||||
? []
|
||||
: [storageFailure.cause]),
|
||||
...(browserFailure === undefined ? [] : [browserFailure.cause]),
|
||||
]);
|
||||
return this.mode === "launch"
|
||||
? ("terminated-owned-process" as const)
|
||||
: ("disconnected-external" as const);
|
||||
|
||||
@@ -0,0 +1,418 @@
|
||||
import { EventEmitter } from "node:events";
|
||||
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import { browserScenarioSchema } from "../domain/browserScenario.js";
|
||||
import { BrowserScenarioSecrets } from "./BrowserScenarioSecrets.js";
|
||||
import { PlaywrightScenarioStorage } from "./PlaywrightScenarioStorage.js";
|
||||
|
||||
class FakeCdp extends EventEmitter {
|
||||
readonly calls: { readonly method: string; readonly params?: unknown }[] = [];
|
||||
storageKey = "https://storage.example.test";
|
||||
failDetachCount = 0;
|
||||
holdDetach = false;
|
||||
resolveDetach: (() => void) | undefined;
|
||||
holdMethod: string | undefined;
|
||||
heldReply: (() => void) | undefined;
|
||||
autoReplyNested = false;
|
||||
readonly pendingNested: {
|
||||
readonly sessionId: string;
|
||||
readonly id: number;
|
||||
}[] = [];
|
||||
detachCalls = 0;
|
||||
|
||||
async send(method: string, params?: unknown): Promise<unknown> {
|
||||
this.calls.push({ method, params });
|
||||
if (method === this.holdMethod) {
|
||||
const response =
|
||||
method === "Page.addScriptToEvaluateOnNewDocument"
|
||||
? { identifier: "late-script" }
|
||||
: method === "Debugger.setBreakpointByUrl"
|
||||
? { breakpointId: "late-breakpoint" }
|
||||
: method === "Storage.getStorageKey"
|
||||
? { storageKey: this.storageKey }
|
||||
: {};
|
||||
return new Promise((resolve) => {
|
||||
this.heldReply = () => resolve(response);
|
||||
});
|
||||
}
|
||||
if (method === "Target.getTargetInfo")
|
||||
return { targetInfo: { targetId: "root-target" } };
|
||||
if (method === "Target.sendMessageToTarget") {
|
||||
const request = params as {
|
||||
readonly sessionId: string;
|
||||
readonly message: string;
|
||||
};
|
||||
const { id } = JSON.parse(request.message) as { readonly id: number };
|
||||
this.pendingNested.push({ sessionId: request.sessionId, id });
|
||||
if (this.autoReplyNested) this.replyNested(request.sessionId, id);
|
||||
}
|
||||
if (method === "Debugger.setBreakpointByUrl")
|
||||
return { breakpointId: "breakpoint" };
|
||||
if (method === "Page.addScriptToEvaluateOnNewDocument")
|
||||
return { identifier: "script" };
|
||||
if (method === "Storage.getStorageKey")
|
||||
return { storageKey: this.storageKey };
|
||||
if (method === "Debugger.evaluateOnCallFrame") return {};
|
||||
return {};
|
||||
}
|
||||
|
||||
async detach(): Promise<void> {
|
||||
this.detachCalls += 1;
|
||||
if (this.failDetachCount > 0) {
|
||||
this.failDetachCount -= 1;
|
||||
throw new Error("one-shot CDP detach failure");
|
||||
}
|
||||
if (this.holdDetach)
|
||||
await new Promise<void>((resolve) => {
|
||||
this.resolveDetach = resolve;
|
||||
});
|
||||
}
|
||||
|
||||
releaseHeldReply(): void {
|
||||
this.heldReply?.();
|
||||
this.heldReply = undefined;
|
||||
this.holdMethod = undefined;
|
||||
}
|
||||
|
||||
releaseDetach(): void {
|
||||
this.holdDetach = false;
|
||||
this.resolveDetach?.();
|
||||
this.resolveDetach = undefined;
|
||||
}
|
||||
|
||||
replyNested(sessionId: string, id: number): void {
|
||||
queueMicrotask(() =>
|
||||
this.emit("Target.receivedMessageFromTarget", {
|
||||
sessionId,
|
||||
message: JSON.stringify({ id, result: {} }),
|
||||
}),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const scenario = () =>
|
||||
browserScenarioSchema.parse({
|
||||
browser: { mode: "launch", executable_path: process.execPath },
|
||||
start_url: { url: "https://storage.example.test/" },
|
||||
actions: [{ step_id: "wait", action: "wait_for_timeout", duration_ms: 1 }],
|
||||
storage: {
|
||||
local_storage: [
|
||||
{
|
||||
origin: "https://storage.example.test",
|
||||
entries: [
|
||||
{
|
||||
name: "local-seed",
|
||||
value: { source: "literal", value: "local-value" },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
session_storage: [
|
||||
{
|
||||
origin: "https://storage.example.test",
|
||||
entries: [
|
||||
{
|
||||
name: "session-seed",
|
||||
value: { source: "literal", value: "session-value" },
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const fixture = (cdp: FakeCdp) => {
|
||||
const page = new EventEmitter();
|
||||
const context = new EventEmitter();
|
||||
const browser = new EventEmitter();
|
||||
return {
|
||||
page,
|
||||
context,
|
||||
browser,
|
||||
contextApi: Object.assign(context, {
|
||||
browser: () => browser,
|
||||
newCDPSession: async () => cdp,
|
||||
}),
|
||||
};
|
||||
};
|
||||
|
||||
const waitForCall = async (cdp: FakeCdp, method: string): Promise<void> => {
|
||||
for (let attempt = 0; attempt < 50; attempt += 1) {
|
||||
if (cdp.calls.some((call) => call.method === method)) return;
|
||||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||||
}
|
||||
throw new Error(`CDP method was not called: ${method}`);
|
||||
};
|
||||
|
||||
const createInitializedStorage = async (
|
||||
context: ReturnType<typeof fixture>["contextApi"],
|
||||
page: ReturnType<typeof fixture>["page"],
|
||||
input = scenario(),
|
||||
options: { readonly signal?: AbortSignal } = {},
|
||||
) => {
|
||||
const owner = await PlaywrightScenarioStorage.create(
|
||||
{ context: context as never, page: page as never },
|
||||
input,
|
||||
BrowserScenarioSecrets.resolve(input, {})!,
|
||||
options,
|
||||
);
|
||||
await owner?.initialize(page as never, context as never);
|
||||
return owner;
|
||||
};
|
||||
|
||||
const emitSeedPause = (cdp: FakeCdp): void => {
|
||||
const addScript = cdp.calls.find(
|
||||
({ method }) => method === "Page.addScriptToEvaluateOnNewDocument",
|
||||
);
|
||||
if (addScript === undefined)
|
||||
throw new Error("Storage seed script is missing");
|
||||
const sourceUrl = (addScript.params as { readonly worldName: string })
|
||||
.worldName;
|
||||
cdp.emit("Runtime.executionContextCreated", {
|
||||
context: {
|
||||
id: 4,
|
||||
origin: "https://storage.example.test",
|
||||
name: sourceUrl,
|
||||
auxData: { frameId: "root-frame" },
|
||||
},
|
||||
});
|
||||
cdp.emit("Debugger.scriptParsed", {
|
||||
scriptId: "seed-script",
|
||||
executionContextId: 4,
|
||||
url: sourceUrl,
|
||||
});
|
||||
cdp.emit("Debugger.paused", {
|
||||
callFrames: [
|
||||
{
|
||||
callFrameId: "seed-frame",
|
||||
location: { scriptId: "seed-script" },
|
||||
},
|
||||
],
|
||||
});
|
||||
};
|
||||
|
||||
it("seeds a browser storage area once before the document script resumes", async () => {
|
||||
const cdp = new FakeCdp();
|
||||
const { page, contextApi } = fixture(cdp);
|
||||
const owner = await createInitializedStorage(contextApi, page);
|
||||
expect(owner).toBeDefined();
|
||||
|
||||
cdp.emit("Debugger.paused", {
|
||||
callFrames: [
|
||||
{
|
||||
callFrameId: "caller-debugger-frame",
|
||||
location: { scriptId: "caller-owned-script" },
|
||||
},
|
||||
],
|
||||
});
|
||||
await owner?.settle();
|
||||
expect(
|
||||
cdp.calls.some(({ method }) => method === "Debugger.evaluateOnCallFrame"),
|
||||
).toBe(false);
|
||||
expect(cdp.calls.some(({ method }) => method === "Debugger.resume")).toBe(
|
||||
false,
|
||||
);
|
||||
|
||||
emitSeedPause(cdp);
|
||||
await owner?.settle();
|
||||
emitSeedPause(cdp);
|
||||
await owner?.settle();
|
||||
|
||||
const evaluations = cdp.calls.filter(
|
||||
({ method }) => method === "Debugger.evaluateOnCallFrame",
|
||||
);
|
||||
expect(evaluations).toHaveLength(1);
|
||||
expect(evaluations[0]?.params).toMatchObject({
|
||||
callFrameId: "seed-frame",
|
||||
expression: expect.stringContaining('"local-seed","local-value"'),
|
||||
});
|
||||
expect(evaluations[0]?.params).toMatchObject({
|
||||
expression: expect.stringContaining('"session-seed","session-value"'),
|
||||
});
|
||||
expect(
|
||||
cdp.calls.findIndex(
|
||||
({ method }) => method === "Debugger.evaluateOnCallFrame",
|
||||
),
|
||||
).toBeLessThan(
|
||||
cdp.calls.findIndex(({ method }) => method === "Debugger.resume"),
|
||||
);
|
||||
await owner?.close();
|
||||
});
|
||||
|
||||
it("seeds session storage once per namespace and StorageKey", async () => {
|
||||
const cdp = new FakeCdp();
|
||||
const { page, contextApi } = fixture(cdp);
|
||||
const base = scenario();
|
||||
const input = browserScenarioSchema.parse({
|
||||
...base,
|
||||
storage: {
|
||||
local_storage: [],
|
||||
session_storage: base.storage.session_storage,
|
||||
},
|
||||
});
|
||||
const owner = await createInitializedStorage(contextApi, page, input);
|
||||
|
||||
emitSeedPause(cdp);
|
||||
await owner?.settle();
|
||||
emitSeedPause(cdp);
|
||||
await owner?.settle();
|
||||
cdp.storageKey = "https://storage.example.test/^0https://top-level.example";
|
||||
emitSeedPause(cdp);
|
||||
await owner?.settle();
|
||||
|
||||
expect(
|
||||
cdp.calls.filter(({ method }) => method === "Debugger.evaluateOnCallFrame"),
|
||||
).toHaveLength(2);
|
||||
await owner?.close();
|
||||
});
|
||||
|
||||
it("retries the owning CDP detach after a failed attempt", async () => {
|
||||
const cdp = new FakeCdp();
|
||||
const { page, contextApi } = fixture(cdp);
|
||||
const owner = await createInitializedStorage(contextApi, page);
|
||||
cdp.failDetachCount = 1;
|
||||
|
||||
await expect(owner?.close()).rejects.toMatchObject({
|
||||
_tag: "BrowserObservationError",
|
||||
reason: "cleanup_failed",
|
||||
});
|
||||
expect(cdp.detachCalls).toBe(1);
|
||||
expect(cdp.listenerCount("Debugger.paused")).toBe(1);
|
||||
|
||||
await owner?.close();
|
||||
expect(cdp.detachCalls).toBe(2);
|
||||
expect(cdp.listenerCount("Debugger.paused")).toBe(0);
|
||||
});
|
||||
|
||||
it("does not create storage instrumentation when there are no storage seeds", async () => {
|
||||
const cdp = new FakeCdp();
|
||||
const { page, contextApi } = fixture(cdp);
|
||||
const empty = browserScenarioSchema.parse({
|
||||
browser: { mode: "launch", executable_path: process.execPath },
|
||||
start_url: { url: "https://storage.example.test/" },
|
||||
actions: [{ step_id: "wait", action: "wait_for_timeout", duration_ms: 1 }],
|
||||
});
|
||||
expect(
|
||||
await PlaywrightScenarioStorage.create(
|
||||
{ context: contextApi as never, page: page as never },
|
||||
empty,
|
||||
BrowserScenarioSecrets.resolve(empty, {})!,
|
||||
),
|
||||
).toBeUndefined();
|
||||
expect(cdp.calls).toEqual([]);
|
||||
});
|
||||
|
||||
it("detaches after setup cancellation even while script installation is pending", async () => {
|
||||
const cdp = new FakeCdp();
|
||||
cdp.holdMethod = "Page.addScriptToEvaluateOnNewDocument";
|
||||
const { page, contextApi } = fixture(cdp);
|
||||
const input = scenario();
|
||||
const controller = new AbortController();
|
||||
const owner = await PlaywrightScenarioStorage.create(
|
||||
{ context: contextApi as never, page: page as never },
|
||||
input,
|
||||
BrowserScenarioSecrets.resolve(input, {})!,
|
||||
{ signal: controller.signal },
|
||||
);
|
||||
expect(owner).toBeDefined();
|
||||
const initialization = owner?.initialize(page as never, contextApi as never);
|
||||
await waitForCall(cdp, "Page.addScriptToEvaluateOnNewDocument");
|
||||
controller.abort();
|
||||
await expect(initialization).rejects.toMatchObject({
|
||||
_tag: "AnalysisCancelledError",
|
||||
});
|
||||
|
||||
await owner?.close();
|
||||
expect(cdp.detachCalls).toBe(1);
|
||||
expect(
|
||||
cdp.calls.some(
|
||||
({ method }) => method === "Page.removeScriptToEvaluateOnNewDocument",
|
||||
),
|
||||
).toBe(false);
|
||||
cdp.releaseHeldReply();
|
||||
});
|
||||
|
||||
it("cancels a pending storage-key reply and still releases its CDP owner", async () => {
|
||||
const cdp = new FakeCdp();
|
||||
const { page, contextApi } = fixture(cdp);
|
||||
const input = scenario();
|
||||
const controller = new AbortController();
|
||||
const owner = await createInitializedStorage(contextApi, page, input, {
|
||||
signal: controller.signal,
|
||||
});
|
||||
expect(owner).toBeDefined();
|
||||
cdp.holdMethod = "Storage.getStorageKey";
|
||||
emitSeedPause(cdp);
|
||||
const settling = owner?.settle();
|
||||
await waitForCall(cdp, "Storage.getStorageKey");
|
||||
controller.abort();
|
||||
await expect(settling).rejects.toMatchObject({
|
||||
_tag: "AnalysisCancelledError",
|
||||
});
|
||||
|
||||
cdp.releaseHeldReply();
|
||||
await owner?.close();
|
||||
expect(cdp.detachCalls).toBe(1);
|
||||
});
|
||||
|
||||
it("releases the CDP owner when an attached target reply remains pending", async () => {
|
||||
const cdp = new FakeCdp();
|
||||
const { page, contextApi } = fixture(cdp);
|
||||
const input = scenario();
|
||||
const owner = await createInitializedStorage(contextApi, page, input);
|
||||
expect(owner).toBeDefined();
|
||||
cdp.emit("Target.attachedToTarget", {
|
||||
sessionId: "iframe-session",
|
||||
targetInfo: { type: "iframe", targetId: "iframe-target" },
|
||||
});
|
||||
await waitForCall(cdp, "Target.sendMessageToTarget");
|
||||
await owner?.close();
|
||||
expect(cdp.detachCalls).toBe(1);
|
||||
});
|
||||
|
||||
it("does not fail root storage capture when an OOPIF detaches during setup", async () => {
|
||||
const cdp = new FakeCdp();
|
||||
const { page, contextApi } = fixture(cdp);
|
||||
const owner = await createInitializedStorage(contextApi, page);
|
||||
expect(owner).toBeDefined();
|
||||
cdp.emit("Target.attachedToTarget", {
|
||||
sessionId: "navigated-away-iframe",
|
||||
targetInfo: { type: "iframe", targetId: "iframe-target" },
|
||||
});
|
||||
await waitForCall(cdp, "Target.sendMessageToTarget");
|
||||
|
||||
cdp.emit("Target.detachedFromTarget", {
|
||||
sessionId: "navigated-away-iframe",
|
||||
});
|
||||
await owner?.settle();
|
||||
|
||||
cdp.autoReplyNested = true;
|
||||
emitSeedPause(cdp);
|
||||
await owner?.settle();
|
||||
expect(
|
||||
cdp.calls.some(({ method }) => method === "Debugger.evaluateOnCallFrame"),
|
||||
).toBe(true);
|
||||
await owner?.close();
|
||||
expect(cdp.detachCalls).toBe(1);
|
||||
});
|
||||
|
||||
it("retains the exact in-flight detach after the bounded wait expires", async () => {
|
||||
const cdp = new FakeCdp();
|
||||
const { page, contextApi } = fixture(cdp);
|
||||
const owner = await createInitializedStorage(contextApi, page);
|
||||
expect(owner).toBeDefined();
|
||||
cdp.holdDetach = true;
|
||||
|
||||
await expect(owner?.close()).rejects.toMatchObject({
|
||||
_tag: "BrowserObservationError",
|
||||
reason: "cleanup_failed",
|
||||
});
|
||||
expect(cdp.detachCalls).toBe(1);
|
||||
const retry = owner?.close();
|
||||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||||
expect(cdp.detachCalls).toBe(1);
|
||||
cdp.releaseDetach();
|
||||
await retry;
|
||||
expect(cdp.detachCalls).toBe(1);
|
||||
});
|
||||
@@ -0,0 +1,678 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
|
||||
import { z } from "zod";
|
||||
import type { BrowserContext, CDPSession, Page } from "playwright-core";
|
||||
|
||||
import type { BrowserScenario } from "../domain/browserScenario.js";
|
||||
import { BrowserObservationError } from "../domain/browserObservationError.js";
|
||||
import type { BrowserScenarioSecrets } from "./BrowserScenarioSecrets.js";
|
||||
import { withPlaywrightExecutionBoundary } from "./PlaywrightExecutionBoundary.js";
|
||||
|
||||
const breakpointResponse = z.object({ breakpointId: z.string() });
|
||||
const scriptIdentifierResponse = z.object({ identifier: z.string() });
|
||||
const storageKeyResponse = z.object({ storageKey: z.string() });
|
||||
const targetInfoResponse = z.object({
|
||||
targetInfo: z.object({ targetId: z.string() }),
|
||||
});
|
||||
type StorageEventMethod =
|
||||
| "Runtime.executionContextCreated"
|
||||
| "Runtime.executionContextDestroyed"
|
||||
| "Runtime.executionContextsCleared"
|
||||
| "Debugger.scriptParsed"
|
||||
| "Debugger.paused"
|
||||
| "Target.attachedToTarget"
|
||||
| "Target.detachedFromTarget"
|
||||
| "Target.receivedMessageFromTarget";
|
||||
const STORAGE_EVENT_METHODS = new Set<string>([
|
||||
"Runtime.executionContextCreated",
|
||||
"Runtime.executionContextDestroyed",
|
||||
"Runtime.executionContextsCleared",
|
||||
"Debugger.scriptParsed",
|
||||
"Debugger.paused",
|
||||
"Target.attachedToTarget",
|
||||
"Target.detachedFromTarget",
|
||||
"Target.receivedMessageFromTarget",
|
||||
]);
|
||||
const attachedTargetEvent = z.object({
|
||||
sessionId: z.string(),
|
||||
targetInfo: z.object({ type: z.string(), targetId: z.string() }),
|
||||
});
|
||||
const targetSessionEvent = z.object({ sessionId: z.string() });
|
||||
const nestedMessageEvent = z.object({
|
||||
sessionId: z.string(),
|
||||
message: z.string(),
|
||||
});
|
||||
const nestedProtocolMessage = z.object({
|
||||
id: z.number().optional(),
|
||||
method: z.string().optional(),
|
||||
params: z.unknown().optional(),
|
||||
result: z.unknown().optional(),
|
||||
error: z.object({ message: z.string() }).optional(),
|
||||
});
|
||||
const executionContextEvent = z.object({
|
||||
context: z.object({
|
||||
id: z.number(),
|
||||
origin: z.string(),
|
||||
name: z.string(),
|
||||
auxData: z.object({ frameId: z.string().optional() }).optional(),
|
||||
}),
|
||||
});
|
||||
const scriptParsedEvent = z.object({
|
||||
scriptId: z.string(),
|
||||
executionContextId: z.number(),
|
||||
url: z.string(),
|
||||
});
|
||||
const pausedEvent = z.object({
|
||||
callFrames: z.array(
|
||||
z.object({
|
||||
callFrameId: z.string(),
|
||||
location: z.object({ scriptId: z.string() }),
|
||||
}),
|
||||
),
|
||||
});
|
||||
const evaluationResponse = z.object({
|
||||
exceptionDetails: z.unknown().optional(),
|
||||
});
|
||||
const STORAGE_CLEANUP_TIMEOUT_MS = 1_000;
|
||||
|
||||
type StorageSeed = {
|
||||
local: [string, string][];
|
||||
session: [string, string][];
|
||||
};
|
||||
|
||||
interface StorageTarget {
|
||||
readonly parentId: string | undefined;
|
||||
readonly topLevelTargetId: string;
|
||||
readonly send: (
|
||||
method: string,
|
||||
params?: Record<string, unknown>,
|
||||
) => Promise<unknown>;
|
||||
readonly contexts: Map<
|
||||
number,
|
||||
z.infer<typeof executionContextEvent>["context"]
|
||||
>;
|
||||
readonly scripts: Map<string, number>;
|
||||
ready: Promise<void>;
|
||||
waitingForDebugger: boolean;
|
||||
}
|
||||
|
||||
type PendingReply = {
|
||||
readonly sessionId: string;
|
||||
readonly resolve: (value: unknown) => void;
|
||||
readonly reject: (error: Error) => void;
|
||||
};
|
||||
type StorageInstallationOptions = {
|
||||
readonly signal?: AbortSignal;
|
||||
readonly retainCleanup?: (close: () => Promise<unknown>) => void;
|
||||
};
|
||||
type StoragePage = { readonly context: BrowserContext; readonly page: Page };
|
||||
|
||||
/** Owns pre-script storage seeding and its CDP instrumentation for one scenario. */
|
||||
export class PlaywrightScenarioStorage {
|
||||
readonly #seeds = new Map<string, StorageSeed>();
|
||||
readonly #initializedLocal = new Set<string>();
|
||||
readonly #initializedSession = new Set<string>();
|
||||
readonly #targets = new Map<string, StorageTarget>();
|
||||
readonly #replies = new Map<number, PendingReply>();
|
||||
readonly #sourceUrl = `rea-storage-${randomUUID()}`;
|
||||
readonly #listeners: {
|
||||
readonly method: StorageEventMethod;
|
||||
readonly listener: (params: unknown) => void;
|
||||
}[] = [];
|
||||
#sequence = 0;
|
||||
#generation = 0;
|
||||
#closed = false;
|
||||
#cleanupCompleted = false;
|
||||
#failure: BrowserObservationError | undefined;
|
||||
#closePromise: Promise<void> | undefined;
|
||||
#detachPromise: Promise<void> | undefined;
|
||||
#rootInitialization: Promise<void> | undefined;
|
||||
#work: Promise<void> = Promise.resolve();
|
||||
#removeListeners: () => void = () => undefined;
|
||||
|
||||
private constructor(
|
||||
private readonly cdp: CDPSession,
|
||||
private readonly signal: AbortSignal | undefined,
|
||||
) {}
|
||||
|
||||
static async create(
|
||||
storagePage: StoragePage,
|
||||
scenario: BrowserScenario,
|
||||
secrets: BrowserScenarioSecrets,
|
||||
options: StorageInstallationOptions = {},
|
||||
): Promise<PlaywrightScenarioStorage | undefined> {
|
||||
const { signal, retainCleanup } = options;
|
||||
const { context, page } = storagePage;
|
||||
if (
|
||||
scenario.storage.local_storage.length === 0 &&
|
||||
scenario.storage.session_storage.length === 0
|
||||
)
|
||||
return undefined;
|
||||
|
||||
const connection = context.newCDPSession(page);
|
||||
let cdp: CDPSession;
|
||||
try {
|
||||
cdp = await withPlaywrightExecutionBoundary(
|
||||
() => connection,
|
||||
undefined,
|
||||
signal,
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
if (signal?.aborted === true) {
|
||||
let detached = false;
|
||||
const lateCleanup = async (): Promise<void> => {
|
||||
let session: CDPSession;
|
||||
try {
|
||||
session = await connection;
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
if (!detached) {
|
||||
await session.detach();
|
||||
detached = true;
|
||||
}
|
||||
};
|
||||
retainCleanup?.(lateCleanup);
|
||||
void withPlaywrightExecutionBoundary(
|
||||
lateCleanup,
|
||||
STORAGE_CLEANUP_TIMEOUT_MS,
|
||||
).catch(() => undefined);
|
||||
}
|
||||
throw cause;
|
||||
}
|
||||
const owner = new PlaywrightScenarioStorage(cdp, signal);
|
||||
owner.addSeeds(scenario.storage.local_storage, "local", secrets);
|
||||
owner.addSeeds(scenario.storage.session_storage, "session", secrets);
|
||||
return owner;
|
||||
}
|
||||
|
||||
async initialize(page: Page, context: BrowserContext): Promise<void> {
|
||||
const initialization = this.initializeRoot(page, context);
|
||||
this.#rootInitialization = initialization;
|
||||
await withPlaywrightExecutionBoundary(
|
||||
() => initialization,
|
||||
undefined,
|
||||
this.signal,
|
||||
);
|
||||
}
|
||||
|
||||
private async initializeRoot(
|
||||
page: Page,
|
||||
context: BrowserContext,
|
||||
): Promise<void> {
|
||||
const rootInfo = targetInfoResponse.parse(
|
||||
await this.cdp.send("Target.getTargetInfo"),
|
||||
);
|
||||
const root = this.createTarget(
|
||||
undefined,
|
||||
rootInfo.targetInfo.targetId,
|
||||
(method, params) =>
|
||||
this.cdp.send(method as Parameters<CDPSession["send"]>[0], params),
|
||||
);
|
||||
this.#targets.set("", root);
|
||||
if (this.#closed) return;
|
||||
this.listen(page, context);
|
||||
const ready = this.initializeTarget(root);
|
||||
root.ready = ready;
|
||||
await ready;
|
||||
}
|
||||
|
||||
private addSeeds(
|
||||
blocks: BrowserScenario["storage"]["local_storage"],
|
||||
kind: "local" | "session",
|
||||
secrets: BrowserScenarioSecrets,
|
||||
): void {
|
||||
for (const { origin, entries } of blocks) {
|
||||
const seed = this.#seeds.get(origin) ?? { local: [], session: [] };
|
||||
seed[kind].push(
|
||||
...entries.map(({ name, value }): [string, string] => [
|
||||
name,
|
||||
secrets.value(value),
|
||||
]),
|
||||
);
|
||||
this.#seeds.set(origin, seed);
|
||||
}
|
||||
}
|
||||
|
||||
private listen(page: Page, context: BrowserContext): void {
|
||||
const browser = context.browser();
|
||||
const detached = (): void => this.targetDetached("");
|
||||
page.on("close", detached);
|
||||
context.on("close", detached);
|
||||
browser?.on("disconnected", detached);
|
||||
this.#removeListeners = () => {
|
||||
page.off("close", detached);
|
||||
context.off("close", detached);
|
||||
browser?.off("disconnected", detached);
|
||||
};
|
||||
for (const method of [
|
||||
"Runtime.executionContextCreated",
|
||||
"Runtime.executionContextDestroyed",
|
||||
"Runtime.executionContextsCleared",
|
||||
"Debugger.scriptParsed",
|
||||
"Debugger.paused",
|
||||
"Target.attachedToTarget",
|
||||
"Target.detachedFromTarget",
|
||||
"Target.receivedMessageFromTarget",
|
||||
] as const) {
|
||||
const listener = (params: unknown): void =>
|
||||
this.event("", method, params);
|
||||
this.#listeners.push({ method, listener });
|
||||
this.cdp.on(method, listener);
|
||||
}
|
||||
}
|
||||
|
||||
private createTarget(
|
||||
parentId: string | undefined,
|
||||
topLevelTargetId: string,
|
||||
send: StorageTarget["send"],
|
||||
): StorageTarget {
|
||||
return {
|
||||
parentId,
|
||||
topLevelTargetId,
|
||||
send,
|
||||
contexts: new Map(),
|
||||
scripts: new Map(),
|
||||
ready: Promise.resolve(),
|
||||
waitingForDebugger: false,
|
||||
};
|
||||
}
|
||||
|
||||
private async initializeTarget(target: StorageTarget): Promise<void> {
|
||||
if (this.#closed) return;
|
||||
await target.send("Page.enable");
|
||||
if (this.#closed) return;
|
||||
await target.send("Runtime.enable");
|
||||
if (this.#closed) return;
|
||||
await target.send("Debugger.enable");
|
||||
if (this.#closed) return;
|
||||
breakpointResponse.parse(
|
||||
await target.send("Debugger.setBreakpointByUrl", {
|
||||
url: this.#sourceUrl,
|
||||
lineNumber: 1,
|
||||
}),
|
||||
);
|
||||
if (this.#closed) return;
|
||||
scriptIdentifierResponse.parse(
|
||||
await target.send("Page.addScriptToEvaluateOnNewDocument", {
|
||||
worldName: this.#sourceUrl,
|
||||
source: `(() => {\n void 0;\n})()\n//# sourceURL=${this.#sourceUrl}`,
|
||||
}),
|
||||
);
|
||||
if (this.#closed) return;
|
||||
await target.send("Target.setAutoAttach", {
|
||||
autoAttach: true,
|
||||
waitForDebuggerOnStart: true,
|
||||
flatten: false,
|
||||
});
|
||||
}
|
||||
|
||||
private sendObserved(
|
||||
target: StorageTarget,
|
||||
method: string,
|
||||
params?: Record<string, unknown>,
|
||||
): Promise<unknown> {
|
||||
return withPlaywrightExecutionBoundary(
|
||||
() => target.send(method, params),
|
||||
undefined,
|
||||
this.signal,
|
||||
);
|
||||
}
|
||||
|
||||
private event(
|
||||
sessionId: string,
|
||||
method: StorageEventMethod,
|
||||
params: unknown,
|
||||
): void {
|
||||
if (this.#closed) return;
|
||||
const target = this.#targets.get(sessionId);
|
||||
if (target === undefined) return;
|
||||
try {
|
||||
if (method === "Runtime.executionContextCreated") {
|
||||
const { context } = executionContextEvent.parse(params);
|
||||
if (context.name === this.#sourceUrl)
|
||||
target.contexts.set(context.id, context);
|
||||
} else if (method === "Runtime.executionContextDestroyed") {
|
||||
const { executionContextId } = z
|
||||
.object({ executionContextId: z.number() })
|
||||
.parse(params);
|
||||
target.contexts.delete(executionContextId);
|
||||
} else if (method === "Runtime.executionContextsCleared") {
|
||||
target.contexts.clear();
|
||||
target.scripts.clear();
|
||||
} else if (method === "Debugger.scriptParsed") {
|
||||
const script = scriptParsedEvent.parse(params);
|
||||
if (script.url === this.#sourceUrl)
|
||||
target.scripts.set(script.scriptId, script.executionContextId);
|
||||
} else if (method === "Debugger.paused") {
|
||||
const pause = pausedEvent.parse(params);
|
||||
const frame = pause.callFrames[0];
|
||||
// CDP resume is target-wide, so identify our private seed frame before
|
||||
// doing any work that will resume the paused target.
|
||||
if (frame === undefined || !target.scripts.has(frame.location.scriptId))
|
||||
return;
|
||||
this.#generation += 1;
|
||||
this.#work = this.#work
|
||||
.then(() => this.seed(target, pause))
|
||||
.catch((cause: unknown) => {
|
||||
if (sessionId === "" || this.#targets.get(sessionId) === target)
|
||||
this.fail(cause);
|
||||
});
|
||||
} else if (method === "Target.attachedToTarget") {
|
||||
this.#generation += 1;
|
||||
this.attach(sessionId, attachedTargetEvent.parse(params));
|
||||
} else if (method === "Target.detachedFromTarget") {
|
||||
this.#generation += 1;
|
||||
this.targetDetached(targetSessionEvent.parse(params).sessionId);
|
||||
} else if (method === "Target.receivedMessageFromTarget") {
|
||||
this.nestedMessage(nestedMessageEvent.parse(params));
|
||||
}
|
||||
} catch (cause: unknown) {
|
||||
this.fail(cause);
|
||||
}
|
||||
}
|
||||
|
||||
private nestedMessage(event: z.infer<typeof nestedMessageEvent>): void {
|
||||
const reply = nestedProtocolMessage.parse(JSON.parse(event.message));
|
||||
if (reply.id !== undefined) {
|
||||
const waiter = this.#replies.get(reply.id);
|
||||
if (waiter?.sessionId !== event.sessionId) return;
|
||||
this.#replies.delete(reply.id);
|
||||
if (reply.error !== undefined)
|
||||
waiter.reject(new Error(reply.error.message));
|
||||
else waiter.resolve(reply.result);
|
||||
return;
|
||||
}
|
||||
if (reply.method !== undefined && STORAGE_EVENT_METHODS.has(reply.method))
|
||||
this.event(
|
||||
event.sessionId,
|
||||
reply.method as StorageEventMethod,
|
||||
reply.params,
|
||||
);
|
||||
}
|
||||
|
||||
private fail(cause: unknown): void {
|
||||
this.#generation += 1;
|
||||
this.#failure ??= new BrowserObservationError(
|
||||
"capture_browser_scenario",
|
||||
"protocol_error",
|
||||
{ cause },
|
||||
);
|
||||
}
|
||||
|
||||
private async seedPausedFrame(
|
||||
target: StorageTarget,
|
||||
frame: z.infer<typeof pausedEvent>["callFrames"][number],
|
||||
): Promise<void> {
|
||||
const contextId = target.scripts.get(frame.location.scriptId);
|
||||
const context =
|
||||
contextId === undefined ? undefined : target.contexts.get(contextId);
|
||||
const seed =
|
||||
context === undefined ? undefined : this.#seeds.get(context.origin);
|
||||
if (
|
||||
this.#closed ||
|
||||
seed === undefined ||
|
||||
context?.auxData?.frameId === undefined
|
||||
)
|
||||
return;
|
||||
|
||||
const storageKey = storageKeyResponse.parse(
|
||||
await this.sendObserved(target, "Storage.getStorageKey", {
|
||||
frameId: context.auxData.frameId,
|
||||
}),
|
||||
).storageKey;
|
||||
const sessionKey = JSON.stringify([target.topLevelTargetId, storageKey]);
|
||||
const local =
|
||||
this.#initializedLocal.has(storageKey) || seed.local.length === 0
|
||||
? []
|
||||
: seed.local;
|
||||
const session =
|
||||
this.#initializedSession.has(sessionKey) || seed.session.length === 0
|
||||
? []
|
||||
: seed.session;
|
||||
if (local.length === 0 && session.length === 0) return;
|
||||
|
||||
const payload = JSON.stringify({ local, session }).replaceAll(
|
||||
"<",
|
||||
"\\u003c",
|
||||
);
|
||||
const response = evaluationResponse.parse(
|
||||
await this.sendObserved(target, "Debugger.evaluateOnCallFrame", {
|
||||
callFrameId: frame.callFrameId,
|
||||
expression: `(() => { const seed = ${payload}; for (const [name, value] of seed.local) localStorage.setItem(name, value); for (const [name, value] of seed.session) sessionStorage.setItem(name, value); })()`,
|
||||
returnByValue: true,
|
||||
}),
|
||||
);
|
||||
if (response.exceptionDetails !== undefined)
|
||||
throw new BrowserObservationError(
|
||||
"capture_browser_scenario",
|
||||
"protocol_error",
|
||||
{ detail: "The browser denied initial storage initialization." },
|
||||
);
|
||||
if (local.length > 0) this.#initializedLocal.add(storageKey);
|
||||
if (session.length > 0) this.#initializedSession.add(sessionKey);
|
||||
}
|
||||
|
||||
private async seed(
|
||||
target: StorageTarget,
|
||||
pause: z.infer<typeof pausedEvent>,
|
||||
): Promise<void> {
|
||||
const frame = pause.callFrames[0];
|
||||
if (frame === undefined) return;
|
||||
|
||||
let primaryFailure: unknown;
|
||||
let hasPrimaryFailure = false;
|
||||
try {
|
||||
await this.seedPausedFrame(target, frame);
|
||||
} catch (cause: unknown) {
|
||||
primaryFailure = cause;
|
||||
hasPrimaryFailure = true;
|
||||
}
|
||||
let resumeFailure: unknown;
|
||||
try {
|
||||
await withPlaywrightExecutionBoundary(
|
||||
() => target.send("Debugger.resume"),
|
||||
STORAGE_CLEANUP_TIMEOUT_MS,
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
resumeFailure = cause;
|
||||
}
|
||||
if (hasPrimaryFailure && resumeFailure !== undefined)
|
||||
throw new AggregateError(
|
||||
[primaryFailure, resumeFailure],
|
||||
"Storage initialization and debugger resume both failed",
|
||||
{ cause: primaryFailure },
|
||||
);
|
||||
if (this.signal?.aborted === true && resumeFailure === undefined) return;
|
||||
if (hasPrimaryFailure) throw primaryFailure;
|
||||
if (resumeFailure !== undefined) throw resumeFailure;
|
||||
}
|
||||
|
||||
private attach(
|
||||
parentId: string,
|
||||
event: z.infer<typeof attachedTargetEvent>,
|
||||
): void {
|
||||
const parent = this.#targets.get(parentId);
|
||||
if (parent === undefined) return;
|
||||
const { sessionId, targetInfo } = event;
|
||||
const target = this.createTarget(
|
||||
parentId,
|
||||
event.targetInfo.type === "iframe"
|
||||
? parent.topLevelTargetId
|
||||
: event.targetInfo.targetId,
|
||||
async (method, params = {}) => {
|
||||
const id = ++this.#sequence;
|
||||
const reply = new Promise<unknown>((resolve, reject) =>
|
||||
this.#replies.set(id, { sessionId, resolve, reject }),
|
||||
);
|
||||
try {
|
||||
const sent = parent.send("Target.sendMessageToTarget", {
|
||||
sessionId,
|
||||
message: JSON.stringify({ id, method, params }),
|
||||
});
|
||||
const [, result] = await Promise.all([sent, reply]);
|
||||
return result;
|
||||
} finally {
|
||||
this.#replies.delete(id);
|
||||
}
|
||||
},
|
||||
);
|
||||
target.waitingForDebugger = true;
|
||||
this.#targets.set(sessionId, target);
|
||||
target.ready = (async () => {
|
||||
try {
|
||||
if (
|
||||
!this.#closed &&
|
||||
(targetInfo.type === "iframe" || targetInfo.type === "page")
|
||||
)
|
||||
await this.initializeTarget(target);
|
||||
} catch (cause: unknown) {
|
||||
if (
|
||||
this.signal?.aborted !== true &&
|
||||
!this.#closed &&
|
||||
this.#targets.get(sessionId) === target
|
||||
)
|
||||
this.fail(cause);
|
||||
}
|
||||
if (this.#targets.get(sessionId) !== target) return;
|
||||
try {
|
||||
await withPlaywrightExecutionBoundary(
|
||||
() => target.send("Runtime.runIfWaitingForDebugger"),
|
||||
STORAGE_CLEANUP_TIMEOUT_MS,
|
||||
);
|
||||
target.waitingForDebugger = false;
|
||||
} catch (cause: unknown) {
|
||||
if (
|
||||
this.signal?.aborted !== true &&
|
||||
!this.#closed &&
|
||||
this.#targets.get(sessionId) === target
|
||||
)
|
||||
this.fail(cause);
|
||||
}
|
||||
})();
|
||||
}
|
||||
|
||||
private targetDetached(sessionId: string): void {
|
||||
for (const [id, target] of this.#targets)
|
||||
if (target.parentId === sessionId) this.targetDetached(id);
|
||||
this.#targets.delete(sessionId);
|
||||
for (const [id, reply] of this.#replies)
|
||||
if (reply.sessionId === sessionId) {
|
||||
this.#replies.delete(id);
|
||||
reply.reject(
|
||||
new BrowserObservationError(
|
||||
"capture_browser_scenario",
|
||||
"disconnected",
|
||||
),
|
||||
);
|
||||
}
|
||||
this.#generation += 1;
|
||||
}
|
||||
|
||||
async settle(): Promise<void> {
|
||||
for (;;) {
|
||||
const generation = this.#generation;
|
||||
const work = this.#work;
|
||||
const targets = [...this.#targets.entries()];
|
||||
await withPlaywrightExecutionBoundary(
|
||||
() => Promise.all(targets.map(([, target]) => target.ready)),
|
||||
undefined,
|
||||
this.signal,
|
||||
);
|
||||
await withPlaywrightExecutionBoundary(() => work, undefined, this.signal);
|
||||
await Promise.resolve();
|
||||
if (this.#failure !== undefined) throw this.#failure;
|
||||
if (
|
||||
generation === this.#generation &&
|
||||
work === this.#work &&
|
||||
targets.every(([id, target]) => this.#targets.get(id) === target)
|
||||
)
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
close(stopLoading = false): Promise<void> {
|
||||
if (this.#cleanupCompleted) return Promise.resolve();
|
||||
if (this.#closePromise !== undefined) return this.#closePromise;
|
||||
const closing = this.closeResources(stopLoading).catch((cause: unknown) => {
|
||||
throw new BrowserObservationError(
|
||||
"capture_browser_scenario",
|
||||
"cleanup_failed",
|
||||
{
|
||||
cause,
|
||||
cleanup: {
|
||||
reason: cause instanceof Error ? cause.message : String(cause),
|
||||
resources: ["browser_transport", "browser_storage_bootstrap"],
|
||||
},
|
||||
},
|
||||
);
|
||||
});
|
||||
this.#closePromise = closing;
|
||||
void closing.catch(() => {
|
||||
if (this.#closePromise === closing) this.#closePromise = undefined;
|
||||
});
|
||||
return closing;
|
||||
}
|
||||
|
||||
private async closeResources(stopLoading: boolean): Promise<void> {
|
||||
this.#closed = true;
|
||||
let stopLoadingFailure: { readonly cause: unknown } | undefined;
|
||||
if (stopLoading) {
|
||||
try {
|
||||
const root = this.#targets.get("");
|
||||
if (root !== undefined)
|
||||
await withPlaywrightExecutionBoundary(
|
||||
() => root.send("Page.stopLoading"),
|
||||
STORAGE_CLEANUP_TIMEOUT_MS,
|
||||
);
|
||||
} catch (cause: unknown) {
|
||||
stopLoadingFailure = { cause };
|
||||
}
|
||||
}
|
||||
|
||||
const work = this.#work;
|
||||
const rootInitialization = this.#rootInitialization;
|
||||
const targets = [...this.#targets.values()];
|
||||
try {
|
||||
await withPlaywrightExecutionBoundary(
|
||||
() =>
|
||||
Promise.all([
|
||||
work.catch(() => undefined),
|
||||
...(rootInitialization === undefined
|
||||
? []
|
||||
: [rootInitialization.catch(() => undefined)]),
|
||||
...targets.map((target) => target.ready.catch(() => undefined)),
|
||||
]),
|
||||
STORAGE_CLEANUP_TIMEOUT_MS,
|
||||
);
|
||||
} catch {
|
||||
// Detaching the owning CDP session below is the terminal release for
|
||||
// scripts, breakpoints, debugger state, and attached child sessions.
|
||||
}
|
||||
|
||||
let detach = this.#detachPromise;
|
||||
if (detach === undefined) {
|
||||
detach = Promise.resolve().then(() => this.cdp.detach());
|
||||
this.#detachPromise = detach;
|
||||
void detach.catch(() => {
|
||||
if (this.#detachPromise === detach) this.#detachPromise = undefined;
|
||||
});
|
||||
}
|
||||
await withPlaywrightExecutionBoundary(
|
||||
() => detach,
|
||||
STORAGE_CLEANUP_TIMEOUT_MS,
|
||||
);
|
||||
|
||||
for (const { method, listener } of this.#listeners)
|
||||
this.cdp.off(method, listener);
|
||||
this.#removeListeners();
|
||||
for (const reply of this.#replies.values())
|
||||
reply.reject(
|
||||
new BrowserObservationError("capture_browser_scenario", "disconnected"),
|
||||
);
|
||||
this.#targets.clear();
|
||||
this.#replies.clear();
|
||||
this.#cleanupCompleted = true;
|
||||
if (stopLoadingFailure !== undefined) throw stopLoadingFailure.cause;
|
||||
}
|
||||
}
|
||||
@@ -21,6 +21,7 @@ import { safeParseJson } from "../domain/safeJson.js";
|
||||
import { jsonParts } from "../domain/jsonSerialization.js";
|
||||
import {
|
||||
inspectSourceMapValue,
|
||||
sourceMapJsonText,
|
||||
SourceMapFormatFailure,
|
||||
type SourceMapLeaf,
|
||||
} from "../javascript/sourceMaps/SourceMapFormat.js";
|
||||
@@ -364,7 +365,9 @@ const readBoundedText = async (
|
||||
offset += chunk.byteLength;
|
||||
}
|
||||
try {
|
||||
return new TextDecoder("utf-8", { fatal: true }).decode(joined);
|
||||
return new TextDecoder("utf-8", { fatal: true, ignoreBOM: true }).decode(
|
||||
joined,
|
||||
);
|
||||
} catch {
|
||||
throw new SourceMapEncodingError();
|
||||
}
|
||||
@@ -455,7 +458,7 @@ const normalizeSourceMap = (
|
||||
fetchedUrl: string,
|
||||
context: SourceMapDecodeContext,
|
||||
): SourceMapItem => {
|
||||
const parsedJson = safeParseJson(text);
|
||||
const parsedJson = safeParseJson(sourceMapJsonText(text));
|
||||
if (!parsedJson.ok)
|
||||
return emptySourceMapItem(
|
||||
request,
|
||||
|
||||
@@ -4,6 +4,10 @@ import { isAbsolute } from "node:path";
|
||||
import { ConfigurationError } from "../domain/configurationErrors.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { analysisProviderSelectorSchema } from "../contracts/providerSelection.js";
|
||||
import {
|
||||
ghidraCompilerSpecIdSchema,
|
||||
ghidraLanguageIdSchema,
|
||||
} from "./ghidraLanguageOverride.js";
|
||||
import { ghidraStartupTimeoutSchema } from "./ghidraStartupTimeout.js";
|
||||
import { mcpInputSchemaProfileSchema } from "./mcpInputSchemaProfile.js";
|
||||
import { mcpResponseBudgetSchema } from "./mcpResponseBudget.js";
|
||||
@@ -33,6 +37,13 @@ const environmentSchema = z.object({
|
||||
.refine(isAbsolute, "REA_GHIDRA_NATIVEAOT_JAR must be absolute")
|
||||
.optional(),
|
||||
REA_GHIDRA_STARTUP_TIMEOUT_MS: ghidraStartupTimeoutSchema,
|
||||
REA_GHIDRA_LANGUAGE_ID: ghidraLanguageIdSchema.optional(),
|
||||
REA_GHIDRA_COMPILER_SPEC_ID: ghidraCompilerSpecIdSchema.optional(),
|
||||
REA_GHIDRA_SEED_FILE: z
|
||||
.string()
|
||||
.min(1)
|
||||
.refine(isAbsolute, "REA_GHIDRA_SEED_FILE must be absolute")
|
||||
.optional(),
|
||||
REA_ILSPY_CMD_PATH: z
|
||||
.string()
|
||||
.min(1)
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
import { z } from "zod";
|
||||
|
||||
/**
|
||||
* Explicit Ghidra import language for targets whose header cannot identify the
|
||||
* producing compiler, such as Borland-built PE images.
|
||||
*/
|
||||
export interface GhidraLanguageOverride {
|
||||
/** Ghidra language ID: `processor:endian:size:variant`. */
|
||||
readonly languageId: string;
|
||||
/** Ghidra compiler-spec ID; absent selects the language's default spec. */
|
||||
readonly compilerSpecId?: string;
|
||||
}
|
||||
|
||||
// Ghidra language IDs are four colon-separated fields; variants may contain
|
||||
// spaces ("x86:LE:16:Real Mode") but never leading or trailing whitespace.
|
||||
export const ghidraLanguageIdSchema = z
|
||||
.string()
|
||||
.regex(
|
||||
/^[A-Za-z0-9_.+-]+:(?:LE|BE):[1-9]\d*:[A-Za-z0-9_.+-](?:[A-Za-z0-9_.+ -]*[A-Za-z0-9_.+-])?$/u,
|
||||
"must be a Ghidra language ID such as x86:LE:32:default",
|
||||
);
|
||||
|
||||
export const ghidraCompilerSpecIdSchema = z
|
||||
.string()
|
||||
.regex(
|
||||
/^[A-Za-z0-9_.+-]+$/u,
|
||||
"must be a Ghidra compiler-spec ID such as default or borlandcpp",
|
||||
);
|
||||
@@ -2,7 +2,7 @@ import { accessSync, constants } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { ConfigurationError } from "../domain/configurationErrors.js";
|
||||
import { ok, type Result } from "../domain/result.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { parseEnvironment } from "./environment.js";
|
||||
import { homeDirectoryFromEnvironment } from "./homeDirectory.js";
|
||||
import { parseStringArray, parseLoaderArgs } from "./parsers.js";
|
||||
@@ -46,6 +46,21 @@ export const parseConfig = (
|
||||
"REA_REFERENCE_SECRET_PATTERNS_JSON",
|
||||
);
|
||||
if (!secretPatterns.ok) return secretPatterns;
|
||||
// Ghidra's analyzeHeadless accepts -cspec only together with -processor.
|
||||
if (
|
||||
env.REA_GHIDRA_COMPILER_SPEC_ID !== undefined &&
|
||||
env.REA_GHIDRA_LANGUAGE_ID === undefined
|
||||
)
|
||||
return err(
|
||||
new ConfigurationError("Invalid REA environment configuration", {
|
||||
settings: [
|
||||
{
|
||||
setting: "REA_GHIDRA_COMPILER_SPEC_ID",
|
||||
constraint: "requires REA_GHIDRA_LANGUAGE_ID",
|
||||
},
|
||||
],
|
||||
}),
|
||||
);
|
||||
return ok({
|
||||
...(env.REA_MCP_MAX_RESPONSE_BYTES === undefined
|
||||
? {}
|
||||
@@ -60,6 +75,19 @@ export const parseConfig = (
|
||||
ghidraInstallDir: env.GHIDRA_INSTALL_DIR,
|
||||
ghidraJavaHome: env.JAVA_HOME,
|
||||
ghidraStartupTimeoutMs: env.REA_GHIDRA_STARTUP_TIMEOUT_MS,
|
||||
...(env.REA_GHIDRA_LANGUAGE_ID === undefined
|
||||
? {}
|
||||
: {
|
||||
ghidraLanguageOverride: {
|
||||
languageId: env.REA_GHIDRA_LANGUAGE_ID,
|
||||
...(env.REA_GHIDRA_COMPILER_SPEC_ID === undefined
|
||||
? {}
|
||||
: { compilerSpecId: env.REA_GHIDRA_COMPILER_SPEC_ID }),
|
||||
},
|
||||
}),
|
||||
...(env.REA_GHIDRA_SEED_FILE === undefined
|
||||
? {}
|
||||
: { ghidraSeedFile: env.REA_GHIDRA_SEED_FILE }),
|
||||
...(env.REA_GHIDRA_NATIVEAOT_JAR === undefined
|
||||
? {}
|
||||
: { ghidraNativeAotJar: env.REA_GHIDRA_NATIVEAOT_JAR }),
|
||||
|
||||
@@ -2,6 +2,7 @@ import type { LogLevel } from "../logger.js";
|
||||
import type { ReferenceSourcePolicy } from "../domain/referenceSourcePolicy.js";
|
||||
import type { AnalysisProviderSelector } from "../contracts/providerSelection.js";
|
||||
|
||||
import type { GhidraLanguageOverride } from "./ghidraLanguageOverride.js";
|
||||
import type { McpInputSchemaProfile } from "./mcpInputSchemaProfile.js";
|
||||
|
||||
export interface AppConfig {
|
||||
@@ -12,6 +13,8 @@ export interface AppConfig {
|
||||
readonly ghidraInstallDir: string | undefined;
|
||||
readonly ghidraJavaHome: string | undefined;
|
||||
readonly ghidraStartupTimeoutMs: number;
|
||||
readonly ghidraLanguageOverride?: GhidraLanguageOverride;
|
||||
readonly ghidraSeedFile?: string;
|
||||
readonly ghidraNativeAotJar?: string;
|
||||
readonly ilspyCmdPath: string | undefined;
|
||||
readonly hopperLauncherPath: string;
|
||||
|
||||
@@ -223,7 +223,7 @@ export const BROWSER_TOOL_CONTRACTS = [
|
||||
name: "compare_web_captures",
|
||||
...toolContractMetadata("compare_web_captures"),
|
||||
description:
|
||||
"Compare passive web captures by providing before and after, each with the complete inspect_web_page normalized_result in inspection and an optional complete discover_webmcp_tools normalized_result in webmcp; or compare recorded scenarios by passing complete capture_browser_scenario normalized_result objects in before_scenario and after_scenario with an optional normalization policy. Use exactly one input group. Scenario comparison aligns exact step IDs, records deterministic literal normalization, and returns every artifact-level difference plus alignment failures inline. Missing or truncated evidence never proves equality.",
|
||||
"Compare passive web captures by providing before and after, each with the complete inspect_web_page normalized_result in inspection and an optional complete discover_webmcp_tools normalized_result in webmcp; or compare recorded scenarios by passing complete capture_browser_scenario normalized_result objects in before_scenario and after_scenario with an optional normalization policy. Use exactly one input group. Network comparison includes observed HTTP requests and WebSocket frame streams; same-URL connections are compared as an unordered collection of ordered streams, retaining connection counts without claiming stable identity between individual connections. Scenario comparison aligns exact step IDs, records deterministic literal normalization, and returns every artifact-level difference plus alignment failures inline. Missing or truncated evidence never proves equality.",
|
||||
kind: "browser-provider",
|
||||
inputSchema: browserCaptureComparisonInputSchema,
|
||||
outputSchema: captureDiffOutputSchema,
|
||||
|
||||
@@ -36,7 +36,7 @@ const functionWorkflow = <Name extends FunctionWorkflowName>(
|
||||
export const FUNCTION_WORKFLOW_TOOL_CONTRACTS = [
|
||||
functionWorkflow(
|
||||
"analyze_function",
|
||||
"Build a dossier for one native function identified by symbol or provider-returned address. Returns identity with complete inclusive body ranges or explicit unknown, provider-specific pseudocode and assembly, comments, calls, references, referenced strings/names, local CFG blocks, and available native API boundary observations.",
|
||||
"Build a dossier for one native function identified by symbol or provider-returned address. Returns identity with complete inclusive analyzed body ranges or explicit unknown, provider-specific pseudocode and assembly, comments, calls, references, referenced strings/names, local CFG blocks, and available native API boundary observations. Ghidra terminal-call assumptions that may cut the analyzed body are reported in limitations with available callee identities.",
|
||||
),
|
||||
functionWorkflow(
|
||||
"inspect_native_api",
|
||||
|
||||
@@ -144,7 +144,7 @@ export const OFFICIAL_TOOL_CONTRACTS = [
|
||||
),
|
||||
official(
|
||||
"procedure_assembly",
|
||||
"Return assembly for one analyzed procedure identified by symbol or hexadecimal address. Use when pseudocode loses calling-convention or instruction-level detail; output is currently returned as one unpaginated string.",
|
||||
"Return assembly for one analyzed procedure identified by symbol or hexadecimal address. Use when pseudocode loses calling-convention or instruction-level detail; output is currently returned as one unpaginated string. Ghidra terminal-call assumptions that may cut the analyzed body are reported in Evidence limitations.",
|
||||
z.object({ procedure, document }),
|
||||
),
|
||||
official(
|
||||
@@ -159,12 +159,12 @@ export const OFFICIAL_TOOL_CONTRACTS = [
|
||||
),
|
||||
official(
|
||||
"procedure_info",
|
||||
"Return provider metadata for one procedure identified by symbol or address: entrypoint, signature, locals, size, block count, and complete inclusive body ranges when observed. Hopper locals preserve observed names and signed stack displacements; source types remain unknown. Body byte count and enclosing span remain distinct; unavailable extent is explicit.",
|
||||
"Return provider metadata for one procedure identified by symbol or address: entrypoint, signature, locals, size, block count, and complete inclusive body ranges when observed. Extents reflect the provider's analyzed body; Ghidra terminal-call assumptions that may cut it are reported in Evidence limitations. Hopper locals preserve observed names and signed stack displacements; source types remain unknown. Body byte count and enclosing span remain distinct; unavailable extent is explicit.",
|
||||
z.object({ procedure, document }),
|
||||
),
|
||||
official(
|
||||
"read_function_instructions",
|
||||
"Return every raw instruction for one analyzed procedure. Instruction text is provider-specific.",
|
||||
"Return every raw instruction for one analyzed procedure. Instruction text and body extent are provider-specific. Ghidra terminal-call assumptions that may cut the analyzed body are reported in limitations.",
|
||||
functionInstructionInputSchema,
|
||||
),
|
||||
...HOPPER_MEMORY_TOOL_DEFINITIONS.map(({ name, description, inputSchema }) =>
|
||||
@@ -181,7 +181,7 @@ export const OFFICIAL_TOOL_CONTRACTS = [
|
||||
),
|
||||
official(
|
||||
"procedure_pseudo_code",
|
||||
"Decompile one analyzed procedure by symbol name or provider-normalized address. Returns provider-specific pseudocode, never original source or cross-provider text equivalence, and may return null; request procedure_assembly when instruction precision matters.",
|
||||
"Decompile one analyzed procedure by symbol name or provider-normalized address. Returns provider-specific pseudocode, never original source or cross-provider text equivalence, and may return null; request procedure_assembly when instruction precision matters. Ghidra no-return assumptions may cut the analyzed body; Evidence limitations report observed terminal calls and available callee identities.",
|
||||
z.object({ procedure, document }),
|
||||
),
|
||||
official(
|
||||
@@ -196,7 +196,7 @@ export const OFFICIAL_TOOL_CONTRACTS = [
|
||||
),
|
||||
official(
|
||||
"search_strings",
|
||||
"Search every analyzed string object using literal matching by default or regex when requested. Results are deterministic and complete over the native inventory. Hopper retains typed byte decoding, native display, and explicit decoding unknowns. Adjacent long-literal fragments match independently. Hopper regex mode uses ECMAScript Unicode syntax in a cancellable worker; a matching deadline leaves its API usable.",
|
||||
"Search every provider-defined string object using literal matching by default or regex when requested. Results are deterministic and complete over those defined objects, not every text sequence in artifact bytes. Ghidra may omit packed or undefined literals, including Rust strings; an empty result does not prove text is absent. If an address is known, read_bytes and xrefs inspect its bytes and references. Hopper retains typed byte decoding, native display, and explicit decoding unknowns. Adjacent long-literal fragments match independently. Hopper regex mode uses ECMAScript Unicode syntax in a cancellable worker; a matching deadline leaves its API usable.",
|
||||
z.object(analysisSearchInput),
|
||||
),
|
||||
official(
|
||||
|
||||
@@ -94,7 +94,7 @@ export const SESSION_TOOL_CONTRACTS = [
|
||||
),
|
||||
session(
|
||||
"capture_process_scenario",
|
||||
"Run one caller-selected command under a PTY and return process capture Evidence with residual unknowns. The renderer admits at most 1,000,000 combined terminal cells, computed as columns × (rows + scrollback); every resize uses the selected scrollback and must fit the same budget. Supply a smaller terminal shape when that product is too large; dimensions are never clamped. A command name resolves through the inherited PATH; the working directory defaults to the caller's current directory; host environment variables are inherited with scenario overrides. Environment keys cannot contain '=' or NUL, and REA_PROCESS_RUN_ID is reserved for process ownership. Filesystem snapshots are opt-in through filesystem_observation_paths. The target runs with the current user's permissions; this is not a security sandbox.",
|
||||
"Run one caller-selected command under a PTY and return process capture Evidence with residual unknowns. The renderer admits at most 1,000,000 combined terminal cells, computed as columns × (rows + scrollback); every resize uses the selected scrollback and must fit the same budget. Supply a smaller terminal shape when that product is too large; dimensions are never clamped. A command name resolves through the inherited PATH; the working directory defaults to the caller's current directory; host environment variables are inherited with scenario overrides. Environment keys cannot contain '=' or NUL, and REA_PROCESS_RUN_ID is reserved for process ownership. Filesystem snapshots are opt-in through filesystem_observation_paths. A positive finalization_ms makes a timeout or idle deadline send SIGTERM to the captured root through its start identity, keep observing, and send SIGKILL once after the interval; exit.finalization records each attempt with its delivery result and the observed exit, and cancellation still ends the run at once. The target runs with the current user's permissions; this is not a security sandbox.",
|
||||
processScenarioSchema,
|
||||
),
|
||||
session(
|
||||
|
||||
@@ -58,6 +58,8 @@ export const TOOL_EXAMPLE_OVERRIDES: Readonly<
|
||||
capture_process_scenario: {
|
||||
executable: "/usr/bin/true",
|
||||
working_directory: "/tmp",
|
||||
timeout_ms: 30_000,
|
||||
finalization_ms: 1_500,
|
||||
},
|
||||
compare_process_captures: {
|
||||
left: PROCESS_CAPTURE_REFERENCE,
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user