Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
502d531f68 | ||
|
|
1ae9000587 | ||
|
|
5d58a98a1f | ||
|
|
02987d8e8c | ||
|
|
b1b0211c28 | ||
|
|
80592b82a6 | ||
|
|
b9b3a9098b | ||
|
|
1f122c02b7 | ||
|
|
bcb60c179e | ||
|
|
c6166d4b56 | ||
|
|
3a4d49406a | ||
|
|
7072f87baf | ||
|
|
1ebbc3b371 | ||
|
|
c86b09a109 | ||
|
|
7e9fb6f1bb | ||
|
|
d6271f0020 | ||
|
|
cd740a6764 | ||
|
|
22552525d5 | ||
|
|
0161a593e5 | ||
|
|
70c238e376 | ||
|
|
f0e86997aa | ||
|
|
3ae02f6a45 | ||
|
|
4acda94b76 | ||
|
|
4d98123279 | ||
|
|
6510d0d57f | ||
|
|
406c44063b | ||
|
|
aa6f74af04 | ||
|
|
1342eddd03 | ||
|
|
14fef73882 | ||
|
|
06739aeaf2 | ||
|
|
bee73fcb67 | ||
|
|
3620074b31 | ||
|
|
ddcdb3c378 | ||
|
|
a1a1c03da9 | ||
|
|
4aa51bfe19 | ||
|
|
2ad2a044d4 | ||
|
|
49e73fc03b | ||
|
|
0bf01dfa59 | ||
|
|
9630c8ff46 | ||
|
|
dd36b4a8a0 | ||
|
|
87a8c8c923 | ||
|
|
7a28dbeec6 | ||
|
|
3cb9d61ec1 | ||
|
|
4f025b61ae | ||
|
|
2320ac23ee | ||
|
|
4c83846947 | ||
|
|
cddf31d55e | ||
|
|
e1897c3c17 | ||
|
|
57649d3d8d | ||
|
|
ed24604ee1 | ||
|
|
20b802f165 | ||
|
|
23dc8fbf83 | ||
|
|
0c6c571a01 | ||
|
|
a9195c3a98 | ||
|
|
4758ba30b6 | ||
|
|
64041e3072 | ||
|
|
39f96d7188 | ||
|
|
6ef33162b8 | ||
|
|
8ee5ca17ee | ||
|
|
e6cf2e354e | ||
|
|
e5a5c17e52 | ||
|
|
a4634ba21b | ||
|
|
14c93a753f | ||
|
|
008c55c835 | ||
|
|
620ac0f5a7 | ||
|
|
0f24c61196 | ||
|
|
8924a3a06c | ||
|
|
5f46da90f9 | ||
|
|
f07446681c | ||
|
|
6f0237af0c | ||
|
|
3e88ebf735 | ||
|
|
073f6c07e9 | ||
|
|
d688507563 | ||
|
|
e5abf4acf4 | ||
|
|
6b167f9812 | ||
|
|
2a7982ce2b | ||
|
|
eac921eb44 | ||
|
|
3c36892ab7 | ||
|
|
56a90afdde | ||
|
|
78b8e28b67 | ||
|
|
fb10bdc546 |
@@ -31,6 +31,49 @@ Diagnose the observed blocker, continue useful supported work, and resume the
|
||||
original task after repair. Client approval, shell PATH, registration and engine
|
||||
availability have different remedies; avoid repeating setup for all of them.
|
||||
|
||||
1. Diagnose without changing REA or client configuration. `npx` may download
|
||||
the package and write to npm's cache. For Codex, run
|
||||
`npx -y rea-agents@latest doctor --client codex --json`. Substitute the current
|
||||
supported client: `claude_code`, `claude_desktop`, `codex`, `cursor`,
|
||||
`gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`, `copilot_cli`,
|
||||
`commandcode`, `vscode`, `grok_build`, `omp`, or `grok_bot`. `grok_bot` has no local
|
||||
file registration: doctor reports the Grok Bot chat step, and setup does not
|
||||
write `mcp.json`. If the client is unknown, use `doctor --json` and
|
||||
inspect its registration results before choosing a setup scope.
|
||||
2. Distinguish the reason. Missing, malformed, or stale registration needs a
|
||||
scoped configuration repair. An aligned registration with no tools in the
|
||||
active session needs a restart/reconnection; doctor cannot prove that the
|
||||
current agent has connected. A missing provider affects only tasks requiring
|
||||
that provider: static JavaScript inspection needs neither Hopper nor Ghidra,
|
||||
and Android inspection has separate bring-your-own JADX/Java prerequisites.
|
||||
3. For a configuration repair, prepare the read-only plan:
|
||||
`npx -y rea-agents@latest setup --client codex --dry-run --json`.
|
||||
Use the current client's ID, show its exact proposed paths, backups, and
|
||||
changes, and obtain approval before setup writes configuration or installs
|
||||
Hopper. Setup normally installs the matching bundled skill too; include that
|
||||
replacement in the reviewed plan. After approval, apply the same scope with
|
||||
`npx -y rea-agents@latest setup --client codex --yes`. Add `--install-hopper`
|
||||
only if that separate installation was needed and explicitly approved.
|
||||
4. Restart/reconnect the affected agent. Verify that REA tools actually appear
|
||||
in the session, then resume the original investigation. If they remain
|
||||
absent, inspect the client's MCP launch error rather than repeating setup.
|
||||
|
||||
REA setup never installs or upgrades Node.js, npm, Homebrew, Java, Ghidra, IDA,
|
||||
JADX, Binwalk, or Unblob. Use existing prerequisites; do not install unrelated
|
||||
software to repair
|
||||
MCP registration. For an unsupported client, use manual stdio registration
|
||||
with a version-pinned `rea-agents` package or continue through the CLI.
|
||||
|
||||
A concrete CLI fallback for an operator-supplied JavaScript tree or ASAR is:
|
||||
|
||||
```bash
|
||||
npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json
|
||||
```
|
||||
|
||||
No MCP registration or native engine is required. Read the returned Evidence,
|
||||
graph, limitations, and unknowns with the same care as an MCP result; the CLI
|
||||
returns the Evidence record directly. This fallback does not establish that
|
||||
MCP is configured. Native CLI tasks still require their selected engine.
|
||||
Prefer an existing `rea` CLI. When none is installed and REA package acquisition
|
||||
is within the authorized scope, use `npx -y rea-agents@latest` in its place.
|
||||
Configuration repair starts with a read-only setup plan; show paths, backups and
|
||||
|
||||
@@ -65,3 +65,7 @@ Use `inspect_analysis_view` with `view: {kind: "api-surface"}` on retained
|
||||
method, origin and path template. Format matches are inferences, not handler
|
||||
identity or schemas. Group pages cite parent record ordinals; consult record
|
||||
coverage and exclusions before claiming completeness. No requests are replayed.
|
||||
When the connected schema advertises `include_query_parameter_names`, select
|
||||
it in the view for distinct observed names with original record ordinals. Query
|
||||
values are omitted; the names establish neither required parameters nor a
|
||||
complete server schema. Keep the original parent Evidence for exact URLs.
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
name: Cache npm package downloads
|
||||
description: Restore npm's download cache and save it only from trusted default-branch events.
|
||||
outputs:
|
||||
can-save:
|
||||
description: Whether this run may update the shared npm download cache.
|
||||
value: ${{ steps.context.outputs.can-save }}
|
||||
path:
|
||||
description: The npm download-cache directory.
|
||||
value: ${{ steps.context.outputs.path }}
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- id: context
|
||||
shell: bash
|
||||
env:
|
||||
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
|
||||
LOCKFILES_HASH: ${{ hashFiles('package-lock.json', 'npm-shrinkwrap.json', '.npmrc') }}
|
||||
run: |
|
||||
cache_path="$(npm config get cache)"
|
||||
runtime_arch="$(node -p 'process.arch')"
|
||||
can_save=false
|
||||
if [[ -n "${DEFAULT_BRANCH}" && "${GITHUB_REF}" == "refs/heads/${DEFAULT_BRANCH}" && ( "${GITHUB_EVENT_NAME}" == "push" || "${GITHUB_EVENT_NAME}" == "schedule" || "${GITHUB_EVENT_NAME}" == "workflow_dispatch" ) ]]; then
|
||||
can_save=true
|
||||
fi
|
||||
cache_key="npm-download-${RUNNER_OS}-${runtime_arch}-${LOCKFILES_HASH}"
|
||||
{
|
||||
printf 'can-save=%s\n' "${can_save}"
|
||||
printf 'path=%s\n' "${cache_path}"
|
||||
printf 'key=%s\n' "${cache_key}"
|
||||
} >> "${GITHUB_OUTPUT}"
|
||||
- name: Restore and save npm package downloads
|
||||
if: steps.context.outputs.can-save == 'true'
|
||||
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
|
||||
with:
|
||||
path: ${{ steps.context.outputs.path }}
|
||||
key: ${{ steps.context.outputs.key }}
|
||||
- name: Restore npm package downloads without saving
|
||||
if: steps.context.outputs.can-save != 'true'
|
||||
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
|
||||
with:
|
||||
path: ${{ steps.context.outputs.path }}
|
||||
key: ${{ steps.context.outputs.key }}
|
||||
@@ -12,12 +12,13 @@ concurrency:
|
||||
permissions:
|
||||
contents: read
|
||||
actions: read
|
||||
pull-requests: read
|
||||
|
||||
jobs:
|
||||
report:
|
||||
name: Collect recent completed workflow attempts
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
# Consume only API data and execute batch code from the trusted default branch.
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
@@ -27,6 +28,18 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
continue-on-error: true
|
||||
uses: ./.github/actions/npm-cache
|
||||
- name: Diagnose missing and startup-failed PR CI
|
||||
continue-on-error: true
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
CI_REPOSITORY: ${{ github.repository }}
|
||||
CI_REPOSITORY_ID: ${{ github.repository_id }}
|
||||
CI_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
|
||||
run: node scripts/ci/pr-ci-health.mjs
|
||||
- name: Record queue, execution, and matched step timing
|
||||
continue-on-error: true
|
||||
env:
|
||||
@@ -44,3 +57,28 @@ jobs:
|
||||
if-no-files-found: error
|
||||
overwrite: true
|
||||
retention-days: 14
|
||||
- name: Install dependencies for static failure attribution
|
||||
id: failure-deps
|
||||
if: ${{ !cancelled() && hashFiles('.cache/ci-job-timings/batch-manifest.json') != '' }}
|
||||
continue-on-error: true
|
||||
run: npm ci
|
||||
- name: Attribute supported static failures against main
|
||||
if: ${{ !cancelled() && steps.failure-deps.outcome == 'success' && hashFiles('.cache/ci-job-timings/batch-manifest.json') != '' }}
|
||||
continue-on-error: true
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
CI_REPOSITORY: ${{ github.repository }}
|
||||
CI_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
|
||||
run: node scripts/ci/failure-report.mjs
|
||||
- name: Upload CI health and failure diagnostics
|
||||
if: ${{ !cancelled() }}
|
||||
continue-on-error: true
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: ci-health-diagnostics
|
||||
path: |
|
||||
.cache/ci-pr-health/
|
||||
.cache/ci-failures/
|
||||
include-hidden-files: true
|
||||
if-no-files-found: warn
|
||||
retention-days: 14
|
||||
|
||||
+128
-23
@@ -42,6 +42,7 @@ jobs:
|
||||
tier: ${{ steps.plan.outputs.tier }}
|
||||
qualification_run_id: ${{ steps.qualification.outputs.run_id }}
|
||||
code: ${{ steps.plan.outputs.code }}
|
||||
reverse_tools: ${{ steps.plan.outputs.reverse_tools }}
|
||||
sqlite: ${{ steps.plan.outputs.sqlite }}
|
||||
runtime: ${{ steps.plan.outputs.runtime }}
|
||||
package: ${{ steps.plan.outputs.package }}
|
||||
@@ -122,11 +123,28 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- name: Restore static task cache
|
||||
if: needs.changes.outputs.code == 'true'
|
||||
id: static-cache
|
||||
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
|
||||
with:
|
||||
path: .turbo/cache/
|
||||
key: rea-static-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.nvmrc', '.npmrc', 'package-lock.json') }}-${{ github.sha }}
|
||||
# Turbo validates each task's inputs after restoring this snapshot.
|
||||
restore-keys: rea-static-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.nvmrc', '.npmrc', 'package-lock.json') }}-
|
||||
- name: Validate workflow syntax and expressions
|
||||
if: needs.changes.outputs.workflows == 'true'
|
||||
run: go run github.com/rhysd/actionlint/cmd/actionlint@v1.7.7 -shellcheck= -pyflakes=
|
||||
- name: Record static task source and execution identity
|
||||
continue-on-error: true
|
||||
if: needs.changes.outputs.code == 'true'
|
||||
env:
|
||||
CI_PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: npm run ci:static-context
|
||||
- if: needs.changes.outputs.code == 'true'
|
||||
run: npm run check:ci
|
||||
- if: needs.changes.outputs.code == 'true'
|
||||
@@ -136,6 +154,15 @@ jobs:
|
||||
run: npm run test:python
|
||||
- if: needs.changes.outputs.code != 'true'
|
||||
run: npm run format:check
|
||||
- name: Save validated static task cache
|
||||
if: >-
|
||||
success() && needs.changes.outputs.code == 'true' &&
|
||||
steps.npm-cache.outputs.can-save == 'true' &&
|
||||
steps.static-cache.outputs.cache-hit != 'true'
|
||||
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
|
||||
with:
|
||||
path: .turbo/cache/
|
||||
key: ${{ steps.static-cache.outputs.cache-primary-key }}
|
||||
- name: Upload complete static diagnostics
|
||||
if: always() && needs.changes.outputs.code == 'true'
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
@@ -146,6 +173,34 @@ jobs:
|
||||
if-no-files-found: ignore
|
||||
retention-days: 7
|
||||
|
||||
test-source:
|
||||
name: Source tests
|
||||
needs: changes
|
||||
if: needs.changes.outputs.code == 'true' && needs.changes.outputs.tier == 'fast'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- name: Run source tests without waiting for build
|
||||
run: npm run test:fast -- --reporter=default --reporter=blob
|
||||
env:
|
||||
VITEST_BLOB_LABEL: source
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
if: ${{ !cancelled() }}
|
||||
with:
|
||||
name: vitest-report-source
|
||||
path: .vitest/blob/*
|
||||
include-hidden-files: true
|
||||
if-no-files-found: error
|
||||
retention-days: 7
|
||||
|
||||
test-shard:
|
||||
name: Test shard (${{ matrix.shard }})
|
||||
needs: [changes, build]
|
||||
@@ -153,15 +208,21 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
strategy:
|
||||
fail-fast: false
|
||||
# Failed PRs release the remaining matrix slots; qualification retains
|
||||
# every shard's diagnostics and coverage. Passing PRs run the full suite.
|
||||
fail-fast: ${{ github.event_name == 'pull_request' && needs.changes.outputs.tier == 'fast' }}
|
||||
matrix:
|
||||
shard: [1/6, 2/6, 3/6, 4/6, 5/6, 6/6]
|
||||
# Fast feedback spends the same six test jobs on one source group and
|
||||
# five built groups; full qualification retains six complete shards.
|
||||
shard: ${{ fromJSON(needs.changes.outputs.tier == 'fast' && '["1/5","2/5","3/5","4/5","5/5"]' || '["1/6","2/6","3/6","4/6","5/6","6/6"]') }}
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
@@ -172,21 +233,42 @@ jobs:
|
||||
run: npm run test:ci:shard:run -- --shard=${{ matrix.shard }}
|
||||
env:
|
||||
REA_CI_TEST_HISTORY: .cache/ci-test-history/test-results.json
|
||||
- name: Run complete portable suite for fast feedback
|
||||
- name: Run built-dependent tests for fast feedback
|
||||
if: needs.changes.outputs.tier == 'fast'
|
||||
run: npm run test:run -- --reporter=default --reporter=blob --shard=${{ matrix.shard }}
|
||||
run: npm run test:built -- --reporter=default --reporter=blob --shard=${{ matrix.shard }}
|
||||
env:
|
||||
VITEST_BLOB_LABEL: built
|
||||
REA_CI_TEST_HISTORY: .cache/ci-test-history/test-results.json
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
if: ${{ !cancelled() }}
|
||||
with:
|
||||
name: vitest-report-${{ strategy.job-index }}
|
||||
name: vitest-report-built-${{ strategy.job-index }}
|
||||
path: .vitest/blob/*
|
||||
include-hidden-files: true
|
||||
if-no-files-found: error
|
||||
overwrite: true
|
||||
retention-days: 7
|
||||
|
||||
reverse-tools-quality:
|
||||
name: Reverse tools coverage and mutation
|
||||
needs: [changes, static]
|
||||
if: needs.changes.outputs.reverse_tools == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 60
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- name: Run reverse tools tests with coverage
|
||||
run: npm run test:coverage:reverse-tools
|
||||
- name: Report reverse tools mutation results
|
||||
run: npm run test:mutation:reverse-tools
|
||||
|
||||
build:
|
||||
name: Build package
|
||||
needs: changes
|
||||
@@ -212,7 +294,9 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- name: Reuse portable build outputs across runs
|
||||
uses: ./.github/actions/portable-build-cache
|
||||
@@ -271,7 +355,7 @@ jobs:
|
||||
|
||||
test:
|
||||
name: Portable tests
|
||||
needs: [changes, test-shard]
|
||||
needs: [changes, test-source, test-shard]
|
||||
if: ${{ !cancelled() && needs.changes.outputs.tier != 'reused' }}
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
@@ -286,13 +370,18 @@ jobs:
|
||||
if: needs.changes.outputs.code == 'true' && success()
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
if: needs.changes.outputs.code == 'true' && success()
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- if: needs.changes.outputs.code == 'true' && success()
|
||||
run: npm ci
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
if: >-
|
||||
!cancelled() && needs.changes.outputs.code == 'true' &&
|
||||
(needs.test-shard.result == 'success' || needs.test-shard.result == 'failure')
|
||||
(needs.test-shard.result == 'success' || needs.test-shard.result == 'failure' ||
|
||||
(needs.changes.outputs.tier == 'fast' &&
|
||||
(needs.test-source.result == 'success' || needs.test-source.result == 'failure')))
|
||||
with:
|
||||
path: .vitest/blob
|
||||
pattern: vitest-report-*
|
||||
@@ -320,7 +409,8 @@ jobs:
|
||||
if: >-
|
||||
!cancelled() && needs.changes.outputs.code == 'true' &&
|
||||
needs.changes.outputs.tier == 'fast' &&
|
||||
(needs.test-shard.result == 'success' || needs.test-shard.result == 'failure')
|
||||
(needs.test-shard.result == 'success' || needs.test-shard.result == 'failure' ||
|
||||
needs.test-source.result == 'success' || needs.test-source.result == 'failure')
|
||||
run: >-
|
||||
npm exec -- vitest --merge-reports
|
||||
--reporter=default
|
||||
@@ -333,6 +423,8 @@ jobs:
|
||||
if: needs.changes.outputs.code == 'true' && !cancelled() && hashFiles('test-timings.json') != ''
|
||||
env:
|
||||
SHARD_RESULT: ${{ needs.test-shard.result }}
|
||||
SOURCE_REQUIRED: ${{ needs.changes.outputs.tier == 'fast' }}
|
||||
SOURCE_RESULT: ${{ needs.test-source.result }}
|
||||
MERGE_RESULT: ${{ needs.changes.outputs.tier == 'full' && steps.merge.outcome || steps.merge-fast.outcome }}
|
||||
CI_VERIFY_TEST_INVENTORY: "true"
|
||||
CI_TEST_SELECTION: ${{ needs.changes.outputs.test_selection }}
|
||||
@@ -365,7 +457,13 @@ jobs:
|
||||
env:
|
||||
CODE_REQUIRED: ${{ needs.changes.outputs.code }}
|
||||
SHARD_RESULT: ${{ needs.test-shard.result }}
|
||||
run: test "${CODE_REQUIRED}" != "true" || test "${SHARD_RESULT}" = "success"
|
||||
SOURCE_REQUIRED: ${{ needs.changes.outputs.tier == 'fast' }}
|
||||
SOURCE_RESULT: ${{ needs.test-source.result }}
|
||||
run: |
|
||||
if [ "${CODE_REQUIRED}" = "true" ]; then
|
||||
test "${SHARD_RESULT}" = "success" || exit 1
|
||||
test "${SOURCE_REQUIRED}" != "true" || test "${SOURCE_RESULT}" = "success"
|
||||
fi
|
||||
|
||||
conformance-fixtures:
|
||||
name: Source-built conformance fixtures
|
||||
@@ -450,7 +548,9 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- name: Verify native runtime identity
|
||||
env:
|
||||
EXPECTED_CI_PLATFORM: ${{ matrix.platform }}
|
||||
@@ -472,11 +572,6 @@ jobs:
|
||||
- run: npm run verify:windows-native:artifact
|
||||
- name: Verify NativeAOT diagnostic applicability on this host
|
||||
run: npm run test:focused -- tests/evaluation/package/nativeAotDiagnostics.test.ts
|
||||
- name: Resolve npm cache before verifier HOME isolation
|
||||
id: npm-cache
|
||||
run: |
|
||||
cache_path="$(npm config get cache)"
|
||||
echo "path=${cache_path}" >> "${GITHUB_OUTPUT}"
|
||||
- run: npm run verify:package
|
||||
env:
|
||||
npm_config_cache: ${{ steps.npm-cache.outputs.path }}
|
||||
@@ -506,7 +601,9 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
@@ -554,7 +651,9 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- name: Run the public CLI/MCP reconstruction journey
|
||||
env:
|
||||
@@ -583,7 +682,9 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
@@ -703,7 +804,9 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- run: npm run docs:check
|
||||
- name: Generation leaves authored files unchanged
|
||||
@@ -792,7 +895,9 @@ jobs:
|
||||
changes,
|
||||
static,
|
||||
build,
|
||||
reverse-tools-quality,
|
||||
test-shard,
|
||||
test-source,
|
||||
sqlite-runtime,
|
||||
test,
|
||||
conformance-fixtures,
|
||||
|
||||
@@ -23,6 +23,8 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- run: npm run docs:build
|
||||
|
||||
@@ -32,7 +32,9 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
|
||||
@@ -25,7 +25,9 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
name: Real upstream Cutter verification
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
repository_dispatch:
|
||||
types: [real-cutter-upstream]
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
windows-native-artifact:
|
||||
name: Build Windows bridge security boundary
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Install the cross compiler used by the Windows native lane
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install --no-install-recommends -y g++-mingw-w64-x86-64
|
||||
- run: npm run build:windows-native
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: cutter-windows-native
|
||||
path: native/windows/build/
|
||||
if-no-files-found: error
|
||||
|
||||
cutter-upstream:
|
||||
name: Cutter upstream smoke (${{ matrix.platform }})
|
||||
needs: windows-native-artifact
|
||||
runs-on: ${{ matrix.runner }}
|
||||
timeout-minutes: 20
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- platform: linux
|
||||
runner: [self-hosted, cutter-upstream, linux]
|
||||
- platform: macOS
|
||||
runner: [self-hosted, cutter-upstream, macOS]
|
||||
- platform: Windows
|
||||
runner: [self-hosted, cutter-upstream, Windows]
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- if: matrix.platform == 'Windows'
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: cutter-windows-native
|
||||
path: native/windows/build/
|
||||
- if: matrix.platform == 'Windows'
|
||||
run: npm run verify:windows-native:artifact
|
||||
- if: matrix.platform == 'Windows'
|
||||
run: npm run build:cached
|
||||
- if: matrix.platform == 'Windows'
|
||||
run: npm run verify:windows-native
|
||||
- name: Verify host and real Cutter bridge
|
||||
env:
|
||||
EXPECTED_PLATFORM: ${{ matrix.platform }}
|
||||
run: npm run verify:cutter
|
||||
- uses: actions/upload-artifact@v4
|
||||
if: always()
|
||||
with:
|
||||
name: cutter-upstream-${{ matrix.platform }}
|
||||
path: cutter-smoke.json
|
||||
if-no-files-found: ignore
|
||||
retention-days: 30
|
||||
@@ -26,7 +26,9 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
|
||||
@@ -48,7 +48,9 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff # v5.6.0
|
||||
with:
|
||||
go-version: "1.26.1"
|
||||
|
||||
@@ -19,7 +19,9 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- name: Pack CLI for setup verification
|
||||
shell: bash
|
||||
@@ -40,7 +42,9 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- name: Install packed CLI in isolation
|
||||
shell: bash
|
||||
|
||||
@@ -19,7 +19,9 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- name: Install packed CLI in isolation
|
||||
shell: bash
|
||||
|
||||
@@ -27,7 +27,9 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
|
||||
@@ -47,7 +47,9 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y tshark util-linux python3
|
||||
- run: npm ci
|
||||
- run: npm run build:cached
|
||||
|
||||
@@ -29,7 +29,9 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
|
||||
@@ -40,7 +40,9 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: sudo apt-get update && sudo apt-get install -y sigrok-cli util-linux python3
|
||||
- run: npm ci
|
||||
- run: npm run build:cached
|
||||
|
||||
@@ -26,7 +26,9 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- name: Use pinned package manager
|
||||
run: npm install --global npm@11.16.0
|
||||
- run: npm ci
|
||||
|
||||
@@ -21,12 +21,14 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- run: npm run verify:browser:runtime
|
||||
- run: env -u DBUS_SESSION_BUS_ADDRESS npm run verify:browser:runtime
|
||||
env:
|
||||
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
|
||||
REA_BROWSER_NO_SANDBOX: "true"
|
||||
|
||||
@@ -21,7 +21,9 @@ jobs:
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
|
||||
@@ -27,7 +27,9 @@ jobs:
|
||||
with:
|
||||
node-version: ${{ matrix.node }}
|
||||
check-latest: true
|
||||
cache: npm
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
|
||||
@@ -33,6 +33,8 @@ codedb.snapshot
|
||||
htmlcov/
|
||||
.coverage
|
||||
coverage/
|
||||
.stryker-tmp/
|
||||
reports/mutation/
|
||||
.vitest/
|
||||
.vitest-reports/
|
||||
test-results.xml
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
**English** · [简体中文](docs/readme/README_zh.md) · [繁體中文](docs/readme/README_zh-TW.md) · [日本語](docs/readme/README_ja.md) · [한국어](docs/readme/README_ko.md) · [Türkçe](docs/readme/README_tr.md) · [Русский](docs/readme/README_ru.md) · [Tiếng Việt](docs/readme/README_vi.md) · [ไทย](docs/readme/README_th.md) · [Deutsch](docs/readme/README_de.md) · [Español](docs/readme/README_es.md) · [Français](docs/readme/README_fr.md) · [Українська](docs/readme/README_uk.md) · [Polski](docs/readme/README_pl.md) · [Português (Brasil)](docs/readme/README_pt-BR.md) · [العربية](docs/readme/README_ar.md) · [فارسی](docs/readme/README_fa.md) · [Bahasa Indonesia](docs/readme/README_id.md) · [Italiano](docs/readme/README_it.md)
|
||||
· [Ελληνικά](docs/readme/README_el.md)
|
||||
|
||||
# REA: Reverse Engineer Anything
|
||||
|
||||
@@ -23,6 +24,12 @@
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Get REA from [morluto/rea](https://github.com/morluto/rea),
|
||||
> [rea.tools](https://rea.tools/), or the [rea-agents npm package](https://www.npmjs.com/package/rea-agents).
|
||||
> Follow the installation and release links from these official sources. A similar
|
||||
> repository name or copied README does not establish that a download is official.
|
||||
|
||||
<br />
|
||||
|
||||
<img src="docs/assets/rea-hopper-analysis.png" alt="REA launching its analysis bridge inside Hopper while inspecting a native binary" width="1200" />
|
||||
@@ -180,6 +187,11 @@ Static JavaScript and .NET inspection read the supplied files without running
|
||||
the application. Runtime capture runs or interacts with the selected target
|
||||
using your user permissions; each runtime guide describes its effects.
|
||||
|
||||
For offline native-binary inspection, REA also supports caller-supplied GNU
|
||||
objdump and Rizin. See the [reverse-engineering tools guide](docs/reverse-engineering-tools.md)
|
||||
for provider setup, CLI commands, and Evidence limitations. REA does not install
|
||||
or update either provider.
|
||||
|
||||
<a id="choosing-a-deep-analysis-provider"></a>
|
||||
|
||||
Native formats and host support vary by provider. See
|
||||
|
||||
@@ -0,0 +1,816 @@
|
||||
"""REA's experimental local bridge plugin for Cutter.
|
||||
|
||||
Install this file in Cutter's user plugin directory. The bridge is REA-owned
|
||||
IPC, not an upstream Cutter transport. It binds to loopback and requires a
|
||||
per-process random token stored in a user-only descriptor file.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import hashlib
|
||||
import hmac
|
||||
import os
|
||||
import secrets
|
||||
import socket
|
||||
import stat
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
if os.name == "nt":
|
||||
import ctypes
|
||||
from ctypes import wintypes
|
||||
|
||||
import cutter
|
||||
|
||||
try:
|
||||
from PySide6.QtCore import QObject, Signal
|
||||
except ImportError:
|
||||
from PySide2.QtCore import QObject, Signal
|
||||
|
||||
|
||||
MAX_FRAME_BYTES = 16 * 1024 * 1024
|
||||
SOCKET_TIMEOUT_SECONDS = 20
|
||||
AUTH_FRAME_TIMEOUT_SECONDS = 1
|
||||
MAX_DESCRIPTOR_BYTES = 64 * 1024
|
||||
MAX_COMPLETION_RECEIPTS = 64
|
||||
|
||||
|
||||
def _windows_api() -> tuple[Any, Any]:
|
||||
kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
|
||||
advapi32 = ctypes.WinDLL("advapi32", use_last_error=True)
|
||||
kernel32.GetCurrentProcess.restype = wintypes.HANDLE
|
||||
kernel32.LocalFree.argtypes = [wintypes.HLOCAL]
|
||||
kernel32.LocalFree.restype = wintypes.HLOCAL
|
||||
kernel32.CloseHandle.argtypes = [wintypes.HANDLE]
|
||||
kernel32.CloseHandle.restype = wintypes.BOOL
|
||||
kernel32.GetFileAttributesW.argtypes = [wintypes.LPCWSTR]
|
||||
kernel32.GetFileAttributesW.restype = wintypes.DWORD
|
||||
kernel32.CreateDirectoryW.argtypes = [wintypes.LPCWSTR, wintypes.LPVOID]
|
||||
kernel32.CreateDirectoryW.restype = wintypes.BOOL
|
||||
kernel32.CreateFileW.argtypes = [wintypes.LPCWSTR, wintypes.DWORD, wintypes.DWORD,
|
||||
wintypes.LPVOID, wintypes.DWORD, wintypes.DWORD, wintypes.HANDLE]
|
||||
kernel32.CreateFileW.restype = wintypes.HANDLE
|
||||
kernel32.WriteFile.argtypes = [wintypes.HANDLE, wintypes.LPVOID, wintypes.DWORD,
|
||||
ctypes.POINTER(wintypes.DWORD), wintypes.LPVOID]
|
||||
kernel32.WriteFile.restype = wintypes.BOOL
|
||||
kernel32.FlushFileBuffers.argtypes = [wintypes.HANDLE]
|
||||
kernel32.FlushFileBuffers.restype = wintypes.BOOL
|
||||
kernel32.MoveFileExW.argtypes = [wintypes.LPCWSTR, wintypes.LPCWSTR, wintypes.DWORD]
|
||||
kernel32.MoveFileExW.restype = wintypes.BOOL
|
||||
kernel32.DeleteFileW.argtypes = [wintypes.LPCWSTR]
|
||||
kernel32.DeleteFileW.restype = wintypes.BOOL
|
||||
advapi32.OpenProcessToken.argtypes = [wintypes.HANDLE, wintypes.DWORD, ctypes.POINTER(wintypes.HANDLE)]
|
||||
advapi32.OpenProcessToken.restype = wintypes.BOOL
|
||||
advapi32.GetTokenInformation.argtypes = [wintypes.HANDLE, ctypes.c_int, wintypes.LPVOID,
|
||||
wintypes.DWORD, ctypes.POINTER(wintypes.DWORD)]
|
||||
advapi32.GetTokenInformation.restype = wintypes.BOOL
|
||||
advapi32.ConvertSidToStringSidW.argtypes = [wintypes.LPVOID, ctypes.POINTER(wintypes.LPWSTR)]
|
||||
advapi32.ConvertSidToStringSidW.restype = wintypes.BOOL
|
||||
advapi32.GetNamedSecurityInfoW.argtypes = [wintypes.LPWSTR, ctypes.c_int, wintypes.DWORD,
|
||||
ctypes.POINTER(wintypes.LPVOID), ctypes.POINTER(wintypes.LPVOID),
|
||||
ctypes.POINTER(wintypes.LPVOID), ctypes.POINTER(wintypes.LPVOID),
|
||||
ctypes.POINTER(wintypes.LPVOID)]
|
||||
advapi32.GetNamedSecurityInfoW.restype = wintypes.DWORD
|
||||
advapi32.GetSecurityDescriptorControl.argtypes = [wintypes.LPVOID, ctypes.POINTER(wintypes.WORD),
|
||||
ctypes.POINTER(wintypes.DWORD)]
|
||||
advapi32.GetSecurityDescriptorControl.restype = wintypes.BOOL
|
||||
advapi32.EqualSid.argtypes = [wintypes.LPVOID, wintypes.LPVOID]
|
||||
advapi32.EqualSid.restype = wintypes.BOOL
|
||||
advapi32.GetAclInformation.argtypes = [wintypes.LPVOID, wintypes.LPVOID, wintypes.DWORD, ctypes.c_int]
|
||||
advapi32.GetAclInformation.restype = wintypes.BOOL
|
||||
advapi32.GetAce.argtypes = [wintypes.LPVOID, wintypes.DWORD, ctypes.POINTER(wintypes.LPVOID)]
|
||||
advapi32.GetAce.restype = wintypes.BOOL
|
||||
advapi32.IsValidSid.argtypes = [wintypes.LPVOID]
|
||||
advapi32.IsValidSid.restype = wintypes.BOOL
|
||||
advapi32.GetLengthSid.argtypes = [wintypes.LPVOID]
|
||||
advapi32.GetLengthSid.restype = wintypes.DWORD
|
||||
advapi32.ConvertStringSecurityDescriptorToSecurityDescriptorW.argtypes = [
|
||||
wintypes.LPCWSTR, wintypes.DWORD, ctypes.POINTER(wintypes.LPVOID), ctypes.POINTER(wintypes.DWORD),
|
||||
]
|
||||
advapi32.ConvertStringSecurityDescriptorToSecurityDescriptorW.restype = wintypes.BOOL
|
||||
return kernel32, advapi32
|
||||
|
||||
|
||||
def _windows_user_sid_text() -> str:
|
||||
kernel32, advapi32 = _windows_api()
|
||||
token = wintypes.HANDLE()
|
||||
if not advapi32.OpenProcessToken(kernel32.GetCurrentProcess(), 0x0008, ctypes.byref(token)):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
try:
|
||||
required = wintypes.DWORD()
|
||||
advapi32.GetTokenInformation(token, 1, None, 0, ctypes.byref(required))
|
||||
if required.value == 0:
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
buffer = ctypes.create_string_buffer(required.value)
|
||||
if not advapi32.GetTokenInformation(token, 1, buffer, required, ctypes.byref(required)):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
|
||||
class SID_AND_ATTRIBUTES(ctypes.Structure):
|
||||
_fields_ = [("Sid", wintypes.LPVOID), ("Attributes", wintypes.DWORD)]
|
||||
|
||||
class TOKEN_USER(ctypes.Structure):
|
||||
_fields_ = [("User", SID_AND_ATTRIBUTES)]
|
||||
|
||||
sid = ctypes.cast(buffer, ctypes.POINTER(TOKEN_USER)).contents.User.Sid
|
||||
text = wintypes.LPWSTR()
|
||||
if not advapi32.ConvertSidToStringSidW(sid, ctypes.byref(text)):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
try:
|
||||
return text.value
|
||||
finally:
|
||||
kernel32.LocalFree(text)
|
||||
finally:
|
||||
kernel32.CloseHandle(token)
|
||||
|
||||
|
||||
def _verify_windows_private_path(path: Path, directory: bool) -> None:
|
||||
kernel32, advapi32 = _windows_api()
|
||||
attributes = kernel32.GetFileAttributesW(str(path))
|
||||
if attributes == 0xFFFFFFFF:
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
if attributes & 0x400 or bool(attributes & 0x10) != directory:
|
||||
raise RuntimeError("Cutter bridge path is a reparse point or has the wrong object type")
|
||||
|
||||
class ACL_SIZE_INFORMATION(ctypes.Structure):
|
||||
_fields_ = [("AceCount", wintypes.DWORD), ("AclBytesInUse", wintypes.DWORD), ("AclBytesFree", wintypes.DWORD)]
|
||||
|
||||
class ACE_HEADER(ctypes.Structure):
|
||||
_fields_ = [("AceType", wintypes.BYTE), ("AceFlags", wintypes.BYTE), ("AceSize", wintypes.WORD)]
|
||||
|
||||
class SID_HEADER(ctypes.Structure):
|
||||
_fields_ = [("Revision", wintypes.BYTE), ("SubAuthorityCount", wintypes.BYTE),
|
||||
("IdentifierAuthority", wintypes.BYTE * 6)]
|
||||
|
||||
class ACCESS_ALLOWED_ACE(ctypes.Structure):
|
||||
_fields_ = [("Header", ACE_HEADER), ("Mask", wintypes.DWORD), ("SidStart", wintypes.DWORD)]
|
||||
|
||||
owner = wintypes.LPVOID()
|
||||
dacl = wintypes.LPVOID()
|
||||
descriptor = wintypes.LPVOID()
|
||||
result = advapi32.GetNamedSecurityInfoW(
|
||||
str(path), 1, 0x1 | 0x4, ctypes.byref(owner), None, ctypes.byref(dacl), None, ctypes.byref(descriptor),
|
||||
)
|
||||
if result != 0:
|
||||
raise ctypes.WinError(result)
|
||||
try:
|
||||
control = wintypes.WORD()
|
||||
revision = wintypes.DWORD()
|
||||
if not advapi32.GetSecurityDescriptorControl(descriptor, ctypes.byref(control), ctypes.byref(revision)):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
if not control.value & 0x1000 or not dacl:
|
||||
raise RuntimeError("Cutter bridge ACL must be present and protected from inheritance")
|
||||
|
||||
token = wintypes.HANDLE()
|
||||
if not advapi32.OpenProcessToken(kernel32.GetCurrentProcess(), 0x0008, ctypes.byref(token)):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
try:
|
||||
required = wintypes.DWORD()
|
||||
advapi32.GetTokenInformation(token, 1, None, 0, ctypes.byref(required))
|
||||
token_user = ctypes.create_string_buffer(required.value)
|
||||
if not advapi32.GetTokenInformation(token, 1, token_user, required, ctypes.byref(required)):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
|
||||
class SID_AND_ATTRIBUTES(ctypes.Structure):
|
||||
_fields_ = [("Sid", wintypes.LPVOID), ("Attributes", wintypes.DWORD)]
|
||||
|
||||
class TOKEN_USER(ctypes.Structure):
|
||||
_fields_ = [("User", SID_AND_ATTRIBUTES)]
|
||||
|
||||
current_sid = ctypes.cast(token_user, ctypes.POINTER(TOKEN_USER)).contents.User.Sid
|
||||
if not advapi32.EqualSid(owner, current_sid):
|
||||
raise RuntimeError("Cutter bridge path owner does not match the current user")
|
||||
finally:
|
||||
kernel32.CloseHandle(token)
|
||||
|
||||
info = ACL_SIZE_INFORMATION()
|
||||
if not advapi32.GetAclInformation(dacl, ctypes.byref(info), ctypes.sizeof(info), 2) or info.AceCount != 2:
|
||||
raise RuntimeError("Cutter bridge ACL must allow only the current user and SYSTEM")
|
||||
expected_user = _windows_user_sid_text()
|
||||
user_allowed = False
|
||||
system_allowed = False
|
||||
for index in range(info.AceCount):
|
||||
ace_pointer = wintypes.LPVOID()
|
||||
if not advapi32.GetAce(dacl, index, ctypes.byref(ace_pointer)):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
header = ctypes.cast(ace_pointer, ctypes.POINTER(ACE_HEADER)).contents
|
||||
if header.AceType != 0 or header.AceSize < ctypes.sizeof(ACE_HEADER) + ctypes.sizeof(wintypes.DWORD) + ctypes.sizeof(SID_HEADER):
|
||||
raise RuntimeError("Cutter bridge ACL contains a malformed or unsupported ACE")
|
||||
ace_address = ctypes.cast(ace_pointer, ctypes.c_void_p).value
|
||||
if ace_address is None:
|
||||
raise RuntimeError("Cutter bridge ACL returned a null ACE")
|
||||
sid_pointer = ctypes.c_void_p(ace_address + ctypes.sizeof(ACE_HEADER) + ctypes.sizeof(wintypes.DWORD))
|
||||
sid_header = ctypes.cast(sid_pointer, ctypes.POINTER(SID_HEADER)).contents
|
||||
sid_size = ctypes.sizeof(SID_HEADER) + sid_header.SubAuthorityCount * ctypes.sizeof(wintypes.DWORD)
|
||||
if sid_size > header.AceSize - ctypes.sizeof(ACE_HEADER) - ctypes.sizeof(wintypes.DWORD) or not advapi32.IsValidSid(sid_pointer) or advapi32.GetLengthSid(sid_pointer) != sid_size:
|
||||
raise RuntimeError("Cutter bridge ACL contains an invalid SID")
|
||||
ace = ctypes.cast(ace_pointer, ctypes.POINTER(ACCESS_ALLOWED_ACE)).contents
|
||||
sid_text = wintypes.LPWSTR()
|
||||
if ace.Mask != 0x001F01FF or ace.Header.AceFlags != 0x03:
|
||||
raise RuntimeError("Cutter bridge ACL contains an unsupported ACE")
|
||||
if not advapi32.ConvertSidToStringSidW(sid_pointer, ctypes.byref(sid_text)):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
try:
|
||||
if sid_text.value == expected_user:
|
||||
user_allowed = True
|
||||
elif sid_text.value == "S-1-5-18":
|
||||
system_allowed = True
|
||||
else:
|
||||
raise RuntimeError("Cutter bridge ACL allows an unrelated principal")
|
||||
finally:
|
||||
kernel32.LocalFree(sid_text)
|
||||
if not user_allowed or not system_allowed:
|
||||
raise RuntimeError("Cutter bridge ACL must allow only the current user and SYSTEM")
|
||||
finally:
|
||||
kernel32.LocalFree(descriptor)
|
||||
|
||||
|
||||
def _windows_security_attributes() -> tuple[Any, Any]:
|
||||
kernel32, advapi32 = _windows_api()
|
||||
sid = _windows_user_sid_text()
|
||||
sddl = f"O:{sid}G:{sid}D:P(A;OICI;FA;;;{sid})(A;OICI;FA;;;SY)"
|
||||
descriptor = wintypes.LPVOID()
|
||||
if not advapi32.ConvertStringSecurityDescriptorToSecurityDescriptorW(sddl, 1, ctypes.byref(descriptor), None):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
|
||||
class SECURITY_ATTRIBUTES(ctypes.Structure):
|
||||
_fields_ = [("nLength", wintypes.DWORD), ("lpSecurityDescriptor", wintypes.LPVOID), ("bInheritHandle", wintypes.BOOL)]
|
||||
|
||||
attributes = SECURITY_ATTRIBUTES(ctypes.sizeof(SECURITY_ATTRIBUTES), descriptor, False)
|
||||
return kernel32, (descriptor, attributes)
|
||||
|
||||
|
||||
def _ensure_windows_bridge_directory(directory: Path) -> None:
|
||||
directory.parent.mkdir(parents=True, exist_ok=True)
|
||||
if not directory.exists():
|
||||
kernel32, (descriptor, attributes) = _windows_security_attributes()
|
||||
try:
|
||||
if not kernel32.CreateDirectoryW(str(directory), ctypes.byref(attributes)):
|
||||
error = ctypes.get_last_error()
|
||||
if error != 183:
|
||||
raise ctypes.WinError(error)
|
||||
finally:
|
||||
kernel32.LocalFree(descriptor)
|
||||
_verify_windows_private_path(directory, True)
|
||||
|
||||
|
||||
def _ensure_posix_bridge_directory(directory: Path) -> None:
|
||||
directory.mkdir(mode=0o700, parents=True, exist_ok=True)
|
||||
if not hasattr(os, "O_NOFOLLOW") or not hasattr(os, "O_DIRECTORY"):
|
||||
raise RuntimeError("Cutter bridge directory requires no-follow directory handles")
|
||||
descriptor = os.open(directory, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)
|
||||
try:
|
||||
directory_info = os.fstat(descriptor)
|
||||
if directory_info.st_uid != os.getuid():
|
||||
raise RuntimeError("Cutter bridge directory must be owned by the current user")
|
||||
os.fchmod(descriptor, 0o700)
|
||||
directory_info = os.fstat(descriptor)
|
||||
if stat.S_IMODE(directory_info.st_mode) != 0o700:
|
||||
raise RuntimeError("Cutter bridge directory must have mode 0700")
|
||||
finally:
|
||||
os.close(descriptor)
|
||||
|
||||
|
||||
def _write_windows_descriptor(directory: Path, destination: Path, payload: bytes) -> None:
|
||||
if len(payload) > MAX_DESCRIPTOR_BYTES:
|
||||
raise RuntimeError("Cutter bridge descriptor exceeds its byte limit")
|
||||
kernel32, (security_descriptor, attributes) = _windows_security_attributes()
|
||||
temporary = directory / f".{destination.name}.{uuid.uuid4().hex}.tmp"
|
||||
handle = wintypes.HANDLE()
|
||||
try:
|
||||
handle = kernel32.CreateFileW(str(temporary), 0x40000000 | 0x00020000 | 0x00040000,
|
||||
0x1 | 0x2 | 0x4, ctypes.byref(attributes), 1, 0x80, None)
|
||||
if handle == wintypes.HANDLE(-1).value:
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
_verify_windows_private_path(temporary, False)
|
||||
if payload:
|
||||
data = ctypes.create_string_buffer(payload)
|
||||
written = wintypes.DWORD()
|
||||
if not kernel32.WriteFile(handle, data, len(payload), ctypes.byref(written), None) or written.value != len(payload):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
if not kernel32.FlushFileBuffers(handle):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
kernel32.CloseHandle(handle)
|
||||
handle = wintypes.HANDLE()
|
||||
if not kernel32.MoveFileExW(str(temporary), str(destination), 0x1 | 0x8):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
finally:
|
||||
if handle and handle != wintypes.HANDLE(-1).value:
|
||||
kernel32.CloseHandle(handle)
|
||||
kernel32.DeleteFileW(str(temporary))
|
||||
kernel32.LocalFree(security_descriptor)
|
||||
|
||||
|
||||
def _bridge_directory() -> Path:
|
||||
configured = os.environ.get("REA_CUTTER_BRIDGE_DIR")
|
||||
if configured:
|
||||
return Path(configured).expanduser()
|
||||
local_app_data = os.environ.get("LOCALAPPDATA")
|
||||
if os.name == "nt" and local_app_data:
|
||||
return Path(local_app_data) / "REA" / "CutterBridge"
|
||||
return Path.home() / ".cache" / "rea" / "cutter-bridge"
|
||||
|
||||
|
||||
class _RequestDispatcher(QObject):
|
||||
dispatch = Signal(object, object)
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
self.dispatch.connect(self._handle)
|
||||
self.generation = 0
|
||||
self.completed_commands: dict[str, str] = {}
|
||||
self._receipt_lock = threading.Lock()
|
||||
self.current_file = self._read_current_file()
|
||||
try:
|
||||
self.cutter_version = str(cutter.core().getVersionInformation())
|
||||
except Exception:
|
||||
self.cutter_version = None
|
||||
|
||||
@staticmethod
|
||||
def _read_current_file() -> str | None:
|
||||
try:
|
||||
value = cutter.core().currentlyOpenFile()
|
||||
return str(value) if value else None
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
def _handle(self, request: dict[str, Any], reply: dict[str, Any]) -> None:
|
||||
kind = request.get("kind")
|
||||
request_id = request.get("request_id")
|
||||
if kind in ("command", "ack"):
|
||||
try:
|
||||
valid_request_id = (
|
||||
isinstance(request_id, str)
|
||||
and str(uuid.UUID(request_id)) == request_id.lower()
|
||||
)
|
||||
except (ValueError, AttributeError):
|
||||
valid_request_id = False
|
||||
if not valid_request_id:
|
||||
reply.update({
|
||||
"ok": False,
|
||||
"error": "invalid-command-request-id",
|
||||
"execution_state": "not_started",
|
||||
})
|
||||
request["_event"].set()
|
||||
return
|
||||
if kind == "ack":
|
||||
with self._receipt_lock:
|
||||
acknowledged = self.completed_commands.pop(request_id, None) is not None
|
||||
reply.update({"ok": True, "acknowledged": acknowledged})
|
||||
request["_event"].set()
|
||||
return
|
||||
if request["_cancelled"].is_set():
|
||||
reply.update({
|
||||
"ok": False,
|
||||
"error": "request-cancelled-before-execution",
|
||||
"execution_state": "not_started",
|
||||
})
|
||||
if kind == "command":
|
||||
self._record_command_completion(request_id, "not_started")
|
||||
request["_event"].set()
|
||||
return
|
||||
self._refresh_document_identity()
|
||||
if kind == "status":
|
||||
with self._receipt_lock:
|
||||
completed_commands = [
|
||||
{"request_id": receipt_id, "execution_state": state}
|
||||
for receipt_id, state in self.completed_commands.items()
|
||||
]
|
||||
reply.update({
|
||||
"ok": True,
|
||||
"session_id": request.get("session_id"),
|
||||
"current_file": self.current_file,
|
||||
"document_generation": self.generation,
|
||||
"cutter_version": self.cutter_version,
|
||||
"identity_status": "partial",
|
||||
"completed_commands": completed_commands,
|
||||
})
|
||||
request["_event"].set()
|
||||
return
|
||||
if kind != "command":
|
||||
reply.update({"ok": False, "error": "unsupported-request-kind"})
|
||||
request["_event"].set()
|
||||
return
|
||||
if request.get("expected_generation") != self.generation:
|
||||
reply.update({
|
||||
"ok": False,
|
||||
"error": "active-document-generation-changed",
|
||||
"execution_state": "not_started",
|
||||
})
|
||||
self._record_command_completion(request_id, "not_started")
|
||||
request["_event"].set()
|
||||
return
|
||||
command = request.get("command")
|
||||
if not isinstance(command, str) or not command.strip():
|
||||
reply.update({
|
||||
"ok": False,
|
||||
"error": "invalid-command",
|
||||
"execution_state": "not_started",
|
||||
})
|
||||
self._record_command_completion(request_id, "not_started")
|
||||
request["_event"].set()
|
||||
return
|
||||
reply["execution_state"] = "running"
|
||||
try:
|
||||
output = cutter.cmdj(command) if request.get("json") is True else cutter.cmd(command)
|
||||
command_error = None
|
||||
except Exception as error:
|
||||
output = None
|
||||
command_error = str(error)
|
||||
self._refresh_document_identity()
|
||||
if command_error is None:
|
||||
reply.update({
|
||||
"ok": True,
|
||||
"output": output,
|
||||
"current_file": self.current_file,
|
||||
"document_generation": self.generation,
|
||||
"cutter_version": self.cutter_version,
|
||||
"identity_status": "partial",
|
||||
"execution_state": "complete",
|
||||
})
|
||||
self._record_command_completion(request_id, "complete")
|
||||
else:
|
||||
reply.update({
|
||||
"ok": False,
|
||||
"error": command_error,
|
||||
"current_file": self.current_file,
|
||||
"document_generation": self.generation,
|
||||
"cutter_version": self.cutter_version,
|
||||
"identity_status": "partial",
|
||||
"execution_state": "unknown",
|
||||
"message": "The command raised an exception; it may have produced partial effects. Do not retry automatically.",
|
||||
})
|
||||
self._record_command_completion(request_id, "failed")
|
||||
request["_event"].set()
|
||||
|
||||
def _record_command_completion(
|
||||
self, request_id: str, execution_state: str
|
||||
) -> None:
|
||||
with self._receipt_lock:
|
||||
if request_id in self.completed_commands or len(self.completed_commands) < MAX_COMPLETION_RECEIPTS:
|
||||
self.completed_commands[request_id] = execution_state
|
||||
|
||||
def can_accept_command(self) -> bool:
|
||||
with self._receipt_lock:
|
||||
return len(self.completed_commands) < MAX_COMPLETION_RECEIPTS
|
||||
|
||||
def _refresh_document_identity(self) -> None:
|
||||
current_file = self._read_current_file()
|
||||
if current_file != self.current_file:
|
||||
self.current_file = current_file
|
||||
self.generation += 1
|
||||
|
||||
|
||||
class _BridgeServer(threading.Thread):
|
||||
def __init__(self, dispatcher: _RequestDispatcher) -> None:
|
||||
super().__init__(name="rea-cutter-bridge", daemon=True)
|
||||
self.dispatcher = dispatcher
|
||||
self.server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
self.server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
self.server.bind(("127.0.0.1", 0))
|
||||
self.server.listen(8)
|
||||
self.server.settimeout(0.5)
|
||||
self.port = self.server.getsockname()[1]
|
||||
self.session_id = str(uuid.uuid4())
|
||||
self.token = secrets.token_urlsafe(32)
|
||||
self.poisoned = False
|
||||
self.connection_ordinal = 0
|
||||
self.directory = _bridge_directory()
|
||||
self.descriptor = self.directory / f"cutter-{os.getpid()}-{self.session_id}.json"
|
||||
|
||||
def publish(self) -> None:
|
||||
if os.name == "nt":
|
||||
_ensure_windows_bridge_directory(self.directory)
|
||||
else:
|
||||
_ensure_posix_bridge_directory(self.directory)
|
||||
if os.name == "nt" and (self.directory.is_symlink() or not self.directory.is_dir()):
|
||||
raise RuntimeError("Cutter bridge directory must be a real directory")
|
||||
payload = {
|
||||
"session_id": self.session_id,
|
||||
"pid": os.getpid(),
|
||||
"host": "127.0.0.1",
|
||||
"port": self.port,
|
||||
"token": self.token,
|
||||
"document_generation": self.dispatcher.generation,
|
||||
"current_file": self.dispatcher.current_file,
|
||||
"cutter_version": self.dispatcher.cutter_version,
|
||||
"identity_status": "partial",
|
||||
}
|
||||
if os.name == "nt":
|
||||
_write_windows_descriptor(self.directory, self.descriptor, json.dumps(payload).encode("utf-8"))
|
||||
return
|
||||
descriptor, temporary_path = tempfile.mkstemp(prefix=f".{self.descriptor.name}.", dir=self.directory)
|
||||
try:
|
||||
os.fchmod(descriptor, 0o600)
|
||||
descriptor_info = os.fstat(descriptor)
|
||||
if descriptor_info.st_uid != os.getuid() or stat.S_IMODE(descriptor_info.st_mode) & 0o077:
|
||||
raise RuntimeError("Cutter bridge descriptor must be owned by the current user with mode 0600")
|
||||
with os.fdopen(descriptor, "w", encoding="utf-8") as stream:
|
||||
stream.write(json.dumps(payload))
|
||||
stream.flush()
|
||||
os.fsync(stream.fileno())
|
||||
os.replace(temporary_path, self.descriptor)
|
||||
except Exception:
|
||||
try:
|
||||
os.close(descriptor)
|
||||
except OSError:
|
||||
pass
|
||||
try:
|
||||
os.unlink(temporary_path)
|
||||
except OSError:
|
||||
pass
|
||||
raise
|
||||
|
||||
def run(self) -> None:
|
||||
try:
|
||||
self.publish()
|
||||
except Exception:
|
||||
self.server.close()
|
||||
return
|
||||
while True:
|
||||
try:
|
||||
connection, _ = self.server.accept()
|
||||
except socket.timeout:
|
||||
if getattr(self, "stopping", False):
|
||||
break
|
||||
continue
|
||||
except OSError:
|
||||
break
|
||||
self.connection_ordinal += 1
|
||||
connection_ordinal = self.connection_ordinal
|
||||
with connection:
|
||||
connection.settimeout(AUTH_FRAME_TIMEOUT_SECONDS)
|
||||
authenticated = False
|
||||
try:
|
||||
hello = self._receive_frame(connection)
|
||||
if not isinstance(hello, dict) or hello.get("kind") != "rea-auth-v2" or not isinstance(hello.get("nonce"), str) or len(hello["nonce"]) != 64:
|
||||
self._send(connection, {"ok": False, "error": "authentication-failed"})
|
||||
continue
|
||||
client_nonce = hello["nonce"]
|
||||
server_nonce = f"{connection_ordinal:016x}{secrets.token_hex(24)}"
|
||||
server_proof = hmac.new(self.token.encode(), f"rea-cutter-server-v2:{self.session_id}:{client_nonce}:{server_nonce}".encode(), hashlib.sha256).hexdigest()
|
||||
self._send(connection, {"session_id": self.session_id, "nonce": server_nonce, "proof": server_proof})
|
||||
authenticated_frame = self._receive_frame(connection)
|
||||
request_bytes = authenticated_frame.get("request") if isinstance(authenticated_frame, dict) else None
|
||||
expected_proof = hmac.new(self.token.encode(), f"rea-cutter-client-v2:{self.session_id}:{client_nonce}:{server_nonce}:{request_bytes}".encode(), hashlib.sha256).hexdigest()
|
||||
if not isinstance(request_bytes, str) or not isinstance(authenticated_frame.get("proof"), str) or not hmac.compare_digest(authenticated_frame["proof"], expected_proof):
|
||||
self._send(connection, {"ok": False, "error": "authentication-failed"})
|
||||
continue
|
||||
request = json.loads(request_bytes)
|
||||
if not isinstance(request, dict) or request.get("session_id") != self.session_id:
|
||||
self._send(connection, {"ok": False, "error": "authentication-failed"})
|
||||
continue
|
||||
authenticated = True
|
||||
connection.settimeout(SOCKET_TIMEOUT_SECONDS)
|
||||
if self.poisoned and request.get("kind") not in ("status", "ack"):
|
||||
self._send_authenticated(connection, {
|
||||
"ok": False,
|
||||
"error": "session-poisoned-after-ui-timeout",
|
||||
"execution_state": "not_started",
|
||||
}, client_nonce, server_nonce)
|
||||
continue
|
||||
if (
|
||||
request.get("kind") == "command"
|
||||
and not self.dispatcher.can_accept_command()
|
||||
):
|
||||
self._send_authenticated(connection, {
|
||||
"ok": False,
|
||||
"error": "completion-receipt-capacity-exceeded",
|
||||
"execution_state": "not_started",
|
||||
}, client_nonce, server_nonce)
|
||||
continue
|
||||
request["session_id"] = self.session_id
|
||||
event = threading.Event()
|
||||
cancelled = threading.Event()
|
||||
reply: dict[str, Any] = {}
|
||||
request["_event"] = event
|
||||
request["_cancelled"] = cancelled
|
||||
self.dispatcher.dispatch.emit(request, reply)
|
||||
if not event.wait(SOCKET_TIMEOUT_SECONDS):
|
||||
cancelled.set()
|
||||
self.poisoned = True
|
||||
self._send_authenticated(connection, {
|
||||
"ok": False,
|
||||
"error": "ui-thread-request-timeout",
|
||||
"execution_state": "unknown",
|
||||
"message": "The UI operation may still complete after this timeout; restart Cutter before issuing further commands.",
|
||||
}, client_nonce, server_nonce)
|
||||
else:
|
||||
if request.get("kind") == "status" and reply.get("ok") is True:
|
||||
reply["completed_connection"] = f"{connection_ordinal - 1:016x}"
|
||||
self._send_authenticated(connection, reply, client_nonce, server_nonce)
|
||||
except socket.timeout:
|
||||
if authenticated:
|
||||
self._send_authenticated(connection, {
|
||||
"ok": False,
|
||||
"error": "request-timeout",
|
||||
"execution_state": "unknown",
|
||||
"message": "The command may have produced partial or persistent effects. Do not retry automatically.",
|
||||
}, client_nonce, server_nonce)
|
||||
except Exception as error:
|
||||
self._send(connection, {"ok": False, "error": str(error)})
|
||||
self.server.close()
|
||||
try:
|
||||
self.descriptor.unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
@staticmethod
|
||||
def _receive_frame(connection: socket.socket) -> dict[str, Any]:
|
||||
data = bytearray()
|
||||
deadline = time.monotonic() + AUTH_FRAME_TIMEOUT_SECONDS
|
||||
while b"\n" not in data:
|
||||
if len(data) >= MAX_FRAME_BYTES:
|
||||
raise ValueError("frame-limit-exceeded")
|
||||
remaining = deadline - time.monotonic()
|
||||
if remaining <= 0:
|
||||
raise socket.timeout("authentication frame timed out")
|
||||
connection.settimeout(remaining)
|
||||
part = connection.recv(min(65536, MAX_FRAME_BYTES + 1 - len(data)))
|
||||
if not part:
|
||||
raise ValueError("incomplete authentication frame")
|
||||
data.extend(part)
|
||||
line, remainder = bytes(data).split(b"\n", 1)
|
||||
if len(line) > MAX_FRAME_BYTES or remainder:
|
||||
raise ValueError("invalid framed request")
|
||||
value = json.loads(line)
|
||||
return value if isinstance(value, dict) else {}
|
||||
|
||||
def _send_authenticated(self, connection: socket.socket, value: dict[str, Any], client_nonce: str, server_nonce: str) -> None:
|
||||
encoded = self._encode_authenticated_response(value, client_nonce, server_nonce)
|
||||
if encoded is None:
|
||||
value = self._truncated_response(value)
|
||||
encoded = self._encode_authenticated_response(value, client_nonce, server_nonce)
|
||||
if encoded is None:
|
||||
# This fixed-size fallback is checked through the same signed framing path.
|
||||
value = {
|
||||
"ok": False,
|
||||
"error": "response-limit-exceeded",
|
||||
"execution_state": value.get("execution_state", "unknown"),
|
||||
"output": None,
|
||||
"output_truncated": True,
|
||||
"message": "The result exceeded the bridge frame limit; command completion state is preserved.",
|
||||
}
|
||||
encoded = self._encode_authenticated_response(value, client_nonce, server_nonce)
|
||||
if encoded is None:
|
||||
return
|
||||
try:
|
||||
connection.sendall(encoded)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
def _encode_authenticated_response(self, value: dict[str, Any], client_nonce: str, server_nonce: str) -> bytes | None:
|
||||
if not _within_response_limit(value):
|
||||
return None
|
||||
prefix = b'{"response":"'
|
||||
suffix = b'","proof":"' + (b"0" * 64) + b'"}\n'
|
||||
frame_size = len(prefix) + len(suffix)
|
||||
response_parts: list[bytes] = []
|
||||
escaped_parts: list[bytes] = []
|
||||
for fragment in json.JSONEncoder(ensure_ascii=False, separators=(",", ":")).iterencode(value):
|
||||
raw = fragment.encode("utf-8")
|
||||
escaped = json.dumps(fragment, ensure_ascii=False)[1:-1].encode("utf-8")
|
||||
frame_size += len(escaped)
|
||||
if frame_size > MAX_FRAME_BYTES:
|
||||
return None
|
||||
response_parts.append(raw)
|
||||
escaped_parts.append(escaped)
|
||||
proof = hmac.new(self.token.encode(), digestmod=hashlib.sha256)
|
||||
proof.update(f"rea-cutter-response-v2:{self.session_id}:{client_nonce}:{server_nonce}:".encode())
|
||||
for part in response_parts:
|
||||
proof.update(part)
|
||||
return prefix + b"".join(escaped_parts) + b'","proof":"' + proof.hexdigest().encode() + b'"}\n'
|
||||
|
||||
@staticmethod
|
||||
def _truncated_response(value: dict[str, Any]) -> dict[str, Any]:
|
||||
response: dict[str, Any] = {
|
||||
"ok": value.get("ok") is True,
|
||||
"output": None,
|
||||
"execution_state": value.get("execution_state", "unknown"),
|
||||
"output_truncated": True,
|
||||
"message": "The command result exceeded the bridge frame limit; completion state is preserved and output was omitted.",
|
||||
}
|
||||
for key in ("current_file", "document_generation", "cutter_version", "identity_status"):
|
||||
if key in value:
|
||||
response[key] = value[key]
|
||||
if value.get("ok") is not True and isinstance(value.get("error"), str):
|
||||
response["error"] = value["error"]
|
||||
return response
|
||||
|
||||
@staticmethod
|
||||
def _send(connection: socket.socket, value: dict[str, Any]) -> None:
|
||||
if not _within_response_limit(value):
|
||||
value = {
|
||||
"ok": False,
|
||||
"error": "response-limit-exceeded",
|
||||
"execution_state": value.get("execution_state", "unknown"),
|
||||
"output_truncated": True,
|
||||
"message": "The command completed or may have completed, but its result exceeded the bridge response limit. Do not retry automatically.",
|
||||
}
|
||||
parts: list[bytes] = []
|
||||
encoded_size = 0
|
||||
for part in json.JSONEncoder(ensure_ascii=False).iterencode(value):
|
||||
encoded_part = part.encode("utf-8")
|
||||
encoded_size += len(encoded_part)
|
||||
if encoded_size > MAX_FRAME_BYTES:
|
||||
fallback = {
|
||||
"ok": False,
|
||||
"error": "response-limit-exceeded",
|
||||
"execution_state": value.get("execution_state", "unknown"),
|
||||
"output_truncated": True,
|
||||
"message": "The command completed or may have completed, but its result exceeded the bridge response limit. Do not retry automatically.",
|
||||
}
|
||||
parts = [json.dumps(fallback, separators=(",", ":")).encode("utf-8")]
|
||||
break
|
||||
parts.append(encoded_part)
|
||||
encoded = b"".join(parts)
|
||||
try:
|
||||
connection.sendall(encoded + b"\n")
|
||||
except OSError:
|
||||
# A disconnected caller must not terminate the bridge thread or
|
||||
# prevent later commands from being served.
|
||||
pass
|
||||
|
||||
def stop(self) -> None:
|
||||
self.stopping = True
|
||||
try:
|
||||
self.server.close()
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
class ReaCutterPlugin(cutter.CutterPlugin):
|
||||
name = "REA Local Bridge"
|
||||
description = "Local REA analysis bridge for the active Cutter session"
|
||||
version = "1.0"
|
||||
author = "REA"
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
self._dispatcher: _RequestDispatcher | None = None
|
||||
self._bridge: _BridgeServer | None = None
|
||||
|
||||
def setupPlugin(self) -> None:
|
||||
return
|
||||
|
||||
def setupInterface(self, main: Any) -> None:
|
||||
self._dispatcher = _RequestDispatcher()
|
||||
self._bridge = _BridgeServer(self._dispatcher)
|
||||
self._bridge.start()
|
||||
|
||||
def terminate(self) -> None:
|
||||
if self._bridge is not None:
|
||||
self._bridge.stop()
|
||||
self._bridge.join(timeout=2)
|
||||
self._bridge = None
|
||||
self._dispatcher = None
|
||||
|
||||
|
||||
def create_cutter_plugin() -> ReaCutterPlugin:
|
||||
return ReaCutterPlugin()
|
||||
|
||||
|
||||
def _within_response_limit(value: Any) -> bool:
|
||||
pending = [value]
|
||||
seen: set[int] = set()
|
||||
size = 0
|
||||
while pending:
|
||||
current = pending.pop()
|
||||
if isinstance(current, str):
|
||||
size += 2
|
||||
if current.isascii():
|
||||
size += len(current)
|
||||
size += current.count('"') + current.count("\\")
|
||||
size += sum(5 for character in current if ord(character) < 0x20)
|
||||
else:
|
||||
size += 4 * len(current)
|
||||
size += current.count('"') + current.count("\\")
|
||||
size += sum(5 for character in current if ord(character) < 0x20)
|
||||
elif isinstance(current, dict):
|
||||
identity = id(current)
|
||||
if identity in seen:
|
||||
return False
|
||||
seen.add(identity)
|
||||
size += 2 + 4 * len(current)
|
||||
if size > MAX_FRAME_BYTES:
|
||||
return False
|
||||
pending.extend(current.keys())
|
||||
pending.extend(current.values())
|
||||
elif isinstance(current, (list, tuple)):
|
||||
identity = id(current)
|
||||
if identity in seen:
|
||||
return False
|
||||
seen.add(identity)
|
||||
size += 2 + 2 * len(current)
|
||||
if size > MAX_FRAME_BYTES:
|
||||
return False
|
||||
pending.extend(current)
|
||||
else:
|
||||
size += 16
|
||||
if size > MAX_FRAME_BYTES:
|
||||
return False
|
||||
return True
|
||||
@@ -1490,9 +1490,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
InstructionScan scan = scanInstructions(function, Integer.MAX_VALUE);
|
||||
DecompileResults decompilation = decompile(function);
|
||||
String pseudocode = decompiledText(decompilation);
|
||||
if (pseudocode == null) {
|
||||
pseudocode = "";
|
||||
}
|
||||
List<Reference> incomingReferences = collectReferences(function, "incoming")
|
||||
.stream()
|
||||
.filter(reference -> !function.getBody().contains(reference.getFromAddress()))
|
||||
@@ -1514,7 +1511,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
procedure.add("locals", functionLocals(function));
|
||||
JsonObject result = new JsonObject();
|
||||
result.add("procedure", procedure);
|
||||
result.addProperty("pseudocode", pseudocode);
|
||||
result.addProperty("pseudocode", pseudocode == null ? "" : pseudocode);
|
||||
result.add("assembly", assembly);
|
||||
result.add("comments", comments);
|
||||
result.add("callers", callers);
|
||||
@@ -1523,6 +1520,20 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
result.add("outgoing_references", outgoing);
|
||||
result.add("referenced_strings", referencedStrings);
|
||||
result.add("referenced_names", referencedNames);
|
||||
JsonObject facets = new JsonObject();
|
||||
JsonObject pseudocodeAvailability = new JsonObject();
|
||||
pseudocodeAvailability.addProperty("available", pseudocode != null);
|
||||
if (pseudocode == null) {
|
||||
pseudocodeAvailability.addProperty("reason", "Ghidra did not supply pseudocode.");
|
||||
}
|
||||
facets.add("pseudocode", pseudocodeAvailability);
|
||||
for (String facet : List.of("assembly", "comments", "callers", "callees",
|
||||
"references", "strings_names", "cfg")) {
|
||||
JsonObject available = new JsonObject();
|
||||
available.addProperty("available", true);
|
||||
facets.add(facet, available);
|
||||
}
|
||||
result.add("facet_availability", facets);
|
||||
boolean swift = swiftProgram();
|
||||
boolean aarch64 = currentProgram.getLanguageID().getIdAsString().startsWith("AARCH64:");
|
||||
SwiftStringLiteralDecode swiftLiterals = swift && aarch64 ? swiftStringLiterals(function) : null;
|
||||
|
||||
@@ -644,7 +644,7 @@ def _analyze_function(document, params):
|
||||
"end": _hex(_basic_block_end(procedure, block)),
|
||||
"successors": sorted(set(successors), key=lambda value: int(value, 16)),
|
||||
})
|
||||
pseudo = _pseudocode(document, procedure) or ""
|
||||
pseudo = _pseudocode(document, procedure)
|
||||
assembly_lines = _assembly(procedure).splitlines()
|
||||
callers = sorted((_procedure_identity(item) for item in procedure.getAllCallerProcedures()), key=lambda item: int(item["address"], 16))
|
||||
callees = sorted((_procedure_identity(item) for item in procedure.getAllCalleeProcedures()), key=lambda item: int(item["address"], 16))
|
||||
@@ -706,7 +706,7 @@ def _analyze_function(document, params):
|
||||
"signature": procedure.signatureString(),
|
||||
"locals": _procedure_locals(procedure),
|
||||
},
|
||||
"pseudocode": pseudo,
|
||||
"pseudocode": pseudo or "",
|
||||
"assembly": assembly_lines,
|
||||
"comments": comments,
|
||||
"callers": callers, "callees": callees,
|
||||
@@ -718,6 +718,12 @@ def _analyze_function(document, params):
|
||||
"basic_blocks": blocks,
|
||||
"native_api": None,
|
||||
"native_value_flow": None,
|
||||
"facet_availability": {
|
||||
"pseudocode": {"available": True} if pseudo is not None else _unavailable("Hopper did not supply pseudocode."),
|
||||
**{facet: {"available": True} for facet in (
|
||||
"assembly", "comments", "callers", "callees", "references", "strings_names", "cfg"
|
||||
)},
|
||||
},
|
||||
"limitations": [
|
||||
"Native CallReference classifications are observed where available; detailed reference flags remain unknown.",
|
||||
"Hopper's public Python API does not expose equivalent external or thunk classification in this dossier.",
|
||||
|
||||
@@ -63,6 +63,7 @@ export default defineConfig({
|
||||
{ text: "Ghidra DOS analysis", link: "/ghidra-dos" },
|
||||
{ text: "Ghidra raw binary", link: "/raw-binary" },
|
||||
{ text: "Ghidra NativeAOT", link: "/ghidra-nativeaot" },
|
||||
{ text: "Ghidra PowerPC", link: "/ghidra-ppc" },
|
||||
{ text: "Ghidra semantics", link: "/ghidra-provider" },
|
||||
],
|
||||
},
|
||||
|
||||
@@ -33,10 +33,40 @@ unverified.
|
||||
## Supply tools
|
||||
|
||||
Use a **full JDK 17 or newer** (including its compiler module) and
|
||||
**jadx-headless-mcp 0.7.1**. REA runs its packaged Java metadata bridge in
|
||||
source-file mode against the unmodified engine JAR. REA does not install Java,
|
||||
an Android SDK, JADX, a device service or an emulator. Set the path in the shell
|
||||
or MCP server environment:
|
||||
[**jadx-headless-mcp 0.7.1**](https://github.com/1013503897/jadx-headless-mcp/releases/download/v0.7.1/jadx-headless-mcp-0.7.1-all.jar).
|
||||
Verify the downloaded JAR's SHA-256 against
|
||||
[REA's pinned release metadata](https://github.com/morluto/rea/blob/main/src/android/JadxRelease.ts)
|
||||
before using it. The following download and verification commands work for npm
|
||||
package consumers as well as repository checkouts; the repository-only
|
||||
`npm run fixtures:android` helper is not included in the npm package.
|
||||
|
||||
On macOS or Linux, choose a destination and verify the pinned digest:
|
||||
|
||||
```sh
|
||||
mkdir -p "$HOME/.local/share/rea/android" &&
|
||||
cd "$HOME/.local/share/rea/android" &&
|
||||
curl -fL -o jadx-headless-mcp-0.7.1-all.jar https://github.com/1013503897/jadx-headless-mcp/releases/download/v0.7.1/jadx-headless-mcp-0.7.1-all.jar &&
|
||||
if command -v sha256sum >/dev/null; then
|
||||
echo '6e5eacf500b64292bfb73c49797c1958f6ee44646e43e868039ae7feb573ff75 jadx-headless-mcp-0.7.1-all.jar' | sha256sum -c -
|
||||
else
|
||||
echo '6e5eacf500b64292bfb73c49797c1958f6ee44646e43e868039ae7feb573ff75 jadx-headless-mcp-0.7.1-all.jar' | shasum -a 256 -c -
|
||||
fi &&
|
||||
export REA_JADX_MCP_JAR="$PWD/jadx-headless-mcp-0.7.1-all.jar"
|
||||
```
|
||||
|
||||
On Windows, run the equivalent in PowerShell:
|
||||
|
||||
```powershell
|
||||
$jar = Join-Path $HOME '.local\share\rea\android\jadx-headless-mcp-0.7.1-all.jar'
|
||||
New-Item -ItemType Directory -Force (Split-Path $jar) | Out-Null
|
||||
Invoke-WebRequest -Uri 'https://github.com/1013503897/jadx-headless-mcp/releases/download/v0.7.1/jadx-headless-mcp-0.7.1-all.jar' -OutFile $jar
|
||||
if ((Get-FileHash -Algorithm SHA256 $jar).Hash.ToLowerInvariant() -ne '6e5eacf500b64292bfb73c49797c1958f6ee44646e43e868039ae7feb573ff75') { throw 'JADX JAR SHA-256 does not match REA pinned metadata' }
|
||||
$env:REA_JADX_MCP_JAR = $jar
|
||||
```
|
||||
|
||||
REA runs its packaged Java metadata bridge in source-file mode against the
|
||||
unmodified engine JAR. REA does not install Java, an Android SDK, JADX, a device
|
||||
service or an emulator. Set the path in the shell or MCP server environment:
|
||||
|
||||
```sh
|
||||
export REA_JADX_MCP_JAR=/absolute/path/jadx-headless-mcp-0.7.1-all.jar
|
||||
|
||||
@@ -38,6 +38,10 @@ graph TD
|
||||
JS_CODEC["src/javascript/sourceMaps/<br/>pinned upstream map codec<br/>owned heap-limited Node process, complete point evidence"]
|
||||
INSPECTOR["src/inspector/<br/>passive V8 Inspector discovery and capture<br/>script and target locations"]
|
||||
DOTNET["src/dotnet/<br/>Managed static metadata provider<br/>PE/CLI metadata inventory, members, IL"]
|
||||
OBJDUMP["src/objdump/<br/>GNU objdump command profile"]
|
||||
RIZIN["src/rizin/<br/>Static command profile and persistent debugger sessions"]
|
||||
GDB["src/gdb/<br/>Persistent GDB/MI sessions<br/>owned process lifecycle and target state"]
|
||||
|
||||
FIRMWARE["src/firmware/<br/>Binwalk / Unblob adapters<br/>owned processes, report normalization, private extraction"]
|
||||
ANDROID["src/android/<br/>Static APK adapter<br/>owned SDK stdio, pinned headless JADX, normalization"]
|
||||
REFERENCE["src/reference/<br/>Filesystem reference-source reader<br/>bounded traversal, validation, errors"]
|
||||
@@ -50,7 +54,7 @@ graph TD
|
||||
end
|
||||
|
||||
subgraph "Contracts Layer"
|
||||
CONTRACTS["src/contracts/<br/>Caller-visible schemas for analysis, managed, browser, Electron, application, and session tools"]
|
||||
CONTRACTS["src/contracts/<br/>Caller-visible schemas for analysis, reverse engineering, managed, browser, Electron, application, and session tools"]
|
||||
ERRORS["src/contracts/errorSchemas.ts<br/>Shared analysis error schemas<br/>CLI diagnostics, generated error catalog"]
|
||||
end
|
||||
|
||||
@@ -122,6 +126,14 @@ graph TD
|
||||
INSPECTOR --> DOMAIN
|
||||
APP --> DOTNET
|
||||
DOTNET --> DOMAIN
|
||||
COMPOSITION --> OBJDUMP
|
||||
OBJDUMP --> DOMAIN
|
||||
COMPOSITION --> RIZIN
|
||||
RIZIN --> DOMAIN
|
||||
CLI --> GDB
|
||||
SERVER --> GDB
|
||||
GDB --> PROCESS
|
||||
GDB --> DOMAIN
|
||||
HOPPER --> HOPPER_BRIDGE
|
||||
GHIDRA --> GHIDRA_BRIDGE
|
||||
HOPPER --> PROCESS
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
# PowerPC ELF analysis through Ghidra
|
||||
|
||||
This lane extends native ELF analysis to 32-bit big-endian PowerPC; it does
|
||||
not execute PowerPC code. It is the first milestone of
|
||||
[#1741](https://github.com/morluto/rea/issues/1741). GameCube/Wii DOL
|
||||
executables, PPC64 and other PowerPC variants are separate follow-ups.
|
||||
|
||||
## Admission boundary
|
||||
|
||||
ELF `EM_PPC` (20) is retained as the provider-neutral `powerpc` family. The
|
||||
target also retains class, byte order, `e_type` and raw `e_flags`. Recognition
|
||||
does not mean every provider or PowerPC variant is supported. `EM_PPC64` (21)
|
||||
is not recognized as this family.
|
||||
|
||||
The Ghidra lane admits ELF32 `ET_EXEC`, big endian. ELF64, little-endian,
|
||||
relocatable and shared objects are refused with a specific unsupported
|
||||
constraint. `EF_PPC_EMB` declares an embedded variant and is refused; the
|
||||
remaining `e_flags` bits only declare relocation conventions and do not change
|
||||
instruction decoding.
|
||||
|
||||
## Bound language
|
||||
|
||||
Ghidra's ELF loader selects `PowerPC:BE:32:e500:default` for a plain `EM_PPC`
|
||||
header with `e_flags` 0. e500 is the embedded Book-E variant whose SPE unit
|
||||
replaces the classic floating-point unit, so it misinterprets ordinary
|
||||
750-family code. Nothing in the header distinguishes the two.
|
||||
|
||||
The lane therefore imports with `PowerPC:BE:32:default` and compiler spec
|
||||
`default`, commits that choice in the analysis profile as
|
||||
`language_selection: "provider-pinned-v1"`, and checks the language Ghidra
|
||||
reports after import. A different language set through
|
||||
`REA_GHIDRA_LANGUAGE_ID` (and optionally `REA_GHIDRA_COMPILER_SPEC_ID`)
|
||||
replaces the lane default and is admitted against the installed language
|
||||
catalog as usual; for example `PowerPC:BE:32:e500` for genuine e500 code.
|
||||
|
||||
PowerPC header facts are committed in the analysis profile so interpretation
|
||||
changes invalidate profile-bound snapshots. The support-lane identity is
|
||||
versioned independently of file identity. Evidence, lifecycle outputs and saved
|
||||
snapshots preserve the `powerpc` family. Hopper and IDA adapters do not
|
||||
implicitly gain PowerPC support. Windows Ghidra P0 remains PE x86/x86-64 only.
|
||||
|
||||
## Use
|
||||
|
||||
With a caller-installed Ghidra and compatible JDK on a supported Linux/macOS host:
|
||||
|
||||
```bash
|
||||
rea analyze /absolute/path/to/ppc.elf --provider ghidra --json
|
||||
rea function /absolute/path/to/ppc.elf function_name --provider ghidra --json
|
||||
```
|
||||
|
||||
MCP uses the existing tools, including `open_binary` with an absolute path and
|
||||
`provider_id: "ghidra"`. No PowerPC-specific tool catalog or emulator is
|
||||
introduced.
|
||||
|
||||
## Verification
|
||||
|
||||
The source-owned freestanding fixture is `tests/conformance/c/ppc.c`. The
|
||||
optional cross-target lane requires a caller-supplied Clang with the PowerPC
|
||||
target, LLD, GNU `readelf`, Ghidra and its compatible JDK; ordinary host-native
|
||||
checks do not acquire these tools. `REA_PPC_CLANG` selects Clang and
|
||||
`REA_PPC_READELF` selects GNU readelf (for example
|
||||
`/opt/homebrew/opt/binutils/bin/readelf` on macOS). These are test-time
|
||||
prerequisites, not REA runtime providers.
|
||||
|
||||
```bash
|
||||
GHIDRA_INSTALL_DIR=/absolute/path/to/ghidra npm run verify:ghidra:ppc
|
||||
```
|
||||
|
||||
The lane compiles the fixture for a 750-family core (`-mcpu=750`) and never
|
||||
executes it. GNU readelf independently supplies header facts and symbol
|
||||
addresses, compared with the production REA target resolver before provider
|
||||
startup. Its version, bounded command output and selected tool path are
|
||||
retained in the verification report. The parser deliberately recognizes only
|
||||
this fixture's GNU output, rejecting missing, ambiguous or unexpected fields
|
||||
rather than substituting defaults.
|
||||
|
||||
Existing public MCP operations check Ghidra's **observed** language and
|
||||
compiler specification and loaded-file hashes; an e500 import fails the lane.
|
||||
The source-owned `ppc-probe.S` supplies fixed `addi` and conditional-branch
|
||||
encodings: verification checks bytes, length, immediate and direct destination,
|
||||
independent of decompiler variable names. Global and string bytes are checked at
|
||||
independently reported symbol addresses.
|
||||
|
||||
CLI/MCP discovery and decompilation, direct calls, target/profile/Evidence
|
||||
identity, snapshot round-trip and final process cleanup remain covered.
|
||||
Pseudocode presence is a liveness check, not proof of semantic equivalence.
|
||||
Instruction decoding and reader agreement do not establish runtime or complete
|
||||
floating-point semantics. Parser/profile tests are narrower checks, not
|
||||
substitutes for this real-provider run. Report the actual lanes run and leave
|
||||
unexecuted verification pending.
|
||||
@@ -209,21 +209,34 @@ native/runtime questions retain their own authority and coverage.
|
||||
Names and semantic labels do not establish durable member identity. Comparison
|
||||
preserves observed differences, structural inferences, and unknowns separately.
|
||||
|
||||
Cross-version matching first prefers exact CIL/signature identity. When that
|
||||
does not match, it pairs an exact declared type, method name, and raw signature
|
||||
before trying structural body shape. The exact-signature key uses names only
|
||||
as part of that full tuple; names alone never select a pair. Duplicate tuples
|
||||
remain ambiguous rather than being paired by token order. Fields follow the
|
||||
same exact tiers. The raw signature is exact whether or not REA decoded it, so
|
||||
undecoded signatures pair by that tuple too, but they never enter structural
|
||||
rounds. An unmatched member is therefore reported as `unknown`, not added or
|
||||
removed, when its own signature was not decoded or when the other side has an
|
||||
unpaired member, one-sided or ambiguous, with the same declared type and name
|
||||
whose signature was not decoded.
|
||||
Cross-version matching first pairs members of the same declared type and name:
|
||||
methods by exact CIL/signature identity, then by raw signature, then by
|
||||
structural body shape; fields by raw signature. Exact CIL/signature identity
|
||||
requires equal raw CIL digests and decoded fingerprints. Equal decoded
|
||||
fingerprints alone cannot establish an exact match: they omit switch targets
|
||||
and collapse distinct NaN encodings. Only then may a remap pair a
|
||||
renamed or moved member across declared types or names: a method by exact
|
||||
CIL/signature identity or structural body shape, a field by exact decoded
|
||||
signature. A remap never takes a member whose own declared type and name remain
|
||||
unpaired on the other side, so a member is never taken from its own changed
|
||||
counterpart. A member whose declaring type is unresolved could belong to any
|
||||
type, so it pairs by name only when no other member shares its exact
|
||||
CIL/signature key, never by structural shape alone, and any remap group it
|
||||
joins stays ambiguous. The exact-signature key uses names only as part of that
|
||||
full tuple; names alone never select a pair. Duplicate tuples remain ambiguous
|
||||
rather than being paired by token order. The raw signature is exact whether or
|
||||
not REA decoded it, so undecoded signatures pair by that tuple too, but they
|
||||
never enter structural rounds. An unmatched member is therefore reported as
|
||||
`unknown`, not added or removed, when its own signature was not decoded or when
|
||||
the other side has an unpaired member, one-sided or ambiguous, with the same
|
||||
declared type and name whose signature was not decoded.
|
||||
|
||||
For a matched method, an unavailable or partial body makes body-shape facets
|
||||
unknown while preserving observed signature differences. Structural identity
|
||||
compares normalized signatures, the limited decoded-CIL tuple fingerprint,
|
||||
unknown while preserving observed signature differences. For complete bodies,
|
||||
a difference in either the raw CIL digest or the decoded fingerprint is reported
|
||||
in the `cil` dimension, including reordered switch targets with equal decoded
|
||||
fingerprints. Structural identity compares normalized signatures, the limited
|
||||
decoded-CIL tuple fingerprint,
|
||||
constants, and bounded shape context. It reports all candidates at the winning
|
||||
score and remains ambiguous when the evidence does not distinguish them. The
|
||||
digest does not itself remap metadata tokens.
|
||||
@@ -360,8 +373,8 @@ Opcode names retain short/long forms and prefix opcodes. This makes the value a
|
||||
reproducible decoded-tuple fingerprint, not a canonical semantic CIL identity:
|
||||
it does not resolve tokens across MVIDs, does not commit full control flow or
|
||||
exception semantics, and does not replace the raw CIL digest. Cross-build
|
||||
comparison uses the value only as one unique-only structural tier and reports
|
||||
ambiguity when that evidence is insufficient.
|
||||
comparison requires equal raw CIL digests as well for exact CIL/signature
|
||||
matches and reports ambiguity when structural evidence is insufficient.
|
||||
|
||||
For example, a decoded `nop; ret` body serializes exactly as
|
||||
`[["nop","none",null],["ret","none",null]]` and has digest
|
||||
|
||||
@@ -58,7 +58,13 @@ side declares a required facet unavailable. An empty legacy facet without a
|
||||
coverage declaration is also unknown. A producer that actually observed an
|
||||
empty facet can declare it available so observed absence remains comparable.
|
||||
Call comparison additionally retains the Evidence relationship coverage
|
||||
checks. IDA declares the facets its selected adapter cannot provide; an empty
|
||||
checks. Ghidra and Hopper declare their observed dossier facets and complete
|
||||
provider-observed caller/callee sets, including empty sets. Missing decompiler
|
||||
output remains unavailable; complete call coverage does not claim every runtime
|
||||
target. Hopper's unavailable reference-kind metadata and reported unresolved
|
||||
calls still prevent equality conclusions for those dimensions. Analysis profiles
|
||||
version these declarations so older snapshots are not reused as current results.
|
||||
IDA declares the facets its selected adapter cannot provide; an empty
|
||||
unsupported array does not prove observed absence.
|
||||
Evidence and bundle imports reject unsupported fields in their envelopes,
|
||||
identities, manifests and other fixed metadata before authentication. Imports
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · **العربية** · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md)
|
||||
· [Ελληνικά](README_el.md)
|
||||
|
||||
# REA: هندسة عكسية لأي شيء
|
||||
|
||||
@@ -19,10 +20,14 @@
|
||||
|
||||
**[الموقع](https://rea.tools/ar/) · [الأدلة](https://rea.tools/ar/guides/) · [دراسات الحالة](https://rea.tools/ar/showcase/)**
|
||||
|
||||
[البدء السريع](#البدء-السريع) · [كيف يعمل REA](#كيف-يعمل-rea) · [ما الذي يمكنك تحليله](#ما-الذي-يمكنك-تحليله) · [دراسات الحالة](#دراسات-الحالة) · [الأسئلة الشائعة](#الأسئلة-الشائعة) · [التوثيق](#التوثيق)
|
||||
[البدء السريع](#البدء-السريع) · [كيف يعمل REA](#كيف-يعمل-rea) · [ما الذي يمكنك تحليله](#ما-الذي-يمكنك-تحليله) · [دراسات الحالة](#دراسات-الحالة) · [الأسئلة الشائعة](#الأسئلة-الشائعة) · [أدوات GNU objdump وRizin](../reverse-engineering-tools.md) · [التوثيق](#التوثيق)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> المصادر الرسمية لـ REA: [morluto/rea](https://github.com/morluto/rea), [rea.tools](https://rea.tools/), [rea-agents](https://www.npmjs.com/package/rea-agents) (npm).
|
||||
> استخدم روابط التثبيت والإصدارات المنشورة في هذه المصادر الرسمية. تشابه اسم المستودع أو نسخ ملف README لا يثبت أن التنزيل رسمي.
|
||||
|
||||
<br />
|
||||
|
||||
<img src="../assets/rea-hopper-analysis.png" alt="REA يشغّل جسر التحليل داخل Hopper لفحص ملف ثنائي للشيفرة الأصلية" width="1200" />
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · **Deutsch** · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md)
|
||||
· [Ελληνικά](README_el.md)
|
||||
|
||||
# REA: Alles reverse-engineeren
|
||||
|
||||
@@ -19,10 +20,14 @@
|
||||
|
||||
**[Website](https://rea.tools/de/) · [Anleitungen](https://rea.tools/de/guides/) · [Fallstudien](https://rea.tools/de/showcase/)**
|
||||
|
||||
[Schnellstart](#schnellstart) · [So funktioniert REA](#so-funktioniert-rea) · [Was du analysieren kannst](#was-du-analysieren-kannst) · [Fallstudien](#fallstudien) · [Häufige Fragen](#häufige-fragen) · [Dokumentation](#dokumentation)
|
||||
[Schnellstart](#schnellstart) · [So funktioniert REA](#so-funktioniert-rea) · [Was du analysieren kannst](#was-du-analysieren-kannst) · [Fallstudien](#fallstudien) · [Häufige Fragen](#häufige-fragen) · [GNU-objdump- und Rizin-Tools](../reverse-engineering-tools.md) · [Dokumentation](#dokumentation)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Offizielle Quellen für REA: [morluto/rea](https://github.com/morluto/rea), [rea.tools](https://rea.tools/), [rea-agents](https://www.npmjs.com/package/rea-agents) (npm).
|
||||
> Folge den Installations- und Release-Links aus diesen offiziellen Quellen. Ein ähnlicher Repository-Name oder eine kopierte README belegt nicht, dass ein Download offiziell ist.
|
||||
|
||||
<br />
|
||||
|
||||
<img src="../assets/rea-hopper-analysis.png" alt="REA startet seine Analysebrücke in Hopper und untersucht eine native Binärdatei" width="1200" />
|
||||
|
||||
@@ -0,0 +1,316 @@
|
||||
<div align="center">
|
||||
|
||||
[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md) · **Ελληνικά**
|
||||
|
||||
# REA: Reverse Engineer Anything
|
||||
|
||||
### Ένα MCP για αντίστροφη μηχανική δυαδικών αρχείων, εφαρμογών και της συμπεριφοράς τους κατά την εκτέλεση.
|
||||
|
||||
**Βρήκες μια λειτουργία που σου αρέσει; Μάθε πώς λειτουργεί, μέχρι το επίπεδο του δυαδικού κώδικα.**
|
||||
|
||||
[](https://www.npmjs.com/package/rea-agents)
|
||||
[](../mcp-contracts.md#generated-catalog)
|
||||
[](#τι-μπορείς-να-αναλύσεις)
|
||||
[](https://skills.sh/morluto/rea/reverse-engineer-anything)
|
||||
[](../../LICENSE)
|
||||
[](https://discord.gg/GkcryMnJDM)
|
||||
|
||||
<a href="https://trendshift.io/repositories/82054?utm_source=repository-badge&utm_medium=badge&utm_campaign=badge-repository-82054" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/repositories/82054" alt="morluto%2Frea | Trendshift" width="250" height="55"/></a>
|
||||
|
||||
**[Ιστότοπος](https://rea.tools/) · [Οδηγοί](https://rea.tools/guides/) · [Μελέτες περίπτωσης](https://rea.tools/showcase/)**
|
||||
|
||||
[Γρήγορη εκκίνηση](#γρήγορη-εκκίνηση) · [Πώς λειτουργεί το REA](#πώς-λειτουργεί-το-rea) · [Τι μπορείς να αναλύσεις](#τι-μπορείς-να-αναλύσεις) · [Μελέτες περίπτωσης](#μελέτες-περίπτωσης) · [Συχνές ερωτήσεις](#συχνές-ερωτήσεις) · [Τεκμηρίωση](#τεκμηρίωση)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Κατέβασε το REA από το [morluto/rea](https://github.com/morluto/rea),
|
||||
> το [rea.tools](https://rea.tools/) ή το [πακέτο rea-agents στο npm](https://www.npmjs.com/package/rea-agents).
|
||||
> Ακολούθησε τους συνδέσμους εγκατάστασης και εκδόσεων από αυτές τις επίσημες πηγές.
|
||||
> Ένα παρόμοιο όνομα αποθετηρίου ή ένα αντιγραμμένο README δεν αποδεικνύει ότι
|
||||
> ένα αρχείο προς λήψη προέρχεται από επίσημη πηγή.
|
||||
|
||||
<br />
|
||||
|
||||
<img src="../assets/rea-hopper-analysis.png" alt="Το REA ξεκινά τη γέφυρα ανάλυσής του μέσα στο Hopper κατά την εξέταση ενός δυαδικού αρχείου με κώδικα μηχανής" width="1200" />
|
||||
|
||||
<br /><br />
|
||||
|
||||
<table aria-label="Κοινότητα REA">
|
||||
<tr>
|
||||
<td align="center" width="360">
|
||||
<a href="https://discord.gg/GkcryMnJDM">
|
||||
<img src="../assets/discord.svg" height="42" alt="Discord" /><br />
|
||||
<strong>Έλα στην κοινότητα αντίστροφης μηχανικής</strong>
|
||||
</a><br />
|
||||
<sub>Discord · Ερωτήσεις και απαντήσεις · Παρουσίαση έργων</sub>
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<br />
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
Είδες σε μια εφαρμογή μια λειτουργία που θα ήθελες στο δικό σου προϊόν; Ζήτησε από τον πράκτορά σου (agent) να τη διερευνήσει με το REA. Μπορεί να εξετάσει την εφαρμογή χωρίς τον πηγαίο κώδικά της, να εξηγήσει πώς λειτουργεί η συγκεκριμένη δυνατότητα, να παρουσιάσει τα τεκμήρια και να υλοποιήσει μια αντίστοιχη έκδοση για το έργο σου.
|
||||
|
||||
Το REA συνδέει τον πράκτορά σου με εργαλεία εξέτασης δυαδικών αρχείων με κώδικα μηχανής, εφαρμογών JavaScript και Electron, assemblies .NET και ιστοτόπων. Μπορείς να χρησιμοποιήσεις τα ίδια εργαλεία και από το τερματικό. Η ανάλυση εκτελείται τοπικά και τα αποτελέσματα περιλαμβάνουν τα τεκμήρια και τους περιορισμούς στους οποίους βασίζεται κάθε συμπέρασμα.
|
||||
|
||||
Η διαδικασία εγκατάστασης καταχωρίζει το REA στον agent σου και εγκαθιστά τις αντίστοιχες οδηγίες εργασίας. Για ανάλυση κώδικα μηχανής μπορείς να χρησιμοποιήσεις μια υπάρχουσα εγκατάσταση του Hopper, του Ghidra ή του IDA. Η διαδικασία εγκατάστασης μπορεί επίσης να εγκαταστήσει το Hopper, εφόσον το εγκρίνεις. Η στατική ανάλυση JavaScript δεν χρειάζεται μηχανή ανάλυσης κώδικα μηχανής.
|
||||
|
||||
> **[Επισκέψου τον ιστότοπο του REA](https://rea.tools/)** για οδηγίες εγκατάστασης, εικονογραφημένους οδηγούς και πραγματικές μελέτες περίπτωσης.
|
||||
|
||||
## Γρήγορη εκκίνηση
|
||||
|
||||
### Ρύθμιση του agent σου
|
||||
|
||||
Αφού εγκαταστήσεις το Node.js και το npm, εκτέλεσε:
|
||||
|
||||
```bash
|
||||
npx rea-agents setup
|
||||
```
|
||||
|
||||
Επίλεξε τους agents σου, έλεγξε τις προτεινόμενες αλλαγές και ενέκρινέ τες. Η διαδικασία προσθέτει τον MCP server του REA και τις αντίστοιχες οδηγίες εργασίας, κρατώντας αντίγραφα ασφαλείας των υφιστάμενων ρυθμίσεων. Στη συνέχεια, επανεκκίνησε τον agent σου.
|
||||
|
||||
Υποστηρίζονται τα Claude Code, Codex, Cursor, Gemini CLI, Grok Build και [άλλοι agents](../installation.md#supported-agents). Για τη ρύθμιση των παρόχων ανάλυσης και τη μη αυτόματη καταχώριση MCP, δες τον οδηγό [εγκατάστασης και ρύθμισης](../installation.md).
|
||||
|
||||
### Ζήτησε από τον agent σου
|
||||
|
||||
```text
|
||||
Διερεύνησε πώς λειτουργεί η αναζήτηση στην εφαρμογή Notes, δείξε μου τα τεκμήρια και υλοποίησε μια παρόμοια λειτουργία για το έργο μου.
|
||||
```
|
||||
|
||||
Αντικατάστησε το Notes με την εφαρμογή που θέλεις να εξετάσεις και προσδιόρισε τη λειτουργία που θέλεις να κατανοήσεις.
|
||||
|
||||
### Χρήση από το τερματικό
|
||||
|
||||
Εξέτασε τον αποσυμπιεσμένο φάκελο μιας εφαρμογής JavaScript/Electron ή ένα αρχείο ASAR:
|
||||
|
||||
```bash
|
||||
npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json
|
||||
```
|
||||
|
||||
Το αποτέλεσμα περιλαμβάνει modules, εισαγωγές, τα όρια μεταξύ των μερών της εφαρμογής Electron και τα αντίστοιχα τεκμήρια. Αντικατάστησε τη διαδρομή με τη δική σου, για παράδειγμα `"D:/apps/example"` στα Windows.
|
||||
|
||||
Για να εγκαταστήσεις την εντολή `rea` για τακτική χρήση:
|
||||
|
||||
```bash
|
||||
npm install --global rea-agents
|
||||
rea --help
|
||||
```
|
||||
|
||||
Για ανάλυση κώδικα μηχανής, ρύθμισε πρώτα έναν πάροχο ανάλυσης. Ο [οδηγός CLI και Evidence](../cli.md) περιγράφει τις σχετικές εντολές, την επιλογή παρόχου, τα στιγμιότυπα και τη χρήση σε scripts.
|
||||
|
||||
### Ενημέρωση του REA
|
||||
|
||||
Το REA εξελίσσεται γρήγορα και οι νέες εκδόσεις περιλαμβάνουν συχνές διορθώσεις σφαλμάτων. Διατήρησε την εγκατάστασή σου ενημερωμένη.
|
||||
|
||||
Για CLI εγκατεστημένο με `npm install --global` ή `bun add --global`:
|
||||
|
||||
```bash
|
||||
rea update
|
||||
```
|
||||
|
||||
Οι ενημερώσεις καθολικής εγκατάστασης με Bun απαιτούν Bun 1.2.17 ή νεότερο. Αν χρησιμοποιείς παλαιότερη έκδοση, αναβάθμισε το Bun χειροκίνητα πριν εκτελέσεις το `rea update`.
|
||||
|
||||
Για να ανανεώσεις τις καταχωρίσεις των agents και το skill, εκτέλεσε την εντολή ρύθμισης που εμφανίζει η ενημέρωση.
|
||||
|
||||
Αν χρησιμοποιείς `npx`, ενημέρωσε τη ρύθμιση του agent με:
|
||||
|
||||
```bash
|
||||
npx rea-agents@latest setup
|
||||
```
|
||||
|
||||
Έλεγξε τις αλλαγές και επανεκκίνησε τον agent σου. Για μεμονωμένες εντολές CLI, χρησιμοποίησε `npx rea-agents@latest` και στη συνέχεια την εντολή που θέλεις. Με Bun, χρησιμοποίησε `bunx rea-agents@latest` αντί για `npx rea-agents@latest`.
|
||||
|
||||
## Πώς λειτουργεί το REA
|
||||
|
||||
Ο agent σου καλεί το REA μέσω MCP για να εξετάσει τον στόχο και να ακολουθήσει τον σχετικό κώδικα. Το REA επιστρέφει ευρήματα μαζί με τα τεκμήριά τους. Ο agent τα χρησιμοποιεί για να θέσει πρόσθετες ερωτήσεις, να εξηγήσει τη συμπεριφορά ή να γράψει και να δοκιμάσει μια υλοποίηση. Οι εντολές CLI χρησιμοποιούν τις ίδιες ροές εργασίας.
|
||||
|
||||

|
||||
|
||||
[Άνοιγμα του διαγράμματος σε πλήρες μέγεθος](../../website/public/assets/figures/rea-investigation-flow.svg).
|
||||
|
||||
<a id="current-status"></a>
|
||||
|
||||
## Τι μπορείς να αναλύσεις
|
||||
|
||||
Το REA απαιτεί Node.js 22.x (>=22.19), 24.x (>=24.11) ή 26+, καθώς και npm. Τα πρόσθετα εργαλεία και τα υποστηριζόμενα συστήματα εξαρτώνται από τον στόχο:
|
||||
|
||||
| Στόχος | Τι επιστρέφει το REA | Απαιτήσεις και οδηγός |
|
||||
| -------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Δυαδικά αρχεία με κώδικα μηχανής | Ψευδοκώδικα, κώδικα assembly, συμβολοσειρές, σύμβολα, κλήσεις και αναφορές | Hopper, Ghidra ή IDA· [ανάλυση κώδικα μηχανής](https://rea.tools/guides/native/) |
|
||||
| Δομή ELF χωρίς εκτέλεση | Ενότητες, τμήματα, αρχικά σύμβολα/εγγραφές μετεγκατάστασης και ενδείξεις πιθανών μέτρων προστασίας από στατική ανάλυση | pwntools που παρέχει ο χρήστης σε Linux x64· [διαγνωστικά δυαδικών αρχείων](../binary-diagnostics.md) |
|
||||
| Δυαδικά αρχεία Go | Ενσωματωμένη έκδοση μεταγλωττιστή, modules, αντικαταστάσεις και ρυθμίσεις μεταγλώττισης | Ενσωματωμένη στατική εξέταση ELF/PE/thin Mach-O· δεν απαιτείται εγκατάσταση του Go |
|
||||
| Δυαδικά αρχεία Rust | Ενσωματωμένα ονόματα, εκδόσεις, πηγές και σχέσεις εξάρτησης των crates από το cargo-auditable | Ενσωματωμένη στατική εξέταση ELF/PE/thin Mach-O· [δυαδικά αρχεία Rust](../rust-binaries.md) |
|
||||
| Αρχείο WASM | Έλεγχο εγκυρότητας, ενότητες, ακριβείς μορφές εισαγωγών/εξαγωγών σε WAT και αποκωδικοποιημένο WAT | WABT 1.0.42 που παρέχει ο χρήστης· `inspect_wasm_artifact` |
|
||||
| Bytecode EVM | Selectors επιλογής συνάρτησης, μετατοπίσεις σε bytes, εκτιμώμενα ορίσματα και δυνατότητα μεταβολής της κατάστασης | Τοπική είσοδος σε ακατέργαστα bytes ή δεκαεξαδική μορφή σε Linux x64· [οδηγός EVM χωρίς εκτέλεση](../evm-bytecode.md) |
|
||||
| Βάσεις δεδομένων SQLite | Πίνακες, στήλες, ευρετήρια, όψεις, triggers και επιλεγμένες γραμμές κανονικών πινάκων | Τοπικό στιγμιότυπο βάσης δεδομένων/WAL· Node.js 24.x >=24.15 ή 26+ με εγγενή όρια SQLite |
|
||||
| Καταγεγραμμένες καταρρεύσεις στο Linux | Ακατέργαστες εγγραφές notes, καταχωρητές/σήματα κάθε καταγεγραμμένου νήματος και προαιρετικές πιθανές αντιστοιχίσεις μνήμης | pwntools που παρέχει ο χρήστης σε Linux x64· προαιρετικά GDB/pwndbg· [καταγεγραμμένες καταρρεύσεις](../recorded-crashes.md) |
|
||||
| JavaScript / Electron | Modules, εισαγωγές, source maps, διαδρομές, IPC και σχέσεις με πρόσθετα κώδικα μηχανής | Node.js και npm· [ανάλυση εφαρμογών](https://rea.tools/guides/javascript/) |
|
||||
| Ιστότοποι | Δομή σελίδων, scripts, παρατηρήσεις δικτύου και ζητούμενα στιγμιότυπα οθόνης | Πρόγραμμα περιήγησης της οικογένειας Chrome· [ανάλυση μέσω προγράμματος περιήγησης](https://rea.tools/guides/browser/) |
|
||||
| Αποθηκευμένες καταγραφές δικτύου | Αιτήματα, αποκρίσεις, διαθέσιμα δεδομένα φορτίου και θέσεις στην πηγή | HAR ή SAZ του Fiddler Everywhere· mitmdump σε Linux για την εγγενή μορφή mitmproxy· [οδηγός καταγραφών](../web-network-captures.md) |
|
||||
| Assemblies .NET | Μεταδεδομένα, εντολές CIL, δηλωμένες εξαρτήσεις από κώδικα μηχανής και συγκρίσεις builds | Καμία πρόσθετη απαίτηση (μόνο στατική ανάλυση)· [οδηγός διαχειριζόμενου κώδικα](../managed-code-analysis.md) |
|
||||
| APK Android | Δηλώσεις manifest, κλάσεις, απομεταγλωττισμένες μέθοδοι και αναφορές | JADX χωρίς γραφικό περιβάλλον και πλήρες JDK σε Linux/macOS/Windows x64· [οδηγός Android](../android-analysis.md) |
|
||||
| Συσκευές Android | Συσκευές, πακέτα, διεργασίες, logs, dumps, στιγμιότυπα οθόνης και μεταφορές APK/αρχείων | adb που παρέχει ο χρήστης (εξομοιωτής ή φυσική συσκευή)· [οδηγός ADB](../adb-device-analysis.md) |
|
||||
| Έργα JEB | Μονάδες έργου από τη μηχανή ανάλυσης, ψηφιακά αποτυπώματα αρχείων και ψευδοκώδικα τύπων/μεθόδων | Client JEB που έχει ξεκινήσει ο χρήστης και παρέχει MCP, επιλεγμένος μέσω REA_JEB_MCP_URL· [οδηγός JEB](../jeb-analysis.md) |
|
||||
| Πόροι Android | Αποκωδικοποιημένο manifest, πίνακες συμβολοσειρών, τοπικές ρυθμίσεις και στοιχεία έκδοσης | Launcher του Apktool που παρέχει ο χρήστης· [οδηγός πόρων](../apktool-resource-analysis.md) |
|
||||
| Εφαρμογές Flutter | Hash του στιγμιότυπου Dart, ψηφιακά αποτυπώματα βιβλιοθηκών ανά ABI και στοιχεία προέλευσης του build της μηχανής | Καμία πρόσθετη απαίτηση (στατική ανάλυση APK)· [οδηγός Flutter](../flutter-analysis.md) |
|
||||
| Firmware | Περιοχές, αποτελέσματα εξαγωγής και παραπομπές για ανάλυση κώδικα μηχανής | Binwalk / Unblob σε Linux· [οδηγός firmware](../firmware-analysis.md) |
|
||||
| Καταγεγραμμένα πακέτα | Επιλεγμένα πεδία, επαναλαμβανόμενες τιμές, φίλτρα και κανόνες decode-as | Linux: TShark 4.x + prlimit· [οδηγός καταγραφών](../packet-captures.md) |
|
||||
| Σύνοψη πακέτων | Τερματικά σημεία, DNS, TLS SNI, κατανομή πρωτοκόλλων και αιτήματα HTTP | tshark και capinfos που παρέχει ο χρήστης (επαληθευμένα σε Linux)· [οδηγός pcap](../pcap-analysis.md) |
|
||||
| Καταγεγραμμένα λογικά σήματα | Σχολιασμούς αποκωδικοποιητή, εύρη δειγμάτων και διαγνωστικά | Linux: sigrok-cli 0.7.2 + prlimit· [καταγραφές σημάτων](../signal-recordings.md) |
|
||||
| Πακέτα και πόροι | Καταλόγους αρχείων, ψηφιακά αποτυπώματα, plists, δομή bundles της Apple και εξαγόμενους πόρους | [Οδηγός αρχείων και JavaScript](../javascript-artifact-reconstruction.md), [εφαρμογές Apple](../apple-application-analysis.md) |
|
||||
| Συμπεριφορά διεργασιών | Έξοδο τερματικού, αλληλεπιδράσεις, παρατηρήσεις τερματισμού και συστήματος αρχείων, καθώς και συγκρίσεις εκτελέσεων | Linux/macOS με εγγενές PTY· [καταγραφή διεργασιών](../process-capture.md) |
|
||||
|
||||
Για εξέταση δυαδικών αρχείων με κώδικα μηχανής χωρίς εκτέλεση, το REA υποστηρίζει
|
||||
επίσης GNU objdump και Rizin που παρέχει ο χρήστης. Δες τον
|
||||
[οδηγό εργαλείων αντίστροφης μηχανικής](../reverse-engineering-tools.md)
|
||||
για τη ρύθμιση των παρόχων, τις εντολές CLI και τους περιορισμούς των Evidence.
|
||||
Το REA δεν εγκαθιστά ούτε ενημερώνει κανέναν από τους δύο παρόχους.
|
||||
|
||||
Η στατική εξέταση JavaScript και .NET διαβάζει τα αρχεία που παρέχονται χωρίς να εκτελεί την εφαρμογή. Η καταγραφή κατά την εκτέλεση εκτελεί τον επιλεγμένο στόχο ή αλληλεπιδρά μαζί του με τα δικαιώματα του χρήστη σου. Κάθε σχετικός οδηγός περιγράφει τις επιπτώσεις της.
|
||||
|
||||
<a id="choosing-a-deep-analysis-provider"></a>
|
||||
|
||||
Οι μορφές κώδικα μηχανής και τα υποστηριζόμενα συστήματα διαφέρουν ανά πάροχο ανάλυσης. Δες τη [ρύθμιση Hopper και Ghidra](../installation.md#hopper), τον [οδηγό IDA](../ida-provider.md) (που περιλαμβάνει προαιρετική γέφυρα MCP για κώδικα μηχανής) και την [πειραματική υποστήριξη Ghidra στα Windows](../windows-ghidra-p0.md). Το Ghidra υποστηρίζει επίσης [ανάλυση DOS 16 bit](../ghidra-dos.md). Για μεγάλα δυαδικά αρχεία, αύξησε το χρονικό όριο εκκίνησης του παρόχου με `REA_GHIDRA_STARTUP_TIMEOUT_MS` ή `REA_HOPPER_STARTUP_TIMEOUT_MS`. Για την επιλογή παρόχου, δες τον [οδηγό CLI](../cli.md#choose-a-provider). Έλεγξε τη [διαθεσιμότητα ανά έκδοση](../installation.md#released-package-and-main) για λειτουργίες που προστέθηκαν μετά την τελευταία έκδοση στο npm.
|
||||
|
||||
## Μελέτες περίπτωσης
|
||||
|
||||
[](https://rea.tools/showcase/)
|
||||
|
||||
### DX-Ball: ανακατασκευή υπολογισμού στερεοφωνικής τοποθέτησης ήχου
|
||||
|
||||
Ακολούθησε μια κλήση ήχου μέχρι τη βοηθητική συνάρτηση που μετατρέπει τη θέση σε στερεοφωνική τοποθέτηση, εξέτασε τις εντολές και μετέτρεψε τον ελλιπή ψευδοκώδικα σε C. Η ανακατασκευή περνά 3.205 δοκιμές με τον αρχικό κώδικα x86 και αναπαράγει και τα 63 bytes της μεταγλωττισμένης συνάρτησης.
|
||||
|
||||
[Διάβασε τη μελέτη περίπτωσης](https://rea.tools/showcase/dx-ball/) · [Αποθετήριο ανακατασκευής](https://github.com/N0zoM1z0/dx-ball)
|
||||
|
||||
### Notion: ιχνηλάτηση της γέφυρας προχείρου στο Electron
|
||||
|
||||
Βρες το API προχείρου του renderer, ακολούθησέ το μέσω του preload και του IPC μέχρι την κύρια διεργασία και εξέτασε τη μορφή εμπλουτισμένου περιεχομένου του προχείρου.
|
||||
|
||||
[Διάβασε τη μελέτη περίπτωσης](https://rea.tools/showcase/notion/)
|
||||
|
||||
### TH04: ανάκτηση υπολογισμού δακτυλίου βλημάτων σε DOS
|
||||
|
||||
Εξέτασε τις εντολές 16 bit του αρχικού παιχνιδιού για PC-98, ανέκτησε τους υπολογισμούς σταθερών γωνιών και γωνιών στόχευσης και σύγκρινε τον ανακατασκευασμένο κώδικα C++ με την έξοδο του μεταγλωττιστή της εποχής.
|
||||
|
||||
[Διάβασε τη μελέτη περίπτωσης](https://rea.tools/showcase/th04/) · [Αποθετήριο ανακατασκευής](https://github.com/N0zoM1z0/th04)
|
||||
|
||||
Αν χρησιμοποίησες το REA για κάτι ενδιαφέρον, θα θέλαμε να το δούμε. Μοιράσου τη μελέτη σου σε ένα [issue](https://github.com/morluto/rea/issues) ή [pull request](https://github.com/morluto/rea/pulls), αναφέροντας τον στόχο, το ερώτημά σου, πώς βοήθησε το REA και τι ανακάλυψες.
|
||||
|
||||
## Συχνές ερωτήσεις
|
||||
|
||||
<details>
|
||||
<summary><strong>Ποιοι agents μπορούν να χρησιμοποιήσουν το REA;</strong></summary>
|
||||
|
||||
Κάθε agent που υποστηρίζει τοπικούς MCP servers. Η διαδικασία εγκατάστασης ρυθμίζει τους [υποστηριζόμενους agents](../installation.md#supported-agents). Οι υπόλοιποι clients μπορούν να χρησιμοποιήσουν τη [μη αυτόματη καταχώριση MCP](../installation.md#mcp-registry).
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Χρειάζομαι Hopper, Ghidra ή IDA;</strong></summary>
|
||||
|
||||
Η εις βάθος ανάλυση κώδικα μηχανής χρησιμοποιεί ένα από αυτά. Η στατική εξέταση JavaScript και .NET λειτουργεί χωρίς μηχανή ανάλυσης κώδικα μηχανής. Η διαδικασία εγκατάστασης μπορεί να εγκαταστήσει το Hopper μετά την έγκρισή σου· το Ghidra και το IDA χρησιμοποιούν τις υπάρχουσες εγκαταστάσεις σου. Δες τη [ρύθμιση παρόχων ανάλυσης](../installation.md#hopper).
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Πρέπει να ξεκινήσω πρώτα το Hopper;</strong></summary>
|
||||
|
||||
Το REA ξεκινά το Hopper όταν το χρειάζεται κάποια λειτουργία. Στο macOS, κατά την πρώτη εκκίνηση μπορεί να εμφανιστεί παράθυρο που ζητά να επιλέξεις τη δοκιμαστική λειτουργία ή να ενεργοποιήσεις την άδειά σου. Ολοκλήρωσε αυτό το βήμα στην ίδια συνεδρία χρήστη πριν από την πρώτη λειτουργία CLI/MCP ή εργασία χωρίς επίβλεψη. Άνοιξε το Hopper χειροκίνητα αν χρειαστεί. Δες την [εκκίνηση και αντιμετώπιση προβλημάτων του Hopper](../installation.md#launcher-paths-and-troubleshooting).
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Τι προσφέρει η εγκατάσταση του skill από το skills.sh;</strong></summary>
|
||||
|
||||
Το skill παρέχει οδηγίες διερεύνησης για τον agent σου. Χρησιμοποίησε `npx rea-agents setup` για να καταχωρίσεις τον MCP server του REA και να εγκαταστήσεις τις αντίστοιχες οδηγίες. Στη συνέχεια, επανεκκίνησε τον agent σου. Δες την [εγκατάσταση μόνο του skill](../installation.md#skill-only-installation).
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Τι κώδικα επιστρέφει το REA;</strong></summary>
|
||||
|
||||
Η ανάλυση κώδικα μηχανής επιστρέφει ψευδοκώδικα και assembly. Η ανάλυση JavaScript/Electron ανακτά modules και τις σχέσεις τους. Ο agent σου χρησιμοποιεί αυτά τα ευρήματα για να γράψει και να δοκιμάσει μια υλοποίηση. Οι [μελέτες περίπτωσης](#μελέτες-περίπτωσης) παρουσιάζουν ολοκληρωμένα παραδείγματα.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Το REA ανεβάζει την εφαρμογή μου στο διαδίκτυο;</strong></summary>
|
||||
|
||||
Το REA αναλύει τους στόχους τοπικά. Ο agent σου λαμβάνει τα αποτελέσματα των εργαλείων και ο πάροχος του μοντέλου του έχει τη δική του πολιτική δεδομένων.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Τι να κάνω αν συναντήσω κάποιο σφάλμα;</strong></summary>
|
||||
|
||||
Κάνε πρώτα ενημέρωση· μια πρόσφατη έκδοση μπορεί να το έχει ήδη διορθώσει.
|
||||
|
||||
Για CLI εγκατεστημένο με `npm install --global` ή `bun add --global`:
|
||||
|
||||
```bash
|
||||
rea update
|
||||
```
|
||||
|
||||
Για ρύθμιση agent μέσω `npx`:
|
||||
|
||||
```bash
|
||||
npx rea-agents@latest setup
|
||||
```
|
||||
|
||||
Αν χρησιμοποιείς agent, ολοκλήρωσε την [ανανέωση της ρύθμισης](#ενημέρωση-του-rea) και επανεκκίνησέ τον. Δοκίμασε ξανά την ίδια εργασία. Αν το πρόβλημα παραμένει, [άνοιξε ένα issue](https://github.com/morluto/rea/issues) με την έκδοση του REA, τον τύπο του στόχου, τα βήματα αναπαραγωγής και την έξοδο του σφάλματος.
|
||||
|
||||
</details>
|
||||
|
||||
## Τεκμηρίωση
|
||||
|
||||
Ξεκίνα από τους [οδηγούς με ολοκληρωμένα παραδείγματα](https://rea.tools/guides/) στον ιστότοπο. Για τις ακριβείς επιλογές, τις προϋποθέσεις και τις προδιαγραφές των αποτελεσμάτων:
|
||||
|
||||
- [Εγκατάσταση και ρύθμιση](../installation.md): καταχώριση agents, ρύθμιση παρόχων ανάλυσης, ενημερώσεις και απεγκατάσταση.
|
||||
- [Έλεγχος ετοιμότητας και αντιμετώπιση προβλημάτων](../installation.md#check-readiness-for-your-task): διάγνωση ενός agent ή μιας μηχανής ανάλυσης.
|
||||
- [CLI και Evidence](../cli.md): εντολές, επιλογή παρόχου, στιγμιότυπα, εισαγωγή/εξαγωγή και κωδικοί εξόδου.
|
||||
- [Προδιαγραφές MCP](../mcp-contracts.md) και [prompts για agents](../mcp-prompts.md): αποτελέσματα εργαλείων, συνεδρίες και καθοδηγούμενες διερευνήσεις.
|
||||
- [Κατάλογος εργαλείων](../mcp-contracts.md#generated-catalog): αυτόματα παραγόμενος κατάλογος εργαλείων, παρόχων και εντολών CLI.
|
||||
- [Οδικός χάρτης](../roadmap.md): προγραμματισμένες εργασίες και παρακολούθηση της ανάπτυξης δυνατοτήτων.
|
||||
|
||||
Ανάφερε ευπάθειες ακολουθώντας το [SECURITY.md](../../SECURITY.md).
|
||||
|
||||
## Ιστορικό αστεριών
|
||||
|
||||
🎉 **180.000 αστέρια στο GitHub — σας ευχαριστούμε!**
|
||||
|
||||
Ευχαριστούμε όλους όσοι χρησιμοποιούν το REA, αναφέρουν σφάλματα, ζητούν νέες λειτουργίες, δοκιμάζουν builds και συνεισφέρουν διορθώσεις.
|
||||
|
||||
<a href="https://www.star-history.com/?repos=morluto%2Frea&type=date">
|
||||
<picture>
|
||||
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=morluto/rea&type=date&theme=dark&legend=top-left" />
|
||||
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=morluto/rea&type=date" />
|
||||
<img alt="Ιστορικό αστεριών του REA στο GitHub" src="https://api.star-history.com/chart?repos=morluto/rea&type=date" />
|
||||
</picture>
|
||||
</a>
|
||||
|
||||
## Αποποίηση ευθύνης
|
||||
|
||||
Δες το [DISCLAIMER.md](../../DISCLAIMER.md) για οδηγίες υπεύθυνης χρήσης.
|
||||
|
||||
Η θέση του έργου σχετικά με την παράνομη ή μη εξουσιοδοτημένη χρήση ισχύει για όλες τις εκδόσεις του REA, τα κανάλια διανομής και τις πλατφόρμες. Κάθε αντίγραφο διατηρεί τις άδειες που το διέπουν.
|
||||
|
||||
Το REA προορίζεται για νόμιμη έρευνα, εκπαίδευση, διαλειτουργικότητα, διατήρηση λογισμικού και ανάλυση λογισμικού που σου ανήκει ή για το οποίο έχεις άδεια εξέτασης.
|
||||
|
||||
## Συνεισφορά
|
||||
|
||||
Θα χαρούμε να βοηθήσεις στο REA! [Άνοιξε ένα issue](https://github.com/morluto/rea/issues) για να αναφέρεις ένα σφάλμα ή να προτείνεις μια λειτουργία, ή [στείλε ένα pull request](https://github.com/morluto/rea/pulls) για να βελτιώσεις τον κώδικα ή την τεκμηρίωση.
|
||||
|
||||
Δες το [CONTRIBUTING.md](../../CONTRIBUTING.md) για τη ρύθμιση του περιβάλλοντος ανάπτυξης και τους ελέγχους, τον [οδηγό δοκιμών](../testing.md) για τις διαδικασίες επαλήθευσης και τον [χάρτη αρχιτεκτονικής](../architecture.mermaid) για τη δομή του έργου.
|
||||
|
||||
## Άδεια χρήσης
|
||||
|
||||
[MIT](../../LICENSE)
|
||||
|
||||
Ο κώδικας που έχει δημιουργηθεί για το REA διατίθεται με την άδεια MIT. Το υλικό τρίτων διατηρεί τις δικές του άδειες και δηλώσεις πνευματικών δικαιωμάτων. Δες τις [δηλώσεις για υλικό τρίτων](../../THIRD_PARTY_NOTICES.md) για τον κατάλογο εξαρτήσεων, τα διατηρημένα κείμενα αδειών και τους όρους των εξωτερικών εργαλείων.
|
||||
|
||||
**Πρέπει να αποκτήσεις νόμιμα τα εμπορικά εργαλεία και να έχεις δικαίωμα χρήσης τους**, συμπεριλαμβανομένων των IDA Pro / Hex-Rays decompilers, Hopper και JEB, σύμφωνα με τις αντίστοιχες άδειές τους. Το REA δεν παρέχει άδεια εργαλείου, κλειδί ή δικαίωμα ενεργοποίησης. Οι δοκιμαστικές εκδόσεις και τα demos εξακολουθούν να υπόκεινται στους όρους του προμηθευτή, συμπεριλαμβανομένων των περιορισμών χρήστη, λειτουργιών, αυτοματοποίησης και κοινοποίησης αποτελεσμάτων. Δεν επιτρέπονται σπασμένες εκδόσεις, κλεμμένα κλειδιά, μη εξουσιοδοτημένη κοινοποίηση κλειδιών ή παράκαμψη περιορισμών αδειοδότησης. Αυτές οι οδηγίες δεν τροποποιούν την άδεια MIT ή τις άδειες τρίτων.
|
||||
@@ -1,6 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · **Español** · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md)
|
||||
· [Ελληνικά](README_el.md)
|
||||
|
||||
# REA: Ingeniería inversa de cualquier cosa
|
||||
|
||||
@@ -19,10 +20,14 @@
|
||||
|
||||
**[Sitio web](https://rea.tools/es/) · [Guías](https://rea.tools/es/guides/) · [Casos prácticos](https://rea.tools/es/showcase/)**
|
||||
|
||||
[Inicio rápido](#inicio-rápido) · [Cómo funciona REA](#cómo-funciona-rea) · [Qué puedes analizar](#qué-puedes-analizar) · [Casos prácticos](#casos-prácticos) · [Preguntas frecuentes](#preguntas-frecuentes) · [Documentación](#documentación)
|
||||
[Inicio rápido](#inicio-rápido) · [Cómo funciona REA](#cómo-funciona-rea) · [Qué puedes analizar](#qué-puedes-analizar) · [Casos prácticos](#casos-prácticos) · [Preguntas frecuentes](#preguntas-frecuentes) · [Herramientas GNU objdump y Rizin](../reverse-engineering-tools.md) · [Documentación](#documentación)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Fuentes oficiales de REA: [morluto/rea](https://github.com/morluto/rea), [rea.tools](https://rea.tools/), [rea-agents](https://www.npmjs.com/package/rea-agents) (npm).
|
||||
> Usa los enlaces de instalación y de versiones publicados en estas fuentes oficiales. Un nombre de repositorio parecido o un README copiado no demuestra que una descarga sea oficial.
|
||||
|
||||
<br />
|
||||
|
||||
<img src="../assets/rea-hopper-analysis.png" alt="REA inicia su puente de análisis dentro de Hopper mientras inspecciona un binario nativo" width="1200" />
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
<div align="center" dir="rtl">
|
||||
|
||||
[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · **فارسی** · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md)
|
||||
· [Ελληνικά](README_el.md)
|
||||
|
||||
# REA: مهندسی معکوس هر چیزی
|
||||
|
||||
@@ -19,10 +20,14 @@
|
||||
|
||||
**[وبسایت](https://rea.tools/fa/) · [راهنماها](https://rea.tools/fa/guides/) · [نمونههای عملی](https://rea.tools/fa/showcase/)**
|
||||
|
||||
[شروع سریع](#شروع-سریع) · [REA چگونه کار میکند؟](#rea-چگونه-کار-میکند) · [چه چیزهایی را میتوان تحلیل کرد؟](#چه-چیزهایی-را-میتوان-تحلیل-کرد) · [نمونههای عملی](#نمونههای-عملی) · [پرسشهای متداول](#پرسشهای-متداول) · [مستندات](#مستندات)
|
||||
[شروع سریع](#شروع-سریع) · [REA چگونه کار میکند؟](#rea-چگونه-کار-میکند) · [چه چیزهایی را میتوان تحلیل کرد؟](#چه-چیزهایی-را-میتوان-تحلیل-کرد) · [نمونههای عملی](#نمونههای-عملی) · [پرسشهای متداول](#پرسشهای-متداول) · [ابزارهای GNU objdump و Rizin](../reverse-engineering-tools.md) · [مستندات](#مستندات)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> منابع رسمی REA: [morluto/rea](https://github.com/morluto/rea), [rea.tools](https://rea.tools/), [rea-agents](https://www.npmjs.com/package/rea-agents) (npm).
|
||||
> از پیوندهای نصب و انتشار در این منابع رسمی استفاده کنید. نام مشابه مخزن یا README کپیشده ثابت نمیکند که فایل دانلودی رسمی است.
|
||||
|
||||
<br />
|
||||
|
||||
<img src="../assets/rea-hopper-analysis.png" alt="اجرای پل تحلیل REA در Hopper هنگام بررسی یک فایل باینری بومی" width="1200" />
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · **Français** · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md)
|
||||
· [Ελληνικά](README_el.md)
|
||||
|
||||
# REA : rétro-ingénierie de tout
|
||||
|
||||
@@ -19,10 +20,14 @@
|
||||
|
||||
**[Site web](https://rea.tools/fr/) · [Guides](https://rea.tools/fr/guides/) · [Études de cas](https://rea.tools/fr/showcase/)**
|
||||
|
||||
[Démarrage rapide](#démarrage-rapide) · [Comment fonctionne REA](#comment-fonctionne-rea) · [Ce que vous pouvez analyser](#ce-que-vous-pouvez-analyser) · [Études de cas](#études-de-cas) · [Questions fréquentes](#questions-fréquentes) · [Documentation](#documentation)
|
||||
[Démarrage rapide](#démarrage-rapide) · [Comment fonctionne REA](#comment-fonctionne-rea) · [Ce que vous pouvez analyser](#ce-que-vous-pouvez-analyser) · [Études de cas](#études-de-cas) · [Questions fréquentes](#questions-fréquentes) · [Outils GNU objdump et Rizin](../reverse-engineering-tools.md) · [Documentation](#documentation)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Sources officielles de REA : [morluto/rea](https://github.com/morluto/rea), [rea.tools](https://rea.tools/), [rea-agents](https://www.npmjs.com/package/rea-agents) (npm).
|
||||
> Utilisez les liens d’installation et de publication de ces sources officielles. Un nom de dépôt similaire ou un README copié ne prouve pas qu’un téléchargement est officiel.
|
||||
|
||||
<br />
|
||||
|
||||
<img src="../assets/rea-hopper-analysis.png" alt="REA lance son pont d'analyse dans Hopper pendant l'inspection d'un binaire natif" width="1200" />
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · **Bahasa Indonesia** · [Italiano](README_it.md)
|
||||
· [Ελληνικά](README_el.md)
|
||||
|
||||
# REA: Rekayasa Balik Apa Pun
|
||||
|
||||
@@ -19,10 +20,14 @@
|
||||
|
||||
**[Situs Web](https://rea.tools/id/) · [Panduan](https://rea.tools/id/guides/) · [Contoh Penggunaan](https://rea.tools/id/showcase/)**
|
||||
|
||||
[Mulai Cepat](#mulai-cepat) · [Cara Kerja REA](#cara-kerja-rea) · [Hal yang Dapat Anda Analisis](#hal-yang-dapat-anda-analisis) · [Contoh Penggunaan](#contoh-penggunaan) · [FAQ](#faq) · [Dokumentasi](#dokumentasi)
|
||||
[Mulai Cepat](#mulai-cepat) · [Cara Kerja REA](#cara-kerja-rea) · [Hal yang Dapat Anda Analisis](#hal-yang-dapat-anda-analisis) · [Contoh Penggunaan](#contoh-penggunaan) · [FAQ](#faq) · [Alat GNU objdump dan Rizin](../reverse-engineering-tools.md) · [Dokumentasi](#dokumentasi)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Sumber resmi REA: [morluto/rea](https://github.com/morluto/rea), [rea.tools](https://rea.tools/), [rea-agents](https://www.npmjs.com/package/rea-agents) (npm).
|
||||
> Gunakan tautan instalasi dan rilis dari sumber resmi ini. Nama repositori yang mirip atau README yang disalin tidak membuktikan bahwa unduhan tersebut resmi.
|
||||
|
||||
<br />
|
||||
|
||||
<img src="../assets/rea-hopper-analysis.png" alt="REA menjalankan jembatan analisisnya di dalam Hopper saat memeriksa biner native" width="1200" />
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · **Italiano**
|
||||
· [Ελληνικά](README_el.md)
|
||||
|
||||
# REA: Reverse Engineer Anything
|
||||
|
||||
@@ -19,10 +20,14 @@
|
||||
|
||||
**[Sito web](https://rea.tools/it/) · [Guide](https://rea.tools/it/guides/) · [Showcase](https://rea.tools/it/showcase/)**
|
||||
|
||||
[Avvio rapido](#avvio-rapido) · [Come funziona REA](#come-funziona-rea) · [Cosa puoi analizzare](#cosa-puoi-analizzare) · [Showcase](#showcase) · [FAQ](#faq) · [Documentazione](#documentazione)
|
||||
[Avvio rapido](#avvio-rapido) · [Come funziona REA](#come-funziona-rea) · [Cosa puoi analizzare](#cosa-puoi-analizzare) · [Showcase](#showcase) · [FAQ](#faq) · [Strumenti GNU objdump e Rizin](../reverse-engineering-tools.md) · [Documentazione](#documentazione)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Fonti ufficiali di REA: [morluto/rea](https://github.com/morluto/rea), [rea.tools](https://rea.tools/), [rea-agents](https://www.npmjs.com/package/rea-agents) (npm).
|
||||
> Segui i link di installazione e delle versioni pubblicate da queste fonti ufficiali. Un nome di repository simile o un README copiato non dimostra che un download sia ufficiale.
|
||||
|
||||
<br />
|
||||
|
||||
<img src="../assets/rea-hopper-analysis.png" alt="REA che avvia il suo bridge di analisi dentro Hopper mentre ispeziona un binario nativo" width="1200" />
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · **日本語** · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md)
|
||||
· [Ελληνικά](README_el.md)
|
||||
|
||||
# REA:あらゆるものをリバースエンジニアリング
|
||||
|
||||
@@ -19,10 +20,14 @@
|
||||
|
||||
**[ウェブサイト](https://rea.tools/ja/) · [ガイド](https://rea.tools/ja/guides/) · [事例](https://rea.tools/ja/showcase/)**
|
||||
|
||||
[クイックスタート](#クイックスタート) · [REA の仕組み](#rea-の仕組み) · [分析できる対象](#分析できる対象) · [事例](#事例) · [よくある質問](#よくある質問) · [ドキュメント](#ドキュメント)
|
||||
[クイックスタート](#クイックスタート) · [REA の仕組み](#rea-の仕組み) · [分析できる対象](#分析できる対象) · [事例](#事例) · [よくある質問](#よくある質問) · [GNU objdump・Rizin ツール](../reverse-engineering-tools.md) · [ドキュメント](#ドキュメント)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> REA の公式配布元: [morluto/rea](https://github.com/morluto/rea), [rea.tools](https://rea.tools/), [rea-agents](https://www.npmjs.com/package/rea-agents) (npm).
|
||||
> インストールとリリースのリンクは、これらの公式配布元からたどってください。似たリポジトリ名やコピーされた README だけでは、公式の配布物とは確認できません。
|
||||
|
||||
<br />
|
||||
|
||||
<img src="../assets/rea-hopper-analysis.png" alt="REA が Hopper 内で分析ブリッジを起動し、ネイティブバイナリを調べる様子" width="1200" />
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · **한국어** · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md)
|
||||
· [Ελληνικά](README_el.md)
|
||||
|
||||
# REA: 무엇이든 리버스 엔지니어링
|
||||
|
||||
@@ -19,10 +20,14 @@
|
||||
|
||||
**[웹사이트](https://rea.tools/ko/) · [가이드](https://rea.tools/ko/guides/) · [사례](https://rea.tools/ko/showcase/)**
|
||||
|
||||
[빠른 시작](#빠른-시작) · [REA의 작동 방식](#rea의-작동-방식) · [분석할 수 있는 대상](#분석할-수-있는-대상) · [사례](#사례) · [자주 묻는 질문](#자주-묻는-질문) · [문서](#문서)
|
||||
[빠른 시작](#빠른-시작) · [REA의 작동 방식](#rea의-작동-방식) · [분석할 수 있는 대상](#분석할-수-있는-대상) · [사례](#사례) · [자주 묻는 질문](#자주-묻는-질문) · [GNU objdump 및 Rizin 도구](../reverse-engineering-tools.md) · [문서](#문서)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> REA 공식 배포처: [morluto/rea](https://github.com/morluto/rea), [rea.tools](https://rea.tools/), [rea-agents](https://www.npmjs.com/package/rea-agents) (npm).
|
||||
> 이 공식 배포처에서 제공하는 설치 및 릴리스 링크를 이용하세요. 비슷한 저장소 이름이나 복사된 README만으로는 공식 배포 파일인지 확인할 수 없습니다.
|
||||
|
||||
<br />
|
||||
|
||||
<img src="../assets/rea-hopper-analysis.png" alt="REA가 Hopper 안에서 분석 브리지를 실행해 네이티브 바이너리를 검사하는 모습" width="1200" />
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · **Polski** · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md)
|
||||
· [Ελληνικά](README_el.md)
|
||||
|
||||
# REA: Reverse Engineer Anything
|
||||
|
||||
@@ -19,10 +20,14 @@
|
||||
|
||||
**[Strona internetowa](https://rea.tools/pl/) · [Przewodniki](https://rea.tools/pl/guides/) · [Przykłady zastosowań](https://rea.tools/pl/showcase/)**
|
||||
|
||||
[Szybki start](#szybki-start) · [Jak działa REA](#jak-działa-rea) · [Co możesz analizować](#co-możesz-analizować) · [Przykłady zastosowań](#przykłady-zastosowań) · [FAQ](#faq) · [Dokumentacja](#dokumentacja)
|
||||
[Szybki start](#szybki-start) · [Jak działa REA](#jak-działa-rea) · [Co możesz analizować](#co-możesz-analizować) · [Przykłady zastosowań](#przykłady-zastosowań) · [FAQ](#faq) · [Narzędzia GNU objdump i Rizin](../reverse-engineering-tools.md) · [Dokumentacja](#dokumentacja)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Oficjalne źródła REA: [morluto/rea](https://github.com/morluto/rea), [rea.tools](https://rea.tools/), [rea-agents](https://www.npmjs.com/package/rea-agents) (npm).
|
||||
> Korzystaj z odnośników do instalacji i wydań w tych oficjalnych źródłach. Podobna nazwa repozytorium lub skopiowany README nie potwierdzają, że pobierany plik jest oficjalny.
|
||||
|
||||
<br />
|
||||
|
||||
<img src="../assets/rea-hopper-analysis.png" alt="REA uruchamia most analityczny w Hopperze podczas analizy natywnego pliku binarnego" width="1200" />
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · **Português (Brasil)** · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md)
|
||||
· [Ελληνικά](README_el.md)
|
||||
|
||||
# REA: Faça engenharia reversa de qualquer coisa
|
||||
|
||||
@@ -19,10 +20,14 @@
|
||||
|
||||
**[Site](https://rea.tools/pt-BR/) · [Guias](https://rea.tools/pt-BR/guides/) · [Casos de uso](https://rea.tools/pt-BR/showcase/)**
|
||||
|
||||
[Início rápido](#início-rápido) · [Como o REA funciona](#como-o-rea-funciona) · [O que você pode analisar](#o-que-você-pode-analisar) · [Casos de uso](#casos-de-uso) · [Perguntas frequentes](#perguntas-frequentes) · [Documentação](#documentação)
|
||||
[Início rápido](#início-rápido) · [Como o REA funciona](#como-o-rea-funciona) · [O que você pode analisar](#o-que-você-pode-analisar) · [Casos de uso](#casos-de-uso) · [Perguntas frequentes](#perguntas-frequentes) · [Ferramentas GNU objdump e Rizin](../reverse-engineering-tools.md) · [Documentação](#documentação)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Fontes oficiais do REA: [morluto/rea](https://github.com/morluto/rea), [rea.tools](https://rea.tools/), [rea-agents](https://www.npmjs.com/package/rea-agents) (npm).
|
||||
> Use os links de instalação e de versões dessas fontes oficiais. Um nome de repositório parecido ou um README copiado não comprova que um download seja oficial.
|
||||
|
||||
<br />
|
||||
|
||||
<img src="../assets/rea-hopper-analysis.png" alt="REA inicia sua ponte de análise dentro do Hopper enquanto inspeciona um binário nativo" width="1200" />
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · **Русский** · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md)
|
||||
· [Ελληνικά](README_el.md)
|
||||
|
||||
# REA: Реверс-инжиниринг чего угодно
|
||||
|
||||
@@ -19,10 +20,14 @@
|
||||
|
||||
**[Сайт](https://rea.tools/ru/) · [Руководства](https://rea.tools/ru/guides/) · [Примеры](https://rea.tools/ru/showcase/)**
|
||||
|
||||
[Быстрый старт](#быстрый-старт) · [Как работает REA](#как-работает-rea) · [Что можно анализировать](#что-можно-анализировать) · [Примеры](#примеры) · [Частые вопросы](#частые-вопросы) · [Документация](#документация)
|
||||
[Быстрый старт](#быстрый-старт) · [Как работает REA](#как-работает-rea) · [Что можно анализировать](#что-можно-анализировать) · [Примеры](#примеры) · [Частые вопросы](#частые-вопросы) · [Инструменты GNU objdump и Rizin](../reverse-engineering-tools.md) · [Документация](#документация)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Официальные источники REA: [morluto/rea](https://github.com/morluto/rea), [rea.tools](https://rea.tools/), [rea-agents](https://www.npmjs.com/package/rea-agents) (npm).
|
||||
> Используйте ссылки на установку и выпуски из этих официальных источников. Похожее название репозитория или скопированный README не подтверждают, что скачиваемый файл является официальным.
|
||||
|
||||
<br />
|
||||
|
||||
<img src="../assets/rea-hopper-analysis.png" alt="REA запускает мост анализа внутри Hopper при исследовании нативного бинарного файла" width="1200" />
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · **ไทย** · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md)
|
||||
· [Ελληνικά](README_el.md)
|
||||
|
||||
# REA: Reverse Engineer Anything
|
||||
|
||||
@@ -19,10 +20,14 @@
|
||||
|
||||
**[เว็บไซต์](https://rea.tools/th/) · [คู่มือ](https://rea.tools/th/guides/) · [ตัวอย่างการใช้งาน](https://rea.tools/th/showcase/)**
|
||||
|
||||
[เริ่มต้นอย่างรวดเร็ว](#เริ่มต้นอย่างรวดเร็ว) · [REA ทำงานอย่างไร](#rea-ทำงานอย่างไร) · [สิ่งที่วิเคราะห์ได้](#สิ่งที่วิเคราะห์ได้) · [ตัวอย่างการใช้งาน](#ตัวอย่างการใช้งาน) · [คำถามที่พบบ่อย](#คำถามที่พบบ่อย) · [เอกสาร](#เอกสาร)
|
||||
[เริ่มต้นอย่างรวดเร็ว](#เริ่มต้นอย่างรวดเร็ว) · [REA ทำงานอย่างไร](#rea-ทำงานอย่างไร) · [สิ่งที่วิเคราะห์ได้](#สิ่งที่วิเคราะห์ได้) · [ตัวอย่างการใช้งาน](#ตัวอย่างการใช้งาน) · [คำถามที่พบบ่อย](#คำถามที่พบบ่อย) · [เครื่องมือ GNU objdump และ Rizin](../reverse-engineering-tools.md) · [เอกสาร](#เอกสาร)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> แหล่งทางการของ REA: [morluto/rea](https://github.com/morluto/rea), [rea.tools](https://rea.tools/), [rea-agents](https://www.npmjs.com/package/rea-agents) (npm).
|
||||
> โปรดใช้ลิงก์ติดตั้งและรุ่นเผยแพร่จากแหล่งทางการเหล่านี้ ชื่อที่เก็บโค้ดที่คล้ายกันหรือ README ที่คัดลอกมาไม่ได้ยืนยันว่าไฟล์ดาวน์โหลดเป็นของทางการ
|
||||
|
||||
<br />
|
||||
|
||||
<img src="../assets/rea-hopper-analysis.png" alt="REA เริ่มบริดจ์การวิเคราะห์ภายใน Hopper ขณะตรวจสอบไบนารีเนทีฟ" width="1200" />
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · **Türkçe** · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md)
|
||||
· [Ελληνικά](README_el.md)
|
||||
|
||||
# REA: Her şeyi tersine mühendislikle inceleyin
|
||||
|
||||
@@ -19,10 +20,14 @@
|
||||
|
||||
**[Web sitesi](https://rea.tools/tr/) · [Rehberler](https://rea.tools/tr/guides/) · [Örnek çalışmalar](https://rea.tools/tr/showcase/)**
|
||||
|
||||
[Hızlı başlangıç](#hızlı-başlangıç) · [REA nasıl çalışır](#rea-nasıl-çalışır) · [Neleri analiz edebilirsiniz](#neleri-analiz-edebilirsiniz) · [Örnek çalışmalar](#örnek-çalışmalar) · [Sık sorulan sorular](#sık-sorulan-sorular) · [Belgeler](#belgeler)
|
||||
[Hızlı başlangıç](#hızlı-başlangıç) · [REA nasıl çalışır](#rea-nasıl-çalışır) · [Neleri analiz edebilirsiniz](#neleri-analiz-edebilirsiniz) · [Örnek çalışmalar](#örnek-çalışmalar) · [Sık sorulan sorular](#sık-sorulan-sorular) · [GNU objdump ve Rizin araçları](../reverse-engineering-tools.md) · [Belgeler](#belgeler)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> REA’nın resmî kaynakları: [morluto/rea](https://github.com/morluto/rea), [rea.tools](https://rea.tools/), [rea-agents](https://www.npmjs.com/package/rea-agents) (npm).
|
||||
> Kurulum ve sürüm bağlantılarını bu resmî kaynaklardan takip edin. Benzer bir depo adı veya kopyalanmış bir README, indirilen dosyanın resmî olduğunu kanıtlamaz.
|
||||
|
||||
<br />
|
||||
|
||||
<img src="../assets/rea-hopper-analysis.png" alt="REA, yerel bir ikili dosyayı incelerken Hopper içinde analiz köprüsünü başlatıyor" width="1200" />
|
||||
@@ -137,7 +142,7 @@ REA; Node.js 22.x (>=22.19), 24.x (>=24.11) veya 26+ ve npm gerektirir. Ek araç
|
||||
| Yerel makine kodu içeren ikili dosyalar | Sözde kod, assembly kodu, dizeler, semboller, çağrılar ve referanslar | Hopper, Ghidra veya IDA; [yerel kod analizi](https://rea.tools/tr/guides/native/) |
|
||||
| Çevrimdışı ELF yapısı | Bölümler, segmentler, özgün semboller/yeniden konumlandırmalar ve statik analizden olası güvenlik önlemleri | Linux x64 üzerinde çağıranın sağladığı pwntools; [ikili dosya tanılama](../binary-diagnostics.md) |
|
||||
| Go ikili dosyaları | Gömülü compiler sürümü, modüller, replacement'lar ve build ayarları | Yerleşik statik ELF/PE/thin Mach-O incelemesi; Go kurulumu gerekmez |
|
||||
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](../rust-binaries.md) |
|
||||
| Rust ikili dosyaları | Gömülü cargo-auditable crate adları, sürümleri, kaynakları ve bağımlılık ilişkileri | Yerleşik statik ELF/PE/thin Mach-O incelemesi; [Rust ikili dosyaları](../rust-binaries.md) |
|
||||
| WASM artefaktı | Doğrulama, bölümler, tam import/export WAT biçimleri ve decode edilmiş WAT | Çağıranın sağladığı WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM bayt kodu | Yönlendirme seçicileri, bayt ofsetleri, çıkarımla belirlenen argümanlar ve durum değiştirilebilirliği | Linux x64 üzerinde yerel ham bayt/onaltılık girdi; [çevrimdışı EVM rehberi](../evm-bytecode.md) |
|
||||
| SQLite veritabanları | Tablolar, sütunlar, index'ler, view'lar, trigger'lar ve seçilen sıradan tablo satırları | Yerel database/WAL snapshot'ı; native SQLite sınırlarıyla Node.js 24.x >=24.15 veya 26+ |
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · **Українська** · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md)
|
||||
· [Ελληνικά](README_el.md)
|
||||
|
||||
# REA: Реверс-інжиніринг будь-чого
|
||||
|
||||
@@ -19,10 +20,14 @@
|
||||
|
||||
**[Сайт](https://rea.tools/uk/) · [Посібники](https://rea.tools/uk/guides/) · [Приклади](https://rea.tools/uk/showcase/)**
|
||||
|
||||
[Швидкий старт](#швидкий-старт) · [Як працює REA](#як-працює-rea) · [Що можна аналізувати](#що-можна-аналізувати) · [Приклади](#приклади) · [Поширені запитання](#поширені-запитання) · [Документація](#документація)
|
||||
[Швидкий старт](#швидкий-старт) · [Як працює REA](#як-працює-rea) · [Що можна аналізувати](#що-можна-аналізувати) · [Приклади](#приклади) · [Поширені запитання](#поширені-запитання) · [Інструменти GNU objdump і Rizin](../reverse-engineering-tools.md) · [Документація](#документація)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Офіційні джерела REA: [morluto/rea](https://github.com/morluto/rea), [rea.tools](https://rea.tools/), [rea-agents](https://www.npmjs.com/package/rea-agents) (npm).
|
||||
> Користуйтеся посиланнями на встановлення та випуски з цих офіційних джерел. Схожа назва репозиторію або скопійований README не підтверджують, що завантажуваний файл є офіційним.
|
||||
|
||||
<br />
|
||||
|
||||
<img src="../assets/rea-hopper-analysis.png" alt="REA запускає міст аналізу всередині Hopper під час дослідження нативного бінарного файла" width="1200" />
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
[English](../../README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · **Tiếng Việt** · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md)
|
||||
· [Ελληνικά](README_el.md)
|
||||
|
||||
# REA: Kỹ thuật dịch ngược cho mọi thứ
|
||||
|
||||
@@ -19,10 +20,14 @@
|
||||
|
||||
**[Trang web](https://rea.tools/vi/) · [Hướng dẫn](https://rea.tools/vi/guides/) · [Các nghiên cứu điển hình](https://rea.tools/vi/showcase/)**
|
||||
|
||||
[Bắt đầu nhanh](#bắt-đầu-nhanh) · [Cách REA hoạt động](#cách-rea-hoạt-động) · [Những gì bạn có thể phân tích](#những-gì-bạn-có-thể-phân-tích) · [Các nghiên cứu điển hình](#các-nghiên-cứu-điển-hình) · [Câu hỏi thường gặp](#câu-hỏi-thường-gặp) · [Tài liệu](#tài-liệu)
|
||||
[Bắt đầu nhanh](#bắt-đầu-nhanh) · [Cách REA hoạt động](#cách-rea-hoạt-động) · [Những gì bạn có thể phân tích](#những-gì-bạn-có-thể-phân-tích) · [Các nghiên cứu điển hình](#các-nghiên-cứu-điển-hình) · [Câu hỏi thường gặp](#câu-hỏi-thường-gặp) · [Công cụ GNU objdump và Rizin](../reverse-engineering-tools.md) · [Tài liệu](#tài-liệu)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Các nguồn chính thức của REA: [morluto/rea](https://github.com/morluto/rea), [rea.tools](https://rea.tools/), [rea-agents](https://www.npmjs.com/package/rea-agents) (npm).
|
||||
> Hãy dùng các liên kết cài đặt và bản phát hành từ những nguồn chính thức này. Tên kho mã tương tự hoặc README được sao chép không chứng minh rằng tệp tải xuống là bản chính thức.
|
||||
|
||||
<br />
|
||||
|
||||
<img src="../assets/rea-hopper-analysis.png" alt="REA khởi chạy cầu nối phân tích bên trong Hopper khi kiểm tra tệp nhị phân mã máy" width="1200" />
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
[English](../../README.md) · [简体中文](README_zh.md) · **繁體中文** · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md)
|
||||
· [Ελληνικά](README_el.md)
|
||||
|
||||
# REA:逆向一切
|
||||
|
||||
@@ -19,10 +20,14 @@
|
||||
|
||||
**[網站](https://rea.tools/zh-TW/) · [指南](https://rea.tools/zh-TW/guides/) · [案例](https://rea.tools/zh-TW/showcase/)**
|
||||
|
||||
[快速開始](#快速開始) · [REA 的工作原理](#rea-的工作原理) · [可以分析什麼](#可以分析什麼) · [案例](#案例) · [常見問題](#常見問題) · [文件](#文件)
|
||||
[快速開始](#快速開始) · [REA 的工作原理](#rea-的工作原理) · [可以分析什麼](#可以分析什麼) · [案例](#案例) · [常見問題](#常見問題) · [GNU objdump 與 Rizin 工具](../reverse-engineering-tools.md) · [文件](#文件)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> REA 官方來源: [morluto/rea](https://github.com/morluto/rea), [rea.tools](https://rea.tools/), [rea-agents](https://www.npmjs.com/package/rea-agents) (npm).
|
||||
> 請透過這些官方來源提供的安裝與發布連結取得 REA。相似的儲存庫名稱或複製的 README 不能證明下載內容來自官方。
|
||||
|
||||
<br />
|
||||
|
||||
<img src="../assets/rea-hopper-analysis.png" alt="REA 在 Hopper 中啟動分析橋,檢查原生二進位檔案" width="1200" />
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
<div align="center">
|
||||
|
||||
[English](../../README.md) · **简体中文** · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md) · [Italiano](README_it.md)
|
||||
· [Ελληνικά](README_el.md)
|
||||
|
||||
# REA:逆向一切
|
||||
|
||||
@@ -19,10 +20,14 @@
|
||||
|
||||
**[网站](https://rea.tools/zh-CN/) · [指南](https://rea.tools/zh-CN/guides/) · [案例](https://rea.tools/zh-CN/showcase/)**
|
||||
|
||||
[快速开始](#快速开始) · [REA 的工作原理](#rea-的工作原理) · [可以分析什么](#可以分析什么) · [案例](#案例) · [常见问题](#常见问题) · [文档](#文档)
|
||||
[快速开始](#快速开始) · [REA 的工作原理](#rea-的工作原理) · [可以分析什么](#可以分析什么) · [案例](#案例) · [常见问题](#常见问题) · [GNU objdump 和 Rizin 工具](../reverse-engineering-tools.md) · [文档](#文档)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> REA 官方来源: [morluto/rea](https://github.com/morluto/rea), [rea.tools](https://rea.tools/), [rea-agents](https://www.npmjs.com/package/rea-agents) (npm).
|
||||
> 请通过这些官方来源提供的安装和发布链接获取 REA。相似的仓库名称或复制的 README 不能证明下载内容来自官方。
|
||||
|
||||
<br />
|
||||
|
||||
<img src="../assets/rea-hopper-analysis.png" alt="REA 在 Hopper 中启动分析桥,检查原生二进制文件" width="1200" />
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
# Reverse-engineering tool integrations
|
||||
|
||||
REA can call caller-supplied GNU objdump, GDB, and Rizin installations. REA does not install or update these tools. Static objdump/Rizin workflows use the same services from CLI and MCP. GDB's persistent MCP sessions and ephemeral CLI command are available only on Linux. Persistent debugger sessions exist only for the lifetime of the REA MCP server process.
|
||||
|
||||
In a shared MCP server, unrestricted Rizin and Cutter commands, GDB console
|
||||
commands, Rizin debugger start/close, and GDB debugger close retire any active
|
||||
binary analysis session after the operation. This prevents a fresh analysis
|
||||
provider from being rebound automatically while command effects may still be
|
||||
writing. After the command and any background writers have stopped, call
|
||||
`open_binary` explicitly before continuing analysis. A successful command
|
||||
response alone does not prove that background writers have stopped. An active
|
||||
Ghidra analysis session must be closed explicitly before an external mutation
|
||||
can run.
|
||||
|
||||
## GNU objdump
|
||||
|
||||
`inspect_with_objdump` and `rea inspect-with-objdump` capture stdout and stderr as inline Evidence. Artifact inspection Evidence records SHA-256 digests before and after the provider call and whether those observed digests differ; this cannot prove the exact bytes consumed if the artifact changes during execution. Choose one artifact operation: `file_headers`, `section_headers`, `symbols`, `relocations`, `dwarf`, `disassemble`, or `disassemble_all`. `disassemble` maps to `-d`; `disassemble_all` maps to `-D` and has different symbol-boundary semantics. The pathless `inspect_with_objdump {"operation":"architectures"}` MCP call or `rea list-objdump-architectures` CLI command maps to `-i` and produces subjectless provider-capability Evidence describing BFD targets compiled into the installed binary, not a guarantee that a target is supported.
|
||||
|
||||
For DWARF, REA does not follow local debug links by default. Set `follow_debug_links` to `true` to pass `-WK`; external debug-file identities are not bound to Evidence when following links is enabled. The CLI option is `--follow-debug-links true`. REA clears `DEBUGINFOD_URLS` in the objdump child environment to disable debuginfod network access on GNU binutils versions that do not support `-WE`. The complete raw output is authoritative; REA does not normalize it into a version-independent disassembly schema.
|
||||
|
||||
```sh
|
||||
rea inspect-with-objdump ./program disassemble
|
||||
rea inspect-with-objdump ./program dwarf --follow-debug-links false
|
||||
```
|
||||
|
||||
Captured stdout and stderr are retained up to a 16 MiB combined limit; `output_truncated` marks output beyond that limit.
|
||||
|
||||
## GDB
|
||||
|
||||
Persistent GDB sessions, including the MCP workflow and one-shot `rea debug-with-gdb` CLI, are available only on Linux. REA verifies a live inferior's output descriptors through `/proc` before detaching it; it does not claim equivalent descriptor verification on macOS. Windows is also unsupported because owned Job Object cleanup can terminate debugger inferiors. On supported Linux hosts, `start_gdb_session`, `gdb_console`, `gdb_session_status`, and `close_gdb_session` manage a persistent GDB/MI session in MCP. The one-shot CLI requires a caller-selected durable TTY: `rea debug-with-gdb --inferior-tty /dev/pts/7 'file ./program'`. The TTY must remain available after GDB exits so a detached inferior can continue writing. MCP starts GDB with `--nx --quiet --interpreter=mi3`, retries initialization with MI2 if MI3 initialization returns an error and cleanup succeeds, waits for its prompt, and disables auto-loading. Pass optional `inferior_tty` to select the terminal that future inferior runs use; REA configures it before the session becomes ready and checks the exact value with GDB/MI. `gdb_session_status` reports the selected path, whether REA still considers the configuration verified, and whether invalid UTF-8 has been observed in MI bytes (`mi_invalid_utf8`) or stderr (`stderr_invalid_utf8`). Commands that can change the inferior TTY or redirect a run revoke that verification permanently for the session, so close retains GDB ownership if a live inferior remains. Closing a session with one live inferior and a verified TTY automatically detaches the target, which can resume execution; MCP clients receive destructive-operation annotations for this behavior. The console remains unrestricted and can run shell commands, change target state, access files, and use network in the REA process user's security context. Raw MI records and console/target/log streams are returned with Evidence; command output is retained up to 16 MiB and explicitly marked when truncated.
|
||||
|
||||
Before a normal close, REA checks for live GDB thread groups. If one remains and a durable `inferior_tty` is verified, REA asks GDB to detach it and confirms the detach before stopping the debugger. Without that sink, select and apply a disposition to each inferior through `gdb_console` (for example, detach or kill as appropriate), then retry close. REA never infers target ownership from a PID or path. When the MCP server shuts down, REA terminates only its directly owned GDB process after confirming no inferior or a sink-backed detach; otherwise it retains the process and its output streams for retry.
|
||||
|
||||
```text
|
||||
start_gdb_session {"inferior_tty":"/dev/pts/7"}
|
||||
gdb_console {"session_id":"…","command":"file ./program"}
|
||||
gdb_console {"session_id":"…","command":"info files"}
|
||||
gdb_session_status {"session_id":"…"}
|
||||
close_gdb_session {"session_id":"…"}
|
||||
```
|
||||
|
||||
## Rizin
|
||||
|
||||
`start_rizin_debug_session` and `rea debug-with-rizin` require an explicit `inferior_tty` on Linux. Rizin launches the target with stdout and stderr routed to that caller-selected TTY and `dbg.exitkills=false`; the target can therefore continue running and writing to the TTY after REA closes the debugger. This is a visible target output effect, and the caller is responsible for selecting a suitable terminal that remains available. Before closing, REA serializes after the active command, parses the structured `olj` descriptor list, detaches only a unique `ptrace://` descriptor, confirms its absence, reads back `dbg.exitkills=false`, and verifies `/proc/<pid>/fd/1` and `/proc/<pid>/fd/2` still identify the selected TTY by device and inode. Any malformed or ambiguous readback keeps the Rizin session owned for retry. Persistent Rizin debugger sessions are unavailable outside Linux because this verification depends on `/proc`.
|
||||
|
||||
Persistent command Evidence retains the caller-selected and resolved startup paths in `parameters.launch_provenance`. These paths describe launch provenance, not the current target after unrestricted commands. `rizin_debug_session_status.output_invalid_utf8` reports whether lossy frame decoding has been observed during the session. Command Evidence sets `output_invalid_utf8` only when loss was observed in frames collected for that command, including retained partial output; earlier startup or command frames do not mark later valid output as lossy.
|
||||
|
||||
```sh
|
||||
rea debug-with-rizin --inferior-tty /dev/pts/7 ./program dr
|
||||
```
|
||||
|
||||
`execute_rizin_command` and `rea execute-rizin-command` run one selected command against a local artifact. The one-shot profile uses `-N -q -c`, suppressing user settings and scripts while leaving plugins available. Commands and plugins run with the REA process user's security context and may access files, the network, or perform other actions. REA preserves raw output and does not claim that command-specific JSON has one universal schema.
|
||||
|
||||
```sh
|
||||
rea execute-rizin-command ./program iI
|
||||
```
|
||||
|
||||
Captured stdout and stderr are retained up to a 16 MiB combined limit; `output_truncated` marks output beyond that limit.
|
||||
|
||||
## Cutter bridge
|
||||
|
||||
Cutter commands use the local REA Python bridge and authenticated loopback messages.
|
||||
The client and plugin must support the current v2 handshake; older peers are
|
||||
rejected before dispatch because they cannot preserve cross-caller recovery proof.
|
||||
The client checks the final encoded request against the bridge's 16 MiB frame
|
||||
limit before dispatch; an oversized command does not retain a cleanup owner.
|
||||
Discovery admits at most 256 descriptor candidates and reads and probes at most
|
||||
eight concurrently. A larger directory reports `bridge_directory_capacity_exceeded`.
|
||||
|
||||
Terminal receipts are recovered before another command, including from a new CLI
|
||||
invocation. A signed status response also confirms completion of older bridge
|
||||
connections after its UI dispatch, preserving recovery proof for a caller whose
|
||||
receipt another caller has acknowledged. Neither receipt recovery nor a status
|
||||
probe retries the original command. Failed acknowledgements remain pending and
|
||||
block new commands until they can be retried successfully.
|
||||
|
||||
## Configuration and coverage
|
||||
|
||||
The optional `REA_OBJDUMP_COMMAND`, `REA_GDB_COMMAND`, and `REA_RIZIN_COMMAND` values select executable names or paths; otherwise REA searches `PATH`. Values are passed directly to process spawning, not expanded as shell command lines. Provider versions and capabilities remain unknown until observed from the installed tool and verified in the corresponding real-provider lane.
|
||||
|
||||
Provider capture limits are separate from the MCP response budget. A result that fits the provider limit may still exceed the default 10 MiB MCP response budget and return a transport `resource_constraint`. Configure `REA_MCP_MAX_RESPONSE_BYTES` to match a larger client receive buffer when that client supports one; otherwise use retained Evidence/export or complete CLI JSON output. See [MCP response limits](./mcp-contracts.md#tool-results).
|
||||
+90
-7
@@ -262,8 +262,33 @@ CI owns coverage; aggregate and domain/contract thresholds are maintained in
|
||||
`vitest.config.ts`. They are glob-specific and never updated automatically.
|
||||
Coverage does not replace named boundary or real-provider scenarios.
|
||||
|
||||
CI runs six native Vitest shards without retries. Each shard emits a blob
|
||||
report; the merge job produces JUnit and JSON diagnostics, a module timing
|
||||
The `reverse-tools` commands run the focused tests and report coverage and
|
||||
mutation results without an additional percentage requirement. Test failures,
|
||||
invalid mutation configuration, and runner failures still fail the check.
|
||||
Mutation targets cover reviewed production behavior, including signal-terminated
|
||||
commands, Cutter descriptor identity, debugger cleanup ownership, GDB command
|
||||
exit and cancellation, and Rizin startup, timeout Evidence, and process exit.
|
||||
A second exhaustive pass runs the focused Cutter and Rizin timeout semantics tests against every compilable
|
||||
selected mutant to avoid relying on per-test mutation coverage associations;
|
||||
Stryker reports compile-error mutants separately because those cannot execute
|
||||
tests. Both passes resolve named syntax owners, scoped declarations, guards,
|
||||
callbacks, and reviewed expressions from the current TypeScript source; no
|
||||
hand-maintained line ranges select mutations. Missing, ambiguous, or malformed
|
||||
source targets stop configuration before mutation generation. The selections
|
||||
retain their review scope rather than expanding to whole source files. The
|
||||
focused suite also exercises affected MCP routes and the Python Cutter bridge.
|
||||
Contract, integration, and real-provider lanes complement these focused tests;
|
||||
simulated providers do not establish real-provider behavior.
|
||||
|
||||
Fast CI starts the source-only projects after classification, independently of
|
||||
build and static checks, and runs five built-dependent shards after the build.
|
||||
The six test jobs still select every portable test. Full qualification keeps six
|
||||
complete Vitest shards with coverage. Neither tier retries tests. Canonical
|
||||
project ownership lives in `scripts/lib/test-projects.mjs`, shared by Vitest,
|
||||
CI group execution, and development-test preparation. A test importing compiled
|
||||
verifier code belongs to the built group even if its directory is conformance.
|
||||
Source and built blobs have separate labels to prevent merge collisions.
|
||||
Each test job emits a blob report; the merge job produces JUnit and JSON diagnostics, a module timing
|
||||
report, plus aggregate coverage during full qualification. It retains available
|
||||
reports even when a shard fails, uploads available diagnostics, and writes the
|
||||
slowest files to the workflow summary. Partial reports are marked diagnostic
|
||||
@@ -298,6 +323,30 @@ portable test shard. These limits leave headroom for cold setup and full coverag
|
||||
while preventing a hung job from holding the active main run for hours. They do
|
||||
not bound runner queue waiting. Timeouts still fail the required gate.
|
||||
|
||||
The separate `CI timings` workflow also diagnoses missing, startup-failed,
|
||||
queued/running, approval-required, canceled, and failed PR CI through bounded,
|
||||
read-only API scans. Incomplete pagination or changing identity stays unknown.
|
||||
It never closes/reopens PRs, reruns workflows, posts comments, or changes gates.
|
||||
The timing manifest retains each run's event and head branch, including explicit
|
||||
unknowns, so PR runs cannot consume the eligible-main comparison budget.
|
||||
Its static failure report compares supported lint/duplication diagnostics with
|
||||
failed main runs using exact source, ancestry, task hashes, and toolchain proof.
|
||||
A match describes those diagnostics only; it does not establish that every
|
||||
failure in a job came from main. Duplication reports retain the actual checker
|
||||
exit result, so passing duplication checks preserve lint failures and caller
|
||||
thresholds remain owned by the producer. Missing checker outcomes leave
|
||||
those duplication diagnostics unclassified. Missing historical receipts remain unknown.
|
||||
The reporter executes only trusted default-branch code and parses whitelisted
|
||||
JSON artifact members without executing artifact content. Reports and limits
|
||||
are retained in `ci-health-diagnostics`; its ten-minute job limit covers the
|
||||
existing four-minute timing collection plus the bounded diagnostic stages.
|
||||
|
||||
Shard-count reduction remains deferred: the available successful timing sample
|
||||
covers an older inventory, and file phase weights are not job wall time. The
|
||||
one-source/five-built split retains six test jobs; it makes no hosted speedup
|
||||
claim. Current complete timings and completed candidate hosted runs are needed
|
||||
before enabling a smaller adaptive matrix.
|
||||
|
||||
The separate `CI timings` workflow batches recent completed CI and Release runs
|
||||
hourly or on manual dispatch. It uploads `ci-job-timings-batch` with raw API pages and
|
||||
a normalized point-in-time report for each collected workflow attempt, plus a
|
||||
@@ -346,6 +395,17 @@ that explicit lane, not the routine iteration requirement.
|
||||
|
||||
## Real-toolchain verification lanes
|
||||
|
||||
Linux Rizin debugger ownership changes require a separate real-provider gate on
|
||||
Rizin 0.7.4. Run
|
||||
`npm run test:focused -- tests/boundary/providers/RizinInferiorTty.test.ts`.
|
||||
This test verifies the caller-selected TTY against the actual inferior's file
|
||||
descriptors, confirms `dbg.exitkills=false`, detaches through the `olj`/`o-`
|
||||
protocol, and checks that the target survives debugger close and writes to the
|
||||
selected TTY. The suite is included in the reverse-tools Vitest and mutation
|
||||
typecheck selections; its real-provider assertions skip when Rizin 0.7.4 is
|
||||
unavailable. Run the focused command above on a host with that provider for the
|
||||
explicit real-provider verification gate.
|
||||
|
||||
Each real-toolchain command must require only the host tools needed to prove
|
||||
its stated claim. Use a host-native fixture for host/provider acceptance, and
|
||||
place optional cross-target formats or platform-specific runners in separate
|
||||
@@ -887,8 +947,13 @@ successful ancestor push run of this CI workflow, rather than the preceding
|
||||
push: changes from failed or cancelled intermediate runs remain in scope.
|
||||
Missing, untrusted, or non-ancestor history selects every lane. The lookup is
|
||||
bounded to the last 100 successful push runs; lookup failures broaden selection.
|
||||
Every code change still runs all six portable shards, without coverage
|
||||
instrumentation in this fast tier. This deliberately avoids relying on an
|
||||
Every code change still selects all six portable shards, without coverage
|
||||
instrumentation in this fast tier. Ordinary PR matrices cancel remaining
|
||||
shards after the first failure; passing PRs execute the complete suite. Main,
|
||||
`ci:full`, scheduled, manual, and release qualification retain every shard's
|
||||
results after a failure. Available failed-shard reports remain diagnostic only:
|
||||
failed, cancelled, skipped, or missing selected shards cannot pass the gate or
|
||||
publish successful timing history. This deliberately avoids relying on an
|
||||
import graph for generated files, filesystem inputs, subprocesses, or bridges.
|
||||
|
||||
Scheduled qualification is admitted hourly at minute 23. A forced full run at
|
||||
@@ -916,10 +981,11 @@ classification job records its selection and broad-fallback reasons in the run
|
||||
summary. Representative Git-diff cases live in
|
||||
`tests/boundary/filesystem/ciChangeScope.test.ts`.
|
||||
|
||||
`CI required` always runs and checks classification and every selected lane,
|
||||
`CI required` checks classification and every selected lane,
|
||||
rejecting failed, cancelled, or unexpectedly skipped work. It rejects missing or
|
||||
unknown tiers and requires every canonical scope to be explicitly `true` or
|
||||
`false`; incomplete classification cannot silently omit verification lanes.
|
||||
The aggregate remains cancellable so superseded runs release their slots.
|
||||
Branch protection
|
||||
should require this aggregate, `Static checks`, `Portable tests`, and
|
||||
`Conventional PR title`. Title validation runs separately on PR opening,
|
||||
@@ -968,7 +1034,7 @@ filename cases.
|
||||
|
||||
POSIX package verification resolves the runner's configured npm cache before
|
||||
creating disposable client homes and passes that location to its child installs.
|
||||
This reuses the download cache restored by `setup-node` while keeping client
|
||||
This reuses the download cache restored by the shared npm-cache action while keeping client
|
||||
configuration isolated. Installs retain their existing registry and integrity
|
||||
checks.
|
||||
|
||||
@@ -998,7 +1064,24 @@ thresholds remain enforced in full qualification.
|
||||
Missing or unusable history falls back to Vitest's default assignment. Only
|
||||
timing data is reused; live provider results are never cached. Turbo outputs are
|
||||
restored by task hash, with trusted default-branch runs as cache writers and PRs
|
||||
as restore-only consumers. npm caches hold downloads, not `node_modules`.
|
||||
as restore-only consumers. The static job also restores its Turbo snapshot;
|
||||
each task revalidates its own input hash before reusing outputs or diagnostics.
|
||||
The Node pin and npm install settings are global hash inputs. Knip includes its ignore rules and VitePress
|
||||
entry, so changes to those inputs cannot reuse an older successful check.
|
||||
|
||||
The shared npm-cache action caches downloads, not `node_modules`, and every
|
||||
consumer still runs `npm ci`. Keys cover the runner OS, actual Node architecture,
|
||||
root lockfiles, and `.npmrc`. Only push, schedule, and manual events on the
|
||||
repository's default branch may save shared npm caches; PRs restore only.
|
||||
Static snapshots use the same write authority and are saved only after the
|
||||
static job's validation succeeds. Main warms the new cache namespace after
|
||||
merge; a cold cache or changed lockfile falls back to the normal install.
|
||||
This prevents each PR from saving another branch-scoped copy of the same
|
||||
download cache. It does not remove runner assignment waits.
|
||||
|
||||
Portable owner-based test selection remains observational. Its reports compare
|
||||
proposed omissions with the full run; incomplete or skipped observations do not
|
||||
establish that fewer tests would preserve regression detection.
|
||||
|
||||
The eight automatic browser, binary-layout, EVM, JavaScript-recovery,
|
||||
recorded-crash, and web provider lanes download `rea-build-cache` from their own
|
||||
|
||||
@@ -224,6 +224,22 @@ are 0 and 50. The view groups request lines by method, WHATWG URL origin and
|
||||
path template, sorted by origin, template and method. Query strings, fragments,
|
||||
credentials, headers and bodies are not copied into groups.
|
||||
|
||||
Select `include_query_parameter_names: true` in the view to include distinct
|
||||
observed query names per group. The default and explicit `false` omit the
|
||||
`query_parameter_names` field. Selected groups include an array of
|
||||
`{name, record_ordinals}` entries, sorted by Unicode code point; each name's
|
||||
ordinals follow the parent's selected record order and occur once per record.
|
||||
For `/products?category=books&page=1` and
|
||||
`/products?category=games&sort=price`, names are `category`, `page` and `sort`.
|
||||
Query values and fragments are not copied into this inventory.
|
||||
|
||||
Names use WHATWG `URLSearchParams` form-query decoding: literal `+` becomes
|
||||
space, `%2B` becomes `+`, percent escapes decode as UTF-8, and empty names are
|
||||
retained. Empty query pairs are ignored. Original URLs stay in parent Evidence;
|
||||
name decoding does not normalize the recorded paths used for grouping.
|
||||
The inventory describes observed names, without inferring required parameters,
|
||||
value types or a complete API schema. Sensitive-parent refusals still apply.
|
||||
|
||||
| Rule | Exact segment format | Placeholder |
|
||||
| ---------- | ---------------------------------------------------- | ------------ |
|
||||
| `numeric` | ASCII digits only | `{numeric}` |
|
||||
@@ -269,13 +285,16 @@ and validation scratch by walking the parent without cloning it. Inline requests
|
||||
borrow the Evidence value through CLI/MCP envelope validation; the application
|
||||
admits the entire portable envelope before authenticated parsing or hashing.
|
||||
The envelope clone, hash scratch and capture clone stay charged during grouping.
|
||||
Temporary URL/path/MIME work and retained state must
|
||||
Temporary URL/path/MIME/query work and retained state must
|
||||
fit together within a 96 MiB projection representation budget. A whole group or
|
||||
merge is admitted before allocating its retained sets, map entries or array
|
||||
values. Fixed group storage, distinct text, exclusions and limitations count as
|
||||
well as matched segments. Per-record reservations are released after each
|
||||
record; repeated values do not accumulate another retained charge. Only the
|
||||
requested page is materialized as output groups.
|
||||
Opt-in query projection reserves parser tuples and the per-record name set
|
||||
before decoding, then admits distinct retained names and source ordinals before
|
||||
changing a group. Repeated names within a request retain that ordinal once.
|
||||
|
||||
The representation budget uses conservative storage and worst-case JSON-escaping
|
||||
allowances; it is not an enforced V8 heap or process-memory limit. It is separate
|
||||
|
||||
@@ -19,6 +19,10 @@ Follow-up: [SQLite existing-control measurements](sqlite-existing-controls.md)
|
||||
tests QGIS's shipped GeoPackage through the existing column selection, preserving
|
||||
full selected-table schema and typed cells before considering another API.
|
||||
|
||||
Follow-up: [small real PCAP recordings](pcap-small-captures.md) compares existing
|
||||
complete delivery with summary/pages. Preserve the negative SDK/model input
|
||||
results: compact captures do not justify another production mechanism.
|
||||
|
||||
Follow-up: [native PE/DOS entry views and real model verification](native-entry-views.md)
|
||||
records the post-merge native continuation, including a small-function case where
|
||||
four selected replies cost more bytes than the complete dossier.
|
||||
|
||||
@@ -180,7 +180,7 @@ def main():
|
||||
'inspect_android_package', 'search_android_classes',
|
||||
'inspect_android_class', 'inspect_android_method',
|
||||
'inspect_firmware_regions', 'inspect_managed_artifact',
|
||||
'inspect_sqlite_database',
|
||||
'inspect_sqlite_database', 'inspect_pcap_capture',
|
||||
'inspect_managed_members', 'inspect_managed_native_boundaries',
|
||||
'inspect_artifact', 'inspect_wasm_artifact',
|
||||
'inspect_evm_interface', 'get_evidence_bundle', 'export_evidence_bundle',
|
||||
|
||||
@@ -70,6 +70,44 @@ for (const [side, answer] of [
|
||||
value: answer.evidence_ids,
|
||||
});
|
||||
}
|
||||
// PCAP delivery observations preserve each actual receipt's identity, limitations
|
||||
// and view coverage. Summary/page scopes differ from complete source scope;
|
||||
// retain them separately rather than equating page exhaustion with capture
|
||||
// coverage. Repeated equivalent deliveries must also match every observation
|
||||
// field except the receipt and parent IDs, which identify their own invocations.
|
||||
if (
|
||||
["wireshark-http.cap", "wireshark-http_redirects.pcapng"].includes(
|
||||
left.artifact,
|
||||
)
|
||||
) {
|
||||
for (const [side, answer] of [
|
||||
["left", a],
|
||||
["right", b],
|
||||
]) {
|
||||
if (
|
||||
!Array.isArray(answer.delivery_observations) ||
|
||||
!answer.delivery_observations.length
|
||||
)
|
||||
throw new Error("Missing original PCAP delivery observations");
|
||||
runSpecific.push({
|
||||
side,
|
||||
path: ["delivery_observations"],
|
||||
value: answer.delivery_observations,
|
||||
});
|
||||
if (left.delivery === right.delivery) {
|
||||
answer.delivery_observations = answer.delivery_observations.map(
|
||||
(observation) =>
|
||||
Object.fromEntries(
|
||||
Object.entries(observation).filter(
|
||||
([key]) => key !== "evidence_id" && key !== "parent_evidence_id",
|
||||
),
|
||||
),
|
||||
);
|
||||
} else {
|
||||
delete answer.delivery_observations;
|
||||
}
|
||||
}
|
||||
}
|
||||
// Binwalk assigns a new opaque region UUID in each invocation. Preserve both
|
||||
// original lookup values in this report; compare the observed region facts
|
||||
// separately. The product and original replies retain those IDs unchanged.
|
||||
|
||||
@@ -263,6 +263,34 @@
|
||||
"url": "https://raw.githubusercontent.com/qgis/QGIS/e1223b8e9e71fc50ee995ed556f711c54ccc101b/resources/data/world_map.gpkg",
|
||||
"derivation": "unmodified official resource file; streaming Git blob and SHA-256 verification"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "wireshark-http.cap",
|
||||
"file": "http.cap",
|
||||
"bytes": 25803,
|
||||
"sha256": "25a72bdf10339f2c29916920c8b9501d294923108de8f29b19aba7cc001ab60d",
|
||||
"provenance": {
|
||||
"project": "Wireshark public sample captures",
|
||||
"source_page": "https://wiki.wireshark.org/SampleCaptures",
|
||||
"url": "https://wiki.wireshark.org/uploads/27707187aeb30df68e70c8fb9d614981/http.cap",
|
||||
"resolved_url": "https://gitlab.com/wireshark/wireshark/-/wikis/uploads/27707187aeb30df68e70c8fb9d614981/http.cap",
|
||||
"upstream_expected_digest": null,
|
||||
"derivation": "Unmodified capture published by official Wireshark wiki. Local digest pins downloaded bytes; no independently published upstream digest is claimed."
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "wireshark-http_redirects.pcapng",
|
||||
"file": "http_redirects.pcapng",
|
||||
"bytes": 47804,
|
||||
"sha256": "ff432a6b71d06cd20720f7763e584014edd0a43c3b55981c682d1b7640207abc",
|
||||
"provenance": {
|
||||
"project": "Wireshark public sample captures",
|
||||
"source_page": "https://wiki.wireshark.org/SampleCaptures",
|
||||
"url": "https://wiki.wireshark.org/uploads/__moin_import__/attachments/SampleCaptures/http_redirects.pcapng",
|
||||
"resolved_url": "https://gitlab.com/wireshark/wireshark/-/wikis/uploads/__moin_import__/attachments/SampleCaptures/http_redirects.pcapng",
|
||||
"upstream_expected_digest": null,
|
||||
"derivation": "Unmodified capture published by official Wireshark wiki. Local digest pins downloaded bytes; no independently published upstream digest is claimed."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -523,7 +523,207 @@ const inspectEvmTarget = async () => {
|
||||
};
|
||||
};
|
||||
|
||||
const PCAP_EVIDENCE_ID = /^ev_[a-f0-9]{64}$/u;
|
||||
const pcapObservation = (evidence, requestedView, parseEvidence) => {
|
||||
// Authenticate with the selected runtime's owner, preserving the original
|
||||
// envelope for measurement and provenance rather than reconstructing it.
|
||||
parseEvidence(evidence);
|
||||
const evidenceId = evidence.evidence_id;
|
||||
const value = normalized(evidence);
|
||||
for (const key of ["limitations", "unknowns"])
|
||||
if (
|
||||
Object.hasOwn(value, key) &&
|
||||
(!Array.isArray(value[key]) ||
|
||||
!value[key].every((item) => typeof item === "string"))
|
||||
)
|
||||
throw new Error("PCAP normalized provenance lacks a valid string list");
|
||||
const target =
|
||||
value.kind === "summary"
|
||||
? value.summary?.target
|
||||
: value.kind === "pcap"
|
||||
? value.artifact
|
||||
: value.target;
|
||||
if (
|
||||
evidence.subject?.digest?.sha256 !== inputSha ||
|
||||
evidence.subject?.local_path !== values.path ||
|
||||
target?.sha256 !== inputSha ||
|
||||
target?.path !== values.path ||
|
||||
(value.kind === "summary" &&
|
||||
(value.artifact?.sha256 !== inputSha ||
|
||||
value.artifact?.path !== values.path))
|
||||
)
|
||||
throw new Error("PCAP Evidence does not identify the selected capture");
|
||||
// Summary delivery delegates to a retained view, whose Evidence operation is
|
||||
// inspect_analysis_view even when inspect_pcap_capture was requested.
|
||||
const operation = requestedView
|
||||
? "inspect_analysis_view"
|
||||
: "inspect_pcap_capture";
|
||||
if (evidence.operation !== operation)
|
||||
throw new Error(
|
||||
"PCAP Evidence operation differs from the requested delivery",
|
||||
);
|
||||
if (
|
||||
requestedView &&
|
||||
(typeof value.parent_evidence_id !== "string" ||
|
||||
!PCAP_EVIDENCE_ID.test(value.parent_evidence_id) ||
|
||||
value.parent_evidence_id === evidenceId ||
|
||||
!Array.isArray(evidence.evidence_links) ||
|
||||
!evidence.evidence_links.includes(value.parent_evidence_id))
|
||||
)
|
||||
throw new Error("PCAP retained parent lacks a distinct Evidence link");
|
||||
if (
|
||||
requestedView &&
|
||||
value.parent_digest !== value.parent_evidence_id.slice(3)
|
||||
)
|
||||
throw new Error("PCAP retained parent digest differs from its identity");
|
||||
const parameters = requestedView
|
||||
? { parent_evidence_id: value.parent_evidence_id, view: requestedView }
|
||||
: { path: values.path };
|
||||
if (
|
||||
!isDeepStrictEqual(evidence.parameters, parameters) ||
|
||||
(requestedView && !isDeepStrictEqual(value.view, requestedView))
|
||||
)
|
||||
throw new Error("PCAP Evidence parameters or view differ from the request");
|
||||
return {
|
||||
evidence_id: evidenceId,
|
||||
limitations: evidence.limitations,
|
||||
...Object.fromEntries(
|
||||
[
|
||||
"kind",
|
||||
"view",
|
||||
"parent_evidence_id",
|
||||
"coverage",
|
||||
"limitations",
|
||||
"unknowns",
|
||||
]
|
||||
.filter((key) => Object.hasOwn(value, key) && key !== "limitations")
|
||||
.map((key) => [key, value[key]]),
|
||||
),
|
||||
...(Object.hasOwn(value, "limitations")
|
||||
? { normalized_limitations: value.limitations }
|
||||
: {}),
|
||||
};
|
||||
};
|
||||
const inspectPcapTarget = async () => {
|
||||
const { parseEvidence } = await import(
|
||||
pathToFileURL(resolve(values["runtime-root"], "dist/domain/evidence.js"))
|
||||
.href
|
||||
);
|
||||
const input = {
|
||||
path: values.path,
|
||||
...(values.delivery === "summary" ? { detail: "summary" } : {}),
|
||||
};
|
||||
const definition = catalogDefinitions.get("inspect_pcap_capture");
|
||||
const ajv = new Ajv2020({ strict: false, validateFormats: false });
|
||||
if (
|
||||
!definition ||
|
||||
!ajv.validateSchema(definition.inputSchema) ||
|
||||
!ajv.validate(definition.inputSchema, input)
|
||||
)
|
||||
throw new Error("Selected PCAP request is not advertised by this runtime");
|
||||
const inspected = await call("inspect_pcap_capture", input);
|
||||
const n = normalized(inspected);
|
||||
const observations = [
|
||||
pcapObservation(
|
||||
inspected,
|
||||
values.delivery === "summary" ? { kind: "summary" } : undefined,
|
||||
parseEvidence,
|
||||
),
|
||||
];
|
||||
let capture, uris;
|
||||
if (values.delivery === "summary") {
|
||||
if (n.kind !== "summary" || n.parent_operation !== "inspect_pcap_capture")
|
||||
throw new Error("Runtime returned no requested PCAP summary");
|
||||
if (
|
||||
!isDeepStrictEqual(n.coverage, {
|
||||
status: "complete-within-view",
|
||||
examined: 1,
|
||||
total: 1,
|
||||
next_offset: null,
|
||||
exhausted: true,
|
||||
})
|
||||
)
|
||||
throw new Error("PCAP summary coverage differs from its complete view");
|
||||
capture = n.summary;
|
||||
const parentEvidenceId = observations[0].parent_evidence_id;
|
||||
const view = { kind: "pcap", facet: "http-uris", offset: 0, limit: 2 };
|
||||
const selected = await call("inspect_analysis_view", {
|
||||
source: { kind: "retained-evidence", evidence_id: parentEvidenceId },
|
||||
view,
|
||||
});
|
||||
const page = normalized(selected);
|
||||
const observation = pcapObservation(selected, view, parseEvidence);
|
||||
const total = capture.retained_counts.http_request_uris;
|
||||
const examined = Math.min(view.limit, total);
|
||||
const expectedCoverage = {
|
||||
status: "page",
|
||||
examined,
|
||||
total,
|
||||
next_offset: examined < total ? examined : null,
|
||||
exhausted: examined >= total,
|
||||
};
|
||||
if (
|
||||
page.parent_evidence_id !== parentEvidenceId ||
|
||||
page.parent_operation !== "inspect_pcap_capture" ||
|
||||
page.kind !== "pcap" ||
|
||||
page.page.facet !== "http-uris" ||
|
||||
page.source_coverage !== capture.coverage ||
|
||||
!isDeepStrictEqual(page.coverage, expectedCoverage)
|
||||
)
|
||||
throw new Error(
|
||||
"PCAP page identity, scope or coverage differs from its parent",
|
||||
);
|
||||
uris = page.page.items;
|
||||
if (
|
||||
observations.some((item) => item.evidence_id === observation.evidence_id)
|
||||
)
|
||||
throw new Error("PCAP retained read reused an Evidence identity");
|
||||
observations.push(observation);
|
||||
} else {
|
||||
const {
|
||||
endpoints,
|
||||
dns_queries,
|
||||
tls_sni_values,
|
||||
protocol_distribution,
|
||||
http_request_uris,
|
||||
unattributed_dns_addresses,
|
||||
diagnostics,
|
||||
...metadata
|
||||
} = n;
|
||||
capture = {
|
||||
...metadata,
|
||||
retained_counts: {
|
||||
endpoints: endpoints.length,
|
||||
dns_queries: dns_queries.length,
|
||||
tls_sni_values: tls_sni_values.length,
|
||||
protocol_distribution: protocol_distribution.length,
|
||||
http_request_uris: http_request_uris.length,
|
||||
unattributed_dns_addresses: unattributed_dns_addresses.length,
|
||||
},
|
||||
diagnostics_code_units:
|
||||
diagnostics === undefined
|
||||
? null
|
||||
: {
|
||||
tshark: diagnostics.tshark_stderr.length,
|
||||
capinfos: diagnostics.capinfos_stderr.length,
|
||||
},
|
||||
};
|
||||
uris = http_request_uris.slice(0, 2);
|
||||
}
|
||||
if (uris.length !== Math.min(2, capture.retained_counts.http_request_uris))
|
||||
throw new Error("Runtime did not return the requested retained URI prefix");
|
||||
return {
|
||||
artifact_sha256: targetSha(inspected),
|
||||
capture,
|
||||
http_request_uris_first2: uris,
|
||||
delivery_observations: observations,
|
||||
evidence_ids: observations.map((item) => item.evidence_id),
|
||||
};
|
||||
};
|
||||
|
||||
const workflows = {
|
||||
"wireshark-http.cap": inspectPcapTarget,
|
||||
"wireshark-http_redirects.pcapng": inspectPcapTarget,
|
||||
"qgis-world-map.gpkg": async () => {
|
||||
const columns = ["fid", "NAME", "ISO_A3"];
|
||||
const input = {
|
||||
|
||||
@@ -0,0 +1,291 @@
|
||||
# Small real PCAP recordings: keep complete delivery
|
||||
|
||||
Continues [#2016](https://github.com/morluto/rea/issues/2016) after the QGIS
|
||||
SQLite round. Measured main is pinned to
|
||||
`3ae826ceba6f25b08bce50bcb51001209bc16b57`, with 23 intervening commits.
|
||||
These include bounded PCAP summaries/pages, authenticated artifact subjects,
|
||||
unknown-snapshot reuse and shared transcript grading. This round measures existing
|
||||
PCAP delivery choices; it adds no production, API, default, cache or skill change.
|
||||
|
||||
## Corpus and question
|
||||
|
||||
Use two unmodified recordings published on the official
|
||||
[Wireshark sample-capture page](https://wiki.wireshark.org/SampleCaptures):
|
||||
|
||||
| Recording | Bytes | SHA-256 |
|
||||
| --------------------------------------- | -----: | ------------------------------------------------------------------ |
|
||||
| HTTP request/response, `http.cap` | 25,803 | `25a72bdf10339f2c29916920c8b9501d294923108de8f29b19aba7cc001ab60d` |
|
||||
| HTTP redirects, `http_redirects.pcapng` | 47,804 | `ff432a6b71d06cd20720f7763e584014edd0a43c3b55981c682d1b7640207abc` |
|
||||
|
||||
`corpus.json` preserves original/resolved URLs. Local digests pin downloaded
|
||||
bytes; no independently published upstream digest or immutable source commit is
|
||||
claimed. These are upstream saved application-protocol recordings, not generated
|
||||
REA fixtures. Recorded clients and current deployments are unknown. No new
|
||||
traffic is captured and no recorded target/URL is executed or fetched.
|
||||
|
||||
The question asks for complete returned identity/client/scalar metadata, every
|
||||
observed count and source coverage, six retained inventory counts, explicit nulls,
|
||||
diagnostic UTF-16 lengths, and the first two retained HTTP URIs in original order.
|
||||
Preserve each used receipt's limitations and identity. Summary/page scope and
|
||||
unknowns remain distinct from source coverage; exhaustion does not establish
|
||||
network completeness. Other inventory lists and diagnostic text are unrequested.
|
||||
|
||||
Compare default `{path}` with existing `{path, detail: "summary"}` plus a retained
|
||||
`http-uris` page, offset 0, limit 2. Ordinary selected-field
|
||||
`inspect_packet_capture` remains a separate boundary, not a substitute claim
|
||||
about complete inventories. `questions/pcap-http.txt` contains the fixed question.
|
||||
|
||||
## SDK measurements
|
||||
|
||||
Frozen dist/bridge file-table SHA-256:
|
||||
`db050545e94dc305a48042e06df96ac6c64b21ee6c5c4707c787d5e46df79e47`.
|
||||
Linux x64, Node 24.18.0, SDK client/server 2.3.1, installed Wireshark 4.0.17.
|
||||
Explicit commands are `/usr/bin/tshark`, `/usr/bin/capinfos`, `/usr/bin/prlimit`;
|
||||
private receipts retain executable hashes. Each workflow starts a fresh server
|
||||
and uses actual named advertised definitions/schema validation. Host page caches
|
||||
are not flushed. Both tools read the owned snapshot.
|
||||
|
||||
Eight trials run complete/summary/summary/complete per recording. Both pairs
|
||||
match selected capture facts, URI prefixes, settings and original source bytes.
|
||||
Each delivery's exact limitations, identities and coverage remain separately
|
||||
recorded, rather than equating source coverage with page exhaustion.
|
||||
|
||||
| Trial | Reply bytes | SDK seconds | Payload CPU seconds | Kernel peak bytes |
|
||||
| -------------------- | ----------: | ----------: | ------------------: | ----------------: |
|
||||
| HTTP complete-1 | 7,601 | 4.349 | 5.336 | 611,778,560 |
|
||||
| HTTP summary-1 | 17,540 | 4.972 | 5.973 | 601,661,440 |
|
||||
| HTTP summary-2 | 17,540 | 5.101 | 6.092 | 611,438,592 |
|
||||
| HTTP complete-2 | 7,601 | 4.463 | 5.503 | 617,148,416 |
|
||||
| Redirects complete-1 | 7,535 | 4.670 | 5.641 | 610,627,584 |
|
||||
| Redirects summary-1 | 17,240 | 5.099 | 6.104 | 615,522,304 |
|
||||
| Redirects summary-2 | 17,240 | 5.060 | 6.056 | 610,385,920 |
|
||||
| Redirects complete-2 | 7,535 | 4.466 | 5.458 | 608,141,312 |
|
||||
|
||||
Bytes are compact client-result JSON summed over one complete or two summary/page
|
||||
calls, not model tokens or raw wire bytes. Summary/page expands replies to more
|
||||
than twice the complete result on both recordings. All four SDK pairs are slower
|
||||
and use more payload CPU; memory differences are small/inconsistent. CPU includes
|
||||
driver/catalog work outside the SDK workflow timer.
|
||||
|
||||
Reproduce with `measure.mjs --artifact wireshark-http.cap` or
|
||||
`wireshark-http_redirects.pcapng`, pinned path/runtime/revision, explicit provider
|
||||
JSON, `--schemas selected`, and `--delivery complete|summary`. Use `compare.mjs`
|
||||
on completed output directories. Original replies, catalogs, comparison receipts
|
||||
and exact measured driver source stay private. Repeated deliveries in the same
|
||||
mode also compare the ordered receipt observations, including limitations, view
|
||||
parameters, coverage and unknowns; only receipt/parent IDs are invocation-specific.
|
||||
Comparator regressions run with `npm run test:local -- tests/evaluation/analysisEfficiencyComparison.test.ts`
|
||||
using synthetic measurement directories; they do not establish provider behavior.
|
||||
|
||||
## Actual model tasks
|
||||
|
||||
Four actual Codex CLI 0.161.0 tasks use requested `gpt-5.6-terra` high, the same
|
||||
redirects question/runtime/artifact/provider settings, direct mode, two exact
|
||||
enabled tools and equal context/discovery capture flags. The requested alias is
|
||||
not verified server-resolved model identity. Equal cohort instructions select
|
||||
complete or summary plus one URI page. The experiment driver applies its existing
|
||||
private per-tool approval policy to `inspect_pcap_capture`; global configuration
|
||||
and REA authority are unchanged.
|
||||
|
||||
| Trial | Input | Cached | Uncached | Output | CLI wall seconds | MCP calls | Facts |
|
||||
| ---------- | -----: | -----: | -------: | -----: | ---------------: | --------: | ----- |
|
||||
| complete-1 | 38,947 | 9,984 | 28,963 | 2,119 | 69.469 | 1 | pass |
|
||||
| summary-1 | 58,779 | 38,144 | 20,635 | 3,241 | 141.304 | 2 | pass |
|
||||
| summary-2 | 58,584 | 38,144 | 20,440 | 4,261 | 50.916 | 2 | pass |
|
||||
| complete-2 | 38,917 | 23,040 | 15,877 | 2,813 | 106.896 | 1 | pass |
|
||||
|
||||
Summary/page total input **increases 50.92% / 50.54%** and output increases
|
||||
52.95% / 51.48%. Uncached input is mixed: 28.75% lower / 28.74% higher. Time is
|
||||
mixed: 103.41% slower / 52.37% faster. No cause is inferred for caching or timing
|
||||
variation. Cached input is included in total; each CLI reports one final aggregate
|
||||
usage. Output includes 1,149 / 1,162 / 2,196 / 1,837 reasoning tokens.
|
||||
|
||||
All calls succeed without extras/retries. Events contain only the selected MCP
|
||||
tools and agent messages. All source-digest and private-profile cleanup checks
|
||||
pass. Model payload CPU seconds/kernel peaks: 7.818/791,060,480;
|
||||
9.793/752,705,536; 7.685/757,891,072; 7.999/750,993,408. These do not establish
|
||||
a general CPU/memory gain.
|
||||
|
||||
The first summary initially fails the grader because it uses explicit equivalent
|
||||
labels and a URI object retaining page/source coverage. Its facts are correct.
|
||||
The grading boundary accepts these known forms while checking exact types,
|
||||
null presence, original limitations and extra scopes against actual receipts.
|
||||
Conflicting duplicates and unsupported claims remain failures. Each cited
|
||||
receipt's own ID must be reported; its parent lookup ID is checked separately.
|
||||
Original answers/reviews/grader source remain intact. No model is rerun. A grader
|
||||
review pause separates the pairs; it is not task time or a cold-cache claim.
|
||||
|
||||
## Verification and decision
|
||||
|
||||
An independent stdlib raw PCAP/PCAPNG/Ethernet/IP/TCP/complete-HTTP-header parser
|
||||
agrees on 43/271 packets, per-packet endpoint counts, first URI prefixes and 2/23
|
||||
distinct HTTP URIs. It invokes no Wireshark. DNS associations, protocol-stack
|
||||
dissection, nullable timestamps and live-network semantics are not independently
|
||||
validated.
|
||||
|
||||
Both real CLI complete results match full MCP normalized results, subjects,
|
||||
parameters, IDs and limitations. Portable CLI summary/URI views match MCP retained
|
||||
normalized facts and subjects; source forms/caller parameters remain distinct.
|
||||
Source bytes stay unchanged. Thirty-four context/grading regressions pass, including
|
||||
cited complete metadata/counts/UTF-16 diagnostics and exact ordered response ID
|
||||
lists that reject parent lookups, duplicates and nested representations. URI
|
||||
prefix coverage must come from its cited page, even if an uncited page has the
|
||||
same items. The SDK driver also checks the returned view against the exact
|
||||
requested kind, facet, offset and limit. Five real-answer
|
||||
mutations—omitted null, false page exhaustion, contradictory source scope,
|
||||
unsupported absence claim and non-string receipt identity—fail correctly.
|
||||
Malformed identities produce failed facts rather than crashing the grader.
|
||||
JS syntax, Python compilation and
|
||||
relevant standard lint pass.
|
||||
|
||||
A fresh real SDK summary/URI smoke matches the original selected facts. Three
|
||||
private consumer fault injections alter the returned view's offset, limit or
|
||||
presence after a real SDK reply; the driver rejects each. These establish the
|
||||
experiment consumer's guard, not provider malformed-reply behavior. The initial
|
||||
injection setup loaded the separate CommonJS client and did not inject a fault;
|
||||
that discarded setup attempt remains private alongside the corrected ESM runs.
|
||||
Original measured driver sources, token totals and timing data remain unchanged.
|
||||
|
||||
The driver also requires a distinct Evidence receipt for the retained URI page.
|
||||
A private SDK consumer injection reuses the summary's response ID after a real
|
||||
MCP reply: the preceding driver accepts it, the corrected driver rejects it
|
||||
without writing an answer, and an unmodified real reply still passes. This
|
||||
checks receipt uniqueness before recording observations; it does not establish
|
||||
that a provider actually returned duplicate IDs.
|
||||
|
||||
The page guard checks the complete expected coverage object, including status,
|
||||
continuation and exhaustion. Private actual-reply injections demonstrate that
|
||||
the preceding driver accepts inconsistent coverage, while the corrected driver
|
||||
rejects wrong status, next offset or exhaustion. Unmodified exhausted and
|
||||
continuable URI pages still pass. The grader also requires each cited receipt's
|
||||
subject digest and local path to identify the reference capture, and uses the
|
||||
reference page's verified coverage. Thirty-four grading/context regressions pass,
|
||||
including missing or malformed subjects and faithful but inconsistent coverage;
|
||||
all four original model answers still pass factual regrading. Strict resource
|
||||
grades remain false where terminal whole-scope counters are unknown.
|
||||
|
||||
PCAP observation production now validates both the Evidence subject and
|
||||
normalized target against the selected path/digest for complete, summary and
|
||||
page replies. Retained parents must be declared in each view receipt's Evidence
|
||||
links and differ from that response's own ID. The grader enforces the same link
|
||||
relationship. Private actual-reply injections and normal complete/summary
|
||||
controls exercise this boundary; these do not establish a provider defect.
|
||||
|
||||
The grader preserves each receipt's producing tool while indexing it; conflicting
|
||||
tool/envelope ownership for one ID remains ambiguous, while identical repeated
|
||||
receipts are allowed. Complete delivery requires the capture tool/operation.
|
||||
Summary delivery uses the capture tool but correctly delegates its Evidence
|
||||
operation to `inspect_analysis_view`; the URI page uses that view tool/operation.
|
||||
Both the grader and SDK observation producer validate this actual contract.
|
||||
Wrong-tool/operation regressions fail before the correction and pass afterward;
|
||||
private SDK operation faults are rejected while real complete/summary controls
|
||||
and all four stored model factual grades still pass. No operation is rewritten
|
||||
to make the tool name and delegated Evidence operation identical.
|
||||
|
||||
The receipt index parses the required envelope/nested-view containers before
|
||||
allowing lookup, so missing or malformed normalized results/limitations become
|
||||
failed facts with a written report. Empty observation lists also remain failed
|
||||
facts and never enter citation checks that require a first receipt. Coverage uses type-sensitive JSON comparison
|
||||
to distinguish booleans, integers and floats. Receipt-identity conflict checks
|
||||
use the same type-sensitive rule, so a scalar type change cannot hide in a
|
||||
repeated ID. Existing PCAP scope-alias checks already preserve JSON types.
|
||||
Requested and cited URI views use the same type-sensitive JSON rule, rejecting
|
||||
boolean or floating-point offsets/limits even when an answer faithfully copies
|
||||
them. The observation producer rejects missing, null or malformed response IDs
|
||||
before recording either capture or page receipts. Private real-reply injections
|
||||
exercise these ID failures alongside normal complete/summary controls; they do
|
||||
not establish malformed provider behavior.
|
||||
Completed-call arguments stay with their producing tool and receipt in the
|
||||
grader's identity index. Cited envelope parameters and actual call arguments
|
||||
must match the selected path or retained parent/view, including delegated
|
||||
summary parameters. The observation producer validates both summary and URI
|
||||
views and envelope parameters at their common boundary. Private real-reply
|
||||
faults reproduce missing-parameter/summary-view acceptance before the fix and
|
||||
are rejected afterward; normal complete/summary controls still pass.
|
||||
Negative regressions reproduce
|
||||
prior crashes and false passes; all original real-model factual grades still
|
||||
pass without model reruns. The SDK workflow is unchanged by these grader fixes.
|
||||
|
||||
**Decision: keep existing complete defaults; add no production optimization.**
|
||||
For these compact recordings and this question, summary/page costs more total
|
||||
input in both real-model pairs. No stable speed or CPU/memory benefit justifies
|
||||
another API, automatic delivery switch, cache or plugin layer. Existing pages
|
||||
remain available; their benefit needs evidence for the actual question/inventory.
|
||||
Large captures, other protocols, Windows/macOS/Bun Wireshark and other model
|
||||
workloads remain unverified here.
|
||||
|
||||
Heavy work stays serial with one CPU, 3 GiB early stop/4 GiB hard memory and no
|
||||
swap. Build kernel peak: 2,575,732,736 bytes. Payload cleanup and zero phase limit
|
||||
events are verified; missing terminal whole-scope counters remain unknown, so
|
||||
strict whole-resource grades stay false. No raw capture, trace, credential,
|
||||
runtime or result JSON is committed. No unrelated REA product bug is identified
|
||||
or fixed, and no release is performed.
|
||||
|
||||
Review follow-up binds the cited URI page to its exact requested facet/range,
|
||||
reported values and the summary retained parent. Small synthetic trial receipts
|
||||
exercise the entire grading entrypoint, including faithful but wrong-page/range/
|
||||
parent citations, missing nulls, false scope and malformed identities. These are
|
||||
grader regressions, not real-provider evidence. All four stored actual model
|
||||
answers still pass; no model reruns are required. Transcript readers close their
|
||||
owned file handles after grading.
|
||||
|
||||
The SDK driver and factual grader require summary coverage to describe its one
|
||||
complete view and require each present normalized limitations/unknowns field to
|
||||
be a string array. Missing fields and valid empty arrays remain distinct.
|
||||
Controlled consumer mutations over actual SDK replies reproduce the previously
|
||||
accepted inconsistent summary coverage and verify rejection after the guard;
|
||||
complete and summary controls still pass. These mutations do not establish a
|
||||
malformed-response defect in the provider. The four stored model answers still
|
||||
pass factual regrading without another model run.
|
||||
|
||||
Summary views must preserve both the outer artifact path/digest and the nested
|
||||
summary target, as well as the Evidence subject. The full grader rejects
|
||||
faithful receipts with missing or incorrect outer artifact identities. Private
|
||||
mutations over actual SDK replies reproduce the previous false success and
|
||||
verify rejection after the shared observation guard; normal deliveries and all
|
||||
four original model answers retain their recorded facts.
|
||||
|
||||
Declared retained parent digests must match the digest encoded in the linked
|
||||
parent Evidence ID for both summary and page views. SDK acceptance and factual
|
||||
grading reject mismatches and missing digests while preserving complete
|
||||
deliveries. Before/after private actual-reply mutations verify both view roles;
|
||||
unmodified deliveries and four stored original-model factual regrades pass.
|
||||
|
||||
The SDK observation boundary additionally authenticates each complete, summary
|
||||
and page envelope with the selected runtime’s existing `parseEvidence` owner.
|
||||
It preserves the original envelope, without duplicating semantic-ID hashing.
|
||||
Format-valid swapped-ID mutations over actual SDK replies pass before this
|
||||
guard and fail afterward; unmodified complete/summary controls pass. The model
|
||||
grader tests selected facts against cited transcript receipts; its synthetic
|
||||
fixtures do not establish Evidence authentication. The original real-model
|
||||
receipts are independently authenticated against the pinned runtime.
|
||||
|
||||
The grader separates truthful cited facts from equivalence of the measured
|
||||
workflow. PCAP workflow checks require the planned ordered capture call (complete)
|
||||
or summary capture followed by the URI-page call, with exact arguments and
|
||||
completed status. Extra, reversed, failed or undecodable attempts fail workflow
|
||||
acceptance even when the final cited facts are correct; their actual cost and
|
||||
errors remain recorded. All four original model trials pass both factual and
|
||||
workflow checks without model reruns. Strict resource grades remain unchanged.
|
||||
|
||||
Each planned call must also decode as an Evidence object with its own cited
|
||||
response ID. An unreadable planned reply cannot independently pass workflow
|
||||
checks merely because tool names, arguments and terminal status match. The
|
||||
regression reproduces the previous workflow-only false success; original model
|
||||
factual/workflow grades remain unchanged.
|
||||
|
||||
Workflow citation cardinality must equal the planned call count before binding
|
||||
per-call IDs. A shorter or longer answer list cannot silently shorten Python
|
||||
zip’s expected plan; missing URI calls and extra citations are rejected.
|
||||
Original complete/summary model workflows continue to pass unchanged.
|
||||
|
||||
The six comparator CLI regressions now live in the normally discovered Vitest
|
||||
evaluation suite, so routine local/PR/CI gates execute them. The original six
|
||||
scenarios and CLI assertions are retained; the former manual Node suite is
|
||||
removed. They remain synthetic consumer checks, not provider evidence.
|
||||
|
||||
Failed calls with scalar/list replies or malformed nested resource-error
|
||||
containers are parsed defensively. The grader writes its report, preserves the
|
||||
original failure and any supported cited facts, and rejects workflow acceptance
|
||||
instead of crashing. Genuine structured resource Evidence references remain
|
||||
observable; no malformed producer behavior is inferred from synthetic cases.
|
||||
@@ -0,0 +1 @@
|
||||
Statically inspect the supplied saved Wireshark capture at {{artifact_path}} using only registered REA inspect_pcap_capture and inspect_analysis_view. Report its original client/target identity and every returned scalar metadata/count/coverage field exactly, preserving explicit nulls. Include retained_counts for endpoints, dns_queries, tls_sni_values, protocol_distribution, http_request_uris and unattributed_dns_addresses; derive array lengths only when reading complete Evidence. Include diagnostics_code_units as exact UTF-16 lengths of retained capinfos/tshark stderr, or null if diagnostics are unrecorded. In capture, omit inventory arrays and diagnostic text; do not omit their observed counts or change source coverage. Report the first two retained HTTP URIs in original order, or all if fewer; do not request other inventory pages. Report delivery_observations in dependency order for the Evidence used: evidence_id, outer limitations, and each present normalized kind, view, parent_evidence_id, coverage, unknowns and normalized_limitations. Preserve presence, explicit nulls, scalar types and original limitation order. Distinguish source coverage from URI-page exhaustion. Final JSON fields: artifact_sha256, capture, http_request_uris_first2, delivery_observations, evidence_ids. Include only actual response IDs. Saved packet/URI observations do not establish current deployment, process identity, missing traffic, decrypted payloads or current DNS resolution. Do not use shell, host file tools, network, other agents, traffic acquisition or execute recorded targets. Stop once these selected facts are supported.
|
||||
@@ -67,9 +67,47 @@ def scope_with_verified_counts(value, expected_scope, returned_counts):
|
||||
return scope, checks
|
||||
|
||||
|
||||
def pcap_observation_fields(value):
|
||||
"""Equivalent explicit labels are accepted; conflicting duplicates remain failures."""
|
||||
if not isinstance(value, dict):
|
||||
return value
|
||||
result = dict(value)
|
||||
for label, canonical in {
|
||||
'outer_limitations': 'limitations', 'normalized_kind': 'kind',
|
||||
'normalized_view': 'view', 'normalized_parent_evidence_id': 'parent_evidence_id',
|
||||
'normalized_coverage': 'coverage', 'normalized_unknowns': 'unknowns',
|
||||
'normalized_source_coverage': 'source_coverage',
|
||||
}.items():
|
||||
if label not in result:
|
||||
continue
|
||||
if canonical in result and json.dumps(result[label], sort_keys=True) != json.dumps(result[canonical], sort_keys=True):
|
||||
return {**result, 'conflicting_fields': [label, canonical]}
|
||||
result[canonical] = result.pop(label)
|
||||
return result
|
||||
|
||||
|
||||
def pcap_complete_capture(value):
|
||||
"""Project the selected metadata/counts from the cited complete receipt."""
|
||||
inventories = ['endpoints', 'dns_queries', 'tls_sni_values', 'protocol_distribution',
|
||||
'http_request_uris', 'unattributed_dns_addresses']
|
||||
if not isinstance(value, dict) or not all(isinstance(value.get(key), list) for key in inventories):
|
||||
return None
|
||||
capture = {key: item for key, item in value.items() if key not in inventories + ['diagnostics']}
|
||||
capture['retained_counts'] = {key: len(value[key]) for key in inventories}
|
||||
diagnostics = value.get('diagnostics')
|
||||
if 'diagnostics' not in value:
|
||||
capture['diagnostics_code_units'] = None
|
||||
elif isinstance(diagnostics, dict) and all(isinstance(diagnostics.get(key + '_stderr'), str) for key in ['tshark', 'capinfos']):
|
||||
capture['diagnostics_code_units'] = {key: len(diagnostics[key + '_stderr'].encode('utf-16-le', errors='surrogatepass')) // 2
|
||||
for key in ['tshark', 'capinfos']}
|
||||
else:
|
||||
return None
|
||||
return capture
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument('--domain', choices=['wasm', 'evm', 'android', 'flutter-aot', 'sqlite', 'managed', 'managed-nonempty', 'firmware', 'inventory', 'javascript', 'native', 'browser'], required=True)
|
||||
parser.add_argument('--domain', choices=['wasm', 'evm', 'android', 'flutter-aot', 'sqlite', 'pcap', 'managed', 'managed-nonempty', 'firmware', 'inventory', 'javascript', 'native', 'browser'], required=True)
|
||||
parser.add_argument('--trial', type=Path, required=True)
|
||||
parser.add_argument('--reference', type=Path)
|
||||
parser.add_argument('--resource-log', type=Path, required=True)
|
||||
@@ -83,25 +121,43 @@ def main():
|
||||
reference = read(args.reference / 'answer.json') if args.reference else None
|
||||
observed_ids = set()
|
||||
producer_results = {}
|
||||
pcap_calls = []
|
||||
pcap_attempts = []
|
||||
workflow_checks = {'pcap_exact_call_sequence': False} if args.domain == 'pcap' else {}
|
||||
tool_errors = []
|
||||
for line in (args.trial / 'events.jsonl').open():
|
||||
event = json.loads(line)
|
||||
item = event.get('item', {})
|
||||
if event.get('type') != 'item.completed' or item.get('type') != 'mcp_tool_call':
|
||||
continue
|
||||
result = item.get('result') or {}
|
||||
parsed = decode(result)
|
||||
if item['status'] == 'completed':
|
||||
# Only response text contributes observed IDs, never model arguments.
|
||||
for text in strings(result):
|
||||
observed_ids.update(re.findall(r'ev_[a-f0-9]{64}', text))
|
||||
if parsed is not None:
|
||||
producer_results.setdefault(item['tool'], []).append(parsed)
|
||||
else:
|
||||
tool_errors.append({'tool': item['tool'], 'arguments': item['arguments'], 'response': parsed if parsed is not None else result})
|
||||
if parsed is not None and parsed.get('error', {}).get('code') == 'resource_constraint':
|
||||
reference_value = parsed['error'].get('details', {}).get('reported_limits', {}).get('evidence_reference')
|
||||
observed_ids.update(evidence_ids(reference_value))
|
||||
with (args.trial / 'events.jsonl').open() as event_lines:
|
||||
for line in event_lines:
|
||||
event = json.loads(line)
|
||||
item = event.get('item', {})
|
||||
if event.get('type') != 'item.completed' or item.get('type') != 'mcp_tool_call':
|
||||
continue
|
||||
result = item.get('result') or {}
|
||||
parsed = decode(result)
|
||||
if args.domain == 'pcap':
|
||||
pcap_attempts.append({'tool': item.get('tool'), 'arguments': item.get('arguments'),
|
||||
'status': item.get('status'),
|
||||
'evidence_id': parsed.get('evidence_id') if isinstance(parsed, dict) else None,
|
||||
'readable_evidence': isinstance(parsed, dict)
|
||||
and isinstance(parsed.get('normalized_result'), dict)
|
||||
and isinstance(parsed.get('evidence_id'), str)
|
||||
and re.fullmatch(r'ev_[a-f0-9]{64}', parsed['evidence_id']) is not None})
|
||||
if item['status'] == 'completed':
|
||||
# Only response text contributes observed IDs, never model arguments.
|
||||
for text in strings(result):
|
||||
observed_ids.update(re.findall(r'ev_[a-f0-9]{64}', text))
|
||||
if parsed is not None:
|
||||
if args.domain == 'pcap':
|
||||
pcap_calls.append((item['tool'], item.get('arguments'), parsed))
|
||||
else:
|
||||
producer_results.setdefault(item['tool'], []).append(parsed)
|
||||
else:
|
||||
tool_errors.append({'tool': item['tool'], 'arguments': item['arguments'], 'response': parsed if parsed is not None else result})
|
||||
error = parsed.get('error') if isinstance(parsed, dict) else None
|
||||
if isinstance(error, dict) and error.get('code') == 'resource_constraint':
|
||||
details = error.get('details')
|
||||
limits = details.get('reported_limits') if isinstance(details, dict) else None
|
||||
if isinstance(limits, dict):
|
||||
observed_ids.update(evidence_ids(limits.get('evidence_reference')))
|
||||
reported_ids = evidence_ids(answer.get('evidence_ids'))
|
||||
checks = {
|
||||
'reported_evidence_ids_observed_in_responses': bool(reported_ids) and reported_ids <= observed_ids,
|
||||
@@ -138,6 +194,187 @@ def main():
|
||||
# Missing is not SQL/JSON null; booleans are not numeric counters.
|
||||
check(key, (key in answer, json.dumps(answer.get(key), sort_keys=True)),
|
||||
(True, json.dumps(reference[key], sort_keys=True)))
|
||||
elif domain == 'pcap':
|
||||
check('capture', ('capture' in answer, json.dumps(answer.get('capture'), sort_keys=True)),
|
||||
(True, json.dumps(reference['capture'], sort_keys=True)))
|
||||
prefix = answer.get('http_request_uris_first2')
|
||||
records = {}
|
||||
for tool, arguments, value in pcap_calls:
|
||||
if not isinstance(value, dict) or not isinstance(value.get('evidence_id'), str):
|
||||
continue
|
||||
identity = value['evidence_id']
|
||||
normalized = value.get('normalized_result')
|
||||
supported = (isinstance(normalized, dict) and isinstance(value.get('limitations'), list)
|
||||
and all(isinstance(item, str) for item in value['limitations']))
|
||||
if supported:
|
||||
supported = all(key not in normalized or (isinstance(normalized[key], list)
|
||||
and all(isinstance(item, str) for item in normalized[key]))
|
||||
for key in ['limitations', 'unknowns'])
|
||||
if supported and ('summary' in normalized or normalized.get('kind') == 'summary'):
|
||||
supported = isinstance(normalized.get('summary'), dict)
|
||||
if supported and ('page' in normalized or normalized.get('kind') == 'pcap'):
|
||||
supported = isinstance(normalized.get('page'), dict)
|
||||
entry = (tool, arguments, value) if supported else None
|
||||
if identity not in records:
|
||||
records[identity] = entry
|
||||
elif json.dumps(records[identity], sort_keys=True) != json.dumps(entry, sort_keys=True):
|
||||
# An ID with conflicting producers or envelopes is ambiguous.
|
||||
records[identity] = None
|
||||
observations = answer.get('delivery_observations')
|
||||
canonical_observations = [pcap_observation_fields(value) for value in observations] if isinstance(observations, list) else observations
|
||||
if isinstance(prefix, dict):
|
||||
checks['pcap_prefix_fields_supported'] = set(prefix) <= {'items', 'coverage', 'source_coverage'} and 'items' in prefix
|
||||
if 'source_coverage' in prefix:
|
||||
check('pcap_prefix_source_coverage', prefix['source_coverage'], reference['capture']['coverage'])
|
||||
if 'coverage' in prefix:
|
||||
last = canonical_observations[-1] if isinstance(canonical_observations, list) and canonical_observations else None
|
||||
identity = last.get('evidence_id') if isinstance(last, dict) else None
|
||||
entry = records.get(identity) if isinstance(identity, str) else None
|
||||
record = entry[2] if entry is not None else None
|
||||
page = record['normalized_result'] if record is not None else {}
|
||||
checks['pcap_prefix_coverage_observed'] = (
|
||||
page.get('kind') == 'pcap'
|
||||
and page.get('page', {}).get('facet') == 'http-uris'
|
||||
and json.dumps(page['page'].get('items'), sort_keys=True) == json.dumps(prefix.get('items'), sort_keys=True)
|
||||
and 'coverage' in page
|
||||
and json.dumps(prefix['coverage'], sort_keys=True) == json.dumps(page['coverage'], sort_keys=True)
|
||||
)
|
||||
prefix = prefix.get('items')
|
||||
check('http_request_uris_first2', ('http_request_uris_first2' in answer, json.dumps(prefix, sort_keys=True)),
|
||||
(True, json.dumps(reference['http_request_uris_first2'], sort_keys=True)))
|
||||
expected_observations = []
|
||||
if isinstance(canonical_observations, list):
|
||||
for observation in canonical_observations:
|
||||
identity = observation.get('evidence_id') if isinstance(observation, dict) else None
|
||||
entry = records.get(identity) if isinstance(identity, str) else None
|
||||
record = entry[2] if entry is not None else None
|
||||
if record is None:
|
||||
expected_observations.append(None)
|
||||
continue
|
||||
value = record['normalized_result']
|
||||
expected = {'evidence_id': record['evidence_id'], 'limitations': record['limitations']}
|
||||
for key in ['kind', 'view', 'parent_evidence_id', 'coverage', 'unknowns']:
|
||||
if key in value:
|
||||
expected[key] = value[key]
|
||||
if 'limitations' in value:
|
||||
expected['normalized_limitations'] = value['limitations']
|
||||
if 'source_coverage' in observation and 'source_coverage' in value:
|
||||
expected['source_coverage'] = value['source_coverage']
|
||||
expected_observations.append(expected)
|
||||
checks['pcap_delivery_observations_nonempty'] = isinstance(observations, list) and bool(observations)
|
||||
check('pcap_original_delivery_observations', json.dumps(canonical_observations, sort_keys=True),
|
||||
json.dumps(expected_observations, sort_keys=True))
|
||||
if isinstance(canonical_observations, list) and canonical_observations and all(isinstance(x, dict) for x in canonical_observations):
|
||||
check('pcap_requested_delivery_chain', [x.get('kind', 'complete') for x in canonical_observations],
|
||||
[x.get('kind', 'complete') for x in reference['delivery_observations']])
|
||||
check('pcap_requested_views', json.dumps([x.get('view') for x in canonical_observations], sort_keys=True),
|
||||
json.dumps([x.get('view') for x in reference['delivery_observations']], sort_keys=True))
|
||||
if len(reference['delivery_observations']) == 2:
|
||||
check('pcap_summary_coverage', json.dumps(canonical_observations[0].get('coverage'), sort_keys=True),
|
||||
json.dumps({'status': 'complete-within-view', 'examined': 1, 'total': 1,
|
||||
'next_offset': None, 'exhausted': True}, sort_keys=True))
|
||||
identities = [x.get('evidence_id') for x in canonical_observations]
|
||||
checks['pcap_observations_have_reported_ids'] = (
|
||||
isinstance(answer.get('evidence_ids'), list)
|
||||
and answer['evidence_ids'] == identities
|
||||
and all(isinstance(value, str) and re.fullmatch(r'ev_[a-f0-9]{64}', value) for value in identities)
|
||||
and len(set(identities)) == len(identities)
|
||||
)
|
||||
entries = [records.get(identity) if isinstance(identity, str) else None for identity in identities]
|
||||
cited = [entry[2] if entry is not None else None for entry in entries]
|
||||
expected_tools = ['inspect_pcap_capture'] + (['inspect_analysis_view'] if len(reference['delivery_observations']) == 2 else [])
|
||||
check('pcap_cited_producer_tools', [entry[0] if entry is not None else None for entry in entries], expected_tools)
|
||||
# Summary delivery delegates to a retained view; its envelope operation
|
||||
# is inspect_analysis_view even though the requested tool is inspect_pcap_capture.
|
||||
expected_operations = ['inspect_pcap_capture'] if len(expected_tools) == 1 else ['inspect_analysis_view'] * 2
|
||||
check('pcap_cited_operations', [record.get('operation') if isinstance(record, dict) else None for record in cited], expected_operations)
|
||||
path = reference['capture']['target']['path']
|
||||
expected_arguments = [{'path': path}]
|
||||
expected_parameters = [{'path': path}]
|
||||
if len(expected_tools) == 2:
|
||||
parent = cited[0]['normalized_result'].get('parent_evidence_id') if isinstance(cited[0], dict) else None
|
||||
views = [observation.get('view') for observation in reference['delivery_observations']]
|
||||
expected_arguments = [{'path': path, 'detail': 'summary'},
|
||||
{'source': {'kind': 'retained-evidence', 'evidence_id': parent}, 'view': views[-1]}]
|
||||
expected_parameters = [{'parent_evidence_id': parent, 'view': view} for view in views]
|
||||
check('pcap_cited_call_arguments', json.dumps([entry[1] if entry is not None else None for entry in entries], sort_keys=True),
|
||||
json.dumps(expected_arguments, sort_keys=True))
|
||||
workflow_checks['pcap_exact_call_sequence'] = (
|
||||
len(identities) == len(expected_tools)
|
||||
and json.dumps(pcap_attempts, sort_keys=True) == json.dumps([
|
||||
{'tool': tool, 'arguments': arguments, 'status': 'completed',
|
||||
'evidence_id': identity, 'readable_evidence': True}
|
||||
for tool, arguments, identity in zip(expected_tools, expected_arguments, identities)
|
||||
], sort_keys=True)
|
||||
)
|
||||
check('pcap_cited_parameters', json.dumps([record.get('parameters') if isinstance(record, dict) else None for record in cited], sort_keys=True),
|
||||
json.dumps(expected_parameters, sort_keys=True))
|
||||
checks['pcap_cited_subjects_match_artifact'] = bool(cited) and all(
|
||||
isinstance(record, dict)
|
||||
and isinstance(record.get('subject'), dict)
|
||||
and isinstance(record['subject'].get('digest'), dict)
|
||||
and record['subject']['digest'].get('sha256') == reference['artifact_sha256']
|
||||
and record['subject'].get('local_path') == reference['capture']['target']['path']
|
||||
for record in cited
|
||||
)
|
||||
checks['pcap_cited_parents_linked'] = bool(cited) and all(isinstance(record, dict) for record in cited) and (
|
||||
len(cited) == 1 or (len(cited) == 2 and all(
|
||||
isinstance(record['normalized_result'].get('parent_evidence_id'), str)
|
||||
and re.fullmatch(r'ev_[a-f0-9]{64}', record['normalized_result']['parent_evidence_id'])
|
||||
and record['normalized_result'].get('parent_digest') == record['normalized_result']['parent_evidence_id'][3:]
|
||||
and record['normalized_result']['parent_evidence_id'] != record['evidence_id']
|
||||
and isinstance(record.get('evidence_links'), list)
|
||||
and record['normalized_result']['parent_evidence_id'] in record['evidence_links']
|
||||
for record in cited
|
||||
))
|
||||
)
|
||||
checks['pcap_cited_source_supports_uri_prefix'] = False
|
||||
checks['pcap_cited_capture'] = False
|
||||
if all(isinstance(record, dict) for record in cited):
|
||||
if len(cited) == 1:
|
||||
source = cited[0]['normalized_result']
|
||||
check('pcap_cited_capture', json.dumps(answer.get('capture'), sort_keys=True),
|
||||
json.dumps(pcap_complete_capture(source), sort_keys=True))
|
||||
checks['pcap_cited_source_supports_uri_prefix'] = (
|
||||
source.get('target') == reference['capture']['target']
|
||||
and isinstance(source.get('http_request_uris'), list)
|
||||
and json.dumps(source['http_request_uris'][:2], sort_keys=True) == json.dumps(prefix, sort_keys=True)
|
||||
)
|
||||
elif len(cited) == 2:
|
||||
source, page = [record['normalized_result'] for record in cited]
|
||||
capture = source.get('summary', {})
|
||||
check('pcap_cited_capture', json.dumps(answer.get('capture'), sort_keys=True),
|
||||
json.dumps(capture, sort_keys=True))
|
||||
selected_page = page.get('page', {})
|
||||
checks['pcap_cited_source_supports_uri_prefix'] = (
|
||||
source.get('kind') == 'summary'
|
||||
and source.get('parent_operation') == 'inspect_pcap_capture'
|
||||
and capture == reference['capture']
|
||||
and source.get('artifact') == {key: capture['target'][key] for key in ['path', 'sha256']}
|
||||
and isinstance(source.get('parent_evidence_id'), str)
|
||||
and page.get('parent_evidence_id') == source['parent_evidence_id']
|
||||
and page.get('parent_operation') == 'inspect_pcap_capture'
|
||||
and json.dumps(page.get('view'), sort_keys=True) == json.dumps({'kind': 'pcap', 'facet': 'http-uris', 'offset': 0, 'limit': 2}, sort_keys=True)
|
||||
and selected_page.get('facet') == 'http-uris'
|
||||
and json.dumps(selected_page.get('items'), sort_keys=True) == json.dumps(prefix, sort_keys=True)
|
||||
and page.get('source_coverage') == capture.get('coverage')
|
||||
and json.dumps(page.get('coverage'), sort_keys=True) == json.dumps(reference['delivery_observations'][-1].get('coverage'), sort_keys=True)
|
||||
and page.get('artifact') == {key: capture['target'][key] for key in ['path', 'sha256']}
|
||||
)
|
||||
else:
|
||||
checks['pcap_requested_delivery_chain'] = False
|
||||
if len(reference['delivery_observations']) == 2:
|
||||
checks['pcap_summary_coverage'] = False
|
||||
checks['pcap_observations_have_reported_ids'] = False
|
||||
checks['pcap_requested_views'] = False
|
||||
checks['pcap_cited_source_supports_uri_prefix'] = False
|
||||
checks['pcap_cited_capture'] = False
|
||||
checks['pcap_cited_subjects_match_artifact'] = False
|
||||
checks['pcap_cited_parents_linked'] = False
|
||||
checks['pcap_cited_producer_tools'] = False
|
||||
checks['pcap_cited_operations'] = False
|
||||
checks['pcap_cited_call_arguments'] = False
|
||||
checks['pcap_cited_parameters'] = False
|
||||
elif domain == 'flutter-aot':
|
||||
for key in ['target', 'abi', 'libapp', 'string_pool', 'coverage', 'limitations']:
|
||||
check(key, answer.get(key), reference[key])
|
||||
@@ -323,6 +560,7 @@ def main():
|
||||
statuses[status] = statuses.get(status, 0) + 1
|
||||
check('network_response_statuses', network['observed_response_statuses'], statuses)
|
||||
factual_checks = dict(checks)
|
||||
checks.update(workflow_checks)
|
||||
scopes = [json.loads(line) for line in args.resource_log.read_text().splitlines() if line.startswith('{')]
|
||||
scope = next(value for value in reversed(scopes) if 'resource_scope' in value)
|
||||
checks['no_resource_abort'] = scope['abort'] is None
|
||||
@@ -342,6 +580,8 @@ def main():
|
||||
'review_source_sha256': hashlib.sha256(Path(__file__).read_bytes()).hexdigest(),
|
||||
'domain': domain, 'checks': checks, 'all_selected_checks_pass': all(checks.values()),
|
||||
'factual_checks': factual_checks, 'all_factual_checks_pass': all(factual_checks.values()),
|
||||
**({'workflow_checks': workflow_checks, 'all_workflow_checks_pass': all(workflow_checks.values())}
|
||||
if domain == 'pcap' else {}),
|
||||
'differences': differences, 'tool_errors': tool_errors, 'summary': summary,
|
||||
'fingerprints': {key: value for key, value in settings.items() if 'sha256' in key or key in ['runtime_revision', 'enabled_tools', 'authority']},
|
||||
'answer_file_sha256': hashlib.sha256((args.trial / 'answer.txt').read_bytes()).hexdigest(),
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
"""Distinct capture/cleanup failures; simulated CLI is not real-model evidence."""
|
||||
|
||||
import importlib.util
|
||||
import copy
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
@@ -38,6 +39,546 @@ class ManagedCoverageTests(unittest.TestCase):
|
||||
self.assertNotEqual(scope, expected)
|
||||
|
||||
|
||||
class PcapCoverageTests(unittest.TestCase):
|
||||
def test_explicit_labels_preserve_original_nulls_and_source_scope(self):
|
||||
canonical = runpy.run_path(str(Path(__file__).with_name('review-model.py')))['pcap_observation_fields']
|
||||
original = {'outer_limitations': [], 'normalized_kind': 'pcap',
|
||||
'normalized_coverage': {'examined': 2, 'total': 23, 'next_offset': 2, 'exhausted': False},
|
||||
'normalized_source_coverage': 'partial', 'normalized_parent_evidence_id': None}
|
||||
result = canonical(original)
|
||||
self.assertEqual(result['limitations'], [])
|
||||
self.assertEqual(result['source_coverage'], 'partial')
|
||||
self.assertEqual(result['coverage']['total'], 23)
|
||||
self.assertIsNone(result['parent_evidence_id'])
|
||||
self.assertIn('outer_limitations', original)
|
||||
self.assertNotIn('parent_evidence_id', canonical({}))
|
||||
|
||||
def test_conflicting_duplicates_and_json_type_substitutions_stay_failures(self):
|
||||
canonical = runpy.run_path(str(Path(__file__).with_name('review-model.py')))['pcap_observation_fields']
|
||||
for wrong in ['partial', None]:
|
||||
result = canonical({'normalized_source_coverage': 'complete', 'source_coverage': wrong})
|
||||
self.assertIn('conflicting_fields', result)
|
||||
self.assertEqual(result['normalized_source_coverage'], 'complete')
|
||||
self.assertEqual(result['source_coverage'], wrong)
|
||||
result = canonical({'normalized_coverage': {'examined': 1}, 'coverage': {'examined': True}})
|
||||
self.assertIn('conflicting_fields', result)
|
||||
self.assertEqual(canonical({'outer_limitations': [], 'limitations': []}), {'limitations': []})
|
||||
|
||||
def test_unsupported_claims_are_retained_for_exact_producer_checking(self):
|
||||
canonical = runpy.run_path(str(Path(__file__).with_name('review-model.py')))['pcap_observation_fields']
|
||||
original = {'normalized_kind': 'pcap', 'unrecorded_traffic_absent': True}
|
||||
self.assertIn('unrecorded_traffic_absent', canonical(original))
|
||||
self.assertIsNone(canonical(None))
|
||||
|
||||
|
||||
class PcapGraderWorkflowTests(unittest.TestCase):
|
||||
"""Synthetic receipts test the grader boundary, not Wireshark or real models."""
|
||||
|
||||
def fixture(self):
|
||||
parent, summary_id, page_id = ['ev_' + value * 64 for value in ['a', 'b', 'c']]
|
||||
capture = {'target': {'path': '/fixture/http.pcapng', 'sha256': 'd' * 64},
|
||||
'capture_start': None, 'coverage': 'complete',
|
||||
'retained_counts': {'http_request_uris': 3}}
|
||||
prefix = ['http://example.test/one', 'http://example.test/two']
|
||||
coverage = {'status': 'page', 'examined': 2, 'total': 3, 'next_offset': 2, 'exhausted': False}
|
||||
subject = {'digest': {'sha256': capture['target']['sha256']}, 'local_path': capture['target']['path']}
|
||||
summary = {'evidence_id': summary_id, 'operation': 'inspect_analysis_view',
|
||||
'subject': copy.deepcopy(subject), 'evidence_links': [parent],
|
||||
'limitations': ['source', 'view'],
|
||||
'normalized_result': {'kind': 'summary', 'view': {'kind': 'summary'},
|
||||
'parent_operation': 'inspect_pcap_capture', 'parent_evidence_id': parent,
|
||||
'summary': capture, 'artifact': copy.deepcopy(capture['target']),
|
||||
'coverage': {'status': 'complete-within-view', 'examined': 1, 'total': 1, 'next_offset': None, 'exhausted': True},
|
||||
'limitations': ['source', 'view'], 'unknowns': ['unrecorded traffic']}}
|
||||
page = {'evidence_id': page_id, 'operation': 'inspect_analysis_view',
|
||||
'subject': copy.deepcopy(subject), 'evidence_links': [parent],
|
||||
'limitations': ['source', 'view'],
|
||||
'normalized_result': {'kind': 'pcap',
|
||||
'view': {'kind': 'pcap', 'facet': 'http-uris', 'offset': 0, 'limit': 2},
|
||||
'parent_operation': 'inspect_pcap_capture', 'parent_evidence_id': parent,
|
||||
'page': {'facet': 'http-uris', 'items': prefix}, 'coverage': coverage,
|
||||
'source_coverage': 'complete', 'artifact': capture['target'],
|
||||
'limitations': ['source', 'view'], 'unknowns': ['unrecorded traffic']}}
|
||||
for record in [summary, page]:
|
||||
record['normalized_result']['parent_digest'] = parent[3:]
|
||||
record['parameters'] = {'parent_evidence_id': parent, 'view': copy.deepcopy(record['normalized_result']['view'])}
|
||||
observations = [self.observation(value) for value in [summary, page]]
|
||||
reference = {'artifact_sha256': 'd' * 64, 'capture': capture,
|
||||
'http_request_uris_first2': prefix, 'delivery_observations': observations}
|
||||
answer = copy.deepcopy({**reference, 'http_request_uris_first2': {
|
||||
'items': prefix, 'coverage': coverage, 'source_coverage': 'complete'},
|
||||
'evidence_ids': [summary_id, page_id]})
|
||||
return answer, [summary, page], copy.deepcopy(reference)
|
||||
|
||||
def observation(self, record):
|
||||
value = record['normalized_result']
|
||||
return {'evidence_id': record['evidence_id'], 'limitations': record['limitations'],
|
||||
'kind': value['kind'], 'view': value['view'],
|
||||
'parent_evidence_id': value['parent_evidence_id'], 'coverage': value['coverage'],
|
||||
'unknowns': value['unknowns'], 'normalized_limitations': value['limitations']}
|
||||
|
||||
def grade(self, answer, records, reference, producer_tools=None, producer_arguments=None, extra_events=None, producer_results=None):
|
||||
review = runpy.run_path(str(Path(__file__).with_name('review-model.py')))
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory);trial = root / 'trial';trial.mkdir()
|
||||
for name, value in [('summary.json', {'completed_mcp_calls': []}),
|
||||
('trial-settings.json', {}), ('cleanup.json', {'profile_removed': True})]:
|
||||
(trial / name).write_text(json.dumps(value))
|
||||
(trial / 'answer.txt').write_text(json.dumps(answer))
|
||||
events = []
|
||||
for index, value in enumerate(records):
|
||||
tool = producer_tools[index] if producer_tools is not None else \
|
||||
('inspect_analysis_view' if value['normalized_result'].get('kind') == 'pcap' else 'inspect_pcap_capture')
|
||||
arguments = {'path': reference['capture']['target']['path']}
|
||||
if tool == 'inspect_analysis_view':
|
||||
normalized = value.get('normalized_result')
|
||||
parent = normalized.get('parent_evidence_id') if isinstance(normalized, dict) else None
|
||||
arguments = {'source': {'kind': 'retained-evidence', 'evidence_id': parent},
|
||||
'view': reference['delivery_observations'][-1].get('view')}
|
||||
elif len(reference['delivery_observations']) == 2:
|
||||
arguments['detail'] = 'summary'
|
||||
if producer_arguments is not None:
|
||||
arguments = producer_arguments[index]
|
||||
events.append({'type': 'item.completed', 'item': {'type': 'mcp_tool_call',
|
||||
'tool': tool, 'arguments': arguments, 'status': 'completed',
|
||||
'result': {'structuredContent': value} if producer_results is None else producer_results[index]}})
|
||||
events.extend(extra_events or [])
|
||||
(trial / 'events.jsonl').write_text(''.join(json.dumps(value) + '\n' for value in events))
|
||||
source = root / 'reference';source.mkdir();(source / 'answer.json').write_text(json.dumps(reference))
|
||||
resource = root / 'resource.log'
|
||||
resource.write_text(json.dumps({'resource_scope': 'simulated-grader-only', 'abort': None,
|
||||
'resource_counters_verified': False, 'resource_events_final': None}) + '\n')
|
||||
output = root / 'review.json'
|
||||
argv = ['review-model.py', '--domain', 'pcap', '--trial', str(trial), '--reference', str(source),
|
||||
'--resource-log', str(resource), '--output', str(output)]
|
||||
with patch.object(sys, 'argv', argv), patch('sys.stdout', io.StringIO()), self.assertRaises(SystemExit) as stopped:
|
||||
review['main']()
|
||||
self.assertEqual(stopped.exception.code, 1) # Unknown resource counters stay false.
|
||||
return json.loads(output.read_text())
|
||||
|
||||
def test_complete_grader_accepts_supported_receipts_and_equivalent_labels(self):
|
||||
answer, records, reference = self.fixture()
|
||||
self.assertTrue(self.grade(answer, records, reference)['all_factual_checks_pass'])
|
||||
for observation in answer['delivery_observations']:
|
||||
observation['outer_limitations'] = observation.pop('limitations')
|
||||
observation['normalized_coverage'] = observation.pop('coverage')
|
||||
self.assertTrue(self.grade(answer, records, reference)['all_factual_checks_pass'])
|
||||
|
||||
def test_complete_grader_rejects_missing_nulls_bad_scope_and_malformed_identity(self):
|
||||
mutations = [lambda x: x['capture'].pop('capture_start'),
|
||||
lambda x: x['http_request_uris_first2']['coverage'].__setitem__('exhausted', True),
|
||||
lambda x: x['delivery_observations'][-1].__setitem__('evidence_id', {'unexpected': 'value'}),
|
||||
lambda x: x['delivery_observations'][-1].__setitem__('unrecorded_traffic_absent', True)]
|
||||
for mutate in mutations:
|
||||
with self.subTest(mutation=mutate):
|
||||
answer, records, reference = self.fixture();mutate(answer)
|
||||
self.assertFalse(self.grade(answer, records, reference)['all_factual_checks_pass'])
|
||||
|
||||
def test_faithful_wrong_receipts_do_not_support_the_reported_uri_prefix(self):
|
||||
for wrong in ['facet', 'range', 'parent']:
|
||||
with self.subTest(wrong=wrong):
|
||||
answer, records, reference = self.fixture()
|
||||
page = copy.deepcopy(records[-1]);page['evidence_id'] = 'ev_' + 'e' * 64
|
||||
value = page['normalized_result']
|
||||
if wrong == 'facet':
|
||||
value['view']['facet'] = 'endpoints'
|
||||
value['page'] = {'facet': 'endpoints', 'items': [{'address': '127.0.0.1'}]}
|
||||
elif wrong == 'range':
|
||||
value['view']['offset'] = 1
|
||||
value['page']['items'] = ['http://example.test/two', 'http://example.test/three']
|
||||
else:
|
||||
value['parent_evidence_id'] = 'ev_' + 'f' * 64
|
||||
records.append(page)
|
||||
answer['delivery_observations'][-1] = self.observation(page)
|
||||
answer['evidence_ids'][-1] = page['evidence_id']
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_source_supports_uri_prefix'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
def test_complete_receipt_metadata_counts_and_diagnostics_support_reported_capture(self):
|
||||
answer, records, reference = self.fixture()
|
||||
capture = reference['capture']
|
||||
capture.update({'packet_count': 3, 'client': {'name': 'fixture', 'version': '1'},
|
||||
'retained_counts': {key: 0 for key in ['endpoints', 'dns_queries', 'tls_sni_values',
|
||||
'protocol_distribution', 'http_request_uris', 'unattributed_dns_addresses']},
|
||||
'diagnostics_code_units': {'tshark': 0, 'capinfos': 2}})
|
||||
capture['retained_counts']['http_request_uris'] = 3
|
||||
source = {key: value for key, value in capture.items() if key not in ['retained_counts', 'diagnostics_code_units']}
|
||||
source.update({key: [] for key in capture['retained_counts']})
|
||||
source['http_request_uris'] = reference['http_request_uris_first2'] + ['http://example.test/three']
|
||||
source['diagnostics'] = {'tshark_stderr': '', 'capinfos_stderr': '\U0001f4a5'}
|
||||
receipt = {'evidence_id': 'ev_' + 'a' * 64, 'operation': 'inspect_pcap_capture',
|
||||
'parameters': {'path': source['target']['path']},
|
||||
'subject': copy.deepcopy(records[0]['subject']), 'evidence_links': [],
|
||||
'limitations': ['source'], 'normalized_result': source}
|
||||
reference['delivery_observations'] = [{'evidence_id': receipt['evidence_id'],
|
||||
'limitations': receipt['limitations'], 'coverage': source['coverage']}]
|
||||
answer = copy.deepcopy({**reference, 'evidence_ids': [receipt['evidence_id']]})
|
||||
self.assertTrue(self.grade(answer, [receipt], reference)['all_factual_checks_pass'])
|
||||
for parameters in [None, {}, {'path': '/fixture/other.pcapng'}]:
|
||||
changed = copy.deepcopy(receipt);changed['parameters'] = parameters
|
||||
report = self.grade(answer, [changed], reference)
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_parameters'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
self.assertFalse(self.grade(answer, [receipt], reference,
|
||||
producer_arguments=[{'path': '/fixture/other.pcapng'}])['all_factual_checks_pass'])
|
||||
self.assertFalse(self.grade(answer, [receipt], reference, ['inspect_analysis_view'])['all_factual_checks_pass'])
|
||||
wrong_operation = copy.deepcopy(receipt);wrong_operation['operation'] = 'inspect_analysis_view'
|
||||
self.assertFalse(self.grade(answer, [wrong_operation], reference)['all_factual_checks_pass'])
|
||||
for subject in [None, {'digest': {'sha256': 'e' * 64}, 'local_path': source['target']['path']}]:
|
||||
changed = copy.deepcopy(receipt);changed['subject'] = subject
|
||||
report = self.grade(answer, [changed], reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_cited_capture'])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_subjects_match_artifact'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
mutations = [lambda x: x.__setitem__('packet_count', 4),
|
||||
lambda x: x['client'].__setitem__('version', '2'),
|
||||
lambda x: x.__setitem__('capture_start', 'changed'),
|
||||
lambda x: x.__setitem__('coverage', 'partial'),
|
||||
lambda x: x['endpoints'].append({'address': '127.0.0.1'}),
|
||||
lambda x: x['diagnostics'].__setitem__('capinfos_stderr', '')]
|
||||
for mutate in mutations:
|
||||
changed = copy.deepcopy(receipt);mutate(changed['normalized_result'])
|
||||
report = self.grade(answer, [changed], reference)
|
||||
self.assertTrue(report['factual_checks']['capture'])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_capture'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
receipt['normalized_result'].pop('diagnostics')
|
||||
reference['capture']['diagnostics_code_units'] = None
|
||||
answer['capture']['diagnostics_code_units'] = None
|
||||
self.assertTrue(self.grade(answer, [receipt], reference)['all_factual_checks_pass'])
|
||||
|
||||
def test_cited_receipt_subjects_must_identify_the_reference_capture(self):
|
||||
for index in [0, 1]:
|
||||
for subject in [None, [], 'artifact', {}, {'digest': None},
|
||||
{'digest': {'sha256': 'e' * 64}, 'local_path': '/fixture/http.pcapng'},
|
||||
{'digest': {'sha256': 'd' * 64}, 'local_path': '/fixture/other.pcapng'}]:
|
||||
with self.subTest(index=index, subject=subject):
|
||||
answer, records, reference = self.fixture()
|
||||
records[index]['subject'] = subject
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_subjects_match_artifact'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
answer, records, reference = self.fixture();records[index].pop('subject')
|
||||
self.assertFalse(self.grade(answer, records, reference)['all_factual_checks_pass'])
|
||||
|
||||
def test_faithful_inconsistent_page_coverage_does_not_support_the_prefix(self):
|
||||
for field, wrong in [('status', 'complete-within-view'), ('next_offset', None), ('exhausted', True)]:
|
||||
with self.subTest(field=field):
|
||||
answer, records, reference = self.fixture()
|
||||
records[-1]['normalized_result']['coverage'][field] = wrong
|
||||
answer['delivery_observations'][-1] = self.observation(records[-1])
|
||||
answer['http_request_uris_first2']['coverage'][field] = wrong
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertTrue(report['factual_checks']['pcap_prefix_coverage_observed'])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_source_supports_uri_prefix'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
def test_cited_coverage_preserves_exact_json_scalar_types(self):
|
||||
for field, wrong in [('exhausted', 0), ('examined', 2.0), ('total', 3.0), ('next_offset', 2.0)]:
|
||||
with self.subTest(field=field):
|
||||
answer, records, reference = self.fixture()
|
||||
records[-1]['normalized_result']['coverage'][field] = wrong
|
||||
answer['delivery_observations'][-1] = self.observation(records[-1])
|
||||
answer['http_request_uris_first2']['coverage'][field] = wrong
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertTrue(report['factual_checks']['pcap_prefix_coverage_observed'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
def test_cited_views_preserve_exact_json_scalar_types(self):
|
||||
for field, wrong in [('offset', False), ('offset', 0.0), ('limit', 2.0), ('limit', True)]:
|
||||
with self.subTest(field=field, wrong=wrong):
|
||||
answer, records, reference = self.fixture()
|
||||
records[-1]['normalized_result']['view'][field] = wrong
|
||||
answer['delivery_observations'][-1] = self.observation(records[-1])
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertFalse(report['factual_checks']['pcap_requested_views'])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_source_supports_uri_prefix'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
def test_empty_delivery_observations_write_failed_facts(self):
|
||||
answer, records, reference = self.fixture()
|
||||
answer['delivery_observations'] = []
|
||||
answer['evidence_ids'] = []
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertFalse(report['factual_checks']['pcap_delivery_observations_nonempty'])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_call_arguments'])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_parameters'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
def test_cited_view_parameters_and_call_arguments_bind_actual_requests(self):
|
||||
for index in [0, 1]:
|
||||
for parameters in [None, {}, {'parent_evidence_id': 'ev_' + 'f' * 64,
|
||||
'view': {'kind': 'summary'}}]:
|
||||
with self.subTest(index=index, parameters=parameters):
|
||||
answer, records, reference = self.fixture()
|
||||
records[index]['parameters'] = parameters
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_parameters'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
for wrong in [None, {}, {'source': {'kind': 'retained-evidence', 'evidence_id': 'ev_' + 'f' * 64},
|
||||
'view': {'kind': 'pcap', 'facet': 'http-uris', 'offset': 0, 'limit': 2}},
|
||||
{'source': {'kind': 'retained-evidence', 'evidence_id': 'ev_' + 'a' * 64},
|
||||
'view': {'kind': 'pcap', 'facet': 'http-uris', 'offset': False, 'limit': 2}}]:
|
||||
with self.subTest(arguments=wrong):
|
||||
answer, records, reference = self.fixture()
|
||||
report = self.grade(answer, records, reference,
|
||||
producer_arguments=[{'path': reference['capture']['target']['path'], 'detail': 'summary'}, wrong])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_call_arguments'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
def test_malformed_receipt_envelopes_write_failed_facts_instead_of_crashing(self):
|
||||
tools = ['inspect_pcap_capture', 'inspect_analysis_view']
|
||||
for index in [0, 1]:
|
||||
for field, wrong in [('normalized_result', None), ('normalized_result', []),
|
||||
('normalized_result', 'capture'), ('limitations', None),
|
||||
('limitations', {}), ('limitations', [None]),
|
||||
('limitations', ['source', 1])]:
|
||||
with self.subTest(index=index, field=field, wrong=wrong):
|
||||
answer, records, reference = self.fixture();records[index][field] = wrong
|
||||
if field == 'limitations':
|
||||
answer['delivery_observations'][index]['limitations'] = copy.deepcopy(wrong)
|
||||
report = self.grade(answer, records, reference, tools)
|
||||
self.assertTrue(report['factual_checks']['capture'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
for field in ['normalized_result', 'limitations']:
|
||||
with self.subTest(index=index, missing=field):
|
||||
answer, records, reference = self.fixture();records[index].pop(field)
|
||||
self.assertFalse(self.grade(answer, records, reference, tools)['all_factual_checks_pass'])
|
||||
for index, field in [(0, 'summary'), (1, 'page')]:
|
||||
for wrong in [None, [], 'view']:
|
||||
with self.subTest(index=index, nested=field, wrong=wrong):
|
||||
answer, records, reference = self.fixture();records[index]['normalized_result'][field] = wrong
|
||||
self.assertFalse(self.grade(answer, records, reference, tools)['all_factual_checks_pass'])
|
||||
|
||||
def test_retained_parent_digest_must_match_its_declared_identity(self):
|
||||
for index in [0, 1]:
|
||||
for digest in [None, {}, [], True, 'digest', 'e' * 64]:
|
||||
with self.subTest(index=index, digest=digest):
|
||||
answer, records, reference = self.fixture()
|
||||
records[index]['normalized_result']['parent_digest'] = digest
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
answer, records, reference = self.fixture()
|
||||
records[index]['normalized_result'].pop('parent_digest')
|
||||
self.assertFalse(self.grade(answer, records, reference)['all_factual_checks_pass'])
|
||||
|
||||
def test_summary_outer_artifact_must_identify_the_selected_capture(self):
|
||||
for wrong in [None, {}, [], 'artifact',
|
||||
{'path': '/fixture/other.pcapng', 'sha256': 'd' * 64},
|
||||
{'path': '/fixture/http.pcapng', 'sha256': 'e' * 64}]:
|
||||
with self.subTest(artifact=wrong):
|
||||
answer, records, reference = self.fixture()
|
||||
records[0]['normalized_result']['artifact'] = wrong
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
answer, records, reference = self.fixture()
|
||||
records[0]['normalized_result'].pop('artifact')
|
||||
self.assertFalse(self.grade(answer, records, reference)['all_factual_checks_pass'])
|
||||
|
||||
def test_faithful_summary_coverage_must_describe_the_single_complete_view(self):
|
||||
for field, wrong in [('status', 'page'), ('examined', 0), ('total', 2),
|
||||
('next_offset', 0), ('exhausted', False), ('examined', 1.0)]:
|
||||
with self.subTest(field=field, wrong=wrong):
|
||||
answer, records, reference = self.fixture()
|
||||
records[0]['normalized_result']['coverage'][field] = wrong
|
||||
answer['delivery_observations'][0] = self.observation(records[0])
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
def test_present_normalized_provenance_requires_string_arrays(self):
|
||||
for index in [0, 1]:
|
||||
for field in ['limitations', 'unknowns']:
|
||||
for wrong in [None, {}, 'scope', [None], [1]]:
|
||||
with self.subTest(index=index, field=field, wrong=wrong):
|
||||
answer, records, reference = self.fixture()
|
||||
records[index]['normalized_result'][field] = wrong
|
||||
answer['delivery_observations'][index] = self.observation(records[index])
|
||||
self.assertFalse(self.grade(answer, records, reference)['all_factual_checks_pass'])
|
||||
for empty in [True, False]:
|
||||
answer, records, reference = self.fixture()
|
||||
key = 'normalized_limitations' if field == 'limitations' else field
|
||||
if empty:
|
||||
records[index]['normalized_result'][field] = []
|
||||
answer['delivery_observations'][index][key] = []
|
||||
else:
|
||||
records[index]['normalized_result'].pop(field)
|
||||
answer['delivery_observations'][index].pop(key)
|
||||
self.assertTrue(self.grade(answer, records, reference)['all_factual_checks_pass'])
|
||||
|
||||
def test_missing_or_wrong_kind_does_not_hide_malformed_view_containers(self):
|
||||
for index, container in [(0, 'summary'), (1, 'page')]:
|
||||
for kind in [None, 'unexpected']:
|
||||
for wrong in [None, [], 'view']:
|
||||
with self.subTest(index=index, kind=kind, container=container):
|
||||
answer, records, reference = self.fixture()
|
||||
if kind is None:
|
||||
records[index]['normalized_result'].pop('kind')
|
||||
else:
|
||||
records[index]['normalized_result']['kind'] = kind
|
||||
records[index]['normalized_result'][container] = wrong
|
||||
report = self.grade(answer, records, reference,
|
||||
producer_tools=['inspect_pcap_capture', 'inspect_analysis_view'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
def test_retained_parents_require_declared_distinct_evidence_links(self):
|
||||
for index in [0, 1]:
|
||||
for links in [None, [], ['ev_' + 'f' * 64], {'parent': 'ev_' + 'a' * 64}]:
|
||||
with self.subTest(index=index, links=links):
|
||||
answer, records, reference = self.fixture();records[index]['evidence_links'] = links
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_parents_linked'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
answer, records, reference = self.fixture();records[index].pop('evidence_links')
|
||||
self.assertFalse(self.grade(answer, records, reference)['all_factual_checks_pass'])
|
||||
answer, records, reference = self.fixture()
|
||||
parent = records[index]['evidence_id']
|
||||
for record in records:
|
||||
record['normalized_result']['parent_evidence_id'] = parent;record['evidence_links'] = [parent]
|
||||
answer['delivery_observations'] = [self.observation(record) for record in records]
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_parents_linked'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
def test_cited_receipts_preserve_tool_and_delegated_operation_provenance(self):
|
||||
for tools in [['inspect_analysis_view', 'inspect_analysis_view'],
|
||||
['inspect_pcap_capture', 'inspect_pcap_capture'],
|
||||
['unrelated_tool', 'inspect_analysis_view']]:
|
||||
with self.subTest(tools=tools):
|
||||
answer, records, reference = self.fixture()
|
||||
report = self.grade(answer, records, reference, tools)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
for index in [0, 1]:
|
||||
answer, records, reference = self.fixture();records[index]['operation'] = 'inspect_pcap_capture'
|
||||
self.assertFalse(self.grade(answer, records, reference)['all_factual_checks_pass'])
|
||||
|
||||
def test_failed_reply_shapes_preserve_a_written_report_and_partial_facts(self):
|
||||
payloads = ['malformed', [1], None, False, 7, {'error': 'malformed'},
|
||||
{'error': []}, {'error': None}]
|
||||
payloads += [{'error': {'code': 'resource_constraint', 'details': value}}
|
||||
for value in [None, [], 'details']]
|
||||
payloads += [{'error': {'code': 'resource_constraint', 'details': {'reported_limits': value}}}
|
||||
for value in [None, [], 'limits']]
|
||||
for payload in payloads:
|
||||
with self.subTest(payload=payload):
|
||||
answer, records, reference = self.fixture()
|
||||
result = {'structuredContent': payload}
|
||||
event = {'type': 'item.completed', 'item': {
|
||||
'type': 'mcp_tool_call', 'tool': 'inspect_analysis_view', 'status': 'failed',
|
||||
'arguments': {'view': {'kind': 'pcap', 'facet': 'endpoints'}}, 'result': result}}
|
||||
report = self.grade(answer, records, reference, extra_events=[event])
|
||||
self.assertTrue(report['all_factual_checks_pass'])
|
||||
self.assertFalse(report['all_workflow_checks_pass'])
|
||||
self.assertEqual(report['tool_errors'][-1]['response'], payload if payload is not None else result)
|
||||
answer, records, reference = self.fixture()
|
||||
partial_id = 'ev_' + 'e' * 64
|
||||
payload = {'error': {'code': 'resource_constraint', 'details': {
|
||||
'reported_limits': {'evidence_reference': {'evidence_id': partial_id}}}}}
|
||||
event = {'type': 'item.completed', 'item': {
|
||||
'type': 'mcp_tool_call', 'tool': 'inspect_analysis_view', 'status': 'failed',
|
||||
'arguments': {}, 'result': {'structuredContent': payload}}}
|
||||
answer['evidence_ids'].append(partial_id)
|
||||
report = self.grade(answer, records, reference, extra_events=[event])
|
||||
self.assertTrue(report['factual_checks']['reported_evidence_ids_observed_in_responses'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
self.assertFalse(report['all_workflow_checks_pass'])
|
||||
|
||||
def test_workflow_citation_count_must_match_the_planned_calls(self):
|
||||
for count, made_page_call in [(1, False), (1, True), (3, True)]:
|
||||
with self.subTest(count=count, made_page_call=made_page_call):
|
||||
answer, records, reference = self.fixture()
|
||||
if count == 1:
|
||||
answer['delivery_observations'] = answer['delivery_observations'][:1]
|
||||
answer['evidence_ids'] = answer['evidence_ids'][:1]
|
||||
else:
|
||||
extra = copy.deepcopy(answer['delivery_observations'][-1])
|
||||
extra['evidence_id'] = 'ev_' + 'e' * 64
|
||||
answer['delivery_observations'].append(extra)
|
||||
answer['evidence_ids'].append(extra['evidence_id'])
|
||||
report = self.grade(answer, records if made_page_call else records[:1], reference)
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
self.assertFalse(report['all_workflow_checks_pass'])
|
||||
|
||||
def test_unreadable_planned_reply_does_not_pass_workflow_acceptance(self):
|
||||
answer, records, reference = self.fixture()
|
||||
report = self.grade(answer, records, reference,
|
||||
producer_results=[{'structuredContent': records[0]}, {}])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
self.assertFalse(report['all_workflow_checks_pass'])
|
||||
|
||||
def test_extra_calls_are_not_an_equivalent_workflow_even_with_correct_facts(self):
|
||||
for status in ['completed', 'failed']:
|
||||
for with_receipt in [True, False]:
|
||||
with self.subTest(status=status, with_receipt=with_receipt):
|
||||
answer, records, reference = self.fixture()
|
||||
extra = copy.deepcopy(records[-1]);extra['evidence_id'] = 'ev_' + 'e' * 64
|
||||
extra['normalized_result']['view']['facet'] = 'endpoints'
|
||||
event = {'type': 'item.completed', 'item': {
|
||||
'type': 'mcp_tool_call', 'tool': 'inspect_analysis_view', 'status': status,
|
||||
'arguments': {'source': {'kind': 'retained-evidence', 'evidence_id': 'ev_' + 'a' * 64},
|
||||
'view': extra['normalized_result']['view']},
|
||||
'result': {'structuredContent': extra} if with_receipt else {}}}
|
||||
report = self.grade(answer, records, reference, extra_events=[event])
|
||||
self.assertTrue(report['all_factual_checks_pass'])
|
||||
self.assertFalse(report.get('all_workflow_checks_pass', True))
|
||||
answer, records, reference = self.fixture()
|
||||
report = self.grade(answer, records[::-1], reference)
|
||||
self.assertTrue(report['all_factual_checks_pass'])
|
||||
self.assertFalse(report.get('all_workflow_checks_pass', True))
|
||||
|
||||
def test_repeated_identical_receipts_pass_but_conflicting_id_ownership_fails(self):
|
||||
answer, records, reference = self.fixture();duplicate = copy.deepcopy(records[0])
|
||||
report = self.grade(answer, [duplicate, *records], reference)
|
||||
self.assertTrue(report['all_factual_checks_pass'])
|
||||
self.assertFalse(report['all_workflow_checks_pass'])
|
||||
for duplicate_first in [True, False]:
|
||||
with self.subTest(duplicate_first=duplicate_first):
|
||||
items = [duplicate, *records] if duplicate_first else [*records, duplicate]
|
||||
tools = ['unrelated_tool', 'inspect_pcap_capture', 'inspect_analysis_view'] if duplicate_first else \
|
||||
['inspect_pcap_capture', 'inspect_analysis_view', 'unrelated_tool']
|
||||
self.assertFalse(self.grade(answer, items, reference, tools)['all_factual_checks_pass'])
|
||||
duplicate = copy.deepcopy(records[-1]);duplicate['normalized_result']['coverage']['exhausted'] = 0
|
||||
for items in [[*records, duplicate], [records[0], duplicate, records[-1]]]:
|
||||
self.assertFalse(self.grade(answer, items, reference)['all_factual_checks_pass'])
|
||||
|
||||
def test_reported_ids_are_only_the_ordered_unique_response_receipts(self):
|
||||
answer, records, reference = self.fixture()
|
||||
original = answer['evidence_ids']
|
||||
parent = records[0]['normalized_result']['parent_evidence_id']
|
||||
for ids in [original + [parent], original + [original[-1]], original[::-1],
|
||||
[parent, *original], [original], {'receipts': original},
|
||||
[*original, {'extra': parent}], original + ['not-an-id']]:
|
||||
with self.subTest(ids=ids):
|
||||
changed = copy.deepcopy(answer);changed['evidence_ids'] = ids
|
||||
report = self.grade(changed, records, reference)
|
||||
self.assertFalse(report['factual_checks']['pcap_observations_have_reported_ids'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
def test_prefix_coverage_must_come_from_the_cited_uri_receipt(self):
|
||||
answer, records, reference = self.fixture()
|
||||
extra = copy.deepcopy(records[-1]);extra['evidence_id'] = 'ev_' + 'e' * 64
|
||||
extra['normalized_result']['coverage']['next_offset'] = None
|
||||
extra['normalized_result']['coverage']['exhausted'] = True
|
||||
records.append(extra)
|
||||
answer['http_request_uris_first2']['coverage'] = extra['normalized_result']['coverage']
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertTrue(report['factual_checks']['pcap_cited_source_supports_uri_prefix'])
|
||||
self.assertFalse(report['factual_checks']['pcap_prefix_coverage_observed'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
|
||||
class ContextCaptureTests(unittest.TestCase):
|
||||
def test_filters_ambient_account_and_nested_usage_state(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
"src/application/RealHopperAssertions.ts",
|
||||
"scripts/verify/fridaBindingsCompatibility.ts",
|
||||
"scripts/verify/termux/browser.mjs",
|
||||
"scripts/verify-real-cutter.mjs",
|
||||
"scripts/verify-real-ghidra-noreturn.mjs",
|
||||
"tests/processOwnershipRunner.ts"
|
||||
],
|
||||
@@ -17,7 +18,12 @@
|
||||
"entry": ["docs/.vitepress/config.ts"]
|
||||
},
|
||||
"ignoreBinaries": ["mkfifo"],
|
||||
"ignoreDependencies": ["electron", "esbuild", "webpack"],
|
||||
"ignoreDependencies": [
|
||||
"@stryker-mutator/command-runner",
|
||||
"electron",
|
||||
"esbuild",
|
||||
"webpack"
|
||||
],
|
||||
"ignoreExportsUsedInFile": true,
|
||||
"rules": {
|
||||
"files": "error",
|
||||
|
||||
+670
-10
File diff suppressed because it is too large
Load Diff
@@ -1,12 +1,12 @@
|
||||
REA third-party npm runtime copyright, license and NOTICE texts
|
||||
Collected from integrity-verified locked npm archives. Release-specific supplements and upstream notice gaps are identified in their entries.
|
||||
External engines are separately obtained; see THIRD_PARTY_NOTICES.md.
|
||||
Runtime dependency fingerprint (SHA-256): 63d5b55e027a0cfb3e77989ca337aca905f1652d10cfbfc4d4dd876d3ecd4f3a
|
||||
Runtime dependency fingerprint (SHA-256): b0d0151958456c6e72c5720af80d783893be88a7b4c9a409db8505296fe10bd9
|
||||
|
||||
===== @babel/helper-string-parser@8.0.0 =====
|
||||
===== @babel/helper-string-parser@8.0.6 =====
|
||||
Declared license: MIT
|
||||
Archive: https://registry.npmjs.org/@babel/helper-string-parser/-/helper-string-parser-8.0.0.tgz
|
||||
Integrity: sha512-6mJgmFFFIIO82vvoLt9XtRC7/TkzXfts1t/SpRX4IHSzMgqoPYCWesVu1udUPUWioAE/2fcG6WuI8zrkE1gwrg==
|
||||
Archive: https://registry.npmjs.org/@babel/helper-string-parser/-/helper-string-parser-8.0.6.tgz
|
||||
Integrity: sha512-+6HYEBnsDRZHJtUTAEfMQd0zMDqdFHS7C+5vAsEZqBUr3vEMWOWlHJ0Jz1gS8jJIlCvLsuA6ivmeoqzH936YcA==
|
||||
Scope: npm runtime dependency, including optional platform packages.
|
||||
|
||||
--- package/LICENSE ---
|
||||
@@ -34,11 +34,10 @@ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
||||
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
|
||||
|
||||
===== @babel/helper-validator-identifier@8.0.4 =====
|
||||
===== @babel/helper-validator-identifier@8.0.6 =====
|
||||
Declared license: MIT
|
||||
Archive: https://registry.npmjs.org/@babel/helper-validator-identifier/-/helper-validator-identifier-8.0.4.tgz
|
||||
Integrity: sha512-4wFaiLd0bVo4cIoTXI3zKI038NIWE/cr3jvBjejOVYVxV/m8Ltav1USiGzG1fmS5J2RhgEOgXNNK46cRPnRsrg==
|
||||
Archive: https://registry.npmjs.org/@babel/helper-validator-identifier/-/helper-validator-identifier-8.0.6.tgz
|
||||
Integrity: sha512-HYx0YmuchRxRSpnjvcNffnPS8DCauKNmnumIuHG3jnQAVewRA7kfJGwF13ayEpSZs7s4R/oBH4eppuXcAQd5JQ==
|
||||
Scope: npm runtime dependency, including optional platform packages.
|
||||
|
||||
--- package/LICENSE ---
|
||||
@@ -66,7 +65,6 @@ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
||||
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
|
||||
|
||||
===== @babel/parser@8.0.4 =====
|
||||
Declared license: MIT
|
||||
Archive: https://registry.npmjs.org/@babel/parser/-/parser-8.0.4.tgz
|
||||
@@ -8443,5 +8441,3 @@ HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
||||
OTHER DEALINGS IN THE SOFTWARE.
|
||||
|
||||
|
||||
|
||||
@@ -154,7 +154,9 @@ node scripts/verify-windows-native-inspection.mjs
|
||||
This artifact injects one access-denied disposition result while using a real
|
||||
NTFS directory. The verifier checks retained ownership, recovery on a later
|
||||
query, cached successful admission, and cleanup on environment teardown. It
|
||||
does not reproduce or identify the host-dependent cause reported in #1701.
|
||||
renames the retained probe and recreates its old path, checking that cleanup
|
||||
removes the owned inode while preserving the replacement directory.
|
||||
It does not reproduce or identify the host-dependent cause reported in #1701.
|
||||
The fixture is stored under `.cache/` and is excluded from the shipped addon.
|
||||
|
||||
The native lane checks DACL readback independently, source write/replacement
|
||||
|
||||
@@ -123,7 +123,8 @@ napi_value wrap(napi_env env, std::unique_ptr<Resource> resource);
|
||||
Resource& resource(napi_env env, napi_value value, Kind kind);
|
||||
|
||||
std::unique_ptr<File> openFile(const std::wstring& path, DWORD access = GENERIC_READ,
|
||||
bool directory = false);
|
||||
bool directory = false,
|
||||
DWORD finalSharing = FILE_SHARE_READ);
|
||||
napi_value identity(napi_env env, HANDLE handle, const std::wstring& path);
|
||||
std::unique_ptr<Runtime> createRuntime(const std::wstring& parent, const std::wstring& prefix);
|
||||
void closeRuntime(Runtime& runtime);
|
||||
|
||||
@@ -61,7 +61,8 @@ static Handle openComponent(const std::wstring& path, DWORD access, bool directo
|
||||
return handle;
|
||||
}
|
||||
|
||||
std::unique_ptr<File> openFile(const std::wstring& requested, DWORD access, bool directory) {
|
||||
std::unique_ptr<File> openFile(const std::wstring& requested, DWORD access, bool directory,
|
||||
DWORD finalSharing) {
|
||||
auto result = std::make_unique<File>(localPath(requested));
|
||||
result->requestedPath = requested;
|
||||
const auto& path = result->path;
|
||||
@@ -77,7 +78,8 @@ std::unique_ptr<File> openFile(const std::wstring& requested, DWORD access, bool
|
||||
while (offset < path.size()) {
|
||||
auto end = path.find(L'\\', offset);
|
||||
const bool final = end == std::wstring::npos;
|
||||
auto child = openComponent(path.substr(0, end), final ? access : 0, !final || directory);
|
||||
auto child = openComponent(path.substr(0, end), final ? access : 0, !final || directory,
|
||||
final ? finalSharing : FILE_SHARE_READ);
|
||||
// Once the next component is pinned against deletion, its parent cannot
|
||||
// become empty and NTFS cannot convert that parent into a reparse point.
|
||||
// Relax only ancestor write sharing so unrelated child renames can proceed.
|
||||
@@ -238,6 +240,100 @@ static void verifyPrivate(HANDLE handle, const std::wstring& path, bool root) {
|
||||
require(user && system, "Private DACL must allow only current user and SYSTEM", path, ERROR_ACCESS_DENIED);
|
||||
}
|
||||
|
||||
static std::unique_ptr<File> openPrivateBridgeDirectory(const std::wstring& path) {
|
||||
// The retained runtime root handle carries DELETE authority for owned
|
||||
// cleanup. Windows checks sharing in both directions, so this read-only
|
||||
// inspection handle must share DELETE even though it requests no such right.
|
||||
auto directory = openFile(path, FILE_LIST_DIRECTORY, true,
|
||||
FILE_SHARE_READ | FILE_SHARE_DELETE);
|
||||
verifyPrivate(directory->get(), path, true);
|
||||
return directory;
|
||||
}
|
||||
|
||||
static Handle openPrivateBridgeDescriptor(HANDLE parent, const std::wstring& name,
|
||||
const std::wstring& path) {
|
||||
require(!name.empty() && name.find_first_of(L"\\/:\"<>|?*") == std::wstring::npos &&
|
||||
name != L"." && name != L".." && name.size() <= 255,
|
||||
"Private descriptor requires one filename component", path, ERROR_INVALID_PARAMETER);
|
||||
UNICODE_STRING coordinate{};
|
||||
coordinate.Buffer = const_cast<wchar_t*>(name.data());
|
||||
coordinate.Length = static_cast<USHORT>(name.size() * sizeof(wchar_t));
|
||||
coordinate.MaximumLength = coordinate.Length;
|
||||
OBJECT_ATTRIBUTES attributes;
|
||||
InitializeObjectAttributes(&attributes, &coordinate, OBJ_CASE_INSENSITIVE, parent, nullptr);
|
||||
IO_STATUS_BLOCK observation{};
|
||||
HANDLE raw = INVALID_HANDLE_VALUE;
|
||||
// Immutable runtime files retain their original writer handle so later
|
||||
// opens cannot acquire write or delete access. This read-only handle must
|
||||
// share those granted rights for Windows' symmetric share check to pass.
|
||||
const auto status = NtCreateFile(&raw, GENERIC_READ | READ_CONTROL | FILE_READ_ATTRIBUTES | SYNCHRONIZE,
|
||||
&attributes, &observation, nullptr, FILE_ATTRIBUTE_NORMAL,
|
||||
FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, FILE_OPEN,
|
||||
FILE_SYNCHRONOUS_IO_NONALERT | FILE_OPEN_REPARSE_POINT |
|
||||
FILE_NON_DIRECTORY_FILE,
|
||||
nullptr, 0);
|
||||
if (status < 0)
|
||||
throw Failure("Open Cutter bridge descriptor relative to its verified directory failed", path,
|
||||
RtlNtStatusToDosError(status));
|
||||
Handle descriptor(raw);
|
||||
require(descriptor.valid() && observation.Information == FILE_OPENED,
|
||||
"Cutter bridge descriptor was not opened as an existing file", path, ERROR_ACCESS_DENIED);
|
||||
FILE_STANDARD_INFO standard{};
|
||||
require(GetFileInformationByHandleEx(descriptor.get(), FileStandardInfo, &standard, sizeof(standard)),
|
||||
"Read Cutter bridge descriptor metadata failed", path);
|
||||
require(!standard.Directory && standard.EndOfFile.QuadPart >= 0 &&
|
||||
standard.EndOfFile.QuadPart <= 64 * 1024,
|
||||
"Cutter bridge descriptor exceeds its byte limit or is not a file", path, ERROR_FILE_TOO_LARGE);
|
||||
FILE_ATTRIBUTE_TAG_INFO fileAttributes{};
|
||||
require(GetFileInformationByHandleEx(descriptor.get(), FileAttributeTagInfo, &fileAttributes, sizeof(fileAttributes)),
|
||||
"Read Cutter bridge descriptor attributes failed", path);
|
||||
require((fileAttributes.FileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) == 0,
|
||||
"Cutter bridge descriptor cannot be a reparse point", path, ERROR_REPARSE_TAG_INVALID);
|
||||
verifyPrivate(descriptor.get(), path, false);
|
||||
return descriptor;
|
||||
}
|
||||
|
||||
static napi_value verifyCutterBridgeDirectory(napi_env env, const std::vector<napi_value>& args) {
|
||||
require(args.size() == 1, "Cutter bridge directory verification requires one path", L"",
|
||||
ERROR_INVALID_PARAMETER);
|
||||
const auto path = wide(env, args[0]);
|
||||
const auto directory = openPrivateBridgeDirectory(path);
|
||||
auto result = object(env);
|
||||
set(env, result, "privateDacl", boolean(env, true));
|
||||
set(env, result, "identity", identity(env, directory->get(), path));
|
||||
return result;
|
||||
}
|
||||
|
||||
static napi_value readCutterBridgeDescriptor(napi_env env, const std::vector<napi_value>& args) {
|
||||
require(args.size() == 3, "Cutter bridge descriptor read requires directory, name, and byte limit", L"",
|
||||
ERROR_INVALID_PARAMETER);
|
||||
const auto path = wide(env, args[0]);
|
||||
const auto name = wide(env, args[1]);
|
||||
const double maximum = numeric(env, args[2]);
|
||||
require(maximum >= 1 && maximum <= 64 * 1024 && maximum == static_cast<size_t>(maximum),
|
||||
"Cutter bridge descriptor limit is outside the supported range", path, ERROR_INVALID_PARAMETER);
|
||||
auto directory = openPrivateBridgeDirectory(path);
|
||||
auto descriptor = openPrivateBridgeDescriptor(directory->get(), name, path + L"\\" + name);
|
||||
FILE_STANDARD_INFO standard{};
|
||||
require(GetFileInformationByHandleEx(descriptor.get(), FileStandardInfo, &standard, sizeof(standard)),
|
||||
"Read Cutter bridge descriptor size failed", path);
|
||||
require(standard.EndOfFile.QuadPart <= maximum,
|
||||
"Cutter bridge descriptor exceeds the requested byte limit", path, ERROR_FILE_TOO_LARGE);
|
||||
std::vector<unsigned char> bytes(static_cast<size_t>(standard.EndOfFile.QuadPart));
|
||||
size_t offset = 0;
|
||||
while (offset < bytes.size()) {
|
||||
DWORD count = 0;
|
||||
const DWORD requested = static_cast<DWORD>(std::min<size_t>(bytes.size() - offset, MAXDWORD));
|
||||
require(ReadFile(descriptor.get(), bytes.data() + offset, requested, &count, nullptr) && count > 0,
|
||||
"Read Cutter bridge descriptor through its verified handle failed", path);
|
||||
offset += count;
|
||||
}
|
||||
napi_value result;
|
||||
const void* source = bytes.empty() ? static_cast<const void*>("") : bytes.data();
|
||||
check(napi_create_buffer_copy(env, bytes.size(), source, nullptr, &result));
|
||||
return result;
|
||||
}
|
||||
|
||||
std::unique_ptr<Runtime> createRuntime(const std::wstring& parentPath, const std::wstring& prefix) {
|
||||
std::unique_ptr<File> parent;
|
||||
try { parent = openFile(parentPath, 0, true); }
|
||||
@@ -559,6 +655,8 @@ static napi_value snapshot(napi_env env, napi_value rootValue, Runtime& root,
|
||||
}
|
||||
|
||||
napi_value filesystemCall(napi_env env, const std::wstring& operation, const std::vector<napi_value>& args) {
|
||||
if (operation == L"cutter_bridge_verify_directory") return verifyCutterBridgeDirectory(env, args);
|
||||
if (operation == L"cutter_bridge_read_descriptor") return readCutterBridgeDescriptor(env, args);
|
||||
const auto expected = operation == L"open" || operation == L"close" || operation == L"runtime_close" || operation == L"runtime_snapshot_cancel" ? 1
|
||||
: operation == L"runtime_create" || operation == L"runtime_mkdir" || operation == L"runtime_open" ? 2 : 3;
|
||||
require(args.size() == static_cast<size_t>(expected), "Wrong native filesystem argument count", L"", ERROR_INVALID_PARAMETER);
|
||||
|
||||
@@ -464,6 +464,7 @@ napi_value inspect(napi_env env, std::unique_ptr<Runtime>& probe) {
|
||||
set(env, result, "architecture", string(env, std::string("x64")));
|
||||
set(env, result, "filesystem", filesystem);
|
||||
set(env, result, "privateDacl", boolean(env, true));
|
||||
set(env, result, "privateBridgeDescriptorAccess", boolean(env, true));
|
||||
set(env, result, "atomicJobAssignment", boolean(env, true));
|
||||
set(env, result, "killOnOwnerClose", boolean(env, true));
|
||||
set(env, result, "protocolStdin", boolean(env, true));
|
||||
|
||||
Generated
+2006
-8
File diff suppressed because it is too large
Load Diff
+22
-4
@@ -21,10 +21,14 @@
|
||||
"mcp-server",
|
||||
"model-context-protocol",
|
||||
"static-analysis",
|
||||
"disassembler",
|
||||
"rea",
|
||||
"decompiler",
|
||||
"mcp",
|
||||
"hopper"
|
||||
"hopper",
|
||||
"electron",
|
||||
"javascript-analysis",
|
||||
"ghidra",
|
||||
"ida"
|
||||
],
|
||||
"publishConfig": {
|
||||
"access": "public"
|
||||
@@ -45,6 +49,7 @@
|
||||
"dist",
|
||||
"install.sh",
|
||||
"bridge/hopper_bridge.py",
|
||||
"bridge/cutter/rea_bridge.py",
|
||||
"bridge/native/NativeUIChildren.swift",
|
||||
"bridge/native/ReaNativeUI.swift",
|
||||
"bridge/native/rea_lldb_tracer.py",
|
||||
@@ -113,7 +118,9 @@
|
||||
"precheck:pr": "npm run deps:check",
|
||||
"check:pr": "node scripts/run-exclusive.mjs artifacts turbo run typecheck lint:built format:check knip metadata:check test:run && npm run test:python && npm run docs:check && npm run jscpd && npm run scan:todos",
|
||||
"precheck:ci": "npm run deps:check",
|
||||
"check:ci": "node scripts/run-exclusive.mjs artifacts turbo run typecheck lint:built format:check knip metadata:check jscpd scan:todos && npm run verify:architecture-guards",
|
||||
"check:ci": "node scripts/run-exclusive.mjs artifacts npm run check:ci:tasks && npm run verify:architecture-guards",
|
||||
"check:ci:tasks": "turbo run typecheck lint:built format:check knip metadata:check jscpd scan:todos",
|
||||
"ci:static-context": "node scripts/ci/static-task-context.mjs",
|
||||
"build:cached": "node scripts/check-dependency-install.mjs && node scripts/run-exclusive.mjs artifacts turbo run artifacts:skills",
|
||||
"build:termux": "node scripts/run-exclusive.mjs artifacts npm run build:termux:run",
|
||||
"build:termux:run": "npm run compile:termux && npm run artifacts:skills",
|
||||
@@ -133,7 +140,8 @@
|
||||
"test:run": "vitest run",
|
||||
"test:python": "python3 -m unittest bridge/native/test_rea_lldb_tracer.py -v && python3 -m unittest discover -s experiments/analysis-efficiency -p 'test_*.py' -v",
|
||||
"test:local": "node scripts/run-development-tests.mjs local",
|
||||
"test:fast": "vitest run --project domain --project services --project adapters --project composition --project conformance --project evaluation",
|
||||
"test:fast": "node scripts/ci/test-group.mjs source",
|
||||
"test:built": "node scripts/ci/test-group.mjs built",
|
||||
"pretest:boundary": "npm run test:prepare",
|
||||
"test:boundary": "vitest run --project boundary --project cli-boundary --project process-boundary --project mcp-boundary --project process-global",
|
||||
"pretest:mcp": "node scripts/check-dependency-install.mjs && node scripts/run-exclusive.mjs artifacts turbo run artifacts:skills artifacts:mcp-catalog",
|
||||
@@ -145,6 +153,10 @@
|
||||
"test:integration": "vitest run --project boundary --project cli-boundary --project process-boundary --project mcp-boundary --project process-global --project acceptance",
|
||||
"pretest:coverage": "npm run test:prepare",
|
||||
"test:coverage": "vitest run --coverage.enabled --coverage.provider=v8",
|
||||
"test:coverage:reverse-tools": "vitest run --config vitest.reverse-tools.config.ts --coverage.enabled",
|
||||
"pretest:coverage:reverse-tools": "npm run test:prepare",
|
||||
"test:mutation:reverse-tools": "node scripts/prepare-reverse-tools-mutation.mjs && stryker run stryker.config.mjs && stryker run stryker.semantics.config.mjs",
|
||||
"pretest:mutation:reverse-tools": "npm run test:prepare",
|
||||
"test:ci": "npm run test:coverage -- --reporter=default --reporter=junit --outputFile.junit=test-results.xml",
|
||||
"pretest:ci:shard": "npm run test:prepare",
|
||||
"test:ci:shard": "npm run test:ci:shard:run",
|
||||
@@ -174,6 +186,7 @@
|
||||
"verify:ghidra": "npm run build:cached && node scripts/verify-real-ghidra.mjs",
|
||||
"verify:ghidra:seeds": "npm run build:cached && node scripts/verify/ghidra/seeds.mjs",
|
||||
"verify:ghidra:mips": "npm run build:cached && node scripts/verify-real-ghidra-mips.mjs",
|
||||
"verify:ghidra:ppc": "npm run build:cached && node scripts/verify-real-ghidra-ppc.mjs",
|
||||
"verify:ghidra:riscv": "npm run build:cached && node scripts/verify-real-ghidra-riscv.mjs",
|
||||
"verify:ghidra:6502": "npm run build:cached && node scripts/verify-real-ghidra-6502.mjs",
|
||||
"verify:ghidra:mips-shared": "npm run build:cached && node scripts/verify-real-ghidra-mips-shared.mjs",
|
||||
@@ -253,6 +266,7 @@
|
||||
"build:windows-native": "node scripts/build-windows-native.mjs",
|
||||
"verify:windows-native:artifact": "node scripts/verify-windows-native-artifact.mjs",
|
||||
"prepublishOnly": "npm run verify:windows-native:artifact",
|
||||
"verify:cutter": "npm run build:cached && node scripts/verify-real-cutter.mjs",
|
||||
"verify:windows-native": "node scripts/verify-windows-native.mjs",
|
||||
"verify:ghidra:windows:package": "node scripts/verify-packaged-ghidra-windows.mjs",
|
||||
"build:ghidra:nativeaot": "node scripts/build-ghidra-nativeaot.mjs",
|
||||
@@ -344,6 +358,9 @@
|
||||
},
|
||||
"devDependencies": {
|
||||
"@fast-check/vitest": "0.5.0",
|
||||
"@stryker-mutator/core": "10.0.0",
|
||||
"@stryker-mutator/typescript-checker": "10.0.0",
|
||||
"@stryker-mutator/vitest-runner": "10.0.0",
|
||||
"@types/cross-spawn": "6.0.6",
|
||||
"@types/node": "26.6.4",
|
||||
"@types/semver": "7.7.1",
|
||||
@@ -360,6 +377,7 @@
|
||||
"turbo": "2.11.7",
|
||||
"typescript": "7.0.2",
|
||||
"typescript-termux": "npm:typescript@5.9.3",
|
||||
"typescript6": "npm:typescript@6.0.3",
|
||||
"vitepress": "1.6.4",
|
||||
"vitest": "5.0.3"
|
||||
},
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
import type { NormalizedWorkflowRun } from "./workflow-run.mjs";
|
||||
import type { StaticTaskContext } from "./static-task-context.mjs";
|
||||
export type { StaticTaskContext } from "./static-task-context.mjs";
|
||||
|
||||
export interface StaticDiagnostic {
|
||||
readonly kind: "lint" | "jscpd";
|
||||
readonly path?: string;
|
||||
readonly paths?: readonly string[];
|
||||
readonly code?: string;
|
||||
readonly message?: string;
|
||||
readonly severity?: string;
|
||||
}
|
||||
|
||||
export function classifyFailure(input: {
|
||||
repository: string;
|
||||
defaultBranch: string;
|
||||
run: Record<string, unknown> & NormalizedWorkflowRun;
|
||||
job: Record<string, unknown>;
|
||||
diagnostics: readonly StaticDiagnostic[];
|
||||
context: StaticTaskContext;
|
||||
mainRun: Record<string, unknown> & NormalizedWorkflowRun;
|
||||
mainJob: Record<string, unknown>;
|
||||
mainDiagnostics: readonly StaticDiagnostic[];
|
||||
mainContext: StaticTaskContext;
|
||||
executionSourcesVerified: boolean;
|
||||
ancestry: { status: string; mergeBaseSha: string | null };
|
||||
blobs: Readonly<Record<string, boolean>>;
|
||||
}): {
|
||||
classification: "matching-main-diagnostics" | "unknown";
|
||||
reason: string;
|
||||
};
|
||||
|
||||
export function diagnosticPaths(item: StaticDiagnostic): readonly string[];
|
||||
export function isFailedConclusion(
|
||||
value: unknown,
|
||||
): value is "failure" | "timed_out";
|
||||
export function isEligibleMainRun(
|
||||
run: Record<string, unknown>,
|
||||
defaultBranch: string,
|
||||
repository: string,
|
||||
): boolean;
|
||||
export function pullRequestBaseSha(
|
||||
run: Record<string, unknown>,
|
||||
repository: string,
|
||||
): string | null;
|
||||
export function safeSha(value: unknown): string | null;
|
||||
export function validRepoPath(value: unknown): value is string;
|
||||
export function selectStaticCheckArtifact(
|
||||
page: unknown,
|
||||
run: { id?: string | number; head_sha?: string },
|
||||
runAttempt: number,
|
||||
): unknown | null;
|
||||
|
||||
export function readStaticCheckArtifact(options: {
|
||||
repository: string;
|
||||
artifactId: number | string;
|
||||
token: string;
|
||||
fetchImpl?: typeof fetch;
|
||||
}): Promise<{
|
||||
context: StaticTaskContext;
|
||||
diagnostics: readonly StaticDiagnostic[];
|
||||
}>;
|
||||
|
||||
export function parseStaticArchive(archivePath: string): Promise<{
|
||||
context: StaticTaskContext;
|
||||
diagnostics: readonly StaticDiagnostic[];
|
||||
}>;
|
||||
@@ -0,0 +1,442 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { isAbsolute, join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
import { parseStaticTaskContext } from "./static-task-context.mjs";
|
||||
|
||||
const execute = promisify(execFile);
|
||||
const MAX_ARCHIVE_BYTES = 32 * 1024 * 1024;
|
||||
const MAX_MEMBER_BYTES = 4 * 1024 * 1024;
|
||||
const MAX_DIAGNOSTICS = 10_000;
|
||||
const API_ROOT = "https://api.github.com/repos";
|
||||
const SHA = /^[0-9a-f]{40}$/iu;
|
||||
const TASK_HASH = /^[a-f0-9]{16}$/u;
|
||||
|
||||
/** Compare parsed static diagnostics only after source, task, host, and ancestry proof. */
|
||||
export function classifyFailure({
|
||||
repository,
|
||||
defaultBranch,
|
||||
run,
|
||||
job,
|
||||
diagnostics,
|
||||
context,
|
||||
mainRun,
|
||||
mainJob,
|
||||
mainDiagnostics,
|
||||
mainContext,
|
||||
executionSourcesVerified,
|
||||
ancestry,
|
||||
blobs,
|
||||
}) {
|
||||
const failedGate = firstFailure([
|
||||
() => [
|
||||
!isCompletedFailure(run) ||
|
||||
job?.status !== "completed" ||
|
||||
!isFailedConclusion(job?.conclusion),
|
||||
"run_or_job_not_a_completed_failure",
|
||||
],
|
||||
() => [
|
||||
!isEligiblePullRun(run, repository) ||
|
||||
!isEligibleMainRun(mainRun, defaultBranch, repository),
|
||||
"run_event_or_repository_identity_unverified",
|
||||
],
|
||||
() => [
|
||||
job?.name !== mainJob?.name || job?.name !== "Static checks",
|
||||
"failed_job_not_the_same_static_gate",
|
||||
],
|
||||
() => [
|
||||
!Array.isArray(diagnostics) ||
|
||||
diagnostics.length === 0 ||
|
||||
!Array.isArray(mainDiagnostics),
|
||||
"structured_diagnostics_missing_or_invalid",
|
||||
],
|
||||
() => [
|
||||
!validContext(context) || !validContext(mainContext),
|
||||
"static_task_context_missing_or_invalid",
|
||||
],
|
||||
() => [
|
||||
executionSourcesVerified !== true,
|
||||
"checkout_sources_not_verified_against_api_commits",
|
||||
],
|
||||
() => [
|
||||
!sameExecutionContext(context, mainContext),
|
||||
"task_or_toolchain_identity_differs",
|
||||
],
|
||||
() => [
|
||||
ancestry?.status !== "ahead" && ancestry?.status !== "identical",
|
||||
"main_head_not_proven_ancestor_of_tested_base",
|
||||
],
|
||||
() => [
|
||||
ancestry?.mergeBaseSha !== mainRun?.head_sha,
|
||||
"main_head_is_not_the_verified_merge_base",
|
||||
],
|
||||
() => [
|
||||
!sameTaskHashes(diagnostics, context, mainContext),
|
||||
"owning_task_hash_differs",
|
||||
],
|
||||
]);
|
||||
if (failedGate) return outcome("unknown", failedGate.reason);
|
||||
const keys = diagnostics.map(diagnosticKey);
|
||||
const mainKeys = new Set(mainDiagnostics.map(diagnosticKey).filter(Boolean));
|
||||
if (keys.some((key) => key === null))
|
||||
return outcome("unknown", "diagnostic_shape_not_supported");
|
||||
if (!keys.every((key) => mainKeys.has(key)))
|
||||
return outcome(
|
||||
"unknown",
|
||||
"main_run_does_not_contain_the_same_structured_diagnostic",
|
||||
);
|
||||
const blobStatus = blobIdentityStatus(diagnostics, blobs);
|
||||
if (blobStatus !== "identical")
|
||||
return outcome(
|
||||
"unknown",
|
||||
blobStatus === "different"
|
||||
? "diagnostic_source_content_not_identical_at_main_and_tested_base"
|
||||
: "diagnostic_source_blob_identity_unavailable",
|
||||
);
|
||||
return outcome(
|
||||
"matching-main-diagnostics",
|
||||
"supported_diagnostics_match_failed_task_and_source_on_verified_main_ancestor",
|
||||
);
|
||||
}
|
||||
|
||||
export function diagnosticPaths(item) {
|
||||
return item.kind === "jscpd" ? item.paths : [item.path];
|
||||
}
|
||||
export function isFailedConclusion(value) {
|
||||
return value === "failure" || value === "timed_out";
|
||||
}
|
||||
export function isEligibleMainRun(run, defaultBranch, repository) {
|
||||
return (
|
||||
isEligibleStaticRun(run) &&
|
||||
run.repository.full_name === repository &&
|
||||
run.head_branch === defaultBranch &&
|
||||
["push", "schedule", "workflow_dispatch"].includes(run.event)
|
||||
);
|
||||
}
|
||||
export function pullRequestBaseSha(run, repository) {
|
||||
if (
|
||||
run?.event !== "pull_request" ||
|
||||
run.repository?.full_name !== repository ||
|
||||
!Array.isArray(run.pull_requests) ||
|
||||
run.pull_requests.length !== 1
|
||||
)
|
||||
return null;
|
||||
const base = run.pull_requests[0].base;
|
||||
return base?.repo?.full_name === repository && SHA.test(base?.sha ?? "")
|
||||
? base.sha
|
||||
: null;
|
||||
}
|
||||
|
||||
export function safeSha(value) {
|
||||
return typeof value === "string" && SHA.test(value)
|
||||
? value.toLowerCase()
|
||||
: null;
|
||||
}
|
||||
export function selectStaticCheckArtifact(page, run, runAttempt) {
|
||||
if (runAttempt !== 1) return null;
|
||||
if (
|
||||
!Array.isArray(page?.artifacts) ||
|
||||
!Number.isSafeInteger(page.total_count) ||
|
||||
page.total_count !== page.artifacts.length ||
|
||||
page.artifacts.length >= 100
|
||||
)
|
||||
return null;
|
||||
const matches = page.artifacts.filter(
|
||||
(artifact) => artifact.name === "static-check-reports",
|
||||
);
|
||||
if (matches.length !== 1) return null;
|
||||
const [artifact] = matches;
|
||||
if (
|
||||
artifact.expired !== false ||
|
||||
String(artifact.workflow_run?.id) !== String(run.id) ||
|
||||
artifact.workflow_run?.head_sha !== run.head_sha
|
||||
)
|
||||
return null;
|
||||
return artifact;
|
||||
}
|
||||
export function validRepoPath(value) {
|
||||
return (
|
||||
typeof value === "string" &&
|
||||
value.length > 0 &&
|
||||
!isAbsolute(value) &&
|
||||
!value.includes("\\") &&
|
||||
value
|
||||
.split("/")
|
||||
.every((part) => part !== "" && part !== "." && part !== "..") &&
|
||||
!value.startsWith("/")
|
||||
);
|
||||
}
|
||||
|
||||
/** Download one named static report archive and parse only whitelisted JSON members. */
|
||||
export async function readStaticCheckArtifact({
|
||||
repository,
|
||||
artifactId,
|
||||
token,
|
||||
fetchImpl = fetch,
|
||||
}) {
|
||||
if (!/^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+$/u.test(repository ?? ""))
|
||||
throw new Error("artifact repository identity is invalid");
|
||||
if (!token) throw new Error("artifact download requires a GitHub token");
|
||||
const url = `${API_ROOT}/${repository}/actions/artifacts/${positiveId(artifactId)}/zip`;
|
||||
let response = await fetchImpl(url, {
|
||||
headers: {
|
||||
Accept: "application/vnd.github+json",
|
||||
Authorization: `Bearer ${token}`,
|
||||
"X-GitHub-Api-Version": "2022-11-28",
|
||||
},
|
||||
signal: AbortSignal.timeout(10_000),
|
||||
redirect: "manual",
|
||||
});
|
||||
if ([301, 302, 303, 307, 308].includes(response.status)) {
|
||||
const location = response.headers.get("location");
|
||||
if (!location || new URL(location).protocol !== "https:")
|
||||
throw new Error("static_artifact_redirect_invalid");
|
||||
response = await fetchImpl(location, {
|
||||
signal: AbortSignal.timeout(10_000),
|
||||
redirect: "error",
|
||||
});
|
||||
}
|
||||
if (!response.ok) throw new Error(`github_http_${response.status}`);
|
||||
const bytes = await readLimitedBytes(response, MAX_ARCHIVE_BYTES);
|
||||
const directory = await mkdtemp(join(tmpdir(), "rea-ci-static-artifact-"));
|
||||
const archive = join(directory, "artifact.zip");
|
||||
try {
|
||||
await writeFile(archive, bytes, { flag: "wx" });
|
||||
return await parseStaticArchive(archive);
|
||||
} finally {
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
/** Accept only the three named producer outputs; no archive paths are written to disk. */
|
||||
export async function parseStaticArchive(archivePath) {
|
||||
const { stdout: listing } = await execute("unzip", ["-Z1", archivePath], {
|
||||
encoding: "utf8",
|
||||
maxBuffer: 128 * 1024,
|
||||
timeout: 5_000,
|
||||
});
|
||||
const allowed = new Set([
|
||||
"oxlint.json",
|
||||
"jscpd/jscpd-report.json",
|
||||
"static-task-context.json",
|
||||
]);
|
||||
const members = new Map();
|
||||
for (const name of listing.split(/\r?\n/u).filter(Boolean)) {
|
||||
const prefix = ".cache/static-checks/";
|
||||
const member = name.startsWith(prefix) ? name.slice(prefix.length) : name;
|
||||
if (!allowed.has(member)) continue;
|
||||
if (members.has(member))
|
||||
throw new Error("static artifact contains duplicate report members");
|
||||
const { stdout } = await execute("unzip", ["-p", archivePath, name], {
|
||||
encoding: "buffer",
|
||||
maxBuffer: MAX_MEMBER_BYTES,
|
||||
timeout: 5_000,
|
||||
});
|
||||
members.set(member, JSON.parse(stdout.toString("utf8")));
|
||||
}
|
||||
const context = parseStaticTaskContext(
|
||||
members.get("static-task-context.json"),
|
||||
);
|
||||
const diagnostics = normalizeDiagnostics(
|
||||
members.get("oxlint.json"),
|
||||
members.get("jscpd/jscpd-report.json"),
|
||||
);
|
||||
if (diagnostics.length > MAX_DIAGNOSTICS)
|
||||
throw new Error("static diagnostic count exceeds the reporting limit");
|
||||
return { context, diagnostics };
|
||||
}
|
||||
|
||||
function normalizeDiagnostics(lint, duplicate) {
|
||||
return [...normalizeLint(lint), ...normalizeDuplicates(duplicate)];
|
||||
}
|
||||
|
||||
function normalizeLint(report) {
|
||||
if (report === undefined) return [];
|
||||
if (!Array.isArray(report?.diagnostics))
|
||||
throw new Error("oxlint diagnostics are malformed");
|
||||
return report.diagnostics
|
||||
.filter((item) => item.severity === "error")
|
||||
.map((item) => {
|
||||
if (
|
||||
typeof item.filename !== "string" ||
|
||||
!repoPath(item.filename) ||
|
||||
typeof item.code !== "string" ||
|
||||
typeof item.message !== "string"
|
||||
)
|
||||
throw new Error("oxlint error diagnostic is malformed");
|
||||
return {
|
||||
kind: "lint",
|
||||
path: item.filename,
|
||||
code: item.code,
|
||||
message: item.message,
|
||||
severity: "error",
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
function normalizeDuplicates(report) {
|
||||
if (report === undefined) return [];
|
||||
if (
|
||||
!Array.isArray(report?.duplicates) ||
|
||||
!Number.isFinite(report.statistics?.total?.percentage) ||
|
||||
report.statistics.total.percentage < 0
|
||||
)
|
||||
throw new Error("jscpd report is malformed");
|
||||
// The producer owns the checker outcome; do not copy or infer its threshold.
|
||||
const exitCode = report.reaCheck?.exitCode;
|
||||
if (exitCode === undefined || exitCode === 0) return [];
|
||||
if (!Number.isSafeInteger(exitCode) || exitCode < 0)
|
||||
throw new Error("jscpd checker outcome is malformed");
|
||||
if (exitCode !== 1) return [];
|
||||
return report.duplicates.map((item) => {
|
||||
const paths = [item.firstFile?.name, item.secondFile?.name].sort();
|
||||
if (paths.some((path) => !repoPath(path)))
|
||||
throw new Error("jscpd duplicate paths are malformed");
|
||||
return { kind: "jscpd", paths };
|
||||
});
|
||||
}
|
||||
|
||||
async function readLimitedBytes(response, limit) {
|
||||
const reader = response.body?.getReader();
|
||||
if (!reader) throw new Error("static artifact response has no body");
|
||||
const chunks = [];
|
||||
let size = 0;
|
||||
try {
|
||||
for (;;) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
size += value.byteLength;
|
||||
if (size > limit) {
|
||||
await reader.cancel();
|
||||
throw new Error("static artifact exceeds the download limit");
|
||||
}
|
||||
chunks.push(value);
|
||||
}
|
||||
} finally {
|
||||
reader.releaseLock();
|
||||
}
|
||||
const result = new Uint8Array(size);
|
||||
let offset = 0;
|
||||
for (const chunk of chunks) {
|
||||
result.set(chunk, offset);
|
||||
offset += chunk.byteLength;
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
function repoPath(value) {
|
||||
return (
|
||||
typeof value === "string" &&
|
||||
value.length > 0 &&
|
||||
!value.startsWith("/") &&
|
||||
!value.includes("\\") &&
|
||||
value
|
||||
.split("/")
|
||||
.every((part) => part !== "" && part !== "." && part !== "..")
|
||||
);
|
||||
}
|
||||
|
||||
function positiveId(value) {
|
||||
const id = Number(value);
|
||||
if (!Number.isSafeInteger(id) || id < 1)
|
||||
throw new Error("artifact ID must be a positive integer");
|
||||
return String(id);
|
||||
}
|
||||
|
||||
function diagnosticKey(item) {
|
||||
if (
|
||||
item?.kind === "lint" &&
|
||||
validRepoPath(item.path) &&
|
||||
typeof item.code === "string" &&
|
||||
typeof item.message === "string" &&
|
||||
item.severity === "error"
|
||||
)
|
||||
return JSON.stringify([
|
||||
item.kind,
|
||||
item.path,
|
||||
item.code,
|
||||
item.message,
|
||||
item.severity,
|
||||
]);
|
||||
if (
|
||||
item?.kind === "jscpd" &&
|
||||
Array.isArray(item.paths) &&
|
||||
item.paths.length === 2 &&
|
||||
item.paths.every(validRepoPath)
|
||||
)
|
||||
return JSON.stringify([item.kind, ...[...item.paths].sort()]);
|
||||
return null;
|
||||
}
|
||||
function blobIdentityStatus(diagnostics, blobs) {
|
||||
const paths = [...new Set(diagnostics.flatMap(diagnosticPaths))];
|
||||
if (paths.some((path) => blobs?.[path] === false)) return "different";
|
||||
return paths.length > 0 && paths.every((path) => blobs?.[path] === true)
|
||||
? "identical"
|
||||
: "unavailable";
|
||||
}
|
||||
function sameTaskHashes(diagnostics, left, right) {
|
||||
return diagnostics.every((diagnostic) => {
|
||||
const task = diagnostic.kind === "jscpd" ? "jscpd" : "lint:built";
|
||||
return (
|
||||
TASK_HASH.test(left.taskHashes[task] ?? "") &&
|
||||
left.taskHashes[task] === right.taskHashes[task]
|
||||
);
|
||||
});
|
||||
}
|
||||
function validContext(value) {
|
||||
return (
|
||||
value?.version === 1 &&
|
||||
SHA.test(value.sourceSha ?? "") &&
|
||||
value.sourceClean === true &&
|
||||
typeof value.nodeVersion === "string" &&
|
||||
typeof value.npmVersion === "string" &&
|
||||
typeof value.turboVersion === "string" &&
|
||||
typeof value.platform === "string" &&
|
||||
typeof value.architecture === "string" &&
|
||||
value.taskHashes &&
|
||||
typeof value.taskHashes === "object" &&
|
||||
!Array.isArray(value.taskHashes)
|
||||
);
|
||||
}
|
||||
function sameExecutionContext(left, right) {
|
||||
return [
|
||||
"nodeVersion",
|
||||
"npmVersion",
|
||||
"turboVersion",
|
||||
"platform",
|
||||
"architecture",
|
||||
].every((key) => left[key] === right[key]);
|
||||
}
|
||||
function isEligiblePullRun(run, repository) {
|
||||
return (
|
||||
isEligibleStaticRun(run) && pullRequestBaseSha(run, repository) !== null
|
||||
);
|
||||
}
|
||||
function isEligibleStaticRun(run) {
|
||||
return (
|
||||
isRecord(run) &&
|
||||
run.status === "completed" &&
|
||||
isFailedConclusion(run.conclusion) &&
|
||||
SHA.test(run.head_sha ?? "") &&
|
||||
typeof run.repository?.full_name === "string" &&
|
||||
run.workflowPath?.file === ".github/workflows/ci.yml"
|
||||
);
|
||||
}
|
||||
function isCompletedFailure(run) {
|
||||
return run?.status === "completed" && isFailedConclusion(run.conclusion);
|
||||
}
|
||||
function isRecord(value) {
|
||||
return typeof value === "object" && value !== null && !Array.isArray(value);
|
||||
}
|
||||
function firstFailure(checks) {
|
||||
for (const check of checks) {
|
||||
const [failed, reason] = check();
|
||||
if (failed) return { reason };
|
||||
}
|
||||
return null;
|
||||
}
|
||||
function outcome(classification, reason) {
|
||||
return { classification, reason };
|
||||
}
|
||||
@@ -0,0 +1,159 @@
|
||||
import type { NormalizedWorkflowRun } from "./workflow-run.mjs";
|
||||
import type { StaticDiagnostic } from "./artifacts.mjs";
|
||||
export type { StaticDiagnostic } from "./artifacts.mjs";
|
||||
import type { StaticTaskContext } from "./static-task-context.mjs";
|
||||
export type { StaticTaskContext } from "./static-task-context.mjs";
|
||||
export interface CiRun {
|
||||
readonly id?: string | number;
|
||||
readonly run_attempt?: number;
|
||||
readonly status?: string;
|
||||
readonly conclusion?: string | null;
|
||||
readonly event?: string;
|
||||
readonly path?: string;
|
||||
readonly head_sha?: string;
|
||||
readonly head_branch?: string;
|
||||
readonly repository?: { readonly full_name?: string };
|
||||
readonly pull_requests?: readonly {
|
||||
readonly base?: {
|
||||
readonly sha?: string;
|
||||
readonly repo?: { readonly full_name?: string };
|
||||
};
|
||||
}[];
|
||||
}
|
||||
|
||||
export interface CiJob {
|
||||
readonly name?: string;
|
||||
readonly status?: string;
|
||||
readonly conclusion?: string | null;
|
||||
readonly run_id?: number;
|
||||
readonly run_attempt?: number;
|
||||
}
|
||||
|
||||
export interface StaticEvidence {
|
||||
readonly context: StaticTaskContext;
|
||||
readonly diagnostics: readonly StaticDiagnostic[];
|
||||
}
|
||||
|
||||
export interface CiFailureReport {
|
||||
readonly version: 1;
|
||||
readonly observedAt: string;
|
||||
readonly repository: string;
|
||||
readonly defaultBranch: string;
|
||||
readonly informationalOnly: true;
|
||||
readonly gateEffect: string;
|
||||
readonly boundedInput: {
|
||||
readonly candidateRuns: number;
|
||||
readonly limit: number;
|
||||
readonly mainCandidateLimit: number;
|
||||
readonly mainCandidateRuns: number;
|
||||
readonly mainRunLookupLimit: number;
|
||||
readonly manifestTruncated: boolean;
|
||||
};
|
||||
readonly counts: Readonly<
|
||||
Record<"matching-main-diagnostics" | "unknown", number>
|
||||
>;
|
||||
readonly failures: readonly {
|
||||
readonly runId: number | string;
|
||||
readonly runAttempt: number;
|
||||
readonly classification: "matching-main-diagnostics" | "unknown";
|
||||
readonly reason: string;
|
||||
readonly candidateFailures?: readonly {
|
||||
readonly runId: number | string;
|
||||
readonly runAttempt: number;
|
||||
readonly reason: string;
|
||||
}[];
|
||||
readonly runHead?: string | null;
|
||||
readonly event?: string | null;
|
||||
readonly job?: string;
|
||||
readonly conclusion?: string | null;
|
||||
}[];
|
||||
}
|
||||
|
||||
export interface FailureReportApi {
|
||||
getRun(repository: string, id: string | number): Promise<CiRun>;
|
||||
getJobs(
|
||||
repository: string,
|
||||
id: string | number,
|
||||
attempt: number,
|
||||
): Promise<readonly CiJob[]>;
|
||||
getStaticEvidence(
|
||||
repository: string,
|
||||
run: CiRun,
|
||||
job: CiJob,
|
||||
): Promise<StaticEvidence | null>;
|
||||
getAncestry(
|
||||
repository: string,
|
||||
ancestor: string,
|
||||
descendant: string,
|
||||
): Promise<{ status: string; mergeBaseSha: string | null }>;
|
||||
verifyExecutionSource(
|
||||
repository: string,
|
||||
run: CiRun,
|
||||
context: StaticTaskContext,
|
||||
): Promise<boolean>;
|
||||
getBlobIdentity(
|
||||
repository: string,
|
||||
ancestor: string,
|
||||
descendant: string,
|
||||
paths: readonly string[],
|
||||
): Promise<Readonly<Record<string, boolean>>>;
|
||||
}
|
||||
|
||||
export function classifyFailure(input: {
|
||||
repository: string;
|
||||
defaultBranch: string;
|
||||
run: CiRun & NormalizedWorkflowRun;
|
||||
job: CiJob;
|
||||
diagnostics: readonly StaticDiagnostic[];
|
||||
context: StaticTaskContext;
|
||||
mainRun: CiRun & NormalizedWorkflowRun;
|
||||
mainJob: CiJob;
|
||||
mainDiagnostics: readonly StaticDiagnostic[];
|
||||
mainContext: StaticTaskContext;
|
||||
executionSourcesVerified: boolean;
|
||||
ancestry: { status: string; mergeBaseSha: string | null };
|
||||
blobs: Readonly<Record<string, boolean>>;
|
||||
}): {
|
||||
classification: "matching-main-diagnostics" | "unknown";
|
||||
reason: string;
|
||||
};
|
||||
|
||||
export function reportCiFailures(options: {
|
||||
repository: string;
|
||||
defaultBranch: string;
|
||||
batchManifest: {
|
||||
readonly repository: string;
|
||||
readonly runs: readonly {
|
||||
readonly id: number | string;
|
||||
readonly runAttempt: number;
|
||||
readonly workflow: string;
|
||||
readonly event?: string | null;
|
||||
readonly headBranch?: string | null;
|
||||
readonly conclusion: string | null;
|
||||
readonly status: string;
|
||||
}[];
|
||||
readonly truncation?: readonly unknown[];
|
||||
readonly coverage?: { readonly exhaustive?: boolean };
|
||||
readonly selection?: { readonly exhaustive?: boolean };
|
||||
};
|
||||
api: FailureReportApi;
|
||||
outputDirectory?: string;
|
||||
summaryPath?: string;
|
||||
observedAt?: string;
|
||||
}): Promise<CiFailureReport>;
|
||||
|
||||
export function createGitHubApi(options?: {
|
||||
token?: string;
|
||||
fetchImpl?: typeof fetch;
|
||||
apiRequest?: (
|
||||
method: string,
|
||||
endpoint: string,
|
||||
budget: { requests: number; startedAt: number; limitMs: number },
|
||||
) => Promise<unknown>;
|
||||
}): FailureReportApi;
|
||||
|
||||
export function writeFailureReport(
|
||||
report: CiFailureReport,
|
||||
outputDirectory?: string,
|
||||
summaryPath?: string,
|
||||
): Promise<{ jsonPath: string; markdownPath: string }>;
|
||||
@@ -0,0 +1,533 @@
|
||||
import { appendFile, mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import { join, resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
import {
|
||||
classifyFailure,
|
||||
diagnosticPaths,
|
||||
isEligibleMainRun,
|
||||
isFailedConclusion,
|
||||
pullRequestBaseSha,
|
||||
readStaticCheckArtifact,
|
||||
safeSha,
|
||||
selectStaticCheckArtifact,
|
||||
validRepoPath,
|
||||
} from "./artifacts.mjs";
|
||||
import { createGhApiRequest } from "./github-api.mjs";
|
||||
import { normalizeWorkflowRun } from "./workflow-run.mjs";
|
||||
|
||||
const OUTPUT_DIRECTORY = join(".cache", "ci-failures");
|
||||
const BATCH_DIRECTORY = join(".cache", "ci-job-timings");
|
||||
const MAX_JSON_BYTES = 8 * 1024 * 1024;
|
||||
const MAX_FAILED_RUNS = 20;
|
||||
const MAX_MAIN_CANDIDATES = 5;
|
||||
const MAX_MAIN_RUN_LOOKUPS = 100;
|
||||
const MAX_PATHS_PER_COMPARISON = 50;
|
||||
const MAX_API_REQUESTS = 250;
|
||||
|
||||
export { classifyFailure } from "./artifacts.mjs";
|
||||
|
||||
/** Uses only failed CI candidates present in the supplied bounded timing manifest. */
|
||||
export async function reportCiFailures({
|
||||
repository,
|
||||
defaultBranch,
|
||||
batchManifest,
|
||||
api,
|
||||
outputDirectory = OUTPUT_DIRECTORY,
|
||||
summaryPath,
|
||||
observedAt = new Date().toISOString(),
|
||||
}) {
|
||||
validateRepository(repository);
|
||||
if (typeof defaultBranch !== "string" || defaultBranch.length === 0)
|
||||
throw new Error("defaultBranch is required");
|
||||
if (
|
||||
!batchManifest ||
|
||||
batchManifest.repository !== repository ||
|
||||
!Array.isArray(batchManifest.runs)
|
||||
)
|
||||
throw new Error("timing manifest repository or runs are invalid");
|
||||
|
||||
const failedRuns = batchManifest.runs.filter(
|
||||
(entry) =>
|
||||
entry.workflow === "CI" &&
|
||||
entry.status === "collected" &&
|
||||
isFailedConclusion(entry.conclusion),
|
||||
);
|
||||
const prRuns = failedRuns.filter((entry) => entry.event === "pull_request");
|
||||
const candidates = prRuns.slice(0, MAX_FAILED_RUNS);
|
||||
const mainEntries = failedRuns.filter(
|
||||
(entry) =>
|
||||
entry.headBranch === defaultBranch &&
|
||||
["push", "schedule", "workflow_dispatch"].includes(entry.event),
|
||||
);
|
||||
const lookupRun = verifiedRunLookup(repository, api);
|
||||
const mainCandidates = [];
|
||||
for (const entry of mainEntries.slice(0, MAX_MAIN_RUN_LOOKUPS)) {
|
||||
try {
|
||||
const run = await lookupRun(entry);
|
||||
if (isEligibleMainRun(run, defaultBranch, repository))
|
||||
mainCandidates.push({
|
||||
run,
|
||||
getEvidence: mainEvidenceLookup(repository, api, entry, run),
|
||||
});
|
||||
if (mainCandidates.length === MAX_MAIN_CANDIDATES) break;
|
||||
} catch {
|
||||
// Unverified metadata cannot consume the eligible-main candidate budget.
|
||||
}
|
||||
}
|
||||
const entries = [];
|
||||
for (const entry of candidates) {
|
||||
try {
|
||||
entries.push(
|
||||
...(await analyzeFailedRun({
|
||||
repository,
|
||||
defaultBranch,
|
||||
entry,
|
||||
run: await lookupRun(entry),
|
||||
mainCandidates,
|
||||
api,
|
||||
})),
|
||||
);
|
||||
} catch (error) {
|
||||
entries.push({
|
||||
runId: entry.id,
|
||||
runAttempt: entry.runAttempt,
|
||||
classification: "unknown",
|
||||
reason: errorCode(error),
|
||||
});
|
||||
}
|
||||
}
|
||||
const report = {
|
||||
version: 1,
|
||||
observedAt,
|
||||
repository,
|
||||
defaultBranch,
|
||||
informationalOnly: true,
|
||||
gateEffect: "none; this report never changes CI conclusions",
|
||||
boundedInput: {
|
||||
candidateRuns: candidates.length,
|
||||
limit: MAX_FAILED_RUNS,
|
||||
mainCandidateLimit: MAX_MAIN_CANDIDATES,
|
||||
mainCandidateRuns: mainCandidates.length,
|
||||
mainRunLookupLimit: MAX_MAIN_RUN_LOOKUPS,
|
||||
manifestTruncated:
|
||||
prRuns.length > MAX_FAILED_RUNS ||
|
||||
batchManifest.truncation?.length > 0 ||
|
||||
batchManifest.selection?.exhaustive === false ||
|
||||
batchManifest.coverage?.exhaustive === false,
|
||||
},
|
||||
counts: Object.fromEntries(
|
||||
["matching-main-diagnostics", "unknown"].map((category) => [
|
||||
category,
|
||||
entries.filter((item) => item.classification === category).length,
|
||||
]),
|
||||
),
|
||||
failures: entries,
|
||||
};
|
||||
await writeFailureReport(report, outputDirectory, summaryPath);
|
||||
return report;
|
||||
}
|
||||
|
||||
async function analyzeFailedRun({
|
||||
repository,
|
||||
defaultBranch,
|
||||
entry,
|
||||
run,
|
||||
mainCandidates,
|
||||
api,
|
||||
}) {
|
||||
const jobs = await api.getJobs(repository, entry.id, entry.runAttempt);
|
||||
if (jobs.some((job) => !hasJobIdentity(job, entry)))
|
||||
throw new Error("workflow_job_identity_mismatch");
|
||||
const failedJobs = jobs.filter(
|
||||
(job) => job.status === "completed" && isFailedConclusion(job.conclusion),
|
||||
);
|
||||
if (failedJobs.length === 0)
|
||||
return [
|
||||
{
|
||||
runId: entry.id,
|
||||
runAttempt: entry.runAttempt,
|
||||
classification: "unknown",
|
||||
reason: "failed_jobs_not_present_in_api_snapshot",
|
||||
},
|
||||
];
|
||||
const failures = [];
|
||||
for (const job of failedJobs) {
|
||||
let classification;
|
||||
try {
|
||||
const evidence = await api.getStaticEvidence(repository, run, job);
|
||||
classification = await attributeToMain({
|
||||
repository,
|
||||
defaultBranch,
|
||||
run,
|
||||
job,
|
||||
evidence,
|
||||
mainCandidates,
|
||||
api,
|
||||
});
|
||||
} catch (error) {
|
||||
classification = outcome("unknown", errorCode(error));
|
||||
}
|
||||
failures.push({
|
||||
runId: entry.id,
|
||||
runAttempt: entry.runAttempt,
|
||||
runHead: safeSha(run.head_sha),
|
||||
event: run.event ?? null,
|
||||
job: job.name,
|
||||
conclusion: job.conclusion,
|
||||
...classification,
|
||||
});
|
||||
}
|
||||
return failures;
|
||||
}
|
||||
|
||||
async function attributeToMain({
|
||||
repository,
|
||||
defaultBranch,
|
||||
run,
|
||||
job,
|
||||
evidence,
|
||||
mainCandidates,
|
||||
api,
|
||||
}) {
|
||||
const baseSha = pullRequestBaseSha(run, repository);
|
||||
if (baseSha === null || evidence === null)
|
||||
return outcome("unknown", "no_matching_verified_main_failure");
|
||||
let result = outcome("unknown", "no_matching_verified_main_failure");
|
||||
const candidateFailures = [];
|
||||
for (const candidate of mainCandidates) {
|
||||
try {
|
||||
const main = await compareCandidate({
|
||||
repository,
|
||||
defaultBranch,
|
||||
run,
|
||||
job,
|
||||
evidence,
|
||||
baseSha,
|
||||
candidate,
|
||||
api,
|
||||
});
|
||||
if (main !== null) result = main;
|
||||
if (main?.classification === "matching-main-diagnostics") break;
|
||||
} catch (error) {
|
||||
candidateFailures.push({
|
||||
runId: candidate.run.id,
|
||||
runAttempt: candidate.run.run_attempt,
|
||||
reason: errorCode(error),
|
||||
});
|
||||
}
|
||||
}
|
||||
return {
|
||||
...result,
|
||||
...(candidateFailures.length > 0 ? { candidateFailures } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
async function compareCandidate({
|
||||
repository,
|
||||
defaultBranch,
|
||||
run,
|
||||
job,
|
||||
evidence,
|
||||
baseSha,
|
||||
candidate,
|
||||
api,
|
||||
}) {
|
||||
const mainRun = candidate.run;
|
||||
const main = await candidate.getEvidence(job.name);
|
||||
if (!main?.evidence) return null;
|
||||
const { job: mainJob, evidence: mainEvidence } = main;
|
||||
const ancestry = await api.getAncestry(repository, mainRun.head_sha, baseSha);
|
||||
const paths = [
|
||||
...new Set(evidence.diagnostics.flatMap(diagnosticPaths)),
|
||||
].slice(0, MAX_PATHS_PER_COMPARISON);
|
||||
const blobs = await api.getBlobIdentity(
|
||||
repository,
|
||||
mainRun.head_sha,
|
||||
baseSha,
|
||||
paths,
|
||||
);
|
||||
const executionSourcesVerified =
|
||||
(await api.verifyExecutionSource(repository, run, evidence.context)) &&
|
||||
(await api.verifyExecutionSource(
|
||||
repository,
|
||||
mainRun,
|
||||
mainEvidence.context,
|
||||
));
|
||||
return classifyFailure({
|
||||
repository,
|
||||
defaultBranch,
|
||||
run,
|
||||
job,
|
||||
diagnostics: evidence.diagnostics,
|
||||
context: evidence.context,
|
||||
mainRun,
|
||||
mainJob,
|
||||
mainDiagnostics: mainEvidence.diagnostics,
|
||||
mainContext: mainEvidence.context,
|
||||
executionSourcesVerified,
|
||||
ancestry,
|
||||
blobs,
|
||||
});
|
||||
}
|
||||
|
||||
// Each verified main run/attempt owns its lazy jobs and parsed evidence for this
|
||||
// report only. Cache nulls and rejected reads too; retries cannot spend the budget
|
||||
// repeatedly on an immutable observation that was unavailable or invalid.
|
||||
function mainEvidenceLookup(repository, api, entry, run) {
|
||||
let jobs;
|
||||
const evidence = new Map();
|
||||
return (name) => {
|
||||
jobs ??= Promise.resolve().then(async () => {
|
||||
const items = await api.getJobs(repository, entry.id, entry.runAttempt);
|
||||
return items.some((item) => !hasJobIdentity(item, entry)) ? [] : items;
|
||||
});
|
||||
if (!evidence.has(name))
|
||||
evidence.set(
|
||||
name,
|
||||
jobs.then(async (items) => {
|
||||
const job = items.find(
|
||||
(item) =>
|
||||
item.name === name &&
|
||||
item.status === "completed" &&
|
||||
isFailedConclusion(item.conclusion),
|
||||
);
|
||||
if (!job) return null;
|
||||
return {
|
||||
job,
|
||||
evidence: await api.getStaticEvidence(repository, run, job),
|
||||
};
|
||||
}),
|
||||
);
|
||||
return evidence.get(name);
|
||||
};
|
||||
}
|
||||
|
||||
// Selection and attribution share one immutable API observation per run/attempt.
|
||||
function verifiedRunLookup(repository, api) {
|
||||
const runs = new Map();
|
||||
return (entry) => {
|
||||
const key = `${positiveId(entry.id)}:${positiveId(entry.runAttempt)}`;
|
||||
if (!runs.has(key))
|
||||
runs.set(
|
||||
key,
|
||||
Promise.resolve().then(async () => {
|
||||
const run = normalizeWorkflowRun(
|
||||
await api.getRun(repository, entry.id),
|
||||
);
|
||||
if (!hasRunIdentity(run, entry))
|
||||
throw new Error("workflow_run_identity_mismatch");
|
||||
return run;
|
||||
}),
|
||||
);
|
||||
return runs.get(key);
|
||||
};
|
||||
}
|
||||
|
||||
function hasRunIdentity(run, entry) {
|
||||
return (
|
||||
String(run?.id) === String(entry.id) &&
|
||||
Number(run.run_attempt) === Number(entry.runAttempt) &&
|
||||
run.workflowPath?.file === ".github/workflows/ci.yml"
|
||||
);
|
||||
}
|
||||
function hasJobIdentity(job, entry) {
|
||||
return (
|
||||
String(job.run_id) === String(entry.id) &&
|
||||
Number(job.run_attempt) === Number(entry.runAttempt)
|
||||
);
|
||||
}
|
||||
|
||||
export function createGitHubApi({
|
||||
token = process.env.GH_TOKEN,
|
||||
fetchImpl = fetch,
|
||||
apiRequest = createGhApiRequest(),
|
||||
} = {}) {
|
||||
if (!token) throw new Error("GH_TOKEN is required");
|
||||
const budget = { requests: 0, startedAt: Date.now(), limitMs: 60_000 };
|
||||
const request = async (endpoint) => {
|
||||
if (budget.requests >= MAX_API_REQUESTS)
|
||||
throw new Error("github_api_request_budget_exhausted");
|
||||
budget.requests += 1;
|
||||
return await apiRequest("GET", endpoint, budget);
|
||||
};
|
||||
return {
|
||||
async getRun(repository, id) {
|
||||
return await request(
|
||||
`repos/${repository}/actions/runs/${positiveId(id)}`,
|
||||
);
|
||||
},
|
||||
async getJobs(repository, id, attempt) {
|
||||
const page = await request(
|
||||
`repos/${repository}/actions/runs/${positiveId(id)}/attempts/${positiveId(attempt)}/jobs?per_page=100&page=1`,
|
||||
);
|
||||
if (!Array.isArray(page.jobs) || page.jobs.length >= 100)
|
||||
throw new Error("job_list_incomplete_or_malformed");
|
||||
return page.jobs;
|
||||
},
|
||||
async getStaticEvidence(repository, run, job) {
|
||||
if (job.name !== "Static checks") return null;
|
||||
if (run.run_attempt !== 1) return null;
|
||||
const page = await request(
|
||||
`repos/${repository}/actions/runs/${positiveId(run.id)}/artifacts?per_page=100&page=1`,
|
||||
);
|
||||
const artifact = selectStaticCheckArtifact(page, run, run.run_attempt);
|
||||
if (!artifact) return null;
|
||||
return await readStaticCheckArtifact({
|
||||
repository,
|
||||
artifactId: artifact.id,
|
||||
token,
|
||||
fetchImpl,
|
||||
});
|
||||
},
|
||||
async getAncestry(repository, ancestor, descendant) {
|
||||
const comparison = await request(
|
||||
`repos/${repository}/compare/${safeSha(ancestor)}...${safeSha(descendant)}`,
|
||||
);
|
||||
return {
|
||||
status: comparison.status,
|
||||
mergeBaseSha: safeSha(comparison.merge_base_commit?.sha),
|
||||
};
|
||||
},
|
||||
async verifyExecutionSource(repository, run, context) {
|
||||
const sourceSha = safeSha(context.sourceSha);
|
||||
if (!sourceSha) return false;
|
||||
const pr =
|
||||
run.event === "pull_request"
|
||||
? pullRequestBaseSha(run, repository)
|
||||
: null;
|
||||
if (
|
||||
run.event === "pull_request" &&
|
||||
(pr === null || context.prHeadSha !== run.head_sha)
|
||||
)
|
||||
return false;
|
||||
const commit = await request(`repos/${repository}/commits/${sourceSha}`);
|
||||
if (safeSha(commit.sha) !== sourceSha) return false;
|
||||
if (sourceSha === run.head_sha) return true;
|
||||
if (
|
||||
pr === null ||
|
||||
!Array.isArray(commit.parents) ||
|
||||
commit.parents.length !== 2
|
||||
)
|
||||
return false;
|
||||
return (
|
||||
safeSha(commit.parents[0]?.sha) === pr &&
|
||||
safeSha(commit.parents[1]?.sha) === run.head_sha
|
||||
);
|
||||
},
|
||||
async getBlobIdentity(repository, ancestor, descendant, paths) {
|
||||
const result = {};
|
||||
for (const path of paths) {
|
||||
if (!validRepoPath(path)) continue;
|
||||
const [left, right] = await Promise.all([
|
||||
getContentBlobSha(repository, ancestor, path, request),
|
||||
getContentBlobSha(repository, descendant, path, request),
|
||||
]);
|
||||
result[path] = left === right;
|
||||
}
|
||||
return result;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function getContentBlobSha(repository, ref, path, request) {
|
||||
const encodedPath = path.split("/").map(encodeURIComponent).join("/");
|
||||
const value = await request(
|
||||
`repos/${repository}/contents/${encodedPath}?ref=${encodeURIComponent(ref)}`,
|
||||
);
|
||||
const sha = safeSha(value?.sha);
|
||||
if (value?.type !== "file" || sha === null)
|
||||
throw new Error("github_content_blob_identity_unavailable");
|
||||
return sha;
|
||||
}
|
||||
|
||||
function outcome(classification, reason) {
|
||||
return { classification, reason };
|
||||
}
|
||||
function validateRepository(value) {
|
||||
if (
|
||||
typeof value !== "string" ||
|
||||
!/^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+$/u.test(value)
|
||||
)
|
||||
throw new Error("repository must be owner/name");
|
||||
}
|
||||
function positiveId(value) {
|
||||
if (!Number.isSafeInteger(Number(value)) || Number(value) < 1)
|
||||
throw new Error(
|
||||
"GitHub run and attempt IDs must be positive safe integers",
|
||||
);
|
||||
return String(Number(value));
|
||||
}
|
||||
function errorCode(error) {
|
||||
return error instanceof Error ? error.message.slice(0, 160) : "unknown_error";
|
||||
}
|
||||
|
||||
export async function writeFailureReport(
|
||||
report,
|
||||
outputDirectory = OUTPUT_DIRECTORY,
|
||||
summaryPath,
|
||||
) {
|
||||
await mkdir(outputDirectory, { recursive: true });
|
||||
const jsonPath = join(outputDirectory, "report.json");
|
||||
const markdownPath = join(outputDirectory, "report.md");
|
||||
const matchedDiagnostics = report.failures.filter(
|
||||
(item) => item.classification === "matching-main-diagnostics",
|
||||
);
|
||||
const lines = [
|
||||
"# CI failure attribution",
|
||||
"",
|
||||
"Informational only. This report never changes a CI conclusion or required gate.",
|
||||
"",
|
||||
`Observed ${report.observedAt}; ${String(matchedDiagnostics.length)} of ${String(report.failures.length)} failure reports contain supported diagnostics matching a failed static task on an ancestor main revision. This does not attribute other failures in the same job. Unknown means evidence was incomplete or did not match.`,
|
||||
"",
|
||||
];
|
||||
for (const item of report.failures)
|
||||
lines.push(
|
||||
`- Run ${item.runId} attempt ${item.runAttempt}${item.job ? `, ${item.job}` : ""}: **${item.classification}** — ${item.reason}.`,
|
||||
);
|
||||
if (report.failures.length === 0)
|
||||
lines.push(
|
||||
"- No completed failed CI attempts were present in the supplied timing snapshot.",
|
||||
);
|
||||
await writeFile(jsonPath, `${JSON.stringify(report, null, 2)}\n`, "utf8");
|
||||
await writeFile(markdownPath, `${lines.join("\n")}\n`, "utf8");
|
||||
if (summaryPath)
|
||||
await appendFile(summaryPath, `${lines.join("\n")}\n`, "utf8");
|
||||
return { jsonPath, markdownPath };
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const repository = process.env.CI_REPOSITORY;
|
||||
const defaultBranch = process.env.CI_DEFAULT_BRANCH;
|
||||
const batchPath =
|
||||
process.env.CI_TIMING_MANIFEST ??
|
||||
join(BATCH_DIRECTORY, "batch-manifest.json");
|
||||
const batchManifest = JSON.parse(await readBoundedFile(batchPath));
|
||||
const api = createGitHubApi();
|
||||
await reportCiFailures({
|
||||
repository,
|
||||
defaultBranch,
|
||||
batchManifest,
|
||||
api,
|
||||
outputDirectory: process.env.CI_FAILURES_DIRECTORY ?? OUTPUT_DIRECTORY,
|
||||
summaryPath: process.env.GITHUB_STEP_SUMMARY,
|
||||
});
|
||||
}
|
||||
async function readBoundedFile(path) {
|
||||
const bytes = await readFile(path);
|
||||
if (bytes.byteLength > MAX_JSON_BYTES)
|
||||
throw new Error("timing manifest exceeds size limit");
|
||||
return bytes.toString("utf8");
|
||||
}
|
||||
|
||||
if (
|
||||
process.argv[1] &&
|
||||
resolve(process.argv[1]) === fileURLToPath(import.meta.url)
|
||||
) {
|
||||
try {
|
||||
await main();
|
||||
} catch (error) {
|
||||
console.error(error instanceof Error ? error.message : String(error));
|
||||
process.exitCode = 1;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
export interface GitHubApiBudget {
|
||||
requests: number;
|
||||
startedAt: number;
|
||||
limitMs: number;
|
||||
}
|
||||
|
||||
export type GitHubApiRequest = (
|
||||
method: string,
|
||||
endpoint: string,
|
||||
budget: GitHubApiBudget,
|
||||
) => Promise<unknown>;
|
||||
|
||||
export function createGhApiRequest(options?: {
|
||||
env?: NodeJS.ProcessEnv;
|
||||
now?: () => number;
|
||||
}): GitHubApiRequest;
|
||||
@@ -0,0 +1,33 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
const MAX_BUFFER = 8 * 1024 * 1024;
|
||||
const REQUEST_TIMEOUT_MS = 10_000;
|
||||
|
||||
/** Create a bounded GitHub CLI JSON transport with GET-only authority. */
|
||||
export function createGhApiRequest({ env = process.env, now = Date.now } = {}) {
|
||||
return async (method, endpoint, budget) => {
|
||||
if (method !== "GET")
|
||||
throw new Error(
|
||||
"GitHub API transport supports read-only GET requests only",
|
||||
);
|
||||
const remaining = budget.limitMs - (now() - budget.startedAt);
|
||||
if (remaining <= 0) throw new Error("lookup_time_budget_exhausted");
|
||||
const { stdout } = await execFileAsync(
|
||||
"gh",
|
||||
["api", "--method", "GET", endpoint],
|
||||
{
|
||||
env,
|
||||
encoding: "utf8",
|
||||
maxBuffer: MAX_BUFFER,
|
||||
timeout: Math.min(REQUEST_TIMEOUT_MS, remaining),
|
||||
},
|
||||
);
|
||||
try {
|
||||
return JSON.parse(stdout);
|
||||
} catch (cause) {
|
||||
throw new Error("github_api_malformed_json", { cause });
|
||||
}
|
||||
};
|
||||
}
|
||||
@@ -103,6 +103,9 @@ export async function collectJobTimingsBatch({
|
||||
id: run.id,
|
||||
runAttempt: run.run_attempt,
|
||||
workflow: workflow.name,
|
||||
event: typeof run.event === "string" ? run.event : null,
|
||||
headBranch:
|
||||
typeof run.head_branch === "string" ? run.head_branch : null,
|
||||
conclusion: run.conclusion ?? null,
|
||||
createdAt: run.created_at ?? null,
|
||||
});
|
||||
|
||||
@@ -62,6 +62,10 @@ function classifyPackage(base, head) {
|
||||
]),
|
||||
].filter((key) => beforeScripts?.[key] !== afterScripts?.[key]);
|
||||
for (const script of changed) {
|
||||
if (/^test:(coverage|mutation):reverse-tools$/.test(script)) {
|
||||
selected.reverse_tools = true;
|
||||
continue;
|
||||
}
|
||||
if (script === "test:sqlite" || script === "test:sqlite:run") {
|
||||
selected.sqlite = true;
|
||||
continue;
|
||||
@@ -162,6 +166,7 @@ if (tier === "reused") {
|
||||
.split("\0")
|
||||
.filter(Boolean);
|
||||
for (const file of files) {
|
||||
if (matchesAny(file, SCOPES.reverse_tools)) selected.reverse_tools = true;
|
||||
if (
|
||||
/^(README(?:_[^/]+)?\.md|AGENTS\.md|CONTRIBUTING\.md|CHANGELOG\.md|SECURITY\.md|native\/windows\/README\.md|\.github\/pull_request_template\.md)$/.test(
|
||||
file,
|
||||
|
||||
@@ -0,0 +1,95 @@
|
||||
export interface PrCiHead {
|
||||
repositoryId: number | null;
|
||||
repository?: string | null;
|
||||
ref: string | null;
|
||||
sha: string | null;
|
||||
}
|
||||
|
||||
export interface PrCiPullRequest {
|
||||
number: number | null;
|
||||
draft?: boolean;
|
||||
identityValid: boolean;
|
||||
mergeable?: boolean;
|
||||
head: PrCiHead;
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
export interface PrCiWorkflowRun {
|
||||
id?: number;
|
||||
run_attempt?: number;
|
||||
event?: string;
|
||||
head_branch?: string;
|
||||
head_repository?: { id?: number | null } | null;
|
||||
head_sha?: string;
|
||||
status?: string;
|
||||
conclusion?: string | null;
|
||||
created_at?: string;
|
||||
html_url?: string;
|
||||
pull_requests?: Array<{
|
||||
number?: number;
|
||||
head?: { sha?: string; repo?: { id?: number | null } } | null;
|
||||
}>;
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
export interface PrCiClassification {
|
||||
category: string;
|
||||
reason: string;
|
||||
runs: Array<Record<string, unknown>>;
|
||||
}
|
||||
|
||||
export interface PrCiHealthReport {
|
||||
schemaVersion: 1;
|
||||
observedAt: string;
|
||||
repository: string;
|
||||
repositoryId: number | null;
|
||||
defaultBranch: string;
|
||||
lookbackDays: number;
|
||||
scan: {
|
||||
status: "complete" | "partial";
|
||||
cutoff: string;
|
||||
pullRequestPages: number;
|
||||
workflowRunPages: number;
|
||||
pullRequestInventoryComplete: boolean;
|
||||
workflowRunInventoryComplete: boolean;
|
||||
pullRequestDetailsFetched: number;
|
||||
pullRequestDetailsLimit: number;
|
||||
apiRequests: number;
|
||||
apiRequestLimit: number;
|
||||
errors: string[];
|
||||
};
|
||||
counts: Record<string, number>;
|
||||
pullRequests: Array<PrCiPullRequest & { ci: PrCiClassification }>;
|
||||
}
|
||||
|
||||
export function collectPrCiHealth(options: {
|
||||
repository: string;
|
||||
repositoryId?: number | string | null;
|
||||
defaultBranch: string;
|
||||
lookbackDays?: number;
|
||||
now?: number;
|
||||
apiRequest?: GitHubApiRequest;
|
||||
}): Promise<PrCiHealthReport>;
|
||||
|
||||
export function classifyPrCi(input: {
|
||||
pull: PrCiPullRequest;
|
||||
runs: Array<{ run: PrCiWorkflowRun; basis: string }>;
|
||||
runsComplete: boolean;
|
||||
pullRequestInventoryComplete: boolean;
|
||||
ambiguousRuns: Array<{ run: PrCiWorkflowRun; basis: string }>;
|
||||
}): PrCiClassification;
|
||||
|
||||
export function matchRunToPullRequest(input: {
|
||||
run: PrCiWorkflowRun;
|
||||
pull: PrCiPullRequest;
|
||||
identityCounts: Map<string, number>;
|
||||
inventoryComplete?: boolean;
|
||||
}): { kind: "none" } | { kind: "ambiguous" } | { kind: "match"; basis: string };
|
||||
|
||||
export function writePrCiHealthReport(
|
||||
report: PrCiHealthReport,
|
||||
outputDirectory?: string,
|
||||
): Promise<{ jsonPath: string; markdownPath: string }>;
|
||||
|
||||
export function renderMarkdownSummary(report: PrCiHealthReport): string;
|
||||
import type { GitHubApiRequest } from "./github-api.mjs";
|
||||
@@ -0,0 +1,718 @@
|
||||
import { appendFile, mkdir, writeFile } from "node:fs/promises";
|
||||
import { join, resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
import { createGhApiRequest } from "./github-api.mjs";
|
||||
|
||||
const OUTPUT_DIRECTORY = join(".cache", "ci-pr-health");
|
||||
const PAGE_SIZE = 100;
|
||||
const MAX_PULL_REQUEST_PAGES = 3;
|
||||
const MAX_WORKFLOW_RUN_PAGES = 5;
|
||||
const MAX_PULL_REQUEST_DETAILS = 40;
|
||||
const MAX_API_REQUESTS =
|
||||
MAX_PULL_REQUEST_PAGES + MAX_WORKFLOW_RUN_PAGES + MAX_PULL_REQUEST_DETAILS;
|
||||
const LOOKBACK_DAYS = 14;
|
||||
const MAX_LOOKBACK_DAYS = 30;
|
||||
const MAX_TOTAL_TIME_MS = 60_000;
|
||||
const SHA_PATTERN = /^[0-9a-f]{40}$/iu;
|
||||
const COMPLETED_CATEGORIES = new Map([
|
||||
["failure", "completed_failure"],
|
||||
["timed_out", "completed_failure"],
|
||||
["cancelled", "completed_canceled"],
|
||||
["success", "completed_success"],
|
||||
]);
|
||||
|
||||
/** Read workflow/PR state and conservatively distinguish absent CI from unknown. */
|
||||
export async function collectPrCiHealth({
|
||||
repository,
|
||||
repositoryId,
|
||||
defaultBranch,
|
||||
lookbackDays = LOOKBACK_DAYS,
|
||||
now = Date.now(),
|
||||
apiRequest = createGhApiRequest(),
|
||||
}) {
|
||||
const identity = validateInputs({
|
||||
repository,
|
||||
repositoryId,
|
||||
defaultBranch,
|
||||
lookbackDays,
|
||||
now,
|
||||
});
|
||||
const observedAt = new Date(now).toISOString();
|
||||
const cutoff = now - identity.lookbackDays * 24 * 60 * 60 * 1_000;
|
||||
const snapshot = await collectApiSnapshot({ apiRequest, identity, cutoff });
|
||||
const classified = classifyPullRequests(snapshot);
|
||||
const { pulls, runs, budget } = snapshot;
|
||||
const errors = [pulls.error, runs.error, ...snapshot.detailsErrors].filter(
|
||||
Boolean,
|
||||
);
|
||||
return {
|
||||
schemaVersion: 1,
|
||||
observedAt,
|
||||
repository: identity.repository,
|
||||
repositoryId: identity.repositoryId,
|
||||
defaultBranch: identity.defaultBranch,
|
||||
lookbackDays: identity.lookbackDays,
|
||||
scan: {
|
||||
status:
|
||||
errors.length === 0 && pulls.complete && runs.complete
|
||||
? "complete"
|
||||
: "partial",
|
||||
cutoff: new Date(cutoff).toISOString(),
|
||||
pullRequestPages: pulls.pages,
|
||||
workflowRunPages: runs.pages,
|
||||
pullRequestInventoryComplete: pulls.complete,
|
||||
workflowRunInventoryComplete: runs.complete,
|
||||
pullRequestDetailsFetched: snapshot.detailsFetched,
|
||||
pullRequestDetailsLimit: MAX_PULL_REQUEST_DETAILS,
|
||||
apiRequests: budget.requests,
|
||||
apiRequestLimit: MAX_API_REQUESTS,
|
||||
errors,
|
||||
},
|
||||
counts: countCategories(classified),
|
||||
pullRequests: classified,
|
||||
};
|
||||
}
|
||||
|
||||
async function collectApiSnapshot({ apiRequest, identity, cutoff }) {
|
||||
const budget = {
|
||||
requests: 0,
|
||||
startedAt: Date.now(),
|
||||
limitMs: MAX_TOTAL_TIME_MS,
|
||||
};
|
||||
const requestGet = async (endpoint) => {
|
||||
if (budget.requests >= MAX_API_REQUESTS)
|
||||
throw new Error("lookup_request_budget_exhausted");
|
||||
if (Date.now() - budget.startedAt >= budget.limitMs)
|
||||
throw new Error("lookup_time_budget_exhausted");
|
||||
budget.requests += 1;
|
||||
return await apiRequest("GET", endpoint, budget);
|
||||
};
|
||||
const pulls = await listOpenPullRequests({
|
||||
requestGet,
|
||||
identity,
|
||||
});
|
||||
const candidates = pulls.items.filter(
|
||||
(pull) =>
|
||||
!pull.draft &&
|
||||
Number.isFinite(Date.parse(pull.createdAt ?? "")) &&
|
||||
Date.parse(pull.createdAt ?? "") >= cutoff &&
|
||||
pull.base.ref === identity.defaultBranch &&
|
||||
(identity.repositoryId === null ||
|
||||
pull.base.repositoryId === identity.repositoryId),
|
||||
);
|
||||
const runs = await listRecentWorkflowRuns({
|
||||
requestGet,
|
||||
identity,
|
||||
cutoff,
|
||||
});
|
||||
|
||||
const detailResult = await resolveMergeability({
|
||||
candidates,
|
||||
requestGet,
|
||||
identity,
|
||||
});
|
||||
return {
|
||||
budget,
|
||||
pulls,
|
||||
runs,
|
||||
candidates,
|
||||
detailsFetched: detailResult.fetched,
|
||||
detailsErrors: detailResult.errors,
|
||||
};
|
||||
}
|
||||
|
||||
function classifyPullRequests(snapshot) {
|
||||
const { candidates, pulls, runs } = snapshot;
|
||||
const identityCounts = countHeadIdentities(pulls.items);
|
||||
return candidates.map((pull) => {
|
||||
const relevant = [];
|
||||
const ambiguousRuns = [];
|
||||
for (const run of runs.items) {
|
||||
const match = matchRunToPullRequest({
|
||||
run,
|
||||
pull,
|
||||
identityCounts,
|
||||
inventoryComplete: pulls.complete,
|
||||
});
|
||||
if (match.kind === "match") relevant.push({ run, basis: match.basis });
|
||||
if (match.kind === "ambiguous")
|
||||
ambiguousRuns.push({ run, basis: "ambiguous_head_identity" });
|
||||
}
|
||||
return {
|
||||
...pull,
|
||||
ci: classifyPrCi({
|
||||
pull,
|
||||
runs: relevant,
|
||||
runsComplete: runs.complete,
|
||||
pullRequestInventoryComplete: pulls.complete,
|
||||
ambiguousRuns,
|
||||
}),
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
async function resolveMergeability({ candidates, requestGet, identity }) {
|
||||
let detailsFetched = 0;
|
||||
const errors = new Set();
|
||||
for (const pull of candidates) {
|
||||
if (pull.mergeable !== undefined) continue;
|
||||
if (detailsFetched >= MAX_PULL_REQUEST_DETAILS) {
|
||||
errors.add("pull_request_mergeability_lookup_budget_exhausted");
|
||||
continue;
|
||||
}
|
||||
const result = await getPullRequestDetails({ requestGet, identity, pull });
|
||||
detailsFetched += 1;
|
||||
if (result.error !== null) {
|
||||
errors.add(result.error);
|
||||
continue;
|
||||
}
|
||||
const { detail } = result;
|
||||
const identityChanged =
|
||||
detail.headSha !== pull.head.sha ||
|
||||
detail.headRef !== pull.head.ref ||
|
||||
detail.headRepositoryId !== pull.head.repositoryId ||
|
||||
detail.baseRef !== pull.base.ref ||
|
||||
detail.baseRepositoryId !== pull.base.repositoryId;
|
||||
if (identityChanged) {
|
||||
pull.identityValid = false;
|
||||
pull.identityChangedDuringScan = true;
|
||||
errors.add("pull_request_identity_changed_during_scan");
|
||||
} else {
|
||||
pull.mergeable = detail.mergeable;
|
||||
pull.detailFetched = true;
|
||||
if (detail.mergeable === undefined)
|
||||
errors.add("pull_request_mergeability_unknown");
|
||||
}
|
||||
}
|
||||
return { fetched: detailsFetched, errors: [...errors] };
|
||||
}
|
||||
|
||||
function countCategories(pulls) {
|
||||
const categories = [
|
||||
"missing",
|
||||
"startup_failure",
|
||||
"queued_pending_executing",
|
||||
"approval_required",
|
||||
"completed_success",
|
||||
"completed_failure",
|
||||
"completed_canceled",
|
||||
"completed_other",
|
||||
"merge_conflict",
|
||||
"unknown",
|
||||
"deferred",
|
||||
];
|
||||
return Object.fromEntries(
|
||||
categories.map((category) => [
|
||||
category,
|
||||
pulls.filter(({ ci }) => ci.category === category).length,
|
||||
]),
|
||||
);
|
||||
}
|
||||
|
||||
/** Pure status classifier; missing requires complete, identity-safe inventories. */
|
||||
export function classifyPrCi({
|
||||
pull,
|
||||
runs,
|
||||
runsComplete,
|
||||
pullRequestInventoryComplete,
|
||||
ambiguousRuns,
|
||||
}) {
|
||||
if (!pull.identityValid)
|
||||
return result("unknown", "pull_request_head_identity_invalid", []);
|
||||
const evidence = [...runs, ...ambiguousRuns];
|
||||
if (pull.mergeable === false)
|
||||
return result("merge_conflict", "github_reports_merge_conflict", evidence);
|
||||
// The newest collected run is not proven current until pagination completes:
|
||||
// an older-created run outside this snapshot may have been retried recently.
|
||||
if (!runsComplete)
|
||||
return result(
|
||||
evidence.length > 0 ? "unknown" : "deferred",
|
||||
evidence.length > 0
|
||||
? "workflow_run_inventory_incomplete"
|
||||
: "lookup_inventory_incomplete",
|
||||
evidence,
|
||||
);
|
||||
const selection = newestRun(runs);
|
||||
if (!selection.known)
|
||||
return result("unknown", "workflow_run_order_unknown", evidence);
|
||||
const latest = selection.latest;
|
||||
if (
|
||||
ambiguousRuns.some((candidate) => {
|
||||
const order = latest ? compareRunEvidence(candidate, latest) : null;
|
||||
return order === null || order >= 0;
|
||||
})
|
||||
)
|
||||
return result("unknown", "workflow_run_identity_ambiguous", evidence);
|
||||
if (latest !== undefined) {
|
||||
const run = latest.run;
|
||||
if (run.conclusion === "startup_failure")
|
||||
return result("startup_failure", "workflow_startup_failure", evidence);
|
||||
if (run.status === "waiting" || run.conclusion === "action_required")
|
||||
return result(
|
||||
"approval_required",
|
||||
"workflow_waiting_for_approval_or_gate",
|
||||
evidence,
|
||||
);
|
||||
if (["queued", "requested", "pending", "in_progress"].includes(run.status))
|
||||
return result(
|
||||
"queued_pending_executing",
|
||||
`workflow_${run.status}`,
|
||||
evidence,
|
||||
);
|
||||
if (run.status === "completed")
|
||||
return result(
|
||||
COMPLETED_CATEGORIES.get(run.conclusion) ?? "completed_other",
|
||||
run.conclusion === "success"
|
||||
? "workflow_succeeded"
|
||||
: `workflow_${run.conclusion ?? "unknown_conclusion"}`,
|
||||
evidence,
|
||||
);
|
||||
return result("unknown", "workflow_status_unrecognized", evidence);
|
||||
}
|
||||
if (!pullRequestInventoryComplete)
|
||||
return result("deferred", "lookup_inventory_incomplete", []);
|
||||
if (pull.mergeable !== true)
|
||||
return result("unknown", "pull_request_mergeability_unknown", []);
|
||||
return result("missing", "no_pull_request_event_run_for_current_head", []);
|
||||
}
|
||||
|
||||
/** Match only a pull_request event proven to belong to this immutable PR head. */
|
||||
export function matchRunToPullRequest({
|
||||
run,
|
||||
pull,
|
||||
identityCounts,
|
||||
inventoryComplete = true,
|
||||
}) {
|
||||
if (run?.event !== "pull_request") return { kind: "none" };
|
||||
if (!pull.identityValid || !isRecord(run)) return { kind: "none" };
|
||||
const association = matchExplicitPullAssociation(run, pull);
|
||||
if (association === "match")
|
||||
return { kind: "match", basis: "event_pr_number_and_head_sha" };
|
||||
if (association === "ambiguous") return { kind: "ambiguous" };
|
||||
if (association === "other" || !sameHeadRepositoryAndRef(run, pull))
|
||||
return { kind: "none" };
|
||||
if (!uniqueHeadRepositoryAndRef(pull, identityCounts, inventoryComplete))
|
||||
return { kind: "ambiguous" };
|
||||
if (isSha(run.head_sha) && run.head_sha.toLowerCase() === pull.head.sha)
|
||||
return { kind: "match", basis: "event_unique_head_ref_and_repository_id" };
|
||||
return { kind: "none" };
|
||||
}
|
||||
|
||||
function matchExplicitPullAssociation(run, pull) {
|
||||
const associations = Array.isArray(run.pull_requests)
|
||||
? run.pull_requests
|
||||
: [];
|
||||
if (associations.length === 0) return "absent";
|
||||
const matchingNumber = associations.filter(
|
||||
(item) => item?.number === pull.number,
|
||||
);
|
||||
if (matchingNumber.length === 0) return "other";
|
||||
const exactHead = matchingNumber.find(
|
||||
(item) => item?.head?.sha === pull.head.sha,
|
||||
);
|
||||
if (exactHead === undefined) {
|
||||
return matchingNumber.some(
|
||||
(item) =>
|
||||
typeof item?.head?.sha === "string" && item.head.sha !== pull.head.sha,
|
||||
)
|
||||
? "other"
|
||||
: "absent";
|
||||
}
|
||||
const associatedHeadRepositoryId = positiveInteger(exactHead.head?.repo?.id);
|
||||
const runHeadRepositoryId = positiveInteger(run.head_repository?.id);
|
||||
if (associatedHeadRepositoryId === null || runHeadRepositoryId === null)
|
||||
return "ambiguous";
|
||||
if (
|
||||
associatedHeadRepositoryId !== pull.head.repositoryId ||
|
||||
runHeadRepositoryId !== pull.head.repositoryId
|
||||
)
|
||||
return "other";
|
||||
const baseRefMatches =
|
||||
typeof exactHead.base?.ref !== "string" ||
|
||||
exactHead.base.ref === pull.base.ref;
|
||||
const baseRepositoryId = positiveInteger(exactHead.base?.repo?.id);
|
||||
const baseRepositoryMatches =
|
||||
baseRepositoryId === null || baseRepositoryId === pull.base.repositoryId;
|
||||
return baseRefMatches && baseRepositoryMatches ? "match" : "other";
|
||||
}
|
||||
|
||||
function sameHeadRepositoryAndRef(run, pull) {
|
||||
return (
|
||||
run.head_branch === pull.head.ref &&
|
||||
run.head_repository?.id === pull.head.repositoryId
|
||||
);
|
||||
}
|
||||
|
||||
function uniqueHeadRepositoryAndRef(pull, identityCounts, inventoryComplete) {
|
||||
if (!inventoryComplete) return false;
|
||||
const key = headIdentityKey(pull.head.repositoryId, pull.head.ref);
|
||||
const unknownRepositoryKey = headIdentityKey(null, pull.head.ref);
|
||||
return (
|
||||
identityCounts.get(key) === 1 &&
|
||||
(identityCounts.get(unknownRepositoryKey) ?? 0) === 0
|
||||
);
|
||||
}
|
||||
|
||||
function normalizePullRequest(raw, identity) {
|
||||
const number = positiveInteger(raw?.number);
|
||||
const base = normalizeBase(raw?.base);
|
||||
const head = normalizeHead(raw?.head);
|
||||
const draft = raw?.draft === true;
|
||||
return {
|
||||
number,
|
||||
draft,
|
||||
createdAt: typeof raw?.created_at === "string" ? raw.created_at : null,
|
||||
url: typeof raw?.html_url === "string" ? raw.html_url : null,
|
||||
base,
|
||||
head,
|
||||
mergeable: typeof raw?.mergeable === "boolean" ? raw.mergeable : undefined,
|
||||
identityValid: hasValidPullIdentity({ raw, identity, number, base, head }),
|
||||
};
|
||||
}
|
||||
|
||||
function normalizeBase(raw) {
|
||||
return {
|
||||
ref: typeof raw?.ref === "string" ? raw.ref : null,
|
||||
repositoryId: positiveInteger(raw?.repo?.id),
|
||||
};
|
||||
}
|
||||
|
||||
function normalizeHead(raw) {
|
||||
return {
|
||||
repositoryId: positiveInteger(raw?.repo?.id),
|
||||
repository:
|
||||
typeof raw?.repo?.full_name === "string" ? raw.repo.full_name : null,
|
||||
ref: typeof raw?.ref === "string" ? raw.ref : null,
|
||||
sha:
|
||||
typeof raw?.sha === "string" && isSha(raw.sha)
|
||||
? raw.sha.toLowerCase()
|
||||
: null,
|
||||
};
|
||||
}
|
||||
|
||||
function hasValidPullIdentity({ raw, identity, number, base, head }) {
|
||||
return (
|
||||
number !== null &&
|
||||
typeof raw?.draft === "boolean" &&
|
||||
base.ref === identity.defaultBranch &&
|
||||
(identity.repositoryId === null ||
|
||||
base.repositoryId === identity.repositoryId) &&
|
||||
head.repositoryId !== null &&
|
||||
head.sha !== null &&
|
||||
head.ref !== null
|
||||
);
|
||||
}
|
||||
|
||||
function countHeadIdentities(pulls) {
|
||||
const counts = new Map();
|
||||
for (const pull of pulls) {
|
||||
if (typeof pull.head.ref !== "string") continue;
|
||||
const key = headIdentityKey(pull.head.repositoryId, pull.head.ref);
|
||||
counts.set(key, (counts.get(key) ?? 0) + 1);
|
||||
}
|
||||
return counts;
|
||||
}
|
||||
|
||||
function headIdentityKey(repositoryId, ref) {
|
||||
return `${String(repositoryId)}\u0000${ref}`;
|
||||
}
|
||||
|
||||
function newestRun(runs) {
|
||||
let latest = runs[0];
|
||||
for (const candidate of runs) {
|
||||
if (latest && (compareRunEvidence(candidate, latest) ?? 0) > 0)
|
||||
latest = candidate;
|
||||
}
|
||||
const known = runs.every((candidate) => {
|
||||
if (candidate === latest) return true;
|
||||
const order = compareRunEvidence(candidate, latest);
|
||||
return order !== null && order <= 0;
|
||||
});
|
||||
return { known, latest };
|
||||
}
|
||||
|
||||
// Attempt ordering is meaningful within one run; different run IDs are not clocks.
|
||||
function compareRunEvidence(left, right) {
|
||||
const leftRun = left.run;
|
||||
const rightRun = right.run;
|
||||
const id = positiveInteger(leftRun.id);
|
||||
if (id !== null && id === positiveInteger(rightRun.id)) {
|
||||
const leftAttempt = positiveInteger(leftRun.run_attempt);
|
||||
const rightAttempt = positiveInteger(rightRun.run_attempt);
|
||||
if (leftAttempt === null || rightAttempt === null) return null;
|
||||
if (leftAttempt !== rightAttempt) return leftAttempt - rightAttempt;
|
||||
return leftRun.status === rightRun.status &&
|
||||
leftRun.conclusion === rightRun.conclusion
|
||||
? 0
|
||||
: null;
|
||||
}
|
||||
// A retry may start long after its original creation; without an attempt
|
||||
// clock, creation dates only order first attempts from different runs.
|
||||
if (
|
||||
positiveInteger(leftRun.run_attempt) !== 1 ||
|
||||
positiveInteger(rightRun.run_attempt) !== 1
|
||||
)
|
||||
return null;
|
||||
const time =
|
||||
Date.parse(leftRun.created_at ?? "") -
|
||||
Date.parse(rightRun.created_at ?? "");
|
||||
return Number.isFinite(time) && time !== 0 ? time : null;
|
||||
}
|
||||
|
||||
function result(category, reason, runs) {
|
||||
return {
|
||||
category,
|
||||
reason,
|
||||
runs: runs.map(({ run, basis }) => ({
|
||||
id: positiveInteger(run.id),
|
||||
attempt: positiveInteger(run.run_attempt),
|
||||
status: typeof run.status === "string" ? run.status : null,
|
||||
conclusion: typeof run.conclusion === "string" ? run.conclusion : null,
|
||||
headSha: isSha(run.head_sha) ? run.head_sha.toLowerCase() : null,
|
||||
createdAt: typeof run.created_at === "string" ? run.created_at : null,
|
||||
url: typeof run.html_url === "string" ? run.html_url : null,
|
||||
identityBasis: basis,
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
async function listOpenPullRequests({ requestGet, identity }) {
|
||||
const items = [];
|
||||
let complete = false;
|
||||
let pages = 0;
|
||||
let error = null;
|
||||
for (let page = 1; page <= MAX_PULL_REQUEST_PAGES; page += 1) {
|
||||
try {
|
||||
const endpoint = `repos/${identity.repository}/pulls?state=open&sort=created&direction=desc&per_page=${PAGE_SIZE}&page=${page}`;
|
||||
const response = await requestGet(endpoint);
|
||||
if (!Array.isArray(response))
|
||||
throw new Error("pull_request_page_malformed");
|
||||
pages += 1;
|
||||
const normalized = response.map((raw) =>
|
||||
normalizePullRequest(raw, identity),
|
||||
);
|
||||
if (
|
||||
normalized.some(
|
||||
(pull) => !Number.isFinite(Date.parse(pull.createdAt ?? "")),
|
||||
)
|
||||
)
|
||||
throw new Error("pull_request_page_malformed");
|
||||
items.push(...normalized);
|
||||
if (response.length < PAGE_SIZE) {
|
||||
complete = true;
|
||||
break;
|
||||
}
|
||||
} catch (cause) {
|
||||
error = safeErrorCode(cause);
|
||||
break;
|
||||
}
|
||||
}
|
||||
return { items, pages, complete, error };
|
||||
}
|
||||
|
||||
async function listRecentWorkflowRuns({ requestGet, identity, cutoff }) {
|
||||
const items = [];
|
||||
let complete = false;
|
||||
let pages = 0;
|
||||
let error = null;
|
||||
for (let page = 1; page <= MAX_WORKFLOW_RUN_PAGES; page += 1) {
|
||||
try {
|
||||
const cutoffDate = new Date(cutoff).toISOString().slice(0, 10);
|
||||
const endpoint = `repos/${identity.repository}/actions/workflows/ci.yml/runs?event=pull_request&created=%3E%3D${cutoffDate}&per_page=${PAGE_SIZE}&page=${page}`;
|
||||
const response = await requestGet(endpoint);
|
||||
if (!isRecord(response) || !Array.isArray(response.workflow_runs))
|
||||
throw new Error("workflow_run_page_malformed");
|
||||
pages += 1;
|
||||
if (!response.workflow_runs.every(isRecord))
|
||||
throw new Error("workflow_run_page_malformed");
|
||||
items.push(...response.workflow_runs);
|
||||
if (response.workflow_runs.length < PAGE_SIZE) {
|
||||
complete = true;
|
||||
break;
|
||||
}
|
||||
} catch (cause) {
|
||||
error = safeErrorCode(cause);
|
||||
break;
|
||||
}
|
||||
}
|
||||
return { items, pages, complete, error };
|
||||
}
|
||||
|
||||
async function getPullRequestDetails({ requestGet, identity, pull }) {
|
||||
if (pull.number === null)
|
||||
return { detail: null, error: "pull_request_detail_identity_missing" };
|
||||
try {
|
||||
const response = await requestGet(
|
||||
`repos/${identity.repository}/pulls/${pull.number}`,
|
||||
);
|
||||
if (!isRecord(response))
|
||||
return { detail: null, error: "pull_request_detail_malformed" };
|
||||
return {
|
||||
detail: {
|
||||
mergeable:
|
||||
typeof response.mergeable === "boolean"
|
||||
? response.mergeable
|
||||
: undefined,
|
||||
headSha: isSha(response.head?.sha)
|
||||
? response.head.sha.toLowerCase()
|
||||
: null,
|
||||
headRef:
|
||||
typeof response.head?.ref === "string" ? response.head.ref : null,
|
||||
headRepositoryId: positiveInteger(response.head?.repo?.id),
|
||||
baseRef:
|
||||
typeof response.base?.ref === "string" ? response.base.ref : null,
|
||||
baseRepositoryId: positiveInteger(response.base?.repo?.id),
|
||||
},
|
||||
error: null,
|
||||
};
|
||||
} catch (cause) {
|
||||
return { detail: null, error: safeErrorCode(cause) };
|
||||
}
|
||||
}
|
||||
|
||||
function validateInputs({
|
||||
repository,
|
||||
repositoryId,
|
||||
defaultBranch,
|
||||
lookbackDays,
|
||||
now,
|
||||
}) {
|
||||
if (
|
||||
typeof repository !== "string" ||
|
||||
!/^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+$/u.test(repository)
|
||||
)
|
||||
throw new Error(
|
||||
"CI_REPOSITORY must be an owner/name repository identifier",
|
||||
);
|
||||
if (
|
||||
typeof defaultBranch !== "string" ||
|
||||
defaultBranch.length === 0 ||
|
||||
/[\r\n]/u.test(defaultBranch)
|
||||
)
|
||||
throw new Error("CI_DEFAULT_BRANCH must be a nonempty branch name");
|
||||
const parsedRepositoryId =
|
||||
repositoryId === undefined || repositoryId === null || repositoryId === ""
|
||||
? null
|
||||
: positiveInteger(repositoryId);
|
||||
if (
|
||||
repositoryId !== undefined &&
|
||||
repositoryId !== null &&
|
||||
repositoryId !== "" &&
|
||||
parsedRepositoryId === null
|
||||
)
|
||||
throw new Error("CI_REPOSITORY_ID must be a positive safe integer");
|
||||
if (
|
||||
!Number.isInteger(lookbackDays) ||
|
||||
lookbackDays < 1 ||
|
||||
lookbackDays > MAX_LOOKBACK_DAYS
|
||||
)
|
||||
throw new Error(
|
||||
`lookbackDays must be an integer from 1 to ${String(MAX_LOOKBACK_DAYS)}`,
|
||||
);
|
||||
if (!Number.isFinite(now)) throw new Error("now must be a finite timestamp");
|
||||
return {
|
||||
repository,
|
||||
repositoryId: parsedRepositoryId,
|
||||
defaultBranch,
|
||||
lookbackDays,
|
||||
};
|
||||
}
|
||||
|
||||
function positiveInteger(value) {
|
||||
const number = typeof value === "number" ? value : Number(value);
|
||||
return Number.isSafeInteger(number) && number > 0 ? number : null;
|
||||
}
|
||||
|
||||
function isSha(value) {
|
||||
return typeof value === "string" && SHA_PATTERN.test(value);
|
||||
}
|
||||
|
||||
function isRecord(value) {
|
||||
return typeof value === "object" && value !== null && !Array.isArray(value);
|
||||
}
|
||||
|
||||
function safeErrorCode(cause) {
|
||||
if (cause?.message === "lookup_time_budget_exhausted")
|
||||
return "lookup_time_budget_exhausted";
|
||||
if (cause?.message === "lookup_request_budget_exhausted")
|
||||
return "lookup_request_budget_exhausted";
|
||||
if (cause?.message === "github_api_malformed_json")
|
||||
return "github_api_malformed_json";
|
||||
if (cause?.message?.endsWith("_malformed")) return cause.message;
|
||||
if (cause?.code === "ETIMEDOUT" || cause?.killed) return "github_api_timeout";
|
||||
return "github_api_request_failed";
|
||||
}
|
||||
|
||||
export async function writePrCiHealthReport(
|
||||
report,
|
||||
outputDirectory = OUTPUT_DIRECTORY,
|
||||
) {
|
||||
const jsonPath = join(outputDirectory, "report.json");
|
||||
const markdownPath = join(outputDirectory, "summary.md");
|
||||
await mkdir(outputDirectory, { recursive: true });
|
||||
await writeFile(jsonPath, `${JSON.stringify(report, null, 2)}\n`, "utf8");
|
||||
await writeFile(markdownPath, renderMarkdownSummary(report), "utf8");
|
||||
return { jsonPath, markdownPath };
|
||||
}
|
||||
|
||||
export function renderMarkdownSummary(report) {
|
||||
const rows = report.pullRequests.map(
|
||||
(pull) =>
|
||||
`| #${String(pull.number ?? "?")} | ${escapeCell(pull.head.repository ?? "unknown")} | ${escapeCell(pull.head.ref ?? "unknown")} | \`${pull.head.sha ?? "unknown"}\` | ${pull.ci.category} | ${escapeCell(pull.ci.reason)} |`,
|
||||
);
|
||||
return [
|
||||
"# Pull request CI health snapshot",
|
||||
"",
|
||||
`Observed ${report.observedAt} for \`${report.repository}\`, target branch \`${escapeCell(report.defaultBranch)}\`; recent window ${String(report.lookbackDays)} days. Scan: **${report.scan.status}**.`,
|
||||
"",
|
||||
"This is read-only point-in-time telemetry. Missing is reported only when PR and workflow-run inventories are complete and the current immutable PR head identity is unambiguous. Incomplete lookups remain deferred or unknown.",
|
||||
"",
|
||||
`API pages: ${String(report.scan.pullRequestPages)} PR, ${String(report.scan.workflowRunPages)} workflow runs; ${String(report.scan.apiRequests)} bounded requests. Errors: ${report.scan.errors.length === 0 ? "none" : report.scan.errors.join(", ")}.`,
|
||||
"",
|
||||
"| PR | Head repository | Head ref | Head SHA | CI classification | Evidence |",
|
||||
"| ---: | --- | --- | --- | --- | --- |",
|
||||
...rows,
|
||||
"",
|
||||
"Categories: missing, startup failure, queued/pending/executing, approval required, completed outcomes, merge conflict, unknown, and deferred.",
|
||||
"",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
function escapeCell(value) {
|
||||
return String(value)
|
||||
.replaceAll("|", "\\|")
|
||||
.replaceAll("\n", " ")
|
||||
.replaceAll("\r", " ");
|
||||
}
|
||||
|
||||
if (
|
||||
process.argv[1] !== undefined &&
|
||||
resolve(process.argv[1]) === fileURLToPath(import.meta.url)
|
||||
) {
|
||||
try {
|
||||
const report = await collectPrCiHealth({
|
||||
repository: process.env.CI_REPOSITORY,
|
||||
repositoryId: process.env.CI_REPOSITORY_ID,
|
||||
defaultBranch: process.env.CI_DEFAULT_BRANCH,
|
||||
});
|
||||
const paths = await writePrCiHealthReport(
|
||||
report,
|
||||
process.env.CI_PR_HEALTH_DIRECTORY ?? OUTPUT_DIRECTORY,
|
||||
);
|
||||
if (process.env.GITHUB_STEP_SUMMARY)
|
||||
await appendFile(
|
||||
process.env.GITHUB_STEP_SUMMARY,
|
||||
renderMarkdownSummary(report),
|
||||
"utf8",
|
||||
);
|
||||
process.stdout.write(
|
||||
`Wrote ${report.pullRequests.length} recent PR CI records to ${paths.jsonPath}\n`,
|
||||
);
|
||||
} catch (cause) {
|
||||
process.stderr.write(
|
||||
`PR CI health telemetry failed: ${cause instanceof Error ? cause.message : "unknown_error"}\n`,
|
||||
);
|
||||
process.exitCode = 1;
|
||||
}
|
||||
}
|
||||
@@ -2,6 +2,7 @@ import { execFileSync } from "node:child_process";
|
||||
import { appendFileSync, mkdirSync, writeFileSync } from "node:fs";
|
||||
import { resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { normalizeWorkflowRun } from "./workflow-run.mjs";
|
||||
|
||||
const ARTIFACT_NAME = "ci-full-qualification-v1";
|
||||
const MAX_AGE_MS = 24 * 60 * 60 * 1000;
|
||||
@@ -119,8 +120,8 @@ function admit() {
|
||||
// Artifact metadata narrows the search before run lookups. Never scan an
|
||||
// unbounded history or treat a missing receipt as a successful qualification.
|
||||
for (const artifact of candidates.slice(0, 3)) {
|
||||
const run = api(
|
||||
`repos/${repository}/actions/runs/${artifact.workflow_run.id}`,
|
||||
const run = normalizeWorkflowRun(
|
||||
api(`repos/${repository}/actions/runs/${artifact.workflow_run.id}`),
|
||||
);
|
||||
// An old run can upload a new artifact when only its gate is rerun.
|
||||
// Bound the original run age as well so retries cannot renew stale proof.
|
||||
@@ -133,7 +134,7 @@ function admit() {
|
||||
run.id === artifact.workflow_run.id &&
|
||||
run.status === "completed" &&
|
||||
run.conclusion === "success" &&
|
||||
run.path === ".github/workflows/ci.yml" &&
|
||||
run.workflowPath?.file === ".github/workflows/ci.yml" &&
|
||||
["schedule", "workflow_dispatch"].includes(run.event) &&
|
||||
run.head_sha === head &&
|
||||
run.head_branch === branch &&
|
||||
|
||||
@@ -30,7 +30,12 @@ if (outputs.tier === "full" && selectedScopes.length !== CI_FLAGS.length)
|
||||
throw new Error("Full qualification requires every canonical CI scope");
|
||||
const required = reused ? ["changes"] : ["changes", "static", "test"];
|
||||
const lanes = {
|
||||
code: ["build", "test-shard"],
|
||||
code: [
|
||||
"build",
|
||||
"test-shard",
|
||||
...(outputs.tier === "fast" ? ["test-source"] : []),
|
||||
],
|
||||
reverse_tools: ["reverse-tools-quality"],
|
||||
runtime: ["windows-native-build", "package-e2e"],
|
||||
package: ["windows-curated"],
|
||||
sqlite: ["sqlite-runtime"],
|
||||
|
||||
@@ -2,6 +2,62 @@
|
||||
// Shared build inputs are classified once in plan.mjs rather than repeated in
|
||||
// every workflow.
|
||||
export const SCOPES = {
|
||||
reverse_tools: [
|
||||
"src/application/reverse/**",
|
||||
"src/application/binary/BinarySession.ts",
|
||||
"src/cutter/**",
|
||||
"src/gdb/**",
|
||||
"src/frida/frida-bindings.d.ts",
|
||||
"src/objdump/**",
|
||||
"src/cli/reverseEngineeringCommands.ts",
|
||||
"src/process/ownedDebuggerSession.ts",
|
||||
"src/rizin/**",
|
||||
"src/contracts/*ToolContracts.ts",
|
||||
"src/server/registerCutterTools.ts",
|
||||
"src/server/externalMutationAdmission.ts",
|
||||
"src/server/registerGdbTools.ts",
|
||||
"src/server/registerReverseEngineeringTools.ts",
|
||||
"src/server/providerEnvironment.ts",
|
||||
"src/process/ExecFileOutput.ts",
|
||||
"src/process/ExecFileOutput.test.ts",
|
||||
"tests/composition/analysis-sessions/binarySession.test.ts",
|
||||
"tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts",
|
||||
"tests/boundary/filesystem/reverseToolsMutationConfig.test.ts",
|
||||
"src/server/registerRizinDebugTools.ts",
|
||||
"tests/boundary/providers/CutterBridge*.test.ts",
|
||||
"tests/boundary/providers/cutterBridgeProducer.test.ts",
|
||||
"tests/boundary/providers/GdbSession*.test.ts",
|
||||
"tests/boundary/providers/GdbCommandQueue.test.ts",
|
||||
"tests/boundary/providers/gdbTestHelpers.ts",
|
||||
"tests/boundary/providers/RizinDebugSessionCoverage.part*.test.ts",
|
||||
"tests/boundary/providers/RizinCommandTimeout.test.ts",
|
||||
"tests/boundary/providers/RizinTimeoutEvidence.test.ts",
|
||||
"tests/boundary/providers/RizinDebugSession*.test.ts",
|
||||
"tests/boundary/providers/RizinInferiorTty.test.ts",
|
||||
"tests/boundary/providers/RizinTestInferiorTty.ts",
|
||||
"tests/boundary/process/ownedDebuggerSession.test.ts",
|
||||
"tests/boundary/cli/gdbStartupCleanupCli.test.ts",
|
||||
"tests/boundary/cli/reverseEngineeringCommands.test.ts",
|
||||
"tests/boundary/mcp/cutter*.test.ts",
|
||||
"tests/boundary/mcp/gdbCancellationMcp.test.ts",
|
||||
"tests/boundary/mcp/debuggerCancellationMcp.test.ts",
|
||||
"tests/boundary/mcp/rizinDebuggerCancellationMcp.test.ts",
|
||||
"tests/boundary/mcp/reverseEngineeringPartialMcp.test.ts",
|
||||
"tests/boundary/mcp/workflowSnapshotReplayMcp.test.ts",
|
||||
"tests/boundary/mcp/reverseEngineeringEnvironmentMcp.test.ts",
|
||||
"tests/support/reverseEngineering*.ts",
|
||||
"tests/support/ownedRizinCommandEvidenceTests.ts",
|
||||
"tests/support/reverseDebuggerMcpTestHelpers.ts",
|
||||
"tests/processOwnershipRunner.ts",
|
||||
"tests/composition/reverseEngineering*.test.ts",
|
||||
"vitest.reverse-tools.config.ts",
|
||||
"vitest.reverse-tools-semantics.config.ts",
|
||||
"stryker.config.mjs",
|
||||
"stryker.semantics.config.mjs",
|
||||
"tsconfig.reverse-tools-mutation.json",
|
||||
"scripts/prepare-reverse-tools-mutation.mjs",
|
||||
"scripts/lib/reverse-tools-mutation-targets.mjs",
|
||||
],
|
||||
// Apple artifact CLI/MCP lane on macOS, including its focused regressions.
|
||||
apple: [
|
||||
"src/native/Apple*",
|
||||
|
||||
@@ -133,6 +133,8 @@ try {
|
||||
for (const clone of report.duplicates)
|
||||
for (const file of [clone.firstFile, clone.secondFile])
|
||||
file.name = relative(process.cwd(), file.name).split(sep).join("/");
|
||||
// Record the actual checker result, including caller-selected thresholds.
|
||||
report.reaCheck = { exitCode: result.status };
|
||||
await writeFile(reportPath, JSON.stringify(report, null, 2));
|
||||
const { sources, clones, percentage } = report.statistics.total;
|
||||
console.log(
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
export interface StaticTaskContext {
|
||||
readonly version: 1;
|
||||
readonly sourceSha: string;
|
||||
readonly sourceClean: boolean;
|
||||
readonly prHeadSha?: string;
|
||||
readonly nodeVersion: string;
|
||||
readonly npmVersion: string;
|
||||
readonly turboVersion: string;
|
||||
readonly platform: string;
|
||||
readonly architecture: string;
|
||||
readonly taskHashes: Readonly<Record<string, string>>;
|
||||
}
|
||||
export function parseStaticTaskContext(value: unknown): StaticTaskContext;
|
||||
export function recordStaticTaskContext(options?: {
|
||||
root?: string;
|
||||
npmCli?: string;
|
||||
outputDirectory?: string;
|
||||
}): Promise<StaticTaskContext>;
|
||||
@@ -0,0 +1,104 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import { join, resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { promisify } from "node:util";
|
||||
import { z } from "zod";
|
||||
|
||||
const execute = promisify(execFile);
|
||||
const contextSchema = z.object({
|
||||
version: z.literal(1),
|
||||
sourceSha: z.string().regex(/^[a-f0-9]{40}$/u),
|
||||
sourceClean: z.boolean(),
|
||||
prHeadSha: z
|
||||
.string()
|
||||
.regex(/^[a-f0-9]{40}$/u)
|
||||
.optional(),
|
||||
nodeVersion: z.string().min(1),
|
||||
npmVersion: z.string().min(1),
|
||||
turboVersion: z.string().min(1),
|
||||
platform: z.string().min(1),
|
||||
architecture: z.string().min(1),
|
||||
taskHashes: z.record(z.string(), z.string().regex(/^[a-f0-9]{16}$/u)),
|
||||
});
|
||||
|
||||
/** Parse the producing check graph's receipt; incomplete provenance is unknown. */
|
||||
export const parseStaticTaskContext = (value) => contextSchema.parse(value);
|
||||
|
||||
/** Retain only named task hashes and tool/source identity, never the environment. */
|
||||
export async function recordStaticTaskContext({
|
||||
root = process.cwd(),
|
||||
npmCli = process.env.npm_execpath,
|
||||
outputDirectory = join(root, ".cache/static-checks"),
|
||||
} = {}) {
|
||||
if (typeof npmCli !== "string" || !npmCli)
|
||||
throw new Error(
|
||||
"Run static context recording through npm run ci:static-context",
|
||||
);
|
||||
const options = {
|
||||
cwd: root,
|
||||
encoding: "utf8",
|
||||
timeout: 30_000,
|
||||
maxBuffer: 32 * 1024 * 1024,
|
||||
};
|
||||
const [graph, source, status, npmVersion, turbo] = await Promise.all([
|
||||
execute(
|
||||
process.execPath,
|
||||
[npmCli, "run", "--silent", "check:ci:tasks", "--", "--dry=json"],
|
||||
options,
|
||||
),
|
||||
execute("git", ["rev-parse", "HEAD"], options),
|
||||
execute(
|
||||
"git",
|
||||
["status", "--porcelain", "--untracked-files=normal"],
|
||||
options,
|
||||
),
|
||||
execute(process.execPath, [npmCli, "--version"], options),
|
||||
readFile(join(root, "node_modules/turbo/package.json"), "utf8"),
|
||||
]);
|
||||
const tasks = z
|
||||
.object({
|
||||
tasks: z
|
||||
.array(
|
||||
z.object({
|
||||
task: z.string().min(1),
|
||||
hash: z.string().regex(/^[a-f0-9]{16}$/u),
|
||||
}),
|
||||
)
|
||||
.min(1),
|
||||
})
|
||||
.parse(JSON.parse(graph.stdout)).tasks;
|
||||
if (new Set(tasks.map(({ task }) => task)).size !== tasks.length)
|
||||
throw new Error("Static graph contains duplicate task identities");
|
||||
const context = parseStaticTaskContext({
|
||||
version: 1,
|
||||
sourceSha: source.stdout.trim(),
|
||||
sourceClean: status.stdout.length === 0,
|
||||
...(process.env.CI_PR_HEAD_SHA
|
||||
? { prHeadSha: process.env.CI_PR_HEAD_SHA }
|
||||
: {}),
|
||||
nodeVersion: process.version,
|
||||
npmVersion: npmVersion.stdout.trim(),
|
||||
turboVersion: z.object({ version: z.string() }).parse(JSON.parse(turbo))
|
||||
.version,
|
||||
platform: process.platform,
|
||||
architecture: process.arch,
|
||||
taskHashes: Object.fromEntries(tasks.map(({ task, hash }) => [task, hash])),
|
||||
});
|
||||
if (process.env.GITHUB_SHA && process.env.GITHUB_SHA !== context.sourceSha)
|
||||
throw new Error(
|
||||
"Static context source does not match the workflow revision",
|
||||
);
|
||||
await mkdir(outputDirectory, { recursive: true });
|
||||
await writeFile(
|
||||
join(outputDirectory, "static-task-context.json"),
|
||||
JSON.stringify(context, null, 2) + "\n",
|
||||
);
|
||||
return context;
|
||||
}
|
||||
|
||||
if (
|
||||
process.argv[1] &&
|
||||
resolve(process.argv[1]) === fileURLToPath(import.meta.url)
|
||||
)
|
||||
await recordStaticTaskContext();
|
||||
@@ -0,0 +1,46 @@
|
||||
import { spawn } from "node:child_process";
|
||||
import { resolve } from "node:path";
|
||||
|
||||
import {
|
||||
BUILT_TEST_PROJECTS,
|
||||
SOURCE_TEST_PROJECTS,
|
||||
} from "../lib/test-projects.mjs";
|
||||
|
||||
const [group, ...arguments_] = process.argv.slice(2);
|
||||
const projects = { source: SOURCE_TEST_PROJECTS, built: BUILT_TEST_PROJECTS }[
|
||||
group
|
||||
];
|
||||
if (!projects) throw new Error("Expected source or built test group");
|
||||
// The group owns project selection; callers can request reporters or shards.
|
||||
if (
|
||||
arguments_.some((argument) => /^--(?:project|exclude)(?:=|$)/u.test(argument))
|
||||
)
|
||||
throw new Error("Test groups cannot override their project inventory");
|
||||
const child = spawn(
|
||||
process.execPath,
|
||||
[
|
||||
resolve("node_modules/vitest/vitest.mjs"),
|
||||
"run",
|
||||
...projects.flatMap(({ name }) => ["--project", name]),
|
||||
...arguments_,
|
||||
],
|
||||
{ stdio: "inherit" },
|
||||
);
|
||||
let shutdownSignal;
|
||||
for (const signal of ["SIGINT", "SIGTERM"])
|
||||
process.on(signal, () => {
|
||||
shutdownSignal ??= signal;
|
||||
child.kill(signal);
|
||||
});
|
||||
child.once("error", (error) => {
|
||||
console.error(error.message);
|
||||
process.exitCode = 1;
|
||||
});
|
||||
child.once("close", (code) => {
|
||||
process.exitCode =
|
||||
shutdownSignal === "SIGINT"
|
||||
? 130
|
||||
: shutdownSignal === "SIGTERM"
|
||||
? 143
|
||||
: (code ?? 1);
|
||||
});
|
||||
@@ -2,6 +2,7 @@ import { execFileSync } from "node:child_process";
|
||||
import { appendFileSync } from "node:fs";
|
||||
import { resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { normalizeWorkflowRun } from "./workflow-run.mjs";
|
||||
|
||||
const ARTIFACT_NAME = "test-timings";
|
||||
const MAX_AGE_MS = 30 * 24 * 60 * 60 * 1000;
|
||||
@@ -68,8 +69,8 @@ function findTimingRun({ repository, branch, repositoryId, now = Date.now() }) {
|
||||
.slice(0, MAX_CANDIDATES);
|
||||
|
||||
for (const artifact of candidates) {
|
||||
const run = api(
|
||||
`repos/${repository}/actions/runs/${artifact.workflow_run.id}`,
|
||||
const run = normalizeWorkflowRun(
|
||||
api(`repos/${repository}/actions/runs/${artifact.workflow_run.id}`),
|
||||
);
|
||||
const runAge = now - Date.parse(run?.created_at);
|
||||
if (
|
||||
@@ -80,7 +81,7 @@ function findTimingRun({ repository, branch, repositoryId, now = Date.now() }) {
|
||||
run.id === artifact.workflow_run.id &&
|
||||
run.status === "completed" &&
|
||||
run.conclusion === "success" &&
|
||||
run.path === ".github/workflows/ci.yml" &&
|
||||
run.workflowPath?.file === ".github/workflows/ci.yml" &&
|
||||
["schedule", "workflow_dispatch"].includes(run.event) &&
|
||||
run.head_branch === branch &&
|
||||
run.head_repository?.full_name === repository
|
||||
|
||||
@@ -41,9 +41,20 @@ const files = report.files.map(({ name, durationMs, state }) => {
|
||||
throw new Error(`Test report path is outside the checkout: ${name}`);
|
||||
return { name: path.split(sep).join("/"), durationMs, state };
|
||||
});
|
||||
const complete = process.env.SHARD_RESULT === "success";
|
||||
// Keep group outcomes intact: an unavailable group is not a completed failure.
|
||||
const groupResults = [
|
||||
process.env.SHARD_RESULT,
|
||||
...(process.env.SOURCE_REQUIRED === "true"
|
||||
? [process.env.SOURCE_RESULT]
|
||||
: []),
|
||||
];
|
||||
const groupsCompleted = groupResults.every((state) =>
|
||||
["success", "failure"].includes(state),
|
||||
);
|
||||
const passed =
|
||||
complete && process.env.MERGE_RESULT === "success" && report.success;
|
||||
groupResults.every((state) => state === "success") &&
|
||||
process.env.MERGE_RESULT === "success" &&
|
||||
report.success;
|
||||
// Successful shards and merged reports must account for the complete portable
|
||||
// inventory. Coverage thresholds alone cannot prove that every file ran.
|
||||
const verifyInventory = process.env.CI_VERIFY_TEST_INVENTORY === "true";
|
||||
@@ -69,7 +80,7 @@ if (process.env.CI_TEST_SELECTION) {
|
||||
comparison.complete &&
|
||||
report.unhandledErrors === 0 &&
|
||||
["passed", "failed"].includes(report.reason) &&
|
||||
["success", "failure"].includes(process.env.SHARD_RESULT) &&
|
||||
groupsCompleted &&
|
||||
["success", "failure"].includes(process.env.MERGE_RESULT),
|
||||
executionSha: process.env.GITHUB_SHA ?? null,
|
||||
prHeadSha: process.env.CI_PR_HEAD_SHA ?? null,
|
||||
@@ -91,7 +102,9 @@ if (passed && verifyInventory) {
|
||||
);
|
||||
}
|
||||
const summary = [
|
||||
complete ? "## Merged test diagnostics" : "## Partial test diagnostics",
|
||||
groupsCompleted
|
||||
? "## Merged test diagnostics"
|
||||
: "## Partial test diagnostics",
|
||||
"",
|
||||
...(selection
|
||||
? [
|
||||
@@ -99,11 +112,14 @@ const summary = [
|
||||
"",
|
||||
`Plan: ${selection.plan.mode}. Proposed files: ${selection.selected.length}; omitted: ${selection.omitted.length}.`,
|
||||
`Full-run comparison: ${selection.complete ? "complete" : "incomplete"}; failures outside proposed selection: ${selection.missedFailures.length}.`,
|
||||
"The complete suite still ran. This report cannot enable omissions or qualify a revision. Skipped, missing, and interrupted observations do not establish parity.",
|
||||
"This observational report cannot enable omissions or qualify a revision. Skipped, missing, and interrupted observations do not establish parity.",
|
||||
"",
|
||||
]
|
||||
: []),
|
||||
`Shard result: ${process.env.SHARD_RESULT ?? "unknown"}; merge result: ${process.env.MERGE_RESULT ?? "unknown"}.`,
|
||||
...(process.env.SOURCE_REQUIRED === "true"
|
||||
? [`Source result: ${process.env.SOURCE_RESULT ?? "unknown"}.`]
|
||||
: []),
|
||||
passed
|
||||
? "Successful timing history is available."
|
||||
: "This report is diagnostic only and is not a successful timing baseline.",
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
export interface WorkflowPath {
|
||||
readonly file: string;
|
||||
readonly ref: string | null;
|
||||
}
|
||||
export interface NormalizedWorkflowRun {
|
||||
readonly workflowPath: WorkflowPath | null;
|
||||
}
|
||||
export function normalizeWorkflowRun<T extends Record<string, unknown>>(
|
||||
value: T,
|
||||
): Omit<T, "workflowPath"> & NormalizedWorkflowRun;
|
||||
@@ -0,0 +1,15 @@
|
||||
/** Keep the API path as evidence and parse its optional ref at the boundary. */
|
||||
export function normalizeWorkflowRun(value) {
|
||||
if (typeof value !== "object" || value === null || Array.isArray(value))
|
||||
throw new Error("workflow_run_malformed");
|
||||
const match =
|
||||
typeof value.path === "string" && !/[\u0000-\u001f\u007f]/u.test(value.path)
|
||||
? /^(\.github\/workflows\/[^/\\@\s]+\.ya?ml)(?:@([^\s]+))?$/u.exec(
|
||||
value.path,
|
||||
)
|
||||
: null;
|
||||
return {
|
||||
...value,
|
||||
workflowPath: match ? { file: match[1], ref: match[2] ?? null } : null,
|
||||
};
|
||||
}
|
||||
@@ -5,6 +5,7 @@ import { fileURLToPath } from "node:url";
|
||||
import { promisify } from "node:util";
|
||||
import { TraceMap, eachMapping } from "@jridgewell/trace-mapping";
|
||||
import { CdpBrowserProvider } from "../../dist/browser/CdpBrowserProvider.js";
|
||||
import { listBrowserTargetsInputSchema } from "../../dist/domain/browserObservation.js";
|
||||
import { CdpConnection } from "../../dist/browser/CdpConnection.js";
|
||||
import { parseEvidence } from "../../dist/domain/evidence.js";
|
||||
import { traceSourceMap } from "../../dist/javascript/sourceMaps/TraceSourceMap.js";
|
||||
@@ -89,10 +90,12 @@ async function verifyCase(executable, entrypoint, scenario) {
|
||||
fixture = await startRuntimeFixtureBrowser(executable, site.origin);
|
||||
const endpoint = `http://127.0.0.1:${fixture.port}`;
|
||||
const provider = new CdpBrowserProvider();
|
||||
const targets = await provider.listTargets({
|
||||
cdp_endpoint: endpoint,
|
||||
allowed_origins: [site.origin],
|
||||
});
|
||||
const targets = await provider.listTargets(
|
||||
listBrowserTargetsInputSchema.parse({
|
||||
cdp_endpoint: endpoint,
|
||||
allowed_origins: [site.origin],
|
||||
}),
|
||||
);
|
||||
if (!targets.ok) throw targets.error;
|
||||
const target = targets.value.targets[0];
|
||||
assert.ok(target, "Owned fixture target missing");
|
||||
|
||||
@@ -8,6 +8,7 @@ import { promisify } from "node:util";
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { CdpBrowserProvider } from "../../dist/browser/CdpBrowserProvider.js";
|
||||
import { listBrowserTargetsInputSchema } from "../../dist/domain/browserObservation.js";
|
||||
import { CdpConnection } from "../../dist/browser/CdpConnection.js";
|
||||
import { parseEvidence } from "../../dist/domain/evidence.js";
|
||||
import { webExecutionSchema } from "../../dist/domain/webExecution.js";
|
||||
@@ -45,10 +46,12 @@ export async function verifyBrowserRuntime(
|
||||
try {
|
||||
fixture = await startRuntimeFixtureBrowser(executable, site.origin);
|
||||
const endpoint = `http://127.0.0.1:${fixture.port}`;
|
||||
const targets = await new CdpBrowserProvider().listTargets({
|
||||
cdp_endpoint: endpoint,
|
||||
allowed_origins: [site.origin],
|
||||
});
|
||||
const targets = await new CdpBrowserProvider().listTargets(
|
||||
listBrowserTargetsInputSchema.parse({
|
||||
cdp_endpoint: endpoint,
|
||||
allowed_origins: [site.origin],
|
||||
}),
|
||||
);
|
||||
if (!targets.ok) throw targets.error;
|
||||
const target = targets.value.targets[0];
|
||||
assert.ok(target, "owned fixture page target missing");
|
||||
|
||||
@@ -19,6 +19,11 @@ export const toolFamilyCatalog = (sources) => {
|
||||
contracts: [
|
||||
...sources.nativeContracts.NATIVE_TOOL_CONTRACTS,
|
||||
...sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS,
|
||||
...sources.cutterContracts.CUTTER_TOOL_CONTRACTS,
|
||||
...sources.reverseEngineeringContracts
|
||||
.REVERSE_ENGINEERING_TOOL_CONTRACTS,
|
||||
...sources.gdbContracts.GDB_TOOL_CONTRACTS,
|
||||
...sources.rizinDebugContracts.RIZIN_DEBUG_TOOL_CONTRACTS,
|
||||
],
|
||||
},
|
||||
{
|
||||
@@ -164,6 +169,32 @@ export const providerCatalog = (sources) => {
|
||||
activeContracts,
|
||||
} = electronContractSlices(sources);
|
||||
return [
|
||||
{
|
||||
identity: sources.cutterProvider.CUTTER_PROVIDER_IDENTITY,
|
||||
contracts: sources.cutterContracts.CUTTER_TOOL_CONTRACTS,
|
||||
},
|
||||
{
|
||||
identity: sources.objdumpProvider.OBJDUMP_PROVIDER_IDENTITY,
|
||||
contracts:
|
||||
sources.reverseEngineeringContracts.REVERSE_ENGINEERING_TOOL_CONTRACTS.filter(
|
||||
({ name }) => name === "inspect_with_objdump",
|
||||
),
|
||||
},
|
||||
{
|
||||
identity: sources.rizinProvider.RIZIN_PROVIDER_IDENTITY,
|
||||
contracts:
|
||||
sources.reverseEngineeringContracts.REVERSE_ENGINEERING_TOOL_CONTRACTS.filter(
|
||||
({ name }) => name === "execute_rizin_command",
|
||||
),
|
||||
},
|
||||
{
|
||||
identity: sources.gdbProvider.GDB_PROVIDER_IDENTITY,
|
||||
contracts: sources.gdbContracts.GDB_TOOL_CONTRACTS,
|
||||
},
|
||||
{
|
||||
identity: sources.rizinDebugProvider.RIZIN_DEBUGGER_PROVIDER_IDENTITY,
|
||||
contracts: sources.rizinDebugContracts.RIZIN_DEBUG_TOOL_CONTRACTS,
|
||||
},
|
||||
{
|
||||
identity: sources.fridaProvider.FRIDA_PROVIDER_IDENTITY,
|
||||
contracts: sources.fridaContracts.FRIDA_TOOL_CONTRACTS,
|
||||
|
||||
@@ -43,6 +43,16 @@ const SOURCE_PATHS = {
|
||||
sessionContracts: "dist/contracts/sessionToolContracts.js",
|
||||
binaryDiagnosticsContracts:
|
||||
"dist/contracts/native/binaryDiagnosticsToolContracts.js",
|
||||
cutterContracts: "dist/contracts/cutterToolContracts.js",
|
||||
cutterProvider: "dist/cutter/CutterBridgeClient.js",
|
||||
reverseEngineeringContracts:
|
||||
"dist/contracts/reverseEngineeringToolContracts.js",
|
||||
gdbContracts: "dist/contracts/gdbToolContracts.js",
|
||||
gdbProvider: "dist/gdb/GdbSessionManager.js",
|
||||
rizinDebugContracts: "dist/contracts/rizinDebugToolContracts.js",
|
||||
objdumpProvider: "dist/objdump/ObjdumpCommand.js",
|
||||
rizinProvider: "dist/rizin/RizinCommand.js",
|
||||
rizinDebugProvider: "dist/rizin/RizinDebugSessionManager.js",
|
||||
fridaContracts: "dist/contracts/fridaToolContracts.js",
|
||||
fridaProvider: "dist/frida/FridaInstrumentationManager.js",
|
||||
pwntoolsProvider: "dist/native/pwntools/PwntoolsRelease.js",
|
||||
|
||||
@@ -1,17 +1,4 @@
|
||||
const sourceProjects = [
|
||||
"domain",
|
||||
"services",
|
||||
"adapters",
|
||||
"composition",
|
||||
"conformance",
|
||||
"evaluation",
|
||||
];
|
||||
|
||||
const sourceTest = (path) =>
|
||||
path.startsWith("src/") ||
|
||||
["composition", "conformance", "evaluation"].some((project) =>
|
||||
path.startsWith(`tests/${project}/`),
|
||||
);
|
||||
import { SOURCE_TEST_PROJECTS, projectForTest } from "./test-projects.mjs";
|
||||
|
||||
/** Parse explicit test paths and an optional branch-comparison base. */
|
||||
export const parseDevelopmentTestRequest = (mode, arguments_) => {
|
||||
@@ -35,10 +22,9 @@ export const parseDevelopmentTestRequest = (mode, arguments_) => {
|
||||
path.split("/").some((part) => part === ".." || part === ".")
|
||||
)
|
||||
throw new Error(`Expected an exact repository test path: ${argument}`);
|
||||
if (mode !== "focused" && !sourceTest(path))
|
||||
throw new Error(
|
||||
`Use test:focused for compiled boundary tests: ${path}`,
|
||||
);
|
||||
const project = projectForTest(path);
|
||||
if (mode !== "focused" && project.group !== "source")
|
||||
throw new Error(`Use test:focused for built test projects: ${path}`);
|
||||
paths.push(path);
|
||||
}
|
||||
}
|
||||
@@ -72,9 +58,14 @@ export const developmentTestPlan = (request, baseCommit) => {
|
||||
!/^[a-f0-9]{40,64}$/u.test(baseCommit ?? "")
|
||||
)
|
||||
throw new Error("Changed tests require a resolved Git merge base");
|
||||
const projects = request.mode === "focused" ? [] : sourceProjects;
|
||||
const projects =
|
||||
request.mode === "focused"
|
||||
? []
|
||||
: SOURCE_TEST_PROJECTS.map(({ name }) => name);
|
||||
return {
|
||||
needsBuild: request.paths.some((path) => !sourceTest(path)),
|
||||
needsBuild: request.paths.some(
|
||||
(path) => projectForTest(path).group === "built",
|
||||
),
|
||||
artifactTasks: Object.entries(artifactConsumers).flatMap(
|
||||
([task, consumers]) =>
|
||||
request.paths.some((path) => consumers.includes(path)) ? [task] : [],
|
||||
|
||||
@@ -235,7 +235,11 @@ async function verifyNativeValueAdapters(
|
||||
);
|
||||
assert.notEqual(comparison.isError, true, JSON.stringify(comparison));
|
||||
const comparisonEvidence = parseEvidence(comparison.structuredContent);
|
||||
assert.equal(comparisonEvidence.normalized_result.status, "unchanged");
|
||||
assert.equal(
|
||||
comparisonEvidence.normalized_result.status,
|
||||
"unchanged",
|
||||
JSON.stringify(comparisonEvidence.normalized_result.dimensions),
|
||||
);
|
||||
assert.deepEqual(
|
||||
comparisonEvidence.normalized_result,
|
||||
comparison.structuredContent?.normalized_result,
|
||||
|
||||
@@ -131,6 +131,25 @@ export async function verifyGhidraBoundaries(
|
||||
const externalDossier = await call("analyze_function", {
|
||||
procedure: external.address,
|
||||
});
|
||||
assert.equal(externalDossier.pseudocode, "");
|
||||
assert.deepEqual(externalDossier.assembly, []);
|
||||
assert.deepEqual(externalDossier.callees, []);
|
||||
assert.deepEqual(externalDossier.facet_availability.pseudocode, {
|
||||
available: false,
|
||||
reason: "Ghidra did not supply pseudocode.",
|
||||
});
|
||||
for (const facet of [
|
||||
"assembly",
|
||||
"comments",
|
||||
"callers",
|
||||
"callees",
|
||||
"references",
|
||||
"strings_names",
|
||||
"cfg",
|
||||
])
|
||||
assert.deepEqual(externalDossier.facet_availability[facet], {
|
||||
available: true,
|
||||
});
|
||||
const externalEntry = await call("resolve_containing_procedure", {
|
||||
address: external.address,
|
||||
});
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user