Compare commits

..
Author SHA1 Message Date
Kaoru0822andClaude Opus 5.5 110151e523 fix(capture): keep URLs that WHATWG rejects instead of redacting userinfo
File URLs cannot carry userinfo (an "@" in a file host fails the parse),
and a special authority whose host or port WHATWG rejects fails the whole
URL. Neither has credentials, so the bridge now keeps their source text
and byte sidecar. Checked against Node's URL over 30,000 spellings: no
under-redaction, over-redaction 20,540 -> 5,865 (remaining: empty
userinfo, IPv4 range, IDNA, non-special and baseless references).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 00:26:29 +00:00
Kaoru0822 b93e585d68 Merge remote-tracking branch 'origin/main' into fix/issue-2671 2026-10-14 00:22:25 +00:00
Xiangfang ChenandGitHub e049ef70f7 feat(setup): add WorkBuddy client support (#3041)
* feat(setup): add WorkBuddy client support

Register rea in the global ~/.workbuddy/mcp.json through a new workbuddy
entry in SUPPORTED_CLIENT_DEFINITIONS (json dialect, ~/.workbuddy marker,
~/.workbuddy/skills skill directory; WorkBuddy AI shares the file).
Document the client, extend the unreleased-client note, and cover
registration and uninstall with boundary tests.

* docs(setup): record verified WorkBuddy tool calls

Real-client verification completed: WorkBuddy loads the registration and
real REA tool calls execute through the registered server.
2026-10-14 08:12:33 +08:00
bbf0638e88 fix(javascript): report EventSource and sendBeacon URLs as network endpoints (#3044)
Co-authored-by: xpeng5278-web <330743750+xpeng5278-web@users.noreply.github.com>
2026-10-14 08:10:04 +08:00
242e2b0001 fix(ghidra): preserve function facet and call coverage (#3002)
* fix(ghidra): preserve observed function facet and call coverage

Declare successful query availability in the Java producer, including explicit unavailable pseudocode reasons. Carry observed static call-set coverage through the parsed provider result and Evidence without inventing facts for legacy omissions. Use Ghidra indexed comment addresses over the function body and entry, retaining cancellation checks and sparse collection.

Closes #2951

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>

* fix(ghidra): retain unresolved call coverage in observations

Share decoded unresolved-call facts between dossiers, callee coverage and reference observations. Combine terminal and unresolved observations in one instruction traversal and retain site records only when requested. Keep missing procedure identities incomplete and validate unresolved locations at the provider boundary.

Bind snapshots to the corrected call coverage profile so older complete claims cannot replay under the new semantics. Add real public MCP callback trace/comparison regressions.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>

* fix(ghidra): separate reference targets from callee coverage

Reuse containing-function ownership for resolved reference identities and independently compare those identities with the actual returned callee set. Keep known interior routes out of unresolved site records while retaining partial inventory coverage.

Advance the coverage profile to v3 and reject interim v2 cache bindings. Add a real Ghidra producer regression with restored reference metadata; the previous bridge fails the known-route assertion.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>

* fix(ghidra): measure incoming call coverage across function bodies

Compare incoming reference source identities across the owned function body and entry with the actual returned caller set. Stream caller-only checks and reuse dossier reference observations. Combine caller/callee coverage from observed counts without reinterpreting serialized metadata, and share the body scope owner with comments and references.

Bind snapshots to function-body coverage semantics v4 and preserve the real off-entry caller regression.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>

* fix(ghidra): preserve whole-scope directional call coverage

Check body-and-entry call references independently of decoded instruction flow. Reuse collected dossier references and stream inventory-only coverage checks. Keep caller-only gaps distinct from incomplete outgoing coverage in offline path reconstruction, and invalidate prior snapshot profiles.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>

* fix(mcp): retain compact Evidence field types

Share optional diagnostics across the three tagged relationship coverage states instead of advertising duplicate branches. Preserve coverage validation while keeping compact Evidence properties within the existing reference expansion budget. Verify invalid direction cardinalities against the actual SDK-advertised schema.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>

---------

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-14 07:52:14 +08:00
morluto 929aa67336 chore(main): merge pinned CI checkpoint 2895a18 2026-10-14 03:12:47 +08:00
morluto 4e8475e965 test(ci): isolate alternate launcher process fixture 2026-10-14 03:02:01 +08:00
morluto 557148ed3f fix(ci): declare process inventory callback 2026-10-14 02:25:19 +08:00
morluto f839046776 chore(merge): refresh PR #2767 with latest main 2026-10-14 02:23:57 +08:00
Morluto 281c4b8ec5 test(ci): isolate Hopper process inventory fixtures 2026-10-14 02:15:33 +08:00
Morluto a0da18d7fa test(capture): cover empty URL userinfo markers 2026-10-14 02:14:51 +08:00
Morluto 2c6e074cac fix(capture): remove empty URL userinfo markers 2026-10-14 02:14:51 +08:00
morluto 29c871e51b chore(merge): refresh PR #2767 with main 2026-10-14 02:10:19 +08:00
REA MaintenanceandKaoru0822 c0018ca403 fix(ci): isolate Linux Hopper process fixture inventory 2026-10-14 01:23:30 +08:00
morlutoandKaoru0822 ad16d250e3 fix(android): make DEX descriptor pattern portable 2026-10-14 01:23:25 +08:00
Lucienandmorluto af2f55cda9 fix(comparison): recheck historical calls-only changes (#2223)
* fix(comparison): preserve incomplete function call evidence

Keep missing, unknown, partial and unresolved call relationships unknown in function comparisons and reconstruction claims. Aggregate IDA caller and callee coverage without overwriting uncertainty.

Validate with 144 focused tests, typecheck/lint, docs checks and test discovery. Real providers were not verified; the full headless IDA session suite requires unavailable Windows native controls.

* fix(comparison): recheck historical calls-only changes

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-14 01:23:19 +08:00
morluto 4d5d1f5daa fix(capture): preserve file-relative path evidence after separators 2026-10-13 23:02:00 +08:00
morluto 0aff997b31 fix(capture): distinguish native paths and URL base authorities 2026-10-13 21:35:04 +08:00
morluto e8361b7715 fix(capture): parse URL controls and mixed network authorities 2026-10-13 20:31:54 +08:00
morluto 72e931f3a3 chore(ci): retain contributor updates alongside verification fixes 2026-10-13 20:21:10 +08:00
morluto 0266e6552b fix(capture): preserve same-scheme relative redirect paths 2026-10-13 20:19:00 +08:00
Kaoru0822 7d972a7e85 Merge remote-tracking branch 'origin/main' into fix/issue-2671 2026-10-13 12:04:14 +00:00
Kaoru0822andClaude Opus 5.5 e1fb40460d fix(capture): skip special-scheme leading slashes before userinfo
Ending the authority at a backslash exposed a credential: for
`https://\user:pass@host/path`, WHATWG parsing skips every leading "/"
or "\" after a special scheme (and in scheme-relative references), so
`user:pass` is userinfo, but the new boundary stopped at the first
backslash and found no "@". The same applied to `https:\\user:pass@host`
and `https:user:pass@host`.

Start the authority after that whole slash run for non-file special
schemes and scheme-relative references. File URLs keep their "//"
boundary because they carry no userinfo. Load the adapter in tests with
fresh mitmproxy stand-ins scoped to its import instead of leaving them
in sys.modules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 11:25:05 +00:00
Kaoru0822andClaude Opus 5.5 553cd6e358 fix(capture): end special-scheme URL authorities at a backslash
The native mitmproxy adapter removed transport userinfo with
`^(...//)[^/?#]*@`, so the authority ran to the last "@" before "/", "?"
or "#". For `https://example.test\part@tail` it treated
`example.test\part` as userinfo, reported `https://tail` as the
Location, and withdrew the original bytes as a transport credential.
WHATWG URL parsing, used by the shared TypeScript owner
(recordedTransportUrl.ts) and by HAR captures, ends a special-scheme or
scheme-relative authority at a backslash, so that "@" is ordinary path
text.

Locate the authority with the same boundary rule before removing
userinfo. Bytes are handled as Latin-1 text so unrelated bytes stay
identical. Userinfo before the backslash is still removed and redacted.
Add source unit tests for the adapter projection to test:python.

Fixes #2671

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 11:06:25 +00:00
79 changed files with 2065 additions and 5173 deletions
-21
View File
@@ -1,21 +0,0 @@
{
"name": "rea",
"interface": {
"displayName": "REA"
},
"plugins": [
{
"name": "rea",
"source": {
"source": "npm",
"package": "rea-agents",
"version": "6.4.0"
},
"policy": {
"installation": "AVAILABLE",
"authentication": "ON_INSTALL"
},
"category": "Developer Tools"
}
]
}
@@ -17,9 +17,6 @@ readiness or provider commands.
Installing this skill supplies instructions; it does not register the REA MCP
server, install analysis engines, or add tools to an already-running agent.
The exception is a Claude Code or Codex plugin install: the plugin bundles
this skill and registers its own `rea` server, outside the state that
`rea setup` and `rea doctor` manage.
If REA tools are available and their registration is not known to be stale,
proceed directly to the target. Do not run diagnostics or setup before every
@@ -13,7 +13,7 @@ main and an older release expose the same arguments.
| Observation | Meaning and next step |
| --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| No REA tools in the active session | If REA was installed as a client plugin, follow the plugin section below. Otherwise inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
| No REA tools in the active session | Inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
| Missing, malformed or stale registration | Prepare a scoped setup plan; apply only authorized changes with backups. |
| Aligned registration, but no connected tools | Restart/reconnect the client. Doctor checks registration; it cannot establish the current agent's connection. Inspect the client's launch error if reconnection fails. |
| `MCP tool call requires approval, but approval policy is never` | Codex rejected the call before REA executed it. Do not install an engine or repeat setup to solve this client-policy decision. See the approval section below. |
@@ -34,33 +34,6 @@ answers part of the question, and name the unavailable evidence separately.
Do not substitute another target. Label observations from ordinary shell tools
as such; only actual REA output supplies REA Evidence.
## REA installed as a client plugin
A Claude Code or Codex plugin owns its `rea` server and this skill. Doctor's
`--client` and `--skill` checks and `rea setup` read and write only
setup-owned state, so for a plugin-only install they report `missing`, and
applying setup would add a second `rea` server. If this skill was loaded from
a plugin, or the client lists a `rea` plugin, do not prepare a setup plan for
missing tools. Instead:
- Check that the plugin is installed and enabled with the client's own
listing: `claude plugin list` and `/mcp` in Claude Code, `codex plugin list`
in Codex.
- Restart or reconnect the client, then read its launch error for the `rea`
server if tools are still absent. The first start downloads the pinned
package through `npx` and can exceed the client's startup timeout; a second
start uses the cache.
- Check analysis engines through the connected server's
`binary_session.result.tool_availability`. `rea doctor --provider <id> --json`
from a shell reports that shell's environment, which can differ from the
environment the client gave the plugin server. If you still run it through
the package runner, use the version pinned in the plugin manifest's server
entry (`rea-agents@<version>`), not `@latest`, so it matches the server the
plugin starts.
Use the setup repair below only when the user wants a setup-owned registration
instead of the plugin, and remove or disable the plugin first.
## Repair registration when needed
For Codex:
-21
View File
@@ -1,21 +0,0 @@
{
"$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
"name": "rea",
"description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
"owner": {
"name": "morluto",
"url": "https://github.com/morluto"
},
"plugins": [
{
"name": "rea",
"description": "Reverse engineer anything from your terminal or agent with one CLI and MCP server.",
"source": {
"source": "npm",
"package": "rea-agents",
"version": "6.4.0"
},
"category": "development"
}
]
}
-28
View File
@@ -1,28 +0,0 @@
{
"name": "rea",
"version": "6.4.0",
"description": "Reverse engineer anything from your terminal or agent with one CLI and MCP server.",
"author": {
"name": "morluto",
"url": "https://github.com/morluto"
},
"homepage": "https://rea.tools/",
"repository": "https://github.com/morluto/rea",
"license": "MIT",
"keywords": [
"reverse-engineering",
"binary-analysis",
"decompiler",
"mcp-server"
],
"skills": "./skills",
"mcpServers": {
"rea": {
"command": "node",
"args": [
"${CLAUDE_PLUGIN_ROOT}/scripts/plugin-mcp-launcher.mjs",
"6.4.0"
]
}
}
}
-38
View File
@@ -1,38 +0,0 @@
{
"name": "rea",
"version": "6.4.0",
"description": "Reverse engineer anything from your terminal or agent with one CLI and MCP server.",
"author": {
"name": "morluto",
"url": "https://github.com/morluto"
},
"homepage": "https://rea.tools/",
"repository": "https://github.com/morluto/rea",
"license": "MIT",
"keywords": [
"reverse-engineering",
"binary-analysis",
"decompiler",
"mcp-server"
],
"skills": "./skills",
"mcpServers": "./.mcp.json",
"interface": {
"displayName": "REA",
"shortDescription": "Reverse engineer anything with agents",
"longDescription": "Use REA to explain shipped behavior from native binaries, managed assemblies, Electron and JavaScript bundles, packages, firmware, and browser targets, with evidence that separates observations from inferences.",
"developerName": "morluto",
"category": "Developer Tools",
"capabilities": [
"Interactive",
"Read",
"Write"
],
"websiteURL": "https://rea.tools/",
"defaultPrompt": [
"Use REA to explain how this binary implements a feature.",
"Use REA to compare two versions of this application.",
"Use REA to recover the sources from this JavaScript bundle."
]
}
}
+1 -1
View File
@@ -14,7 +14,7 @@ runs:
shell: bash
env:
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
LOCKFILES_HASH: ${{ hashFiles('npm-shrinkwrap.json', '.npmrc') }}
LOCKFILES_HASH: ${{ hashFiles('package-lock.json', 'npm-shrinkwrap.json', '.npmrc') }}
run: |
cache_path="$(npm config get cache)"
runtime_arch="$(node -p 'process.arch')"
@@ -21,7 +21,7 @@ runs:
if: inputs.build_cache_artifact == ''
shell: bash
env:
BUILD_CACHE_PREFIX: rea-portable-build-v2-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.npmrc', '.nvmrc', 'npm-shrinkwrap.json') }}
BUILD_CACHE_PREFIX: rea-portable-build-v2-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.npmrc', '.nvmrc', 'package-lock.json') }}
run: |
npm run --silent test:prepare -- --dry=json |
node --input-type=module -e '
+2 -2
View File
@@ -133,9 +133,9 @@ jobs:
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: .turbo/cache/
key: rea-static-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.nvmrc', '.npmrc', 'npm-shrinkwrap.json') }}-${{ github.sha }}
key: rea-static-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.nvmrc', '.npmrc', 'package-lock.json') }}-${{ github.sha }}
# Turbo validates each task's inputs after restoring this snapshot.
restore-keys: rea-static-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.nvmrc', '.npmrc', 'npm-shrinkwrap.json') }}-
restore-keys: rea-static-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.nvmrc', '.npmrc', 'package-lock.json') }}-
- name: Validate workflow syntax and expressions
if: needs.changes.outputs.workflows == 'true'
run: go run github.com/rhysd/actionlint/cmd/actionlint@v1.7.7 -shellcheck= -pyflakes=
+1 -1
View File
@@ -25,7 +25,7 @@ on:
- "scripts/lib/mcp-verifier-results.mjs"
- "scripts/rea.mjs"
- "package.json"
- "npm-shrinkwrap.json"
- "package-lock.json"
- ".nvmrc"
permissions:
-10
View File
@@ -1,10 +0,0 @@
{
"mcpServers": {
"rea": {
"command": "node",
"args": ["./scripts/plugin-mcp-launcher.mjs", "6.4.0"],
"cwd": ".",
"startup_timeout_sec": 30
}
}
}
+1 -3
View File
@@ -9,9 +9,7 @@
"docs/api/**",
"node_modules/**",
"CHANGELOG.md",
"npm-shrinkwrap.json",
".claude-plugin/plugin.json",
".codex-plugin/plugin.json",
"package-lock.json",
"_reference/**",
"third_party/**"
]
-3
View File
@@ -131,9 +131,6 @@ npm ci
npm run check:fast
```
`npm-shrinkwrap.json` is the sole dependency lock for development and the
published plugins. Do not add a root `package-lock.json`; npm would ignore it.
`npm ci` installs the exact dependencies and prepares the Husky hooks without
building the project. Run `npm run build:cached` when you need the standalone
CLI or MCP server; it compiles the runtime and bundles the authored skill.
+1 -1
View File
@@ -15,7 +15,7 @@ notices supplied with the software you actually obtain or redistribute.
| Material | Use and licensing records |
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| npm dependencies | [Complete inventory](licenses/npm-dependencies.txt) of every entry in `npm-shrinkwrap.json`, including transitive, optional platform, development, and website dependencies. [Runtime notices](licenses/npm-runtime-notices.txt) preserve the copyright, license, and NOTICE documents from integrity-verified release archives, plus the MIT notice for Rolldown's generated runtime helpers. Development dependencies are inventoried but are not all distributed in REA's npm package. |
| npm dependencies | [Complete inventory](licenses/npm-dependencies.txt) of every entry in `package-lock.json`, including transitive, optional platform, development, and website dependencies. [Runtime notices](licenses/npm-runtime-notices.txt) preserve the copyright, license, and NOTICE documents from integrity-verified release archives, plus the MIT notice for Rolldown's generated runtime helpers. Development dependencies are inventoried but are not all distributed in REA's npm package. |
| NativeAOT parser | REA's derived parser uses the MIT-licensed [ghidra-nativeaot](https://github.com/Washi1337/ghidra-nativeaot/tree/effeb734fc570c32650f88b159608979dc7b423e) work of Washi and the NativeAOT layout definitions in [dotnet/runtime](https://github.com/dotnet/runtime/tree/a3b2d40328be0be3f8dd43f0e5cc925b8827b044). Preserve [Washi's MIT notice](licenses/nativeaot-parser-MIT.txt) and the [.NET Foundation MIT notice](licenses/dotnet-runtime-MIT.txt). |
| EVM analysis | `evmole@0.9.3`, MIT, copyright 2023 Maxim Andreev. The published npm archive omits its license document; REA retains the [original upstream license](third_party/evmole/LICENSE) and [provenance](third_party/evmole/README.md). It is also included in the runtime notices. |
| Windows native addon | The release addon is compiled with MinGW-w64 and statically links GCC runtime libraries. Preserve the [MinGW-w64 runtime notices](licenses/MinGW-w64-runtime.txt), [GPLv3 text](licenses/GPL-3.0.txt), and [GCC Runtime Library Exception 3.1](licenses/GCC-Runtime-Library-Exception-3.1.txt). The exception governs eligible compiled combinations; it does not relicense the runtime itself under MIT. The Node-API build headers are from Node 22.19.0; its [original license and component notices](licenses/Node-22.19.0-LICENSE.txt) are retained. Node itself is supplied by the user. Snapshot provenance and release-toolchain review requirements are in [licenses/README.md](licenses/README.md). |
+282 -97
View File
@@ -81,6 +81,7 @@ import ghidra.program.model.listing.Function;
import ghidra.program.model.listing.FunctionIterator;
import ghidra.program.model.listing.Instruction;
import ghidra.program.model.listing.InstructionIterator;
import ghidra.program.model.listing.Listing;
import ghidra.program.model.listing.Variable;
import ghidra.program.model.mem.MemoryBlock;
import ghidra.program.model.mem.MemoryBlockSourceInfo;
@@ -405,7 +406,9 @@ public final class ReaGhidraBridge extends HeadlessScript {
case "xrefs" -> xrefs(request.params);
case "analyze_function" -> {
FunctionAnalysisObservation observation = analyzeFunctionObservation(request.params);
yield functionObservation(observation.dossier(), observation.limitations());
JsonObject result = functionObservation(observation.dossier(), observation.limitations());
result.add("relationship_coverage", observation.relationshipCoverage());
yield result;
}
case "set_address_name", "set_addresses_names" -> {
if (!descriptor.transport.equals("unix-socket"))
@@ -1016,12 +1019,19 @@ public final class ReaGhidraBridge extends HeadlessScript {
for (Function item : ordered) {
result.add(canonicalAddress(item.getEntryPoint()));
}
return functionObservation(
result,
noReturnCallLimitations(
if (callers) {
JsonObject observation = functionObservation(result, noReturnCallLimitations(
currentProgram.getListing().getInstructions(function.getBody(), true)
)
));
observation.add("relationship_coverage", callRelationshipCoverage(null,
unrepresentedIncomingCalls(function, observed)));
monitor.checkCancelled();
return observation;
}
CallBoundaryObservation calls = callBoundaryObservation(
function, currentProgram.getListing().getInstructions(function.getBody(), true), observed, false, null
);
return callRelationshipObservation(result, calls);
}
private JsonObject procedureInfo(JsonObject params) throws Exception {
@@ -1071,18 +1081,10 @@ public final class ReaGhidraBridge extends HeadlessScript {
result.addProperty("direction", direction);
result.add("references", edges);
result.addProperty("reference_kinds_available", true);
JsonArray unresolvedCalls = new JsonArray();
if (direction.equals("outgoing")) for (Instruction instruction : scanInstructions(function, Integer.MAX_VALUE).instructions) {
if (!instruction.getFlowType().isCall()) continue;
boolean resolved = false;
for (Reference reference : instruction.getReferencesFrom()) if (reference.getReferenceType().isCall()) resolved = true;
if (!resolved) {
JsonObject site = new JsonObject();
site.addProperty("address", canonicalAddress(instruction.getAddress()));
site.addProperty("reason", "Instruction decoder identifies a call but ReferenceManager supplies no call destination");
unresolvedCalls.add(site);
}
}
JsonArray unresolvedCalls = direction.equals("outgoing")
? callBoundaryObservation(
function, currentProgram.getListing().getInstructions(function.getBody(), true), null, true, references
).unresolvedCalls() : new JsonArray();
result.add("unresolved_calls", unresolvedCalls);
return result;
}
@@ -1490,16 +1492,28 @@ public final class ReaGhidraBridge extends HeadlessScript {
InstructionScan scan = scanInstructions(function, Integer.MAX_VALUE);
DecompileResults decompilation = decompile(function);
String pseudocode = decompiledText(decompilation);
List<Reference> incomingReferences = collectReferences(function, "incoming")
.stream()
JsonObject availability = new JsonObject();
availability.add("pseudocode", facetAvailability(
pseudocode != null,
function.isExternal() ? "Ghidra does not decompile external functions."
: function.getBody().isEmpty() ? "Ghidra has no function body to decompile."
: "Ghidra completed decompilation without returning pseudocode."
));
if (pseudocode == null) {
pseudocode = "";
}
List<Reference> bodyIncoming = collectReferences(function, "incoming");
List<Reference> incomingReferences = bodyIncoming.stream()
.filter(reference -> !function.getBody().contains(reference.getFromAddress()))
.toList();
List<Reference> outgoingReferences = collectReferences(function, "outgoing");
JsonArray incoming = referenceEdges(incomingReferences);
JsonArray outgoing = referenceEdges(outgoingReferences);
JsonArray comments = comments(scan.instructions);
JsonArray callers = procedureIdentities(function.getCallingFunctions(monitor));
JsonArray callees = procedureIdentities(function.getCalledFunctions(monitor));
JsonArray comments = comments(function);
Set<Function> callerFunctions = function.getCallingFunctions(monitor);
JsonArray callers = procedureIdentities(callerFunctions);
Set<Function> calleeFunctions = function.getCalledFunctions(monitor);
JsonArray callees = procedureIdentities(calleeFunctions);
JsonArray referencedStrings = referencedStrings(outgoingReferences);
JsonArray referencedNames = referencedNames(outgoingReferences);
JsonArray blocks = basicBlockValues(function);
@@ -1511,7 +1525,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
procedure.add("locals", functionLocals(function));
JsonObject result = new JsonObject();
result.add("procedure", procedure);
result.addProperty("pseudocode", pseudocode == null ? "" : pseudocode);
result.addProperty("pseudocode", pseudocode);
result.add("assembly", assembly);
result.add("comments", comments);
result.add("callers", callers);
@@ -1520,20 +1534,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
result.add("outgoing_references", outgoing);
result.add("referenced_strings", referencedStrings);
result.add("referenced_names", referencedNames);
JsonObject facets = new JsonObject();
JsonObject pseudocodeAvailability = new JsonObject();
pseudocodeAvailability.addProperty("available", pseudocode != null);
if (pseudocode == null) {
pseudocodeAvailability.addProperty("reason", "Ghidra did not supply pseudocode.");
}
facets.add("pseudocode", pseudocodeAvailability);
for (String facet : List.of("assembly", "comments", "callers", "callees",
"references", "strings_names", "cfg")) {
JsonObject available = new JsonObject();
available.addProperty("available", true);
facets.add(facet, available);
}
result.add("facet_availability", facets);
boolean swift = swiftProgram();
boolean aarch64 = currentProgram.getLanguageID().getIdAsString().startsWith("AARCH64:");
SwiftStringLiteralDecode swiftLiterals = swift && aarch64 ? swiftStringLiterals(function) : null;
@@ -1547,6 +1547,15 @@ public final class ReaGhidraBridge extends HeadlessScript {
nativeApiBoundary(function, decompilation)
);
result.add("native_value_flow", nativeValueFlow(decompilation));
// Successful full queries distinguish collected empty sets from missing facts.
// Availability does not claim that Ghidra resolved every runtime relationship.
for (String facet : List.of("comments", "callers", "callees", "references", "strings_names", "cfg")) {
availability.add(facet, facetAvailability(true, null));
}
availability.add("assembly", facetAvailability(
!scan.truncated(), "The function instruction scan was truncated."
));
result.add("facet_availability", availability);
JsonArray limitations = new JsonArray();
limitations.add(
"Unresolved computed or indirect flows without target addresses are not represented as reference edges."
@@ -1574,8 +1583,10 @@ public final class ReaGhidraBridge extends HeadlessScript {
"Swift String literal decoding is implemented for AArch64 register pairs only; no Swift literals were decoded for this architecture."
);
}
List<String> boundaryLimitations = noReturnCallLimitations(scan.instructions);
for (String limitation : boundaryLimitations) {
CallBoundaryObservation calls = callBoundaryObservation(
function, scan.instructions, calleeFunctions, true, outgoingReferences);
result.add("unresolved_calls", calls.unresolvedCalls());
for (String limitation : calls.limitations()) {
limitations.add(limitation);
}
if (decompilation != null && decompilation.getErrorMessage() != null &&
@@ -1583,7 +1594,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
limitations.add("Ghidra decompiler diagnostic: " + decompilation.getErrorMessage());
}
result.add("limitations", limitations);
return new FunctionAnalysisObservation(result, boundaryLimitations);
monitor.checkCancelled();
int missingCallers = 0;
for (Reference reference : bodyIncoming) {
monitor.checkCancelled();
if (unrepresentedCaller(reference, callerFunctions)) missingCallers++;
}
return new FunctionAnalysisObservation(result, calls.limitations(),
callRelationshipCoverage(calls, missingCallers));
}
private JsonObject nativeValueFlow(DecompileResults decompilation) throws Exception {
@@ -3287,54 +3305,202 @@ public final class ReaGhidraBridge extends HeadlessScript {
return result;
}
private static JsonObject facetAvailability(boolean available, String reason) {
JsonObject result = new JsonObject();
result.addProperty("available", available);
if (!available) result.addProperty("reason", reason);
return result;
}
private JsonObject callRelationshipObservation(JsonElement value, CallBoundaryObservation calls)
throws Exception {
monitor.checkCancelled();
JsonObject result = functionObservation(value, calls.limitations());
result.add("relationship_coverage", callRelationshipCoverage(calls, null));
return result;
}
private static JsonObject callRelationshipCoverage(
CallBoundaryObservation calls, Integer missingCallers) {
JsonObject coverage = completeResolvedCallCoverage();
Integer omitted = calls == null ? null : calls.unrepresentedCalleeCalls();
JsonArray directions = new JsonArray();
if (missingCallers != null && missingCallers > 0) directions.add("callers");
String callerLimitation = missingCallers != null && missingCallers > 0
? "Ghidra's caller inventory omits sources of " + missingCallers +
" call reference(s) into this function's body." : null;
if ((calls != null && omitted == null) || (calls == null && missingCallers == null)) {
coverage.addProperty("status", "unknown");
String reason = calls == null ? "No call inventory was examined."
: "The callee inventory was not examined alongside decoded call targets.";
coverage.addProperty("reason", reason + (callerLimitation == null ? "" : " " + callerLimitation));
if (calls == null && missingCallers == null) directions.add("callers");
directions.add("callees");
coverage.add("limited_directions", directions);
return coverage;
}
JsonArray limitations = new JsonArray();
if (callerLimitation != null) limitations.add(callerLimitation);
if (omitted != null && omitted > 0) {
limitations.add("Ghidra's callee inventory omits targets of " + omitted +
" call reference(s) or decoded call(s) without destinations.");
directions.add("callees");
}
if (!limitations.isEmpty()) {
coverage.addProperty("status", "partial");
coverage.add("limitations", limitations);
coverage.add("limited_directions", directions);
}
return coverage;
}
private static JsonObject completeResolvedCallCoverage() {
JsonObject coverage = new JsonObject();
coverage.addProperty("status", "complete");
JsonArray diagnostics = new JsonArray();
diagnostics.add(
"Only resolved Ghidra call references are returned; unresolved computed or indirect calls remain unknown, while function classifications distinguish thunks and externals."
);
coverage.add("diagnostics", diagnostics);
return coverage;
}
private boolean unrepresentedCaller(Reference reference, Set<Function> callers) {
if (!reference.getReferenceType().isCall()) return false;
Function caller = containingFunction(reference.getFromAddress());
return caller == null || !callers.contains(caller);
}
private int unrepresentedIncomingCalls(Function function, Set<Function> callers)
throws Exception {
int[] omitted = { 0 };
visitReferences(function, "incoming", reference -> {
if (unrepresentedCaller(reference, callers)) omitted[0]++;
});
return omitted[0];
}
private List<String> noReturnCallLimitations(Iterable<Instruction> instructions)
throws Exception {
List<String> limitations = new ArrayList<>();
for (Instruction instruction : instructions) {
monitor.checkCancelled();
if (!instruction.getFlowType().isCall() || !instruction.getFlowType().isTerminal()) {
continue;
}
Address[] destinations = instruction.getFlows();
StringBuilder limitation = new StringBuilder();
limitation.append("Ghidra Listing reports a terminal call at ")
.append(canonicalAddress(instruction.getAddress()));
if (destinations.length == 0) {
limitation.append(" with no resolved callee; its no-return basis is unknown.");
}
else {
limitation.append(" to ");
for (int index = 0; index < destinations.length; index++) {
if (index > 0) limitation.append(", ");
Address destination = destinations[index];
limitation.append(canonicalAddress(destination));
Function target = currentProgram.getFunctionManager().getFunctionAt(destination);
if (target == null) {
limitation.append(" (callee identity unknown)");
}
else {
limitation.append(" ").append(procedureName(target));
limitation.append(" (external=").append(target.isExternal());
limitation.append(", thunk=").append(target.isThunk());
limitation.append(", hasNoReturn=").append(target.hasNoReturn()).append(")");
}
}
limitation.append("; fallthrough is excluded by Ghidra's flow model.");
Function target = destinations.length == 1
? currentProgram.getFunctionManager().getFunctionAt(destinations[0])
: null;
if (target != null && target.hasNoReturn()) {
limitation.append(" This records Ghidra's FunctionManager flag, not an independent verification that the callee cannot return.");
}
else {
limitation.append(" The reason for the terminal flow is not established by this observation.");
}
}
limitations.add(limitation.toString());
String limitation = terminalCallLimitation(instruction);
if (limitation != null) limitations.add(limitation);
}
return limitations;
}
private CallBoundaryObservation callBoundaryObservation(
Function function, Iterable<Instruction> instructions, Set<Function> returnedCallees,
boolean includeUnresolvedSites, List<Reference> observedReferences)
throws Exception {
List<String> limitations = new ArrayList<>();
CallBoundaryReferences calls = new CallBoundaryReferences(returnedCallees, includeUnresolvedSites);
if (observedReferences == null) visitReferences(function, "outgoing", calls::observe);
else for (Reference reference : observedReferences) {
monitor.checkCancelled();
calls.observe(reference);
}
for (Instruction instruction : instructions) {
monitor.checkCancelled();
String limitation = terminalCallLimitation(instruction);
if (limitation != null) limitations.add(limitation);
if (!instruction.getFlowType().isCall()) continue;
boolean destination = false;
for (Reference reference : instruction.getReferencesFrom()) {
if (!reference.getReferenceType().isCall()) continue;
destination = true;
break;
}
if (!destination) {
calls.unrepresented++;
calls.unresolved(instruction.getAddress(),
"Instruction decoder identifies a call but ReferenceManager supplies no call destination");
}
}
return new CallBoundaryObservation(List.copyOf(limitations),
returnedCallees == null ? null : calls.unrepresented, calls.unresolvedCalls());
}
private final class CallBoundaryReferences {
private final Set<Function> returnedCallees;
private final TreeMap<Address, String> unresolved;
private int unrepresented;
CallBoundaryReferences(Set<Function> returnedCallees, boolean includeUnresolvedSites) {
this.returnedCallees = returnedCallees;
unresolved = includeUnresolvedSites ? new TreeMap<>() : null;
}
void observe(Reference reference) {
if (!reference.getReferenceType().isCall()) return;
// Reference identity follows the same containing-function owner as referenceEdge.
Function target = containingFunction(reference.getToAddress());
if (returnedCallees != null && (target == null || !returnedCallees.contains(target)))
unrepresented++;
if (target == null) unresolved(reference.getFromAddress(),
"A call destination has no known containing Ghidra procedure");
}
void unresolved(Address address, String reason) {
if (unresolved != null) unresolved.put(address, reason);
}
JsonArray unresolvedCalls() {
if (unresolved == null) return null;
JsonArray result = new JsonArray();
for (var item : unresolved.entrySet()) {
JsonObject site = new JsonObject();
site.addProperty("address", canonicalAddress(item.getKey()));
site.addProperty("reason", item.getValue());
result.add(site);
}
return result;
}
}
private String terminalCallLimitation(Instruction instruction) {
if (!instruction.getFlowType().isCall() || !instruction.getFlowType().isTerminal())
return null;
Address[] destinations = instruction.getFlows();
StringBuilder limitation = new StringBuilder();
limitation.append("Ghidra Listing reports a terminal call at ")
.append(canonicalAddress(instruction.getAddress()));
if (destinations.length == 0) {
limitation.append(" with no resolved callee; its no-return basis is unknown.");
}
else {
limitation.append(" to ");
for (int index = 0; index < destinations.length; index++) {
if (index > 0) limitation.append(", ");
Address destination = destinations[index];
limitation.append(canonicalAddress(destination));
Function target = currentProgram.getFunctionManager().getFunctionAt(destination);
if (target == null) {
limitation.append(" (callee identity unknown)");
}
else {
limitation.append(" ").append(procedureName(target));
limitation.append(" (external=").append(target.isExternal());
limitation.append(", thunk=").append(target.isThunk());
limitation.append(", hasNoReturn=").append(target.hasNoReturn()).append(")");
}
}
limitation.append("; fallthrough is excluded by Ghidra's flow model.");
Function target = destinations.length == 1
? currentProgram.getFunctionManager().getFunctionAt(destinations[0])
: null;
if (target != null && target.hasNoReturn()) {
limitation.append(" This records Ghidra's FunctionManager flag, not an independent verification that the callee cannot return.");
}
else {
limitation.append(" The reason for the terminal flow is not established by this observation.");
}
}
return limitation.toString();
}
private static String renderAssembly(List<Instruction> instructions) {
return String.join("\n", renderAssemblyLines(instructions));
}
@@ -3358,15 +3524,27 @@ public final class ReaGhidraBridge extends HeadlessScript {
private List<Reference> collectReferences(
Function function,
String direction) throws Exception {
TreeMap<String, Reference> observed = new TreeMap<>();
visitReferences(function, direction, reference -> addReference(observed, reference));
List<Reference> result = new ArrayList<>(observed.values());
result.sort(REFERENCE_ORDER);
return result;
}
@FunctionalInterface
private interface ReferenceVisitor {
void accept(Reference reference) throws Exception;
}
private void visitReferences(Function function, String direction, ReferenceVisitor visitor)
throws Exception {
// References can target instruction interiors or originate in embedded
// data. Use actual body ownership, including an external's exact entry.
AddressSet scope = new AddressSet(function.getBody());
scope.add(function.getEntryPoint());
AddressSet scope = functionObservationScope(function);
boolean outgoing = direction.equals("outgoing");
AddressIterator addresses = outgoing
? currentProgram.getReferenceManager().getReferenceSourceIterator(scope, true)
: currentProgram.getReferenceManager().getReferenceDestinationIterator(scope, true);
TreeMap<String, Reference> observed = new TreeMap<>();
while (addresses.hasNext()) {
monitor.checkCancelled();
Address address = addresses.next();
@@ -3374,7 +3552,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
for (Reference reference : currentProgram.getReferenceManager()
.getReferencesFrom(address)) {
monitor.checkCancelled();
addReference(observed, reference);
if (!reference.isEntryPointReference()) visitor.accept(reference);
}
}
else {
@@ -3382,19 +3560,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
.getReferencesTo(address);
while (iterator.hasNext()) {
monitor.checkCancelled();
addReference(observed, iterator.next());
Reference reference = iterator.next();
if (!reference.isEntryPointReference()) visitor.accept(reference);
}
}
}
List<Reference> result = new ArrayList<>(observed.values());
result.sort(REFERENCE_ORDER);
return result;
}
private static void addReference(TreeMap<String, Reference> observed, Reference reference) {
if (reference.isEntryPointReference()) {
return;
}
String key = canonicalAddress(reference.getFromAddress()) + "\u0000" +
canonicalAddress(reference.getToAddress()) + "\u0000" +
reference.getReferenceType().getName() + "\u0000" +
@@ -3403,6 +3576,12 @@ public final class ReaGhidraBridge extends HeadlessScript {
observed.putIfAbsent(key, reference);
}
private static AddressSet functionObservationScope(Function function) {
AddressSet scope = new AddressSet(function.getBody());
scope.add(function.getEntryPoint());
return scope;
}
private JsonArray referenceEdges(List<Reference> references) {
JsonArray result = new JsonArray();
for (Reference reference : references) {
@@ -3556,23 +3735,28 @@ public final class ReaGhidraBridge extends HeadlessScript {
return result;
}
private static JsonArray comments(List<Instruction> instructions) {
private JsonArray comments(Function function) throws Exception {
Listing listing = currentProgram.getListing();
AddressSet scope = functionObservationScope(function);
AddressIterator addresses = listing.getCommentAddressIterator(scope, true);
TreeMap<String, JsonObject> observed = new TreeMap<>();
for (Instruction instruction : instructions) {
while (addresses.hasNext()) {
monitor.checkCancelled();
Address address = addresses.next();
for (CommentType type : CommentType.values()) {
String text = instruction.getComment(type);
String text = listing.getComment(type, address);
if (text == null || text.isEmpty()) {
continue;
}
String kind = type == CommentType.EOL ? "inline" : "comment";
JsonObject item = new JsonObject();
item.addProperty("address", canonicalAddress(instruction.getAddress()));
item.addProperty("address", canonicalAddress(address));
item.addProperty("kind", kind);
item.addProperty("text", text);
String key = canonicalAddress(instruction.getAddress()) + "\u0000" +
String key = canonicalAddress(address) + "\u0000" +
kind + "\u0000" + type.name() + "\u0000" + text;
observed.put(key, item);
}
}
}
JsonArray result = new JsonArray();
observed.values().forEach(result::add);
@@ -4508,7 +4692,8 @@ public final class ReaGhidraBridge extends HeadlessScript {
private record InventoryItem(Address address, String value, JsonObject facts) {}
private record AddressNameRequest(String inputAddress, Address address, String name) {}
private record FunctionEntry(Function function, InventoryItem item) {}
private record FunctionAnalysisObservation(JsonObject dossier, List<String> limitations) {}
private record FunctionAnalysisObservation(JsonObject dossier, List<String> limitations, JsonObject relationshipCoverage) {}
private record CallBoundaryObservation(List<String> limitations, Integer unrepresentedCalleeCalls, JsonArray unresolvedCalls) {}
private record SwiftStringLiteralDecode(JsonArray literals, JsonArray omissions) {}
private record SwiftLiteralLayout(
String representation,
+133 -12
View File
@@ -11,6 +11,7 @@ import math
import re
import resource
from pathlib import Path
from urllib.parse import unquote_to_bytes
from mitmproxy import ctx, version
from mitmproxy.io import tnetstring
@@ -41,6 +42,113 @@ class BoundedReader:
return self.handle.read(count)
SPECIAL_SCHEMES = {"ftp", "file", "http", "https", "ws", "wss"}
SCHEME_PREFIX = r"^[\x00-\x20]*([a-z][a-z0-9+.-]*):"
NETWORK_PREFIX = r"^[\x00-\x20]*[/\\]{2}"
URL_IGNORABLE_CONTROLS = "\t\n\r"
URL_CONTROL_TRANSLATION = {ord(char): None for char in URL_IGNORABLE_CONTROLS}
FORBIDDEN_DOMAIN_CODE_POINTS = {chr(code) for code in range(0x21)} | set("#%/:<>?@[\\]^|\x7f")
def authority_start(text, base_scheme=None):
"""Return where an authority begins and whether it is special, or None.
Follows WHATWG URL parsing as src/domain/recordedTransportUrl.ts does:
special schemes (and scheme-relative references) skip every leading "/"
or "\\" before the authority, so `https://\\user:pass@host` still has
userinfo. File URLs never carry userinfo: an "@" in a file host makes
WHATWG reject the whole URL, so there are no credentials to remove.
"""
scheme = re.match(SCHEME_PREFIX, text, flags=re.I)
name = scheme.group(1).lower() if scheme is not None else None
if name == "file" or (name is None and base_scheme == "file"):
return None
if name is not None and name in SPECIAL_SCHEMES:
# A same-scheme Location without two slash-equivalent separators is
# relative to the recorded request, so its @ belongs to the path.
following = text[scheme.end():scheme.end() + 2]
if name == base_scheme and (len(following) < 2 or any(char not in "/\\" for char in following)):
return None
start = scheme.end()
else:
if scheme is not None:
if text[scheme.end():scheme.end() + 2] != "//":
return None
return scheme.end() + 2, False
# A non-special base treats backslashes as authority data and only
# recognizes two forward slashes as a network reference.
if base_scheme is not None and base_scheme not in SPECIAL_SCHEMES:
prefix = re.match(r"^[\x00-\x20]*//", text)
return None if prefix is None else (prefix.end(), False)
prefix = re.match(NETWORK_PREFIX, text)
if prefix is None:
return None
start = prefix.end()
while start < len(text) and text[start] in "/\\":
start += 1
return start, True
def special_host_parses(host_port):
"""Whether WHATWG could accept a special authority's host and port.
A rejected host fails the whole URL, so it has no credentials to remove.
This mirrors the forbidden-code-point, empty-host and port rules; IDNA,
IPv4 range and IPv6 syntax failures are not modeled and still redact.
"""
if host_port.startswith("["):
host_end = host_port.find("]") + 1
if host_end == 0:
return False
else:
colon = host_port.find(":")
host_end = len(host_port) if colon < 0 else colon
host = unquote_to_bytes(host_port[:host_end]).decode("utf-8", errors="replace")
if host == "" or any(char in FORBIDDEN_DOMAIN_CODE_POINTS for char in host):
return False
port = host_port[host_end:]
if port == "":
return True
digits = port[1:]
return port[0] == ":" and (digits == "" or (digits.isascii() and digits.isdigit() and int(digits) <= 65535))
def without_userinfo(value, base_scheme=None):
"""Remove an authority's userinfo using the WHATWG authority boundary.
Special schemes and scheme-relative references also end the authority
at a backslash, so later path text containing "@" is not userinfo.
"""
binary = isinstance(value, bytes)
text = value.decode("latin-1") if binary else value
# WHATWG removes these controls before interpreting scheme/separators.
# Keep that parse view separate from the exact source spelling and bytes.
parsed_text = text.translate(URL_CONTROL_TRANSLATION)
located = authority_start(parsed_text, base_scheme)
if located is None:
return value
start, special = located
terminators = "/?#\\" if special else "/?#"
end = start
while end < len(parsed_text) and parsed_text[end] not in terminators:
end += 1
at = parsed_text.rfind("@", start, end)
if at < start or (special and not special_host_parses(parsed_text[at + 1 : end])):
return value
parsed_offset = 0
raw_start = 0
for raw_offset, char in enumerate(text):
if char in URL_IGNORABLE_CONTROLS:
continue
if parsed_offset == start:
raw_start = raw_offset
if parsed_offset == at:
safe = text[:raw_start] + text[raw_offset + 1 :]
return safe.encode("latin-1") if binary else safe
parsed_offset += 1
raise CaptureFailure("decoder", "Normalized URL authority could not be bound to its source.")
def project_record(state, secrets):
binaries, numbers, redactions = [], [], []
byte_secrets = []
@@ -58,16 +166,23 @@ def project_record(state, secrets):
return None
return text
def transport_value(value, path, transport_url):
def transport_value(value, path, transport_url, base_scheme):
if not (transport_url or re.fullmatch(r"/(?:backup/)*request/(?:path|authority)", path)):
return value, False
binary = isinstance(value, bytes)
pattern = rb"^([\x00-\x20]*(?:[a-z][a-z0-9+.-]*:)?//)[^/?#]*@" if binary else r"^([\x00-\x20]*(?:[a-z][a-z0-9+.-]*:)?//)[^/?#]*@"
replacement = rb"\1" if binary else r"\1"
separator = b"@" if binary else "@"
safe_url = re.sub(pattern, replacement, value, flags=re.I)
if path.endswith("/authority") and separator in safe_url:
safe_url = safe_url.rsplit(separator, 1)[1]
if transport_url:
safe_url = without_userinfo(value, base_scheme)
elif path.endswith("/authority"):
safe_url = value.rsplit(separator, 1)[-1]
else:
# Native request.path is origin-form unless it explicitly carries
# a scheme (absolute-form). Leading separators are path evidence,
# not a header URL reference's network authority.
text = value.decode("latin-1") if binary else value
safe_url = without_userinfo(value) if re.match(
SCHEME_PREFIX, text.translate(URL_CONTROL_TRANSLATION), flags=re.I
) else value
changed = safe_url != value
if changed:
redactions.append({"pointer": path, "reason": "transport-credential"})
@@ -80,7 +195,7 @@ def project_record(state, secrets):
del numbers[before[1]:]
del redactions[before[2]:]
def visit(value, path, depth=0, credential=False, transport_url=False):
def visit(value, path, depth=0, credential=False, transport_url=False, base_scheme=None):
if depth > 64:
raise CaptureFailure("input-limit", "Native capture exceeds the 64-level evidence nesting budget.", path)
if credential:
@@ -90,7 +205,7 @@ def project_record(state, secrets):
binaries.append({"pointer": path, "representation": "producer-bytes", "state": "redacted", "content_base64": None, "bytes": None, "sha256": None})
return None
if isinstance(value, bytes):
safe_bytes, credential_url = transport_value(value, path, transport_url)
safe_bytes, credential_url = transport_value(value, path, transport_url, base_scheme)
content_base64 = base64.b64encode(value).decode("ascii")
digest = hashlib.sha256(value).hexdigest()
hidden = any(secret in value or secret in safe_bytes for secret in byte_secrets) or any(secret in content_base64 or secret in digest or secret in str(len(value)) for secret in secrets)
@@ -107,7 +222,7 @@ def project_record(state, secrets):
return None
return safe_text
if isinstance(value, str):
safe_text, _ = transport_value(value, path, transport_url)
safe_text, _ = transport_value(value, path, transport_url, base_scheme)
return redact(safe_text, path, value)
if isinstance(value, bool) or value is None:
return value
@@ -125,10 +240,16 @@ def project_record(state, secrets):
for index, field in enumerate(value):
if not isinstance(field, (list, tuple)) or len(field) != 2 or not isinstance(field[0], bytes):
raise CaptureFailure("format", "Native header/trailer is not an ordered byte pair.", pointer(path, index))
result.append([visit(field[0], pointer(pointer(path, index), 0), depth + 2), visit(field[1], pointer(pointer(path, index), 1), depth + 2, field[0].lower() in CREDENTIALS, field[0].lower() in {b"location", b"referer", b"origin"})])
result.append([visit(field[0], pointer(pointer(path, index), 0), depth + 2), visit(field[1], pointer(pointer(path, index), 1), depth + 2, field[0].lower() in CREDENTIALS, field[0].lower() in {b"location", b"referer", b"origin"}, base_scheme if field[0].lower() == b"location" else None)])
return result
return [visit(item, pointer(path, index), depth + 1) for index, item in enumerate(value)]
return [visit(item, pointer(path, index), depth + 1, base_scheme=base_scheme) for index, item in enumerate(value)]
if isinstance(value, dict):
if path == "" or re.fullmatch(r"/backup(?:/backup)*", path):
request = value.get("request")
scheme = request.get("scheme") if isinstance(request, dict) else None
if isinstance(scheme, bytes):
scheme = scheme.decode("latin-1")
base_scheme = scheme.lower() if isinstance(scheme, str) and re.fullmatch(r"[a-z][a-z0-9+.-]*", scheme, flags=re.I) else None
result = {}
for key, item in value.items():
if not isinstance(key, str):
@@ -139,7 +260,7 @@ def project_record(state, secrets):
validate_omitted(item, pointer(path, key), depth + 1)
redactions.append({"pointer": path, "reason": "explicit-sensitive-value", "scope": "property-name"})
continue
result[key] = visit(item, pointer(path, key), depth + 1)
result[key] = visit(item, pointer(path, key), depth + 1, base_scheme=base_scheme)
return result
raise CaptureFailure("decoder", "Native parser returned an unexpected value type: " + type(value).__name__ + ".", path)
+214
View File
@@ -0,0 +1,214 @@
"""Source-owned tests for the mitmproxy adapter's record projection."""
import importlib.util
import os
import sys
import types
import unittest
from unittest import mock
def load_capture():
"""Load the adapter with fresh mitmproxy stand-ins scoped to its import."""
fake_mitmproxy = types.ModuleType("mitmproxy")
fake_mitmproxy.ctx = types.SimpleNamespace()
fake_mitmproxy.version = types.SimpleNamespace(VERSION="12.2.3")
fake_io = types.ModuleType("mitmproxy.io")
fake_io.tnetstring = types.ModuleType("mitmproxy.io.tnetstring")
stand_ins = {
"mitmproxy": fake_mitmproxy,
"mitmproxy.io": fake_io,
"mitmproxy.io.tnetstring": fake_io.tnetstring,
}
spec = importlib.util.spec_from_file_location(
"rea_mitmproxy_capture", os.path.join(os.path.dirname(__file__), "capture.py")
)
module = importlib.util.module_from_spec(spec)
with mock.patch.dict(sys.modules, stand_ins):
spec.loader.exec_module(module)
return module
capture = load_capture()
def location(value, scheme=None):
state = {"response": {"headers": [[b"Location", value]]}}
if scheme is not None:
state["request"] = {"scheme": scheme}
record = capture.project_record(state, [])
sidecar = next(
field for field in record["binary_fields"] if field["pointer"] == "/response/headers/0/1"
)
return record["reported"]["response"]["headers"][0][1], sidecar, record["redactions"]
class TransportUserinfoTests(unittest.TestCase):
def test_ignorable_controls_cannot_hide_authority_credentials(self):
for control in [b"\t", b"\n", b"\r"]:
for value in [b"https://" + control + b"/user:pass@host/path", b"ht" + control + b"tps://user:" + control + b"pass@host/path"]:
with self.subTest(value=value):
text, sidecar, redactions = location(value, b"https")
self.assertNotIn("user", text)
self.assertNotIn("pass", text)
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(len(redactions), 1)
def test_mixed_network_separators_remove_credentials(self):
for prefix in [b"\\\\", b"/\\", b"\\/"]:
with self.subTest(prefix=prefix):
text, sidecar, redactions = location(prefix + b"user:pass@host/path", b"https")
self.assertEqual(text, prefix.decode("ascii") + "host/path")
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(len(redactions), 1)
def test_non_special_base_keeps_backslashes_inside_network_authorities(self):
text, sidecar, redactions = location(b"//user\\name:pass@host/x", b"foo")
self.assertEqual(text, "//host/x")
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(len(redactions), 1)
text, sidecar, redactions = location(b"/\\invoice@customer", b"foo")
self.assertEqual(text, "/\\invoice@customer")
self.assertEqual(sidecar["state"], "retained")
self.assertEqual(redactions, [])
def test_native_request_paths_preserve_origin_form_and_redact_absolute_form(self):
for value in [b"/\\invoice@customer", b"//invoice@customer", b"\\\\invoice@customer"]:
with self.subTest(value=value):
record = capture.project_record({"request": {"path": value}}, [])
self.assertEqual(record["reported"]["request"]["path"], value.decode("ascii"))
self.assertEqual(record["binary_fields"][0]["state"], "retained")
self.assertEqual(record["redactions"], [])
record = capture.project_record({"request": {"path": b"https://user:pass@host/x"}}, [])
self.assertEqual(record["reported"]["request"]["path"], "https://host/x")
self.assertEqual(record["binary_fields"][0]["state"], "redacted")
def test_same_scheme_locations_preserve_relative_path_text_and_bytes(self):
for value in [b"https:invoice@customer", b"https:/invoice@customer", b"https:\\invoice@customer"]:
with self.subTest(value=value):
text, sidecar, redactions = location(value, b"https")
self.assertEqual(text, value.decode("ascii"))
self.assertEqual(sidecar["state"], "retained")
self.assertEqual(redactions, [])
def test_location_authorities_still_remove_credentials_with_a_base_scheme(self):
for value, scheme, expected in [
(b"https://user:pass@host/path", b"https", "https://host/path"),
(b"https:\\\\user:pass@host/path", b"https", "https:\\\\host/path"),
(b"https:user:pass@host/path", b"http", "https:host/path"),
]:
with self.subTest(value=value, scheme=scheme):
text, sidecar, redactions = location(value, scheme)
self.assertEqual(text, expected)
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(len(redactions), 1)
def test_backup_location_uses_its_own_request_scheme(self):
record = capture.project_record({
"request": {"scheme": b"http"},
"backup": {
"request": {"scheme": b"https"},
"response": {"headers": [[b"Location", b"https:invoice@customer"]]},
},
}, [])
self.assertEqual(record["reported"]["backup"]["response"]["headers"][0][1], "https:invoice@customer")
self.assertEqual(record["redactions"], [])
def test_backslash_ends_a_special_scheme_authority(self):
text, sidecar, redactions = location(b"https://example.test\\part@tail")
self.assertEqual(text, "https://example.test\\part@tail")
self.assertEqual(sidecar["state"], "retained")
self.assertIsNotNone(sidecar["content_base64"])
self.assertEqual(redactions, [])
def test_userinfo_before_a_backslash_is_still_removed(self):
text, sidecar, redactions = location(b"https://user:pw@example.test\\path")
self.assertEqual(text, "https://example.test\\path")
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(
redactions,
[{"pointer": "/response/headers/0/1", "reason": "transport-credential"}],
)
def test_ordinary_userinfo_is_removed(self):
text, _sidecar, _redactions = location(b"https://user:pw@example.test/a@b")
self.assertEqual(text, "https://example.test/a@b")
def test_empty_userinfo_marker_is_removed(self):
self.assertEqual(
capture.without_userinfo("https://@host"), "https://host"
)
def test_special_schemes_skip_leading_slashes_before_userinfo(self):
for value, expected in [
(b"https://\\user:pass@host/path", b"https://\\host/path"),
(b"https:\\\\user:pass@host/path", b"https:\\\\host/path"),
(b"https:user:pass@host", b"https:host"),
(b"//\\user:pass@host", b"//\\host"),
]:
with self.subTest(value=value):
self.assertEqual(capture.without_userinfo(value), expected)
text, sidecar, redactions = location(b"https://\\user:pass@host/path")
self.assertEqual(text, "https://\\host/path")
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(len(redactions), 1)
def test_file_base_keeps_paths_after_the_fixed_network_boundary(self):
for value in [b"///u@h", b"////u@h"]:
with self.subTest(value=value):
text, sidecar, redactions = location(value, b"file")
self.assertEqual(text, value.decode("ascii"))
self.assertEqual(sidecar["state"], "retained")
self.assertIsNotNone(sidecar["content_base64"])
self.assertEqual(redactions, [])
def test_file_urls_keep_path_text(self):
self.assertEqual(capture.without_userinfo("file:///a@b"), "file:///a@b")
def test_file_urls_reject_userinfo_and_keep_their_source(self):
# WHATWG rejects an "@" in a file host, so nothing parses as credentials.
self.assertEqual(
capture.without_userinfo("file://user:pass@host/x"), "file://user:pass@host/x"
)
for value in [b"\\\\user:pass@host/x", b"/\\user:pass@host/x", b"//user:pass@host/x"]:
with self.subTest(value=value):
text, sidecar, redactions = location(value, b"file")
self.assertEqual(text, value.decode("ascii"))
self.assertEqual(sidecar["state"], "retained")
self.assertEqual(redactions, [])
def test_rejected_special_hosts_keep_their_source(self):
# A rejected host or port fails the whole URL, so it has no userinfo.
for value in [
"https:user@%",
"https://user@",
"https://user:pass@h^st/x",
"https://user@ho st/x",
"https://user@host:99999/x",
"https://user@host:12a",
"https://user@[::1",
]:
with self.subTest(value=value):
self.assertEqual(capture.without_userinfo(value), value)
text, sidecar, redactions = location(b"https://user:pass@h^st/x", b"https")
self.assertEqual(text, "https://user:pass@h^st/x")
self.assertEqual(sidecar["state"], "retained")
self.assertEqual(redactions, [])
def test_accepted_special_hosts_still_remove_userinfo(self):
for value, expected in [
("https:user@%41", "https:%41"),
("https://user@[::1]:8443/x", "https://[::1]:8443/x"),
("https://user@host:/x", "https://host:/x"),
("wss://user:pass@host:65535", "wss://host:65535"),
]:
with self.subTest(value=value):
self.assertEqual(capture.without_userinfo(value), expected)
def test_non_special_schemes_keep_backslashes_in_the_authority(self):
self.assertEqual(capture.without_userinfo("foo://a\\b@host/x"), "foo://host/x")
self.assertEqual(capture.without_userinfo(b"//a\\b@host"), b"//a\\b@host")
if __name__ == "__main__":
unittest.main()
+15
View File
@@ -265,6 +265,21 @@ The user-level MCP paths follow the editor's VS Code layout. Trae's official
documentation describes the raw JSON editor without specifying those paths;
loading this registration in Trae remains unverified.
## WorkBuddy
For [WorkBuddy](https://www.workbuddy.cn/docs/workbuddy/From-Beginner-to-Expert-Guide/Function-Description/MCP-Guide), setup writes `mcpServers.rea` into the
global `~/.workbuddy/mcp.json` (a standard `mcpServers` JSON file at the user
home root on every platform). Entries carry `command`, `args`, and `env`;
stdio is the default transport. WorkBuddy discovers personal skills from
`~/.workbuddy/skills` (`%userprofile%\.workbuddy\skills` on Windows), so
setup installs its skill copy there.
The global MCP path is a plain JSON file; setup preserves every other server
and top-level key when it upserts `rea`. WorkBuddy also reads a project-level
`<project>/.workbuddy/mcp.json`, which setup does not touch: it registers the
user-level file only. Loading this registration and real REA tool calls
through WorkBuddy are verified.
## ZCode
For [ZCode](https://zcode.z.ai/en/docs/mcp-services), setup registers `rea` in the nested `mcp.servers` table of the
+3 -80
View File
@@ -288,11 +288,12 @@ hosted connector; see [its configuration](client-configuration.md#other-client-s
| Qoder | `qoder` |
| Factory Droid | `droid` |
| Trae | `trae` |
| WorkBuddy | `workbuddy` |
| ZCode | `zcode` |
| DeepSeek Harness | `dsh` |
`dsh` and `kimi` require a release containing them. The registry check above does
not cover them.
`dsh`, `kimi`, and `workbuddy` require a release containing them. The registry
check above does not cover them.
See [client configuration and troubleshooting](client-configuration.md) for
configuration paths, profiles, trust policies, and schema-limit workarounds.
@@ -805,84 +806,6 @@ read-only operations, digest identity, and runtime cleanup. The separate
ordinary user. It requires the built Windows native bundle and an existing
Ghidra/JDK installation; neither is inferred from startup alone.
## Agent plugins and marketplaces
REA's npm package is the shared Claude Code and Codex plugin artifact. It
contains `.claude-plugin/plugin.json`, `.codex-plugin/plugin.json`, and the
canonical generated `skills/` tree. Both manifests select `./skills` and run a plain Node bootstrap pinned to
that exact package version. The client must have a supported standalone `node`
on its PATH. Claude expands its documented `CLAUDE_PLUGIN_ROOT` in the script
argument; Codex resolves `cwd: "."` from the installed plugin root before
launching the relative script. The shared bootstrap preserves the pinned
package-runner arguments and stdio, handles Windows npm shims, and uses an owned
empty working directory so an unbuilt local checkout cannot shadow the exact
registry release. The existing release skill
generator also pins CLI recovery commands to the same version.
The repository's single-plugin marketplaces are declared in
`.claude-plugin/marketplace.json` and `.agents/plugins/marketplace.json`.
Both select an exact `rea-agents` npm version instead of mutable source-checkout
contents. Release Please updates their selectors alongside the plugin manifests.
No plugin-only publication or generated skill files committed to Git are needed.
This distribution becomes installable after the first qualified npm release
containing these plugin manifests. Earlier published packages are not changed
by this source proposal. For local source development, run `npm ci && npm run build:cached` to generate
the runtime and packaged skill before loading the repository as a plugin.
On Windows, also build the Windows x64 native addon following the
[Windows source-checkout guide](https://github.com/morluto/rea/blob/main/native/windows/README.md#build);
the registry fallback requires that addon and the built process supervisor to
own and clean up the package-runner process tree. The source plugin always
launches its pinned registry release, even when the checkout has been built. The authored
`.agents/skills/` tree is not the plugin's shipped skill layout.
In Claude Code:
```text
/plugin marketplace add morluto/rea
/plugin install rea@rea
```
In Codex, add the marketplace and then install REA from the plugin list:
```bash
codex plugin marketplace add morluto/rea
```
Each release updates both manifest versions and their pinned server version
together, which is when installed copies refresh the bundled skill and move to
the new server. A plugin does not run `rea setup`, so provider
configuration still follows the sections above. Use the plugin or a
`rea setup` registration for a client, not both, to avoid two `rea` servers.
`rea doctor` does not see plugin-owned state. Its client and skill checks read
only the registrations and skill locations that `rea setup` manages, so for a
plugin-only install `rea doctor --client <id>` and `rea doctor --skill` report
`missing` and recommend `rea setup`. Do not follow that remediation. Check the
plugin with the client instead (`claude plugin list` and `/mcp` in Claude
Code, `codex plugin list` in Codex).
For analysis engines, ask the connected server: have the agent call
`binary_session` and read `result.tool_availability`, which reports what the
server the plugin started can use, with reasons and remediation. See
[MCP contracts](mcp-contracts.md).
A plugin does not put `rea` on your `PATH`. To check an engine from a
terminal, use the package runner at the version the plugin pins:
<!-- x-release-please-start-version -->
```bash
npx -y rea-agents@6.4.0 doctor --provider ghidra --json
```
<!-- x-release-please-end -->
If your installed plugin is older, use the version in its manifest instead.
That report describes the terminal's environment, not the plugin server's.
The server inherits the client's environment, so a setting such as
`GHIDRA_INSTALL_DIR` that exists only in your terminal does not reach it.
## MCP Registry
REA is published in the official MCP Registry as `io.github.morluto/rea`. Registry
+2 -5
View File
@@ -240,11 +240,8 @@ Review and test this synchronization PR, then use a merge commit so the release
tag remains in main's ancestry. Keep the released tag unchanged.
The synchronization must include `.release-please-manifest.json`,
`package.json`, both root versions in `npm-shrinkwrap.json`, `CHANGELOG.md`,
`server.json`, both plugin manifests (`.claude-plugin/plugin.json` and
`.codex-plugin/plugin.json`), the Codex server declaration (`.mcp.json`), both
marketplaces (`.claude-plugin/marketplace.json` and
`.agents/plugins/marketplace.json`), and the versioned documentation examples. Run
`package.json`, both root versions in `package-lock.json`, `CHANGELOG.md`,
`server.json`, and the versioned documentation examples. Run
`npm run docs:generate`, `npm run docs:check`, and the release configuration
tests against the combined tree. After the merge, verify that the released
tag is an ancestor of main:
+1 -1
View File
@@ -111,7 +111,7 @@ or acquire that reduction automatically.
Use the repository's pinned Node/npm and locked dependencies. Build each selected
revision, then copy its `dist/`, `bridge/`, `scripts/`, `skills/`, `package.json`
and `npm-shrinkwrap.json` to a separate runtime directory before editing production
and `package-lock.json` to a separate runtime directory before editing production
sources. That copy needs the matching locked dependencies. The initial frozen
baseline is `42b0ed126df7d5b908ac369715378d5e519051b7`; this study does not
silently substitute a newer `main` or the previous Code Mode experiment.
@@ -216,7 +216,7 @@ def main():
'authority': args.authority,
'runtime_files_sha256': runtime_digest,
'runtime_dispatcher_sha256': file_digest(repository / 'scripts/rea.mjs'),
'package_lock_sha256': file_digest(repository / 'npm-shrinkwrap.json'),
'package_lock_sha256': file_digest(repository / 'package-lock.json'),
'experiment_sources_sha256': {
name: hashlib.sha256((experiment / name).read_bytes()).hexdigest()
for name in ['agent-trial.py', 'codex_context.py', 'provider-environment.json']
+1 -1
View File
@@ -147,7 +147,7 @@ const fingerprints = {
resolve(values["runtime-root"], "scripts/rea.mjs"),
),
package_lock_sha256: await fileSha(
resolve(values["runtime-root"], "npm-shrinkwrap.json"),
resolve(values["runtime-root"], "package-lock.json"),
),
};
const clientVersion = JSON.parse(
@@ -109,7 +109,7 @@ class ContextCaptureTests(unittest.TestCase):
runtime = root / 'runtime'
(runtime / 'scripts').mkdir(parents=True)
(runtime / 'scripts' / 'rea.mjs').write_text('')
(runtime / 'npm-shrinkwrap.json').write_text('{}')
(runtime / 'package-lock.json').write_text('{}')
auth_home = root / 'source-auth'
auth_home.mkdir()
(auth_home / 'auth.json').write_text('synthetic-credential')
File diff suppressed because it is too large Load Diff
+197 -2333
View File
File diff suppressed because it is too large Load Diff
+6 -7
View File
@@ -47,7 +47,6 @@
"packageManager": "npm@11.16.0",
"files": [
"dist",
"npm-shrinkwrap.json",
"install.sh",
"bridge/hopper_bridge.py",
"bridge/cutter/rea_bridge.py",
@@ -68,16 +67,12 @@
"bridge/android/ReaJadxBridge.java",
"bridge/mitmproxy/capture.py",
"scripts/rea.mjs",
"scripts/plugin-mcp-launcher.mjs",
"scripts/rea-android-workspace-guardian.mjs",
"scripts/electron-active-hook.cjs",
"scripts/electron-active-hook-boundaries.cjs",
"scripts/hopper-demo-x11.py",
"scripts/prepack.mjs",
"scripts/prepare.mjs",
".claude-plugin/plugin.json",
".codex-plugin/plugin.json",
".mcp.json",
"skills",
"README.md",
"DISCLAIMER.md",
@@ -143,7 +138,7 @@
"test": "npm run test:with-build",
"test:with-build": "npm run test:prepare && vitest run",
"test:run": "vitest run",
"test:python": "python3 -m unittest bridge/native/test_rea_lldb_tracer.py -v && python3 -m unittest discover -s experiments/analysis-efficiency -p 'test_*.py' -v",
"test:python": "python3 -m unittest bridge/native/test_rea_lldb_tracer.py bridge/mitmproxy/test_capture.py -v && python3 -m unittest discover -s experiments/analysis-efficiency -p 'test_*.py' -v",
"test:local": "node scripts/run-development-tests.mjs local",
"test:fast": "node scripts/ci/test-group.mjs source",
"test:built": "node scripts/ci/test-group.mjs built",
@@ -356,6 +351,11 @@
"yaml": "2.9.0",
"zod": "4.4.3"
},
"overrides": {
"vitepress": {
"vite": "6.4.4"
}
},
"devDependencies": {
"@fast-check/vitest": "0.5.0",
"@stryker-mutator/core": "10.0.0",
@@ -373,7 +373,6 @@
"lint-staged": "16.4.0",
"oxfmt": "0.72.0",
"oxlint": "1.87.0",
"release-please": "17.3.0",
"rolldown": "1.2.12",
"turbo": "2.11.7",
"typescript": "7.0.2",
-30
View File
@@ -31,36 +31,6 @@
"type": "json",
"path": "server.json",
"jsonpath": "$.packages[0].version"
},
{
"type": "json",
"path": ".claude-plugin/plugin.json",
"jsonpath": "$.version"
},
{
"type": "json",
"path": ".claude-plugin/plugin.json",
"jsonpath": "$.mcpServers.rea.args[1]"
},
{
"type": "json",
"path": ".codex-plugin/plugin.json",
"jsonpath": "$.version"
},
{
"type": "json",
"path": ".mcp.json",
"jsonpath": "$.mcpServers.rea.args[1]"
},
{
"type": "json",
"path": ".claude-plugin/marketplace.json",
"jsonpath": "$.plugins[0].source.version"
},
{
"type": "json",
"path": ".agents/plugins/marketplace.json",
"jsonpath": "$.plugins[0].source.version"
}
]
}
+3 -3
View File
@@ -17,7 +17,7 @@ const installedPackageKey = (name) => `node_modules/${name}`;
export const dependencyInstallProblems = (rootLock, installedLock) => {
const rootPackage = rootLock.packages?.[""];
if (rootPackage === undefined)
return ["npm-shrinkwrap.json does not contain a root package entry"];
return ["package-lock.json does not contain a root package entry"];
const installedPackages = installedLock.packages ?? {};
return Object.keys(directDependencies(rootPackage))
.sort()
@@ -26,7 +26,7 @@ export const dependencyInstallProblems = (rootLock, installedLock) => {
const expected = rootLock.packages?.[key]?.version;
const actual = installedPackages[key]?.version;
if (expected === undefined)
return [`${name}: missing resolved version in npm-shrinkwrap.json`];
return [`${name}: missing resolved version in package-lock.json`];
if (actual === undefined)
return [`${name}: missing from the installed dependency lockfile`];
return actual === expected
@@ -39,7 +39,7 @@ const main = async () => {
const root = process.cwd();
try {
const [rootLock, installedLock] = await Promise.all([
readJson(resolve(root, "npm-shrinkwrap.json")),
readJson(resolve(root, "package-lock.json")),
readJson(resolve(root, "node_modules/.package-lock.json")),
]);
const problems = dependencyInstallProblems(rootLock, installedLock);
+2 -2
View File
@@ -14,7 +14,7 @@ const packageJson = JSON.parse(
await readFile(join(root, "package.json"), "utf8"),
);
const lock = JSON.parse(
await readFile(join(root, "npm-shrinkwrap.json"), "utf8"),
await readFile(join(root, "package-lock.json"), "utf8"),
);
const skill = await readFile(
join(root, ".agents/skills/reverse-engineer-anything/SKILL.md"),
@@ -29,7 +29,7 @@ const versionOf = (name) => {
throw new Error(`Missing locked ${name} version`);
return value;
};
const source = `/** Generated from package.json and npm-shrinkwrap.json; do not edit. */
const source = `/** Generated from package.json and package-lock.json; do not edit. */
export const PACKAGE_METADATA = {
name: ${JSON.stringify(packageJson.name)},
version: ${JSON.stringify(packageJson.version)}, // x-release-please-version
+2 -2
View File
@@ -11,7 +11,7 @@ for (const argument of arguments_)
throw new Error(`Unknown inventory option: ${argument}`);
const root = join(dirname(fileURLToPath(import.meta.url)), "..");
const lock = JSON.parse(
await readFile(join(root, "npm-shrinkwrap.json"), "utf8"),
await readFile(join(root, "package-lock.json"), "utf8"),
);
const entries = Object.entries(lock.packages)
.filter(([path]) => path !== "")
@@ -38,7 +38,7 @@ const runtimeFingerprint = createHash("sha256")
.digest("hex");
const source = [
"REA npm dependency inventory",
"Prepared from the root npm-shrinkwrap.json; refresh this inventory when locked dependencies change.",
"Prepared from the root package-lock.json; refresh this inventory when locked dependencies change.",
"License identifiers are upstream lockfile declarations, not a legal assessment.",
"Runtime includes transitive and optional platform packages; development includes build, test and website tooling.",
`Locked package entries: ${entries.length}; runtime: ${runtime.length}.`,
@@ -1,4 +1,5 @@
export function assertCleanLinuxHopperRuntime(
launcherPaths?: readonly string[],
procRoot?: string,
listProcesses?: () => Promise<{ stdout: string }>,
): Promise<void>;
+2 -1
View File
@@ -17,8 +17,9 @@ export const assertCleanLinuxHopperRuntime = async (
: [process.env.HOPPER_LAUNCHER_PATH]),
],
procRoot = "/proc",
listProcesses = () => execFileAsync("ps", ["-ax", "-o", "command="]),
) => {
const { stdout } = await execFileAsync("ps", ["-ax", "-o", "command="]);
const { stdout } = await listProcesses();
if (/\/opt\/hopper\/bin\/Hopper|\bXvfb\b|hopper-demo-x11\.py/u.test(stdout))
throw new Error("Clean Linux runner retained a Hopper demo process");
const launchers = [];
+82
View File
@@ -215,6 +215,27 @@ async function verifyNativeValueAdapters(
const functionEvidence = parseEvidence(analyzed.structuredContent);
assert.equal(functionEvidence.operation, "analyze_function");
assert.equal(functionEvidence.provider.id, "ghidra");
assert.deepEqual(
functionEvidence.normalized_result.facet_availability,
Object.fromEntries(
[
"pseudocode",
"assembly",
"comments",
"callers",
"callees",
"references",
"strings_names",
"cfg",
].map((facet) => [facet, { available: true }]),
),
);
assert.equal(functionEvidence.relationship_coverage?.status, "complete");
assert.ok(
functionEvidence.relationship_coverage.diagnostics.some((diagnostic) =>
diagnostic.includes("indirect calls remain unknown"),
),
);
const functionProfile = analysisProfileSchema.parse(
functionEvidence.analysis_profile,
);
@@ -247,6 +268,7 @@ async function verifyNativeValueAdapters(
assert.ok(
comparisonEvidence.evidence_links.includes(functionEvidence.evidence_id),
);
await verifyUnresolvedCallEvidence(client);
const bundle = await client.callTool({
name: "get_evidence_bundle",
arguments: {},
@@ -307,3 +329,63 @@ async function verifyNativeValueAdapters(
unknowns: cliGraph.unknowns.length,
};
}
async function verifyUnresolvedCallEvidence(client) {
const inventory = await client.callTool({
name: "list_procedures",
arguments: {},
});
assert.notEqual(inventory.isError, true, JSON.stringify(inventory));
const procedures = parseEvidence(
inventory.structuredContent,
).normalized_result;
const indirect = procedures.find((item) =>
item.value.endsWith("rea_ghidra_inventory_indirect"),
);
const leaf = procedures.find((item) =>
item.value.endsWith("rea_ghidra_inventory_leaf"),
);
assert.ok(
indirect && leaf,
"Source fixture lacks the unresolved callback and its possible target",
);
const analyzed = await client.callTool(
{ name: "analyze_function", arguments: { procedure: indirect.address } },
{ timeout: 180000 },
);
assert.notEqual(analyzed.isError, true, JSON.stringify(analyzed));
const evidence = parseEvidence(analyzed.structuredContent);
assert.equal(evidence.relationship_coverage?.status, "partial");
assert.ok(evidence.normalized_result.unresolved_calls.length > 0);
const traced = await client.callTool(
{
name: "trace_call_path",
arguments: {
start: indirect.address,
goal: leaf.address,
direction: "forward",
},
},
{ timeout: 180000 },
);
assert.notEqual(traced.isError, true, JSON.stringify(traced));
const trace = parseEvidence(traced.structuredContent).normalized_result;
assert.equal(trace.goal_status, "unknown");
assert.equal(trace.truncated, true);
assert.ok(trace.residual_unknowns.length > 0);
const compared = await client.callTool(
{
name: "compare_functions",
arguments: {
left: analyzed.structuredContent,
right: analyzed.structuredContent,
},
},
{ timeout: 180000 },
);
assert.notEqual(compared.isError, true, JSON.stringify(compared));
assert.equal(
parseEvidence(compared.structuredContent).normalized_result.status,
"unknown",
);
}
+1 -1
View File
@@ -136,7 +136,7 @@ export async function verifyGhidraBoundaries(
assert.deepEqual(externalDossier.callees, []);
assert.deepEqual(externalDossier.facet_availability.pseudocode, {
available: false,
reason: "Ghidra did not supply pseudocode.",
reason: "Ghidra does not decompile external functions.",
});
for (const facet of [
"assembly",
+3 -59
View File
@@ -4,37 +4,6 @@ import { z } from "zod";
const shaSchema = z.string().regex(/^[a-f0-9]{40}$/u);
const versionSchema = z.object({ version: z.string() });
const manifestSchema = z.object({ ".": z.string() });
const PLUGIN_MANIFEST_PATHS = [
".claude-plugin/plugin.json",
".codex-plugin/plugin.json",
];
const inlinePluginManifestSchema = versionSchema.extend({
mcpServers: z.object({
rea: z.object({
command: z.literal("node"),
args: z.tuple([
z.literal("${CLAUDE_PLUGIN_ROOT}/scripts/plugin-mcp-launcher.mjs"),
z.string(),
]),
}),
}),
});
const codexPluginManifestSchema = versionSchema.extend({
mcpServers: z.literal("./.mcp.json"),
});
const codexServerConfigSchema = z.object({
mcpServers: z.object({
rea: z.object({
command: z.literal("node"),
args: z.tuple([
z.literal("./scripts/plugin-mcp-launcher.mjs"),
z.string(),
]),
cwd: z.literal("."),
startup_timeout_sec: z.number().int().positive(),
}),
}),
});
const sectionsSchema = z.object({
versioning: z.string().optional(),
"changelog-sections": z.array(
@@ -192,14 +161,14 @@ async function validateCandidate(git, sha, version, commits) {
const lock = await readJson(
git,
sha,
"npm-shrinkwrap.json",
"package-lock.json",
versionSchema.extend({
packages: z.object({ "": versionSchema }),
}),
);
versions.push(
["npm-shrinkwrap.json", lock.version],
["npm-shrinkwrap.json packages.root", lock.packages[""].version],
["package-lock.json", lock.version],
["package-lock.json packages.root", lock.packages[""].version],
);
const server = await readJson(
git,
@@ -220,21 +189,6 @@ async function validateCandidate(git, sha, version, commits) {
["server.json", server.version],
["server.json rea-agents package", npmPackage.version],
);
// A stale manifest keeps the previous plugin cache key or launches the
// previous server, so both are version authority like server.json.
const launches = [];
for (const path of PLUGIN_MANIFEST_PATHS) {
const manifest =
path === ".codex-plugin/plugin.json"
? await readJson(git, sha, path, codexPluginManifestSchema)
: await readJson(git, sha, path, inlinePluginManifestSchema);
versions.push([path, manifest.version]);
const server =
path === ".codex-plugin/plugin.json"
? await readJson(git, sha, ".mcp.json", codexServerConfigSchema)
: manifest;
launches.push([path, server.mcpServers.rea.args[1]]);
}
// Catalogs are generated and checked from this candidate by the release job;
// they are not version authority stored in the immutable source tree.
const generated = await git([
@@ -256,16 +210,6 @@ async function validateCandidate(git, sha, version, commits) {
);
}
}
// Plugin JSON stores only a scalar version. The launcher owns the fixed
// package name and constructs the complete registry specifier at runtime.
const requiredLaunch = version;
for (const [location, launched] of launches) {
if (launched !== requiredLaunch) {
throw new Error(
`${location}@${sha} launches version ${launched ?? "no version"}; release branch requires ${requiredLaunch}.`,
);
}
}
const notes = releaseNotes(
await git(["show", `${sha}:CHANGELOG.md`]),
version,
+1 -65
View File
@@ -365,60 +365,6 @@ export function verifyReleaseNotes(raw, text, body) {
};
}
const VERSION_ONLY_PLUGIN_METADATA = new Set([
".claude-plugin/plugin.json",
".codex-plugin/plugin.json",
".mcp.json",
".claude-plugin/marketplace.json",
".agents/plugins/marketplace.json",
]);
/** Serialize plugin metadata with only its release-owned version fields blanked. */
function pluginMetadataWithoutVersions(path, text) {
const manifest = JSON.parse(text);
if (manifest === null || typeof manifest !== "object") return text;
if (path.endsWith("marketplace.json")) {
if (Array.isArray(manifest.plugins))
for (const plugin of manifest.plugins)
if (
plugin?.name === "rea" &&
plugin?.source?.source === "npm" &&
plugin.source.package === "rea-agents" &&
typeof plugin.source.version === "string"
)
plugin.source.version = "";
return JSON.stringify(manifest);
}
if ("version" in manifest) manifest.version = "";
const launch = manifest.mcpServers?.rea?.args;
// Only the scalar pinned version is release-owned; the launcher owns the
// fixed package name and the executable/path remain product behavior.
if (Array.isArray(launch) && typeof launch[1] === "string") launch[1] = "";
return JSON.stringify(manifest);
}
/**
* Plugin metadata also carries behavior, so a release may change only
* its version, pinned server launch, and exact REA marketplace version. Any other difference, an unreadable
* side, or a manifest added by the release counts as a product change.
*/
async function changesOnlyPluginVersions(run, path, from, to) {
try {
return (
pluginMetadataWithoutVersions(
path,
await run("git", ["show", `${from}:${path}`]),
) ===
pluginMetadataWithoutVersions(
path,
await run("git", ["show", `${to}:${path}`]),
)
);
} catch {
return false;
}
}
/** Reject stale inventories and product changes made after the editorial target. */
export async function verifyInventoryHistory(run, raw, source) {
const inventory = releaseInventorySchema.parse(raw);
@@ -470,7 +416,6 @@ export async function verifyInventoryHistory(run, raw, source) {
".release-please-manifest.json",
"package.json",
"package-lock.json",
"npm-shrinkwrap.json",
"server.json",
"src/generatedPackageMetadata.ts",
"docs/installation.md",
@@ -488,16 +433,7 @@ export async function verifyInventoryHistory(run, raw, source) {
)
.split("\0")
.filter(Boolean);
const product = [];
for (const path of paths) {
if (allowed.has(path)) continue;
if (
VERSION_ONLY_PLUGIN_METADATA.has(path) &&
(await changesOnlyPluginVersions(run, path, inventory.target, resolved))
)
continue;
product.push(path);
}
const product = paths.filter((path) => !allowed.has(path));
if (product.length)
throw new Error(
`Product changed after notes target; regenerate inventory: ${product.join(", ")}`,
-274
View File
@@ -1,274 +0,0 @@
#!/usr/bin/env node
import { spawn } from "node:child_process";
import { randomUUID } from "node:crypto";
import { access, mkdtemp, rm } from "node:fs/promises";
import { constants, tmpdir } from "node:os";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
// Keep the package name at the launcher boundary so release automation updates
// only a scalar version in plugin JSON, never a compound package specifier.
// Git plugins launch the registry release. Windows source plugins additionally
// require a build and native addon for the owned process supervisor.
const [version, ...extra] = process.argv.slice(2);
if (
extra.length !== 0 ||
!/^\d+\.\d+\.\d+(?:-[A-Za-z0-9.-]+)?$/u.test(version ?? "")
) {
process.stderr.write(
"REA plugin requires one exact rea-agents@VERSION pin.\n",
);
process.exitCode = 1;
} else {
await launch(`rea-agents@${version}`);
}
async function launch(packageSpecifier) {
// npm installs already carry the release-pinned runtime. Run it in this
// process so the plugin keeps the caller's working directory and the
// runtime retains ownership of provider and worker cleanup.
const { default: bundledPackage } = await import("../package.json", {
with: { type: "json" },
});
const sourceCheckout = await isSourceCheckout();
let bundledRuntimeMatchesPin =
!sourceCheckout &&
packageSpecifier === `${bundledPackage.name}@${bundledPackage.version}`;
if (bundledRuntimeMatchesPin) {
try {
await access(new URL("../dist/mcp/main.js", import.meta.url));
} catch (cause) {
if (!isMissing(cause)) throw cause;
bundledRuntimeMatchesPin = false;
}
}
if (bundledRuntimeMatchesPin) {
process.argv = [
process.execPath,
fileURLToPath(new URL("./rea.mjs", import.meta.url)),
"mcp",
];
await import("./rea.mjs");
return;
}
const runnerModule = await import(
sourceCheckout
? "../src/application/PackageRunnerCommand.mjs"
: "../dist/application/PackageRunnerCommand.mjs"
);
const { packageRunnerCommand } = runnerModule;
const [command, ...args] = packageRunnerCommand(
"npx",
packageSpecifier,
process.platform,
);
// npm can satisfy the exact pin from a same-version local checkout. An
// empty owned cwd prevents a Git-installed plugin or caller project from
// shadowing the registry package with unbuilt/different local files.
let workingDirectory;
try {
workingDirectory = await mkdtemp(join(tmpdir(), "rea-plugin-launcher-"));
} catch (cause) {
process.stderr.write(
`REA plugin could not create its working directory: ${cause.message}\n`,
);
process.exitCode = 1;
return;
}
if (process.platform === "win32") {
await launchWindowsFallback(command, args, workingDirectory);
return;
}
const child = spawn(command, args, {
cwd: workingDirectory,
// A dedicated POSIX process group gives this launcher ownership of the
// package runner and any server it starts.
detached: true,
stdio: "inherit",
shell: false,
});
let spawnFailed = false;
const signals = new Map();
for (const signal of ["SIGINT", "SIGTERM", "SIGHUP"]) {
const forward = () => {
if (child.exitCode !== null || child.signalCode !== null) return;
try {
if (process.platform !== "win32" && child.pid !== undefined)
process.kill(-child.pid, signal);
else child.kill(signal);
} catch (cause) {
if (!isMissingProcess(cause))
process.stderr.write(
`REA plugin could not forward ${signal}: ${cause.message}\n`,
);
}
};
signals.set(signal, forward);
process.on(signal, forward);
}
const stopForwarding = () => {
for (const [signal, listener] of signals) process.off(signal, listener);
};
child.once("error", (cause) => {
spawnFailed = true;
stopForwarding();
process.stderr.write(
`REA plugin could not start ${command}: ${cause.message}\n`,
);
process.exitCode = 1;
});
child.once("close", async (code, signal) => {
stopForwarding();
let cleanupFailed = false;
try {
await rm(workingDirectory, { recursive: true });
} catch (cause) {
cleanupFailed = true;
process.stderr.write(
`REA plugin retained its working directory at ${workingDirectory}: ${cause.message}\n`,
);
}
if (spawnFailed) return;
if (signal !== null && process.platform !== "win32") {
// Preserve signal termination after the owned child has closed its streams.
process.kill(process.pid, signal);
} else {
process.exitCode =
code === 0 && cleanupFailed
? 1
: (code ?? 128 + (constants.signals[signal] ?? 1));
}
});
}
async function isSourceCheckout() {
// Published artifacts exclude src/. Also reject Git worktrees (whose .git is
// a file) and source exports where Git metadata has been stripped.
for (const path of ["../.git", "../src"]) {
try {
await access(new URL(path, import.meta.url));
return true;
} catch (cause) {
if (!isMissing(cause)) throw cause;
}
}
return false;
}
async function launchWindowsFallback(command, args, workingDirectory) {
let spawnOwnedProviderProcess;
try {
({ spawnOwnedProviderProcess } =
await import("../dist/process/ProviderProcess.js"));
} catch (cause) {
await rm(workingDirectory, { recursive: true }).catch(() => {});
process.stderr.write(
`REA Windows source plugins require npm ci && npm run build:cached and the Windows x64 native addon before loading the plugin. The owned process supervisor is unavailable: ${cause.message}\n`,
);
process.exitCode = 1;
return;
}
let owned;
try {
owned = await spawnOwnedProviderProcess({
command,
arguments: args,
runId: randomUUID(),
cwd: workingDirectory,
stdin: "pipe",
platform: "win32",
});
} catch (cause) {
await rm(workingDirectory, { recursive: true }).catch(() => {});
process.stderr.write(
`REA plugin could not start the owned Windows process tree: ${cause.message}\n`,
);
process.exitCode = 1;
return;
}
const child = owned.process;
child.stdout?.pipe(process.stdout);
child.stderr?.pipe(process.stderr);
if (child.stdin === null || child.stdin === undefined) {
const cleanup = await owned.cleanup?.();
if (cleanup?.cleaned === true)
await rm(workingDirectory, { recursive: true }).catch(() => {});
else
process.stderr.write(
`REA plugin retained its Windows process ownership directory at ${workingDirectory} because cleanup was not verified.\n`,
);
process.stderr.write(
"REA plugin Windows process supervisor did not provide protocol stdin.\n",
);
process.exitCode = 1;
return;
}
process.stdin.pipe(child.stdin);
let cleanupFailed = false;
let cleanupPromise;
const signals = new Map();
for (const signal of ["SIGINT", "SIGTERM", "SIGHUP"]) {
const forward = () => {
if (child.exitCode !== null || child.signalCode !== null) return;
cleanupPromise ??= owned
.cleanup?.()
.then((result) => {
if (!result.cleaned) {
cleanupFailed = true;
process.stderr.write(
`REA plugin could not verify Windows process-tree cleanup: ${result.reason}\n`,
);
}
})
.catch((cause) => {
cleanupFailed = true;
process.stderr.write(
`REA plugin Windows process-tree cleanup failed: ${cause.message}\n`,
);
});
};
signals.set(signal, forward);
process.on(signal, forward);
}
child.once("error", (cause) => {
process.stderr.write(
`REA plugin Windows process supervisor failed: ${cause.message}\n`,
);
process.exitCode = 1;
});
child.once("close", async (code) => {
for (const [signal, listener] of signals) process.off(signal, listener);
await cleanupPromise;
if (!cleanupFailed) {
try {
await rm(workingDirectory, { recursive: true });
} catch (cause) {
cleanupFailed = true;
process.stderr.write(
`REA plugin retained its working directory at ${workingDirectory}: ${cause.message}\n`,
);
}
}
process.exitCode = code === 0 && cleanupFailed ? 1 : (code ?? 1);
});
}
function isMissing(cause) {
return (
typeof cause === "object" &&
cause !== null &&
"code" in cause &&
cause.code === "ENOENT"
);
}
function isMissingProcess(cause) {
return (
typeof cause === "object" &&
cause !== null &&
"code" in cause &&
cause.code === "ESRCH"
);
}
+5 -10
View File
@@ -3,14 +3,13 @@ import {
requireMcpToolError,
} from "./lib/mcp-verifier-results.mjs";
import assert from "node:assert/strict";
import { dirname, join } from "node:path";
import { join } from "node:path";
import { execFile } from "node:child_process";
import { promisify } from "node:util";
import { readFile, realpath, rm } from "node:fs/promises";
import { readFile, rm } from "node:fs/promises";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { PRODUCT_IDENTITY } from "../dist/identity.js";
import { MCP_STARTUP_POLICY } from "../dist/mcpStartupPolicy.js";
import { parseEvidence } from "../dist/domain/evidence.js";
import * as prompts from "./verify-package-prompts.mjs";
@@ -247,7 +246,7 @@ const verifyMcpInlineArtifactEvidence = async (
}
};
/** Check CLI diagnostics, then verify catalog and Evidence through the installed plugin. */
/** Connect to the packaged MCP server and verify catalog and Evidence composition. */
export async function verifyPackageMcp({
cli,
environment,
@@ -268,13 +267,9 @@ export async function verifyPackageMcp({
diagnosed.inventory?.tools?.expected
)
throw new Error("packaged production MCP doctor failed");
const scriptsDirectory = dirname(await realpath(cli));
const transport = new StdioClientTransport({
command: process.execPath,
args: [
join(scriptsDirectory, "plugin-mcp-launcher.mjs"),
PRODUCT_IDENTITY.packageVersion,
],
command: cli,
args: ["mcp"],
env: {
...environment,
},
-70
View File
@@ -25,23 +25,6 @@ export async function verifyPackagePack({ root, workspace }) {
const packedManifest = JSON.parse(
(await exec("tar", ["-xOf", tarball, "package/package.json"])).stdout,
);
const packedLock = JSON.parse(
(await exec("tar", ["-xOf", tarball, "package/npm-shrinkwrap.json"]))
.stdout,
);
if (
packedManifest.overrides !== undefined ||
![2, 3].includes(packedLock.lockfileVersion) ||
packedLock.version !== packedManifest.version ||
packedLock.packages?.[""].version !== packedManifest.version ||
JSON.stringify(packedLock.packages?.[""].dependencies) !==
JSON.stringify(packedManifest.dependencies) ||
JSON.stringify(packedLock.packages?.[""].optionalDependencies) !==
JSON.stringify(packedManifest.optionalDependencies)
)
throw new Error(
"package is not installable under the Claude plugin dependency policy",
);
if (
packedManifest.scripts?.postinstall !== undefined ||
packedManifest.dependencies?.["node-pty"] !== undefined ||
@@ -59,10 +42,6 @@ export async function verifyPackagePack({ root, workspace }) {
throw new Error(
"package included authored skill sources instead of only the generated bundle",
);
if (packedFiles.some((path) => path.startsWith("package/src/")))
throw new Error(
"Packaged runtime must not masquerade as a source checkout",
);
if (packedFiles.some((path) => path.startsWith("package/.cache/")))
throw new Error("package included development cache or test metadata");
const skill = (
@@ -81,55 +60,6 @@ export async function verifyPackagePack({ root, workspace }) {
throw new Error(
"packaged skill metadata did not match the catalog shipped in the package",
);
for (const path of [
".claude-plugin/plugin.json",
".codex-plugin/plugin.json",
]) {
const manifest = JSON.parse(
(await exec("tar", ["-xOf", tarball, `package/${path}`])).stdout,
);
const serverConfig = path.startsWith(".codex")
? JSON.parse(
(await exec("tar", ["-xOf", tarball, "package/.mcp.json"])).stdout,
)
: manifest;
const expectedArgs = [
`${path.startsWith(".claude") ? "${CLAUDE_PLUGIN_ROOT}/" : "./"}scripts/plugin-mcp-launcher.mjs`,
packedManifest.version,
];
if (
manifest.version !== packedManifest.version ||
manifest.skills !== "./skills" ||
(path.startsWith(".codex")
? manifest.mcpServers !== "./.mcp.json" ||
serverConfig.mcpServers?.rea?.cwd !== "." ||
serverConfig.mcpServers?.rea?.startup_timeout_sec !== 30
: false) ||
serverConfig.mcpServers?.rea?.command !== "node" ||
JSON.stringify(serverConfig.mcpServers?.rea?.args) !==
JSON.stringify(expectedArgs)
)
throw new Error(
`Packaged plugin ${path} does not match the exact runtime and canonical skill release`,
);
}
if (!packedFiles.includes("package/.mcp.json"))
throw new Error("Packaged Codex MCP server declaration is missing");
for (const path of [
"scripts/plugin-mcp-launcher.mjs",
"dist/application/PackageRunnerCommand.mjs",
])
if (!packedFiles.includes(`package/${path}`))
throw new Error(
`Packaged plugin launcher dependency is missing: ${path}`,
);
if (
skill.includes(`${packedManifest.name}@latest`) ||
!skill.includes(`${packedManifest.name}@${packedManifest.version}`)
)
throw new Error(
"Packaged plugin skill recovery is not pinned to the shipped runtime",
);
if (
packedFiles.some(
(path) => path.includes("__pycache__") || path.endsWith(".pyc"),
@@ -2,7 +2,6 @@ import { basename, isAbsolute, resolve } from "node:path";
import { valid } from "semver";
import { PRODUCT_IDENTITY } from "../identity.js";
import { packageRunnerCommand } from "./PackageRunnerCommand.mjs";
/** Package runners whose pinned launcher setup can register. */
export type PackageRunner = "npx" | "bunx";
@@ -54,12 +53,14 @@ export const withoutPackageRunnerIdentity = (
export const packageRunnerRegistrationCommand = (
runner: PackageRunner,
platform: NodeJS.Platform = process.platform,
): readonly string[] =>
packageRunnerCommand(
runner,
PRODUCT_IDENTITY.registrationPackageSpecifier,
platform,
);
): readonly string[] => {
const command =
runner === "npx"
? ["npx", "-y", PRODUCT_IDENTITY.registrationPackageSpecifier, "mcp"]
: ["bunx", PRODUCT_IDENTITY.registrationPackageSpecifier, "mcp"];
// Both runners may be .cmd shims on Windows, which clients cannot spawn directly.
return platform === "win32" ? ["cmd.exe", "/d", "/c", ...command] : command;
};
/** Resolve a pinned npx launcher that stdio clients can spawn on the host. */
export const npxRegistrationCommand = (
@@ -1,6 +0,0 @@
/** Exact host argv for a pinned registry package's MCP entry point. */
export function packageRunnerCommand(
runner: "npx" | "bunx",
packageSpecifier: string,
platform: NodeJS.Platform,
): readonly string[];
-16
View File
@@ -1,16 +0,0 @@
/** Host argv shared by setup registrations and the source-only plugin launcher. */
export const packageRunnerCommand = (runner, packageSpecifier, platform) => {
if (runner !== "npx" && runner !== "bunx")
throw new Error("Unsupported package runner");
// Windows cmd.exe consumes these tokens itself. Package registrations use
// exact registry references, never shell expressions or arbitrary operands.
if (!/^[@A-Za-z0-9_./+-]+$/u.test(packageSpecifier))
throw new Error("Package runner requires a shell-safe registry reference");
const command =
runner === "npx"
? ["npx", "-y", packageSpecifier, "mcp"]
: ["bunx", packageSpecifier, "mcp"];
// npm's .cmd shim is not a native executable. Disable cmd AutoRun, matching
// the setup registration policy, and keep all dynamic values out of syntax.
return platform === "win32" ? ["cmd.exe", "/d", "/c", ...command] : command;
};
+8
View File
@@ -606,6 +606,14 @@ export const SUPPORTED_CLIENT_DEFINITIONS = [
markerPath: traeUserDirectory,
format: "json",
},
{
name: "workbuddy",
displayName: "WorkBuddy",
skillPath: [".workbuddy", "skills"],
configPath: [".workbuddy", "mcp.json"],
markerPath: [".workbuddy"],
format: "json",
},
{
name: "grok_build",
displayName: "Grok Build",
+36
View File
@@ -337,6 +337,42 @@ describe("call path topology and capacity", () => {
});
});
describe("directional call relationship coverage", () => {
it.each([
{
directions: ["callers"] as ["callers"],
status: "not_found",
exhaustive: true,
},
{
directions: ["callees"] as ["callees"],
status: "unknown",
exhaustive: false,
},
{
directions: ["callers", "callees"] as ["callers", "callees"],
status: "unknown",
exhaustive: false,
},
])(
"uses $directions coverage for the forward search",
({ directions, status, exhaustive }) => {
const limitation = "The inventory omitted a reference.";
const relationshipCoverage: RelationshipCoverage = {
status: "partial",
limitations: [limitation],
limited_directions: directions,
};
const result = run([
observe("0x1000", ["0x2000"], "rea-workflow", { relationshipCoverage }),
observe("0x2000", [], "rea-workflow", { relationshipCoverage }),
]);
expect(result).toMatchObject({ status, search_scope: { exhaustive } });
expect(result.limitations).toContain(limitation);
},
);
});
describe("explicit call relationship coverage", () => {
it("accumulates 150,000 partial reasons without argument-stack exhaustion", () => {
const result = run([
+2 -2
View File
@@ -10,7 +10,7 @@ import {
parseFunctionEvidence,
type FunctionSnapshot,
} from "./functionDossierEvidence.js";
import { isCallerOnlyUnknownCoverage } from "./functionCallCoverage.js";
import { isCallerOnlyIncompleteCoverage } from "./functionCallCoverage.js";
import { compareUnicodeCodePoints } from "./unicodeCodePointOrder.js";
/** Minimal directed caller-to-callee adjacency for call-path search. */
@@ -268,7 +268,7 @@ const inspectSearch = ({
coverage === null
? snapshot.limitations.length > 0
: coverage.status !== "complete" &&
!isCallerOnlyUnknownCoverage(coverage)
!isCallerOnlyIncompleteCoverage(coverage)
)
incompleteExpandedCoverage = true;
} else if (node === goal) {
+33 -7
View File
@@ -2,7 +2,7 @@ import { describe, expect, it } from "vitest";
import {
combineFunctionRelationshipCoverage,
isCallerOnlyUnknownCoverage,
isCallerOnlyIncompleteCoverage,
} from "./functionCallCoverage.js";
import type { RelationshipCoverage } from "./relationshipCoverage.js";
@@ -22,14 +22,14 @@ const cappedCallees: RelationshipCoverage = {
};
const complete: RelationshipCoverage = { status: "complete" };
describe("isCallerOnlyUnknownCoverage", () => {
describe("isCallerOnlyIncompleteCoverage", () => {
it("detects unknown callers beside complete callees", () => {
const combined = combineFunctionRelationshipCoverage(
unknownCallers,
complete,
);
expect(combined.status).toBe("unknown");
expect(isCallerOnlyUnknownCoverage(combined)).toBe(true);
expect(isCallerOnlyIncompleteCoverage(combined)).toBe(true);
});
it("rejects unknown callees beside complete callers", () => {
@@ -37,7 +37,7 @@ describe("isCallerOnlyUnknownCoverage", () => {
complete,
unknownCallees,
);
expect(isCallerOnlyUnknownCoverage(combined)).toBe(false);
expect(isCallerOnlyIncompleteCoverage(combined)).toBe(false);
});
it("rejects unknown coverage on both sides", () => {
@@ -45,7 +45,7 @@ describe("isCallerOnlyUnknownCoverage", () => {
unknownCallers,
unknownCallees,
);
expect(isCallerOnlyUnknownCoverage(combined)).toBe(false);
expect(isCallerOnlyIncompleteCoverage(combined)).toBe(false);
});
it("rejects unknown callers beside IDA-capped callees", () => {
@@ -54,11 +54,37 @@ describe("isCallerOnlyUnknownCoverage", () => {
cappedCallees,
);
expect(combined.status).toBe("unknown");
expect(isCallerOnlyUnknownCoverage(combined)).toBe(false);
expect(isCallerOnlyIncompleteCoverage(combined)).toBe(false);
});
it("rejects complete coverage", () => {
const combined = combineFunctionRelationshipCoverage(complete, complete);
expect(isCallerOnlyUnknownCoverage(combined)).toBe(false);
expect(isCallerOnlyIncompleteCoverage(combined)).toBe(false);
});
it("retains caller-only partial coverage without relying on limitation wording", () => {
const combined = combineFunctionRelationshipCoverage(
{ status: "partial", limitations: ["Missing incoming references."] },
complete,
);
expect(combined).toMatchObject({ limited_directions: ["callers"] });
expect(isCallerOnlyIncompleteCoverage(combined)).toBe(true);
});
it("uses explicit directions even when explanation text contains direction labels", () => {
expect(
isCallerOnlyIncompleteCoverage({
status: "unknown",
reason: "Callers: unavailable. Callees: diagnostic text.",
limited_directions: ["callers"],
}),
).toBe(true);
expect(
isCallerOnlyIncompleteCoverage({
status: "unknown",
reason: "Callers: unavailable.",
limited_directions: ["callees"],
}),
).toBe(false);
});
});
+27 -10
View File
@@ -22,6 +22,16 @@ export const combineFunctionRelationshipCoverage = (
),
);
const details = diagnostics.length > 0 ? { diagnostics } : {};
const callersLimited = callers?.status !== "complete";
const calleesLimited = callees?.status !== "complete";
const limited_directions: Extract<
RelationshipCoverage,
{ status: "partial" }
>["limited_directions"] = callersLimited
? calleesLimited
? ["callers", "callees"]
: ["callers"]
: ["callees"];
const reasons = reads.flatMap(({ direction, coverage }) => {
const label =
direction === "callers" ? callerCoveragePrefix : calleeCoveragePrefix;
@@ -43,30 +53,37 @@ export const combineFunctionRelationshipCoverage = (
return {
status: "unknown",
reason: uniqueSorted(reasons).join(" "),
limited_directions,
...details,
};
if (reasons.length > 0)
return {
status: "partial",
limitations: uniqueSorted(reasons),
limited_directions,
...details,
};
return { status: "complete", ...details };
};
/**
* Whether combined relationship coverage is unknown only on the caller side.
* The combine helper above prefixes each direction's reasons, so a reason
* that carries the caller prefix and no callee prefix records observed
* callees with unobservable callers. Forward (callee-edge) searches stay
* exhaustive under that caveat; the caller reason remains a limitation.
* Forward searches depend on callee coverage. Preserve the historical
* unknown-reason encoding when reading observations without explicit directions.
*/
export const isCallerOnlyUnknownCoverage = (
export const isCallerOnlyIncompleteCoverage = (
coverage: RelationshipCoverage,
): boolean =>
coverage.status === "unknown" &&
coverage.reason.startsWith(callerCoveragePrefix) &&
!coverage.reason.includes(calleeCoveragePrefix);
): boolean => {
if (coverage.status === "complete") return false;
if (coverage.limited_directions !== undefined)
return !coverage.limited_directions.some(
(direction) => direction === "callees",
);
return (
coverage.status === "unknown" &&
coverage.reason.startsWith(callerCoveragePrefix) &&
!coverage.reason.includes(calleeCoveragePrefix)
);
};
/** Explain why two resolved call sets cannot establish call-set equality. */
export const functionCallCoverageLimitations = (
@@ -0,0 +1,40 @@
import { describe, expect, it } from "vitest";
import { analyzeJavaScriptStaticSource } from "./javascriptStaticAnalysis.js";
const endpoints = (source: string) =>
analyzeJavaScriptStaticSource(source).endpoints.map(
({ kind, value, mechanism }) => ({ kind, value, mechanism }),
);
describe("server-sent event endpoints", () => {
it("reports EventSource URLs like WebSocket URLs", () => {
expect(
endpoints(`
new WebSocket("wss://chat.example/ws");
new EventSource("/api/chat/stream");
new window.EventSource("https://chat.example/events", { withCredentials: true });
`),
).toEqual([
{
kind: "network",
value: "wss://chat.example/ws",
mechanism: "call:WebSocket",
},
{
kind: "network",
value: "/api/chat/stream",
mechanism: "call:EventSource",
},
{
kind: "network",
value: "https://chat.example/events",
mechanism: "call:window.EventSource",
},
]);
});
it("keeps a dynamic EventSource URL unreported", () => {
expect(endpoints("new EventSource(url);")).toEqual([]);
});
});
@@ -0,0 +1,34 @@
import { describe, expect, it } from "vitest";
import { analyzeJavaScriptStaticSource } from "./javascriptStaticAnalysis.js";
const endpoints = (source: string) =>
analyzeJavaScriptStaticSource(source).endpoints.map(
({ kind, value, mechanism }) => ({ kind, value, mechanism }),
);
describe("beacon endpoints", () => {
it("reports navigator.sendBeacon URLs", () => {
expect(
endpoints(`
navigator.sendBeacon("/api/telemetry", payload);
window.navigator.sendBeacon("https://metrics.example/collect");
`),
).toEqual([
{
kind: "network",
value: "/api/telemetry",
mechanism: "call:navigator.sendBeacon",
},
{
kind: "network",
value: "https://metrics.example/collect",
mechanism: "call:window.navigator.sendBeacon",
},
]);
});
it("keeps a dynamic beacon URL unreported", () => {
expect(endpoints("navigator.sendBeacon(url, payload);")).toEqual([]);
});
});
@@ -554,7 +554,13 @@ export const endpointArgument = (
receiverFact?: JavaScriptOpenReceiverFact,
): string | undefined => {
if (receiverFact === "window") return undefined;
if (name === "fetch" || name.endsWith(".fetch") || name === "WebSocket")
if (
name === "fetch" ||
name.endsWith(".fetch") ||
name === "WebSocket" ||
name === "EventSource" ||
name.endsWith(".EventSource")
)
return stringValue(args[0]);
if (name.endsWith(".open"))
return xhrOpenUrl(name, args[0], args[1], receiverFact);
@@ -574,6 +580,9 @@ export const endpointArgument = (
? undefined
: value;
}
// navigator.sendBeacon(url, data) queues a POST to its first argument.
if (name === "sendBeacon" || name.endsWith(".sendBeacon"))
return stringValue(args[0]);
if (name.endsWith("loadURL")) return stringValue(args[0]);
return undefined;
};
+11
View File
@@ -0,0 +1,11 @@
import { expect, it } from "vitest";
import { parseRecordedTransportUrl } from "./recordedTransportUrl.js";
it("recognizes an empty userinfo marker at the authority start", () => {
expect(parseRecordedTransportUrl("https://@host")).toMatchObject({
origin: "https://host",
path: "/",
credentials: { start: 8, end: 9 },
});
});
+17 -6
View File
@@ -1,24 +1,35 @@
import { z } from "zod";
const diagnostics = z.array(z.string().min(1)).min(1);
const completeCoverage = z.strictObject({ status: z.literal("complete") });
const diagnostics = z.array(z.string().min(1)).min(1).exactOptional();
const completeCoverage = z.strictObject({
status: z.literal("complete"),
diagnostics,
});
const limitedDirections = z.union([
z.tuple([z.literal("callers")]).check(z.minLength(1), z.maxLength(1)),
z.tuple([z.literal("callees")]).check(z.minLength(1), z.maxLength(1)),
z
.tuple([z.literal("callers"), z.literal("callees")])
.check(z.minLength(2), z.maxLength(2)),
]);
const partialCoverage = z.strictObject({
status: z.literal("partial"),
limitations: z.array(z.string().min(1)).min(1),
limited_directions: limitedDirections.exactOptional(),
diagnostics,
});
const unknownCoverage = z.strictObject({
status: z.literal("unknown"),
reason: z.string().min(1),
limited_directions: limitedDirections.exactOptional(),
diagnostics,
});
/** Explicit coverage state for a provider's observed caller/callee set. */
export const relationshipCoverageSchema = z.union([
export const relationshipCoverageSchema = z.discriminatedUnion("status", [
completeCoverage,
completeCoverage.extend({ diagnostics }),
partialCoverage,
partialCoverage.extend({ diagnostics }),
unknownCoverage,
unknownCoverage.extend({ diagnostics }),
]);
export type RelationshipCoverage = z.infer<typeof relationshipCoverageSchema>;
+1 -1
View File
@@ -1,4 +1,4 @@
/** Generated from package.json and npm-shrinkwrap.json; do not edit. */
/** Generated from package.json and package-lock.json; do not edit. */
export const PACKAGE_METADATA = {
name: "rea-agents",
version: "6.4.0", // x-release-please-version
+4 -2
View File
@@ -40,6 +40,8 @@ import {
NATIVE_AOT_PROFILE_CONTRACT_REVISION,
} from "../domain/native/nativeAotPe.js";
const callRelationshipCoverage = "function-body-call-coverage-v5";
/**
* The processor language REA pins for this target, or undefined when Ghidra may
* select from the header. The committed profile and the session readback both
@@ -125,7 +127,7 @@ export const resolveGhidraAnalysisProfile = async (
load_image_observations: "source-mappings-entry-context-v2",
function_body_evidence: "complete-inclusive-ranges-v1",
function_references: "complete-body-and-entry-reference-manager-v2",
call_relationship_coverage: "resolved-call-reference-diagnostics-v1",
call_relationship_coverage: callRelationshipCoverage,
function_dossier_coverage:
"observed-facets-and-call-relationships-v1",
string_inventory_evidence: "defined-data-coverage-v1",
@@ -222,7 +224,7 @@ export const resolveGhidraAnalysisProfile = async (
load_image_observations: "source-mappings-entry-context-v2",
function_body_evidence: "complete-inclusive-ranges-v1",
function_references: "complete-body-and-entry-reference-manager-v2",
call_relationship_coverage: "resolved-call-reference-diagnostics-v1",
call_relationship_coverage: callRelationshipCoverage,
function_dossier_coverage: "observed-facets-and-call-relationships-v1",
string_inventory_evidence: "defined-data-coverage-v1",
location_resolution: "explicit-address-exact-entry-symbol-first-v3",
+113
View File
@@ -43,7 +43,120 @@ const parseFunctionValueFixture = (
return parsed.ok ? { ok: true as const, value: parsed.value.value } : parsed;
};
describe("Ghidra unresolved call locations", () => {
it("rejects a noncanonical unresolved call site rather than retaining unverifiable location evidence", () => {
expect(
parseGhidraFunctionResult("analyze_function", {
value: {
...functionDossierSchema.parse(ghidraFunctionDossier()),
unresolved_calls: [
{ address: "not-an-address", reason: "No call destination." },
],
},
limitations: [],
relationship_coverage: {
status: "partial",
limitations: ["An unresolved call remains."],
},
}),
).toMatchObject({ ok: false, error: { _tag: "AnalysisOutputError" } });
});
});
describe("Ghidra function-analysis result values", () => {
it("retains unavailable pseudocode instead of treating its empty representation as observed source", () => {
const availability = {
pseudocode: {
available: false,
reason: "Ghidra does not decompile external functions.",
},
comments: { available: true },
};
const parsed = parseGhidraFunctionResult("analyze_function", {
value: {
...functionDossierSchema.parse(ghidraFunctionDossier()),
pseudocode: "",
facet_availability: availability,
},
limitations: [],
});
expect(parsed).toMatchObject({
ok: true,
value: { value: { pseudocode: "", facet_availability: availability } },
});
});
it.each([
{ status: "complete", diagnostics: ["Resolved static references only."] },
{
status: "partial",
limitations: ["Only the first instruction range was examined."],
},
{ status: "unknown", reason: "Relationship inventory was unavailable." },
{
status: "partial",
limitations: ["Missing callers."],
limited_directions: ["callers"],
},
{
status: "unknown",
reason: "Missing callees.",
limited_directions: ["callees"],
},
])("preserves observed $status relationship coverage", (coverage) => {
const parsed = parseGhidraFunctionResult("analyze_function", {
value: ghidraFunctionDossier(),
limitations: [],
relationship_coverage: coverage,
});
expect(parsed).toMatchObject({
ok: true,
value: { relationshipCoverage: coverage },
});
});
it("preserves omitted relationship coverage for legacy observations", () => {
const parsed = parseGhidraFunctionResult("analyze_function", {
value: ghidraFunctionDossier(),
limitations: [],
});
if (!parsed.ok) throw parsed.error;
expect(parsed.value).not.toHaveProperty("relationshipCoverage");
});
});
describe("Ghidra relationship metadata validation", () => {
it.each([
{ status: "partial", limitations: [] },
{ status: "unknown", reason: "" },
{ status: "complete", limitations: ["Truncated."] },
{ status: "partial", limitations: ["Truncated."], limited_directions: [] },
{
status: "partial",
limitations: ["Truncated."],
limited_directions: ["callers", "callers"],
},
{ status: "unknown", reason: "Missing.", limited_directions: ["incoming"] },
{ status: "complete", limited_directions: ["callers"] },
])("rejects malformed relationship coverage %j", (coverage) => {
expect(
parseGhidraFunctionResult("analyze_function", {
value: ghidraFunctionDossier(),
limitations: [],
relationship_coverage: coverage,
}),
).toMatchObject({ ok: false, error: { _tag: "AnalysisOutputError" } });
});
it("rejects relationship metadata on unrelated function operations", () => {
expect(
parseGhidraFunctionResult("procedure_assembly", {
value: "ret",
limitations: [],
relationship_coverage: { status: "complete" },
}),
).toMatchObject({ ok: false, error: { _tag: "AnalysisOutputError" } });
});
it("preserves file-offset resolution in Evidence locations and rejects inexact offsets", () => {
const selection = {
file_offset: 0x210,
+16 -1
View File
@@ -27,6 +27,10 @@ import {
import { nativeApiBoundarySchema } from "../domain/native/nativeApiBoundary.js";
import { nativeValueFlowSchema } from "../domain/native/nativeValueFlow.js";
import type { EvidenceLocation } from "../domain/evidence.js";
import {
relationshipCoverageSchema,
type RelationshipCoverage,
} from "../domain/relationshipCoverage.js";
import { err, ok, type Result } from "../domain/result.js";
import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js";
import {
@@ -307,6 +311,7 @@ const ghidraFunctionDossier = functionDossierSchema
...value.comments.map(({ address }) => address),
...value.callers.map(({ address }) => address),
...value.callees.map(({ address }) => address),
...(value.unresolved_calls ?? []).map(({ address }) => address),
...value.incoming_references.flatMap(referenceAddresses),
...value.outgoing_references.flatMap(referenceAddresses),
...value.referenced_strings.flatMap(({ address, source_address }) => [
@@ -419,6 +424,7 @@ const functionBoundaryObservationOperations: ReadonlySet<GhidraFunctionOperation
const functionObservationEnvelope = z.strictObject({
value: z.unknown(),
limitations: z.array(z.string()),
relationship_coverage: relationshipCoverageSchema.exactOptional(),
selection: z
.strictObject({
file_offset: z.number().int().min(0).max(Number.MAX_SAFE_INTEGER),
@@ -432,6 +438,7 @@ export interface ParsedGhidraFunctionResult {
readonly value: JsonValue;
readonly limitations: readonly string[];
readonly locations?: readonly EvidenceLocation[];
readonly relationshipCoverage?: RelationshipCoverage;
}
/** Require exact, bounded Java-bridge function output before creating Evidence. */
@@ -442,6 +449,7 @@ export const parseGhidraFunctionResult = (
let observation: {
readonly value: unknown;
readonly limitations: readonly string[];
readonly relationship_coverage?: RelationshipCoverage;
readonly selection?: z.infer<
typeof functionObservationEnvelope
>["selection"];
@@ -450,13 +458,17 @@ export const parseGhidraFunctionResult = (
const envelope = functionObservationEnvelope.safeParse(value);
if (
!envelope.success ||
(operation !== "analyze_function" &&
operation !== "procedure_callers" &&
operation !== "procedure_callees" &&
envelope.data.relationship_coverage !== undefined) ||
(operation !== "procedure_pseudo_code" &&
envelope.data.selection !== undefined)
)
return err(
new AnalysisOutputError(
operation,
"Ghidra bridge omitted the function observation envelope",
"Ghidra bridge returned an invalid function observation envelope for this operation",
envelope.success ? undefined : { cause: envelope.error },
),
);
@@ -469,6 +481,9 @@ export const parseGhidraFunctionResult = (
? ok({
value: jsonValueSchema.parse(parsed.data),
limitations: observation.limitations,
...(observation.relationship_coverage === undefined
? {}
: { relationshipCoverage: observation.relationship_coverage }),
...(observation.selection === undefined
? {}
: {
+11 -2
View File
@@ -633,7 +633,7 @@ describe("Ghidra client projection", () => {
expect(resolved.value.profile.parameters).toMatchObject({
language_id: "auto-from-header",
compiler_spec_id: "auto-default",
call_relationship_coverage: "resolved-call-reference-diagnostics-v1",
call_relationship_coverage: "function-body-call-coverage-v5",
function_dossier_coverage: "observed-facets-and-call-relationships-v1",
});
@@ -870,7 +870,16 @@ describe("Ghidra relationship coverage", () => {
Promise.resolve(
ok(
operation === "procedure_callees"
? { value: ["0x2000"], limitations: [] }
? {
value: ["0x2000"],
limitations: [],
relationship_coverage: {
status: "complete",
diagnostics: [
"Only resolved Ghidra call references are returned; unresolved computed or indirect calls remain unknown, while function classifications distinguish thunks and externals.",
],
},
}
: { value: [], limitations: [] },
),
),
-17
View File
@@ -2,7 +2,6 @@ import type {
CapabilityDescriptor,
ProviderIdentity,
} from "../application/AnalysisProvider.js";
import type { RelationshipCoverage } from "../domain/relationshipCoverage.js";
import {
hasWindowsNativeAuthority,
windowsNativeAuthorityUnavailableReason,
@@ -44,22 +43,6 @@ const stringInventoryLimitation =
const searchLimitation =
"Regex mode uses Java Pattern semantics. Matcher or compiler stack exhaustion returns a resource constraint without closing the session; literal mode avoids regex recursion and returns complete matching values.";
const resolvedCallReferenceLimitation =
"Only resolved Ghidra call references are returned; unresolved computed or indirect calls remain unknown, while function classifications distinguish thunks and externals.";
/** Relationship-only caveats belong to each observed caller/callee result. */
export const relationshipCoverageFor = (
operation: string,
): RelationshipCoverage | undefined =>
operation === "procedure_callers" ||
operation === "procedure_callees" ||
operation === "analyze_function"
? {
status: "complete",
diagnostics: [resolvedCallReferenceLimitation],
}
: undefined;
/** Limitation text for one admitted operation, including the common base. */
export const limitationsFor = (operation: string): readonly string[] => {
const common = [
+5 -2
View File
@@ -34,6 +34,7 @@ import {
} from "../domain/analysisErrorCore.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import type { EvidenceLocation } from "../domain/evidence.js";
import type { RelationshipCoverage } from "../domain/relationshipCoverage.js";
import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js";
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
import { err, ok, type Result } from "../domain/result.js";
@@ -86,7 +87,6 @@ import {
healthLimitations,
windowsP0Limitations,
limitationsFor,
relationshipCoverageFor,
} from "./GhidraProviderCapabilities.js";
import type { GhidraSessionError } from "./GhidraSessionError.js";
import {
@@ -733,8 +733,8 @@ export const createGhidraProviderClient = (input: {
rawResult: JsonValue,
observationLimitations: readonly string[] = [],
locations: readonly EvidenceLocation[] = [],
relationshipCoverage?: RelationshipCoverage,
) => {
const relationshipCoverage = relationshipCoverageFor(operation);
return createAnalysisExecution(normalized, committedProfile.provider, {
rawResult,
locations,
@@ -838,12 +838,14 @@ export const createGhidraProviderClient = (input: {
let normalized: JsonValue;
let observationLimitations: readonly string[] = [];
let observationLocations: readonly EvidenceLocation[] = [];
let relationshipCoverage: RelationshipCoverage | undefined;
if (isGhidraFunctionOperation(operation)) {
const result = parseGhidraFunctionResult(operation, called.value);
if (!result.ok) return result;
normalized = result.value.value;
observationLimitations = result.value.limitations;
observationLocations = result.value.locations ?? [];
relationshipCoverage = result.value.relationshipCoverage;
} else {
const result = parseGhidraInventoryResult(operation, called.value);
if (!result.ok) return result;
@@ -939,6 +941,7 @@ export const createGhidraProviderClient = (input: {
called.value,
observationLimitations,
observationLocations,
relationshipCoverage,
),
);
},
+121 -37
View File
@@ -11,6 +11,7 @@ import { silentLogger } from "../logger.js";
import { GhidraProvider } from "./GhidraProvider.js";
import { GhidraSessionError } from "./GhidraSessionError.js";
import { GHIDRA_SESSION_CAPABILITIES } from "./GhidraSessionValues.js";
import type { RelationshipCoverage } from "../domain/relationshipCoverage.js";
const TARGET: BinaryTarget = {
path: "/tmp/ghidra-cache-fixture",
@@ -23,7 +24,7 @@ const TARGET: BinaryTarget = {
// Exercise the real provider descriptors, profile, input/output projection and
// session cache; only the installed Ghidra process is replaced at its seam.
const fixture = (version = "12.1.4") => {
const fixture = (version = "12.1.4", coverage?: RelationshipCoverage) => {
const config = parseConfig({ GHIDRA_INSTALL_DIR: "/opt/ghidra" });
if (!config.ok) throw config.error;
const calls: string[] = [];
@@ -113,7 +114,13 @@ const fixture = (version = "12.1.4") => {
return ok(
operation === "procedure_pseudo_code" ||
operation === "analyze_function"
? { value, limitations: [] }
? {
value,
limitations: [],
...(operation === "analyze_function" && coverage !== undefined
? { relationship_coverage: coverage }
: {}),
}
: value,
);
},
@@ -133,42 +140,119 @@ const open = async (session: ReturnType<typeof fixture>["session"]) => {
expect((await session.openResolvedTarget(preview.value)).ok).toBe(true);
};
describe("Ghidra snapshot cache", () => {
it("refuses snapshots whose profile predates typed incoming references", async () => {
const initial = fixture();
await open(initial.session);
const snapshot = initial.session.exportAnalysisSnapshot();
if (!snapshot.ok) throw snapshot.error;
await initial.session.close();
const profile = snapshot.value.binding.analysis_profile;
expect(profile.parameters.instruction_references).toBe(
"exact-address-incoming-v1",
);
const legacy = {
...snapshot.value,
binding: {
...snapshot.value.binding,
analysis_profile: createAnalysisProfile(
profile.provider,
Object.fromEntries(
Object.entries(profile.parameters).filter(
([key]) => key !== "instruction_references",
),
),
),
},
};
const replay = fixture();
const imported = replay.session.importAnalysisSnapshot(legacy);
if (!imported.ok) throw imported.error;
expect(await replay.session.previewTarget(TARGET)).toMatchObject({
ok: false,
error: { _tag: "EvidenceIntegrityError" },
});
expect(replay.calls).toEqual([]);
await replay.session.close();
});
describe("Ghidra relationship coverage replay", () => {
it.each([
undefined,
{ status: "complete", diagnostics: ["Resolved static references only."] },
{ status: "partial", limitations: ["Instruction coverage was truncated."] },
{ status: "unknown", reason: "Relationship inventory was unavailable." },
{
status: "partial",
limitations: ["Missing callers."],
limited_directions: ["callers"],
},
] satisfies (RelationshipCoverage | undefined)[])(
"preserves relationship coverage %j through live execution, cache and snapshot replay",
async (coverage) => {
const initial = fixture("12.1.4", coverage);
await open(initial.session);
const parameters = { procedure: "0x1000" };
for (let i = 0; i < 2; i++) {
const execution = await initial.session.execute(
"analyze_function",
parameters,
);
if (!execution.ok) throw execution.error;
expect(execution.value.relationshipCoverage).toEqual(coverage);
}
expect(initial.calls).toEqual(["analyze_function"]);
const snapshot = initial.session.exportAnalysisSnapshot();
if (!snapshot.ok) throw snapshot.error;
await initial.session.close();
const replay = fixture();
expect(replay.session.importAnalysisSnapshot(snapshot.value).ok).toBe(
true,
);
await open(replay.session);
const execution = await replay.session.execute(
"analyze_function",
parameters,
);
if (!execution.ok) throw execution.error;
expect(execution.value.relationshipCoverage ?? undefined).toEqual(
coverage,
);
expect(replay.calls).toEqual([]);
await replay.session.close();
},
);
});
describe("Ghidra snapshot profile admission", () => {
it.each([
{
parameter: "instruction_references",
current: "exact-address-incoming-v1",
legacy: undefined,
},
{
parameter: "call_relationship_coverage",
current: "function-body-call-coverage-v5",
legacy: "resolved-call-reference-diagnostics-v1",
},
{
parameter: "call_relationship_coverage",
current: "function-body-call-coverage-v5",
legacy: "decoded-call-target-coverage-v2",
},
{
parameter: "call_relationship_coverage",
current: "function-body-call-coverage-v5",
legacy: "decoded-call-target-coverage-v3",
},
{
parameter: "call_relationship_coverage",
current: "function-body-call-coverage-v5",
legacy: "function-body-call-coverage-v4",
},
])(
"refuses snapshots with outdated $parameter semantics",
async ({ parameter, current, legacy: prior }) => {
const initial = fixture();
await open(initial.session);
const snapshot = initial.session.exportAnalysisSnapshot();
if (!snapshot.ok) throw snapshot.error;
await initial.session.close();
const profile = snapshot.value.binding.analysis_profile;
expect(profile.parameters[parameter]).toBe(current);
const legacy = {
...snapshot.value,
binding: {
...snapshot.value.binding,
analysis_profile: createAnalysisProfile(profile.provider, {
...Object.fromEntries(
Object.entries(profile.parameters).filter(
([key]) => key !== parameter,
),
),
...(prior === undefined ? {} : { [parameter]: prior }),
}),
},
};
const replay = fixture();
const imported = replay.session.importAnalysisSnapshot(legacy);
if (!imported.ok) throw imported.error;
expect(await replay.session.previewTarget(TARGET)).toMatchObject({
ok: false,
error: { _tag: "EvidenceIntegrityError" },
});
expect(replay.calls).toEqual([]);
await replay.session.close();
},
);
});
describe("Ghidra snapshot cache", () => {
it.each([
{ operation: "analyze_function", parameters: { procedure: "0x1000" } },
{
@@ -130,7 +130,7 @@ printf '%s\n' 'C:\Users\runner with spaces\AppData\Local\npm-cache'
"${{ github.event.repository.default_branch }}",
);
expect(policy?.env?.LOCKFILES_HASH).toBe(
"${{ hashFiles('npm-shrinkwrap.json', '.npmrc') }}",
"${{ hashFiles('package-lock.json', 'npm-shrinkwrap.json', '.npmrc') }}",
);
expect(policy?.run).toContain(
'cache_key="npm-download-${RUNNER_OS}-${runtime_arch}-${LOCKFILES_HASH}"',
@@ -634,7 +634,7 @@ it.each([
{ code: true, package: true, evm: true, browser: true },
],
[
"npm-shrinkwrap.json",
"package-lock.json",
{ code: true, package: true, evm: true, browser: true },
],
[
@@ -67,6 +67,7 @@ describe("client configuration filesystem lifecycle", () => {
"kimi",
"vscode",
"trae",
"workbuddy",
"grok_build",
"omp",
"pi",
@@ -23,7 +23,7 @@ const fixture = async (
const root = await createTestTempDirectory("rea-deps-");
await mkdir(join(root, "node_modules"));
await writeFile(
join(root, "npm-shrinkwrap.json"),
join(root, "package-lock.json"),
JSON.stringify({
packages: {
"": {
@@ -103,7 +103,7 @@ describe("dependency install freshness", () => {
expect(seeded.status, `${seeded.stdout}${seeded.stderr}`).toBe(0);
const rootLock = asRecord(
JSON.parse(await readFile(join(root, "npm-shrinkwrap.json"), "utf8")),
JSON.parse(await readFile(join(root, "package-lock.json"), "utf8")),
);
const lockPackages = asRecord(rootLock?.packages);
const rootPackage = asRecord(lockPackages?.[""]);
@@ -191,7 +191,7 @@ const createBuildCacheFixture = async (): Promise<string> => {
throw new Error("package is missing its cached build command");
const paths = [
".nvmrc",
"npm-shrinkwrap.json",
"package-lock.json",
"turbo.json",
"tsconfig.json",
"tsconfig.build.json",
@@ -1,5 +1,5 @@
import { cp, mkdir, readFile, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { afterAll, expect, it } from "vitest";
import { z } from "zod";
@@ -19,32 +19,6 @@ const verifier = fileURLToPath(
new URL("../../../scripts/verify-release-checkpoint.mjs", import.meta.url),
);
const pluginManifestPaths = [
".claude-plugin/plugin.json",
".codex-plugin/plugin.json",
] as const;
const pluginManifest = (version: string, launch = version) => ({
version,
mcpServers: {
rea: {
command: "node",
args: ["${CLAUDE_PLUGIN_ROOT}/scripts/plugin-mcp-launcher.mjs", launch],
},
},
});
const codexServerConfig = (version: string, launch = version) => ({
mcpServers: {
rea: {
command: "node",
args: ["./scripts/plugin-mcp-launcher.mjs", launch],
cwd: ".",
startup_timeout_sec: 30,
},
},
});
async function metadata(
directory: string,
version: string,
@@ -53,17 +27,9 @@ async function metadata(
) {
await mkdir(join(directory, "docs"), { recursive: true });
await mkdir(join(directory, "src"), { recursive: true });
for (const path of pluginManifestPaths)
await mkdir(join(directory, dirname(path)), { recursive: true });
const files = {
".claude-plugin/plugin.json": pluginManifest(version),
".codex-plugin/plugin.json": {
version,
mcpServers: "./.mcp.json",
},
".mcp.json": codexServerConfig(version),
"package.json": { name: "rea-agents", version },
"npm-shrinkwrap.json": { version, packages: { "": { version } } },
"package-lock.json": { version, packages: { "": { version } } },
".release-please-manifest.json": { ".": version },
"server.json": {
version,
@@ -568,7 +534,7 @@ it("validates candidate options before inspecting the working directory", async
});
it.each([
"npm-shrinkwrap.json",
"package-lock.json",
"server.json",
"src/generatedPackageMetadata.ts",
])("rejects inconsistent version metadata in %s", async (path) => {
@@ -579,7 +545,7 @@ it.each([
path.endsWith(".ts")
? 'export const METADATA = {\n version: "5.1.0",\n};\n'
: JSON.stringify(
path === "npm-shrinkwrap.json"
path === "package-lock.json"
? { version: "6.0.0", packages: { "": { version: "5.1.0" } } }
: path === "server.json"
? {
@@ -599,51 +565,6 @@ it.each([
});
});
it.each(
pluginManifestPaths.map((path) => ({
path,
stale: "version",
manifest:
path === ".codex-plugin/plugin.json"
? { version: "5.1.0", mcpServers: "./.mcp.json" }
: pluginManifest("5.1.0"),
failure: "declares 5.1.0; release branch requires 6.0.0",
})),
)("rejects a plugin $stale in $path", async ({ path, manifest, failure }) => {
const f = await fixture();
await candidate(f, "6.0.0");
await writeFile(join(f.directory, path), JSON.stringify(manifest));
await git(f.directory, ["add", "."]);
await git(f.directory, ["commit", "-m", "chore: stale plugin manifest"]);
await expect(verify(f)).rejects.toMatchObject({
code: 1,
stderr: expect.stringContaining(failure),
});
});
it.each([
{
path: ".claude-plugin/plugin.json",
manifest: pluginManifest("6.0.0", "5.1.0"),
},
{
path: ".mcp.json",
manifest: codexServerConfig("6.0.0", "5.1.0"),
},
])("reads the release pin from $path", async ({ path, manifest }) => {
const f = await fixture();
await candidate(f, "6.0.0");
await writeFile(join(f.directory, path), JSON.stringify(manifest));
await git(f.directory, ["add", "."]);
await git(f.directory, ["commit", "-m", "chore: stale plugin launch"]);
await expect(verify(f)).rejects.toMatchObject({
code: 1,
stderr: expect.stringContaining(
`launches version 5.1.0; release branch requires 6.0.0`,
),
});
});
it("checks a publish merge using its pre-release first parent baseline", async () => {
const f = await fixture();
const sourceSha = await candidate(f, "6.0.0");
+2 -182
View File
@@ -6,7 +6,7 @@ import {
readFile,
writeFile,
} from "node:fs/promises";
import { delimiter, dirname, join } from "node:path";
import { delimiter, join } from "node:path";
import { fileURLToPath } from "node:url";
import { afterAll, beforeAll, expect, it } from "vitest";
import { z } from "zod";
@@ -43,59 +43,6 @@ interface ReleaseNotesSeed {
let seed: ReleaseNotesSeed | undefined;
let seedWorkspaceRoot: string | undefined;
const pluginManifests = [
".claude-plugin/plugin.json",
".codex-plugin/plugin.json",
] as const;
const pluginServerConfig = ".mcp.json";
const pluginMarketplaces = [
".claude-plugin/marketplace.json",
".agents/plugins/marketplace.json",
] as const;
const pluginMarketplace = (version: string, packageName = "rea-agents") => ({
plugins: [
{ name: "rea", source: { source: "npm", package: packageName, version } },
],
});
const pluginServer = (version: string, command = "node") => ({
mcpServers: {
rea: {
command,
args: ["./scripts/plugin-mcp-launcher.mjs", version],
startup_timeout_sec: 30,
},
},
});
const pluginManifest = (
version: string,
{ skills = ["./skill"], command = "node" } = {},
) => ({
version,
skills,
mcpServers: {
rea: {
command,
args: ["${CLAUDE_PLUGIN_ROOT}/scripts/plugin-mcp-launcher.mjs", version],
},
},
});
const pluginManifestFor = (
path: (typeof pluginManifests)[number],
version: string,
options: { skills?: string[]; command?: string } = {},
) =>
path === ".codex-plugin/plugin.json"
? {
version,
skills: options.skills ?? ["./skill"],
mcpServers: "./.mcp.json",
}
: pluginManifest(version, options);
beforeAll(async () => {
const workspace = await createTestWorkspace("rea-release-notes-seed-");
seedWorkspaceRoot = workspace.root;
@@ -111,24 +58,6 @@ beforeAll(async () => {
join(directory, ".release-please-manifest.json"),
JSON.stringify({ ".": "6.3.0" }),
);
for (const path of pluginManifests) {
await mkdir(join(directory, dirname(path)), { recursive: true });
await writeFile(
join(directory, path),
JSON.stringify(pluginManifestFor(path, "6.3.0")),
);
}
await writeFile(
join(directory, pluginServerConfig),
JSON.stringify(pluginServer("6.3.0")),
);
for (const path of pluginMarketplaces) {
await mkdir(join(directory, dirname(path)), { recursive: true });
await writeFile(
join(directory, path),
JSON.stringify(pluginMarketplace("6.3.0")),
);
}
await git(directory, ["add", "."]);
await git(directory, ["commit", "-m", "chore: baseline"]);
const base = await git(directory, ["rev-parse", "HEAD"]);
@@ -439,30 +368,9 @@ it("rejects lost grouped credits, fabricated references, bots, and stale PR bodi
it("binds publication to complete recorded history and the reviewed base", async () => {
const f = await fixture();
for (const path of pluginManifests)
await writeFile(
join(f.directory, path),
JSON.stringify(pluginManifestFor(path, "6.4.0")),
);
await writeFile(
join(f.directory, pluginServerConfig),
JSON.stringify(pluginServer("6.4.0")),
);
for (const path of pluginMarketplaces)
await writeFile(
join(f.directory, path),
JSON.stringify(pluginMarketplace("6.4.0")),
);
await git(f.directory, [
"add",
"CHANGELOG.md",
...pluginManifests,
pluginServerConfig,
...pluginMarketplaces,
]);
await git(f.directory, ["add", "CHANGELOG.md"]);
await git(f.directory, ["commit", "-m", "chore: release notes"]);
const source = await git(f.directory, ["rev-parse", "HEAD"]);
// Release PRs bump the plugin manifest version and pinned launch only.
await f.invoke(["check", "--manifest", f.manifest, "--source", source]);
await writeFile(join(f.directory, "product.txt"), "later product change");
await git(f.directory, ["add", "product.txt"]);
@@ -483,74 +391,6 @@ it("binds publication to complete recorded history and the reviewed base", async
});
});
it.each(pluginManifests)(
"treats a release that changes the skill path in %s as a product change",
async (path) => {
const f = await fixture();
await writeFile(
join(f.directory, path),
JSON.stringify(
pluginManifestFor(path, "6.4.0", { skills: ["./another-skill"] }),
),
);
await git(f.directory, ["add", "CHANGELOG.md", path]);
await git(f.directory, ["commit", "-m", "chore: release notes"]);
await expect(
f.invoke(["check", "--manifest", f.manifest, "--source", "HEAD"]),
).rejects.toMatchObject({
stderr: expect.stringContaining(
`Product changed after notes target; regenerate inventory: ${path}`,
),
});
},
);
it("accepts Release Please lockfile refreshes without allowing product changes", async () => {
const f = await fixture();
await writeFile(
join(f.directory, "package-lock.json"),
'{"lockfileVersion":3}',
);
await writeFile(
join(f.directory, "npm-shrinkwrap.json"),
'{"lockfileVersion":3}',
);
await git(f.directory, [
"add",
"CHANGELOG.md",
"package-lock.json",
"npm-shrinkwrap.json",
]);
await git(f.directory, ["commit", "-m", "chore: release lockfiles"]);
await f.invoke(["check", "--manifest", f.manifest, "--source", "HEAD"]);
});
it.each([
{
path: ".claude-plugin/plugin.json",
content: pluginManifest("6.4.0", { command: "other-launcher" }),
},
{
path: pluginServerConfig,
content: pluginServer("6.4.0", "other-launcher"),
},
])(
"keeps launcher behavior in $path outside version-only metadata",
async ({ path, content }) => {
const f = await fixture();
await writeFile(join(f.directory, path), JSON.stringify(content));
await git(f.directory, ["add", "CHANGELOG.md", path]);
await git(f.directory, ["commit", "-m", "chore: release notes"]);
await expect(
f.invoke(["check", "--manifest", f.manifest, "--source", "HEAD"]),
).rejects.toMatchObject({
stderr: expect.stringContaining(
`Product changed after notes target; regenerate inventory: ${path}`,
),
});
},
);
it("checks the live merged PR and exact Git snapshot before either publication route", async () => {
const f = await fixture();
await mkdir(join(f.directory, "docs/releases"), { recursive: true });
@@ -696,23 +536,3 @@ it("fails closed on partial GitHub errors and non-advancing pagination without s
});
await expect(access(output)).rejects.toMatchObject({ code: "ENOENT" });
});
it.each(pluginMarketplaces)(
"treats a marketplace package change in %s as product behavior",
async (path) => {
const f = await fixture();
await writeFile(
join(f.directory, path),
JSON.stringify(pluginMarketplace("6.4.0", "another-package")),
);
await git(f.directory, ["add", "CHANGELOG.md", path]);
await git(f.directory, ["commit", "-m", "chore: release notes"]);
await expect(
f.invoke(["check", "--manifest", f.manifest, "--source", "HEAD"]),
).rejects.toMatchObject({
stderr: expect.stringContaining(
`Product changed after notes target; regenerate inventory: ${path}`,
),
});
},
);
@@ -78,7 +78,7 @@ const writeTaskFixture = async (
}),
);
await writeFile(
join(root, "npm-shrinkwrap.json"),
join(root, "package-lock.json"),
JSON.stringify({
name: "rea-static-task-cache-fixture",
version: "1.0.0",
@@ -288,38 +288,3 @@ it("invalidates checker task hashes for Node, ignore, and VitePress config chang
`${task} must ignore vitest.config.ts`,
).toBe(initial.get(task));
});
it.each(["compile", "typecheck"])(
"invalidates %s for JavaScript owners, declarations, and the authoritative lock",
async (task) => {
const root = await createTestTempDirectory("rea-mjs-cache-inputs-");
const cache = await createTestTempDirectory("rea-mjs-cache-store-");
await mkdir(join(root, "src/application"), { recursive: true });
await writeTaskFixture(root, [task], {
[task]: 'node -e "process.exit(0)"',
});
const owner = join(root, "src/application/PackageRunnerCommand.mjs");
const declaration = join(
root,
"src/application/PackageRunnerCommand.d.mts",
);
await writeFile(owner, "export const command = 1;\n");
await writeFile(declaration, "export declare const command: number;\n");
await workflowGit(root, ["init", "--quiet"]);
await workflowGit(root, ["add", "."]);
const initial = taskHash(root, task, cache);
expect(taskHash(root, task, cache)).toBe(initial);
await writeFile(owner, "export const command = 2;\n");
expect(taskHash(root, task, cache)).not.toBe(initial);
await writeFile(owner, "export const command = 1;\n");
await writeFile(declaration, "export declare const command: string;\n");
expect(taskHash(root, task, cache)).not.toBe(initial);
await writeFile(declaration, "export declare const command: number;\n");
expect(taskHash(root, task, cache)).toBe(initial);
const lockPath = join(root, "npm-shrinkwrap.json");
const lock = JSON.parse(await readFile(lockPath, "utf8"));
lock.packages["node_modules/fixture"] = { version: "1.0.0" };
await writeFile(lockPath, JSON.stringify(lock));
expect(taskHash(root, task, cache)).not.toBe(initial);
},
);
@@ -28,7 +28,7 @@ test("generates locked inventory and rejects stale notice metadata or preserved
optional: true,
};
await writeFile(
resolve(root, "npm-shrinkwrap.json"),
resolve(root, "package-lock.json"),
JSON.stringify({
packages: {
"": { name: "fixture", version: "1.0.0" },
@@ -0,0 +1,159 @@
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { readClientRegistrationStatuses } from "../../../src/application/ClientRegistrationStatus.js";
import { npxRegistrationCommand } from "../../../src/application/ClientRegistrationIdentity.js";
import { configureClientConfiguration } from "../../../src/application/SetupClientConfiguration.js";
import { skillDestinations } from "../../../src/application/SetupSkill.js";
import { supportedClients } from "../../../src/application/SupportedClients.js";
import { systemUninstallHost } from "../../../src/application/Uninstall.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { clearClientLocationEnvironment } from "../../fixtures/clientEnvironment.js";
/**
* The registration command setup writes on the host running the tests. Windows
* wraps it in `cmd.exe /d /c`, so the expected entry follows the host too.
*/
const command = npxRegistrationCommand(process.platform);
beforeEach(clearClientLocationEnvironment);
afterEach(() => vi.unstubAllEnvs());
const workbuddyPath = (
home: string,
platform: NodeJS.Platform = process.platform,
env: Parameters<typeof supportedClients>[2] = {},
) =>
supportedClients(home, platform, env).find(
({ name }) => name === "workbuddy",
);
describe("WorkBuddy configuration paths", () => {
it("resolves the global mcp.json under the home root", () => {
expect(workbuddyPath("/home/a")?.configPath).toBe(
join("/home/a", ".workbuddy", "mcp.json"),
);
expect(workbuddyPath("/Users/a")?.markerPath).toBe(
join("/Users/a", ".workbuddy"),
);
expect(workbuddyPath("C:\\Users\\a", "win32")?.configPath).toBe(
join("C:\\Users\\a", ".workbuddy", "mcp.json"),
);
});
it("resolves the registration format as the plain mcpServers JSON dialect", () => {
expect(workbuddyPath("/home/a")?.format).toBe("json");
});
it("resolves the personal skills directory under the home root", () => {
expect(
skillDestinations("/home/a", ["workbuddy"]).map(({ path }) => path),
).toEqual([
join("/home/a", ".workbuddy", "skills", "reverse-engineer-anything"),
]);
expect(
skillDestinations("C:\\Users\\a", ["workbuddy"], {}, "win32").map(
({ path }) => path,
),
).toEqual([
join("C:\\Users\\a", ".workbuddy", "skills", "reverse-engineer-anything"),
]);
});
});
describe("WorkBuddy JSON registration", () => {
const workbuddyClient = async () => {
const home = await createTestTempDirectory("rea-workbuddy-");
const client = workbuddyPath(home, process.platform);
if (client?.markerPath === undefined)
throw new Error("missing WorkBuddy client");
await mkdir(client.markerPath, { recursive: true });
return { home, client };
};
it("upserts rea into mcpServers while preserving unrelated settings", async () => {
const { home, client } = await workbuddyClient();
await writeFile(
client.configPath,
`{
"mcpServers": {
"other": { "command": "other-server" }
}
}`,
);
expect(
(await configureClientConfiguration(client, {}, command)).status,
).toBe("configured");
const parsed = JSON.parse(await readFile(client.configPath, "utf8"));
expect(parsed.mcpServers.other).toEqual({ command: "other-server" });
expect(parsed.mcpServers.rea).toEqual({
command: command[0],
args: command.slice(1),
});
expect(await configureClientConfiguration(client, {}, command)).toEqual({
status: "unchanged",
});
expect(
(
await readClientRegistrationStatuses(home, undefined, {
platform: process.platform,
environment: {},
})
).find(({ client }) => client === "workbuddy"),
).toMatchObject({ state: "aligned", command });
});
it("creates the mcpServers table in an existing mcp.json without servers", async () => {
const { client } = await workbuddyClient();
await writeFile(
client.configPath,
JSON.stringify({ unrelated: { setting: true } }, null, 2),
);
expect(
(await configureClientConfiguration(client, {}, command)).status,
).toBe("configured");
const parsed = JSON.parse(await readFile(client.configPath, "utf8"));
expect(parsed.unrelated).toEqual({ setting: true });
expect(parsed.mcpServers.rea).toEqual({
command: command[0],
args: command.slice(1),
});
});
it("registers into an empty mcp.json", async () => {
const { client } = await workbuddyClient();
await writeFile(client.configPath, "");
expect(
(await configureClientConfiguration(client, {}, command)).status,
).toBe("configured");
const parsed = JSON.parse(await readFile(client.configPath, "utf8"));
expect(parsed.mcpServers.rea).toEqual({
command: command[0],
args: command.slice(1),
});
});
it("removes only the rea entry on uninstall", async () => {
const { home, client } = await workbuddyClient();
await writeFile(
client.configPath,
JSON.stringify(
{ mcpServers: { other: { command: "other-server" } } },
null,
2,
),
);
await configureClientConfiguration(client, {}, command);
expect((await systemUninstallHost(home).removeClient(client)).status).toBe(
"removed",
);
const parsed = JSON.parse(await readFile(client.configPath, "utf8"));
expect(parsed.mcpServers.other).toEqual({ command: "other-server" });
expect(parsed.mcpServers.rea).toBeUndefined();
});
});
@@ -0,0 +1,138 @@
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
import { Ajv2020 } from "ajv/dist/2020.js";
import { expect, it } from "vitest";
import { FUNCTION_COMPARISON_EXAMPLE } from "../../../src/contracts/functionComparisonExample.js";
import { createEvidence } from "../../../src/domain/evidence.js";
import { functionDossierSchema } from "../../../src/domain/hopperValues.js";
import type { RelationshipCoverage } from "../../../src/domain/relationshipCoverage.js";
import { createServer } from "../../../src/server/createServer.js";
import { createTestBinarySession } from "../../fixtures/binarySession.js";
const observe = (relationshipCoverage: RelationshipCoverage) => {
const template = FUNCTION_COMPARISON_EXAMPLE.left;
if (template.subject === null)
throw new Error("Expected artifact-bound fixture");
const dossier = functionDossierSchema.parse(template.normalized_result);
return createEvidence(
{
path: template.subject.local_path,
sha256: template.subject.digest.sha256,
format: template.subject.format,
},
template.provider,
{
operation: "analyze_function",
parameters: template.parameters,
result: { ...dossier, callees: [], unresolved_calls: [] },
confidence: template.confidence,
authority: template.authority,
relationshipCoverage,
},
);
};
it("preserves directional gaps through advertised schemas and public path reconstruction", async () => {
const session = createTestBinarySession(() => {
throw new Error("Offline call paths must not launch a provider");
});
const server = createServer({ kind: "session", session });
const client = new Client({ name: "function-call-directions", version: "1" });
const [clientTransport, serverTransport] =
InMemoryTransport.createLinkedPair();
try {
await server.connect(serverTransport);
await client.connect(clientTransport);
const advertised = (await client.listTools()).tools.find(
({ name }) => name === "build_call_path",
);
if (advertised === undefined)
throw new Error("Missing build_call_path contract");
const ajv = new Ajv2020({ strict: false, validateFormats: false });
expect(ajv.validateSchema(advertised.inputSchema)).toBe(true);
const validate = ajv.compile(advertised.inputSchema);
for (const { directions, status, exhaustive } of [
{
directions: ["callers"] as ["callers"],
status: "not_found",
exhaustive: true,
},
{
directions: ["callees"] as ["callees"],
status: "unknown",
exhaustive: false,
},
{
directions: ["callers", "callees"] as ["callers", "callees"],
status: "unknown",
exhaustive: false,
},
]) {
const limitation =
"An observed call reference is absent from the inventory.";
const coverage: RelationshipCoverage = {
status: "partial",
limitations: [limitation],
limited_directions: directions,
};
const observation = observe(coverage);
expect(session.recordEvidence(observation).ok).toBe(true);
const input = {
functions: [observation],
start: { address: "0x1000" },
goal: { address: "0x9999" },
};
expect(validate(input), JSON.stringify(validate.errors)).toBe(true);
for (const limited_directions of [
[],
["callers", "callers"],
["callees", "callees"],
["callers", "callees", "callers"],
["incoming"],
])
expect(
validate({
...input,
functions: [
{
...observation,
relationship_coverage: {
...coverage,
limited_directions,
},
},
],
}),
).toBe(false);
const result = await client.callTool({
name: "build_call_path",
arguments: input,
});
if (result.isError) throw new Error(JSON.stringify(result));
expect(result.structuredContent).toMatchObject({
normalized_result: {
status,
search_scope: { exhaustive },
limitations: expect.arrayContaining([limitation]),
},
});
const comparison = await client.callTool({
name: "compare_functions",
arguments: { left: observation, right: observation },
});
if (comparison.isError) throw new Error(JSON.stringify(comparison));
expect(comparison.structuredContent).toMatchObject({
normalized_result: {
dimensions: expect.arrayContaining([
expect.objectContaining({ dimension: "calls", status: "unknown" }),
]),
},
});
// Each scenario owns its Evidence and residual-unknown ledger.
const closed = await session.close();
if (!closed.ok) throw closed.error;
}
} finally {
await Promise.allSettled([session.close(), client.close(), server.close()]);
}
});
+21 -11
View File
@@ -15,9 +15,17 @@ import {
type GhidraFunctionOperation,
} from "../../../src/ghidra/GhidraFunctionValues.js";
it.each([true, false])(
"compares observed empty Ghidra facets while preserving pseudocode availability (%s)",
async (pseudocodeAvailable) => {
it.each([
{ pseudocodeAvailable: true, reportsCoverage: true },
{ pseudocodeAvailable: false, reportsCoverage: true },
{ pseudocodeAvailable: true, reportsCoverage: false },
{ pseudocodeAvailable: false, reportsCoverage: false },
])(
"compares observed empty Ghidra facets with pseudocode=$pseudocodeAvailable and coverage=$reportsCoverage",
async ({ pseudocodeAvailable, reportsCoverage }) => {
const coverage = reportsCoverage
? { status: "complete" as const }
: undefined;
const dossier = functionDossierSchema.parse(ghidraFunctionDossier());
Object.assign(dossier, {
comments: [],
@@ -47,7 +55,13 @@ it.each([true, false])(
if (operation !== "analyze_function")
throw new Error(`Unexpected operation: ${operation}`);
return Promise.resolve(
ok({ value: jsonValueSchema.parse(dossier), limitations: [] }),
ok({
value: jsonValueSchema.parse(dossier),
limitations: [],
...(coverage === undefined
? {}
: { relationship_coverage: coverage }),
}),
);
},
);
@@ -61,9 +75,7 @@ it.each([true, false])(
harness.session,
analyzed.structuredContent,
);
expect(evidence.relationship_coverage).toMatchObject({
status: "complete",
});
expect(evidence.relationship_coverage).toEqual(coverage ?? null);
expect(evidence.normalized_result).toEqual(dossier);
const reply = await harness.mcp.callTool({
name: "compare_functions",
@@ -86,7 +98,7 @@ it.each([true, false])(
pseudocode: pseudocodeAvailable ? "unchanged" : "unknown",
assembly: "unchanged",
comments: "unchanged",
calls: "unchanged",
calls: reportsCoverage ? "unchanged" : "unknown",
references: "unchanged",
strings_names: "unchanged",
cfg: "unchanged",
@@ -96,9 +108,7 @@ it.each([true, false])(
evidence.parameters,
);
if (!replay.ok) throw replay.error;
expect(replay.value.relationshipCoverage).toEqual(
evidence.relationship_coverage,
);
expect(replay.value.relationshipCoverage).toEqual(coverage);
expect(harness.calls).toEqual(["analyze_function"]);
} finally {
await harness.close();
+1 -3
View File
@@ -94,9 +94,7 @@ const statusCapability = (
describe("server and catalog identity", () => {
it("derives package and SDK versions from canonical package metadata", async () => {
const packageJson = JSON.parse(await readFile("package.json", "utf8"));
const packageLock = JSON.parse(
await readFile("npm-shrinkwrap.json", "utf8"),
);
const packageLock = JSON.parse(await readFile("package-lock.json", "utf8"));
expect(PACKAGE_METADATA).toMatchObject({
name: packageJson.name,
version: packageJson.version,
@@ -1,237 +0,0 @@
import { spawn } from "node:child_process";
import { copyFile, lstat, mkdir, readFile, writeFile } from "node:fs/promises";
import { delimiter, join } from "node:path";
import { fileURLToPath } from "node:url";
import { expect, it, onTestFinished } from "vitest";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
const launcher = fileURLToPath(
new URL("../../../scripts/plugin-mcp-launcher.mjs", import.meta.url),
);
const start = (path: string, version = "6.4.0-rc.1", entry = launcher) => {
const child = spawn(process.execPath, [entry, version], {
env: {
...process.env,
PATH: `${path}${delimiter}${process.env.PATH ?? ""}`,
},
stdio: "pipe",
});
let stdout = "";
let stderr = "";
child.stdout.on(
"data",
(bytes: Buffer) => (stdout += bytes.toString("utf8")),
);
child.stderr.on(
"data",
(bytes: Buffer) => (stderr += bytes.toString("utf8")),
);
const closed = new Promise<{
code: number | null;
signal: NodeJS.Signals | null;
stdout: string;
stderr: string;
}>((resolve, reject) => {
child.once("error", reject);
child.once("close", (code, signal) =>
resolve({ code, signal, stdout, stderr }),
);
});
onTestFinished(async () => {
if (child.exitCode === null && child.signalCode === null)
child.kill("SIGKILL");
await closed;
});
return { child, closed };
};
const fakeRunner = async (body: string) => {
const root = await createTestTempDirectory("rea-plugin-launcher-");
const bin = join(root, "plugin with spaces", "bin");
await mkdir(bin, { recursive: true });
await writeFile(join(bin, "npx"), `#!${process.execPath}\n${body}\n`, {
mode: 0o755,
});
return bin;
};
it.skipIf(process.platform === "win32")(
"passes the exact pin and protocol streams through the source-only plugin launcher",
async () => {
const bin = await fakeRunner(`
process.stderr.write(JSON.stringify(process.argv.slice(2)));
process.stdin.pipe(process.stdout);
process.stdin.on("end", () => { process.exitCode = 7; });
`);
const { child, closed } = start(bin, "6.4.0-rc.1");
child.stdin.end('{"jsonrpc":"2.0","id":1}\n');
expect(await closed).toEqual({
code: 7,
signal: null,
stdout: '{"jsonrpc":"2.0","id":1}\n',
stderr: '["-y","rea-agents@6.4.0-rc.1","mcp"]',
});
},
);
it.skipIf(process.platform === "win32")(
"forwards termination to the live package-runner child before exiting",
async () => {
const bin = await fakeRunner(`
process.on("SIGTERM", () => { process.stderr.write("stopped"); process.exit(0); });
process.stdout.write("ready");
process.stdin.resume();
`);
const { child, closed } = start(bin);
await new Promise<void>((resolve, reject) => {
child.stdout.once("data", () => resolve());
child.once("error", reject);
});
child.kill("SIGTERM");
expect(await closed).toMatchObject({ code: 0, stderr: "stopped" });
},
);
it.skipIf(process.platform === "win32")(
"stops a package runner and its long-lived server with stdin still open",
async () => {
const root = await createTestTempDirectory("rea-plugin-process-tree-");
const runnerMarker = join(root, "runner-stopped");
const serverMarker = join(root, "server-stopped");
const bin = await fakeRunner(`
const { appendFileSync } = require("node:fs");
const { spawn } = require("node:child_process");
const server = spawn(process.execPath, ["-e", ${JSON.stringify(`
const { appendFileSync } = require("node:fs");
process.on("SIGTERM", () => {
appendFileSync(${JSON.stringify(serverMarker)}, "stopped\\n");
process.exit(0);
});
process.stdin.resume();
process.stdout.write("server-ready\\n");
`)}], { stdio: "inherit" });
let stopping = false;
process.on("SIGTERM", () => {
stopping = true;
appendFileSync(${JSON.stringify(runnerMarker)}, "stopped\\n");
});
process.stdin.resume();
server.once("close", () => { if (stopping) process.exit(0); });
`);
const { child, closed } = start(bin);
await new Promise<void>((resolve, reject) => {
const timeout = setTimeout(
() => reject(new Error("package server did not become ready")),
5_000,
);
child.stdout.on("data", (bytes: Buffer) => {
if (bytes.toString("utf8").includes("server-ready\n")) {
clearTimeout(timeout);
resolve();
}
});
child.once("error", reject);
});
// Keep the protocol pipe open: shutdown must reach the package runner and
// its server process independently of EOF on stdin.
child.kill("SIGTERM");
expect(await closed).toMatchObject({ code: 0, signal: null });
await expect(readFile(runnerMarker, "utf8")).resolves.toBe("stopped\n");
await expect(readFile(serverMarker, "utf8")).resolves.toBe("stopped\n");
},
);
it.skipIf(process.platform === "win32")(
"isolates local package resolution and removes the owned runner directory",
async () => {
const bin = await fakeRunner("process.stdout.write(process.cwd());");
const { child, closed } = start(bin);
child.stdin.end();
const result = await closed;
expect(result.code).toBe(0);
expect(result.stdout).toContain("rea-plugin-launcher-");
expect(result.stdout).not.toBe(process.cwd());
await expect(lstat(result.stdout)).rejects.toMatchObject({
code: "ENOENT",
});
},
);
it("rejects a shell expression instead of treating it as the pinned package", async () => {
const root = await createTestTempDirectory("rea-plugin-invalid-pin-");
const { child, closed } = start(root, "6.4.0&echo wrong");
child.stdin.end();
expect(await closed).toMatchObject({
code: 1,
stdout: "",
stderr: "REA plugin requires one exact rea-agents@VERSION pin.\n",
});
});
const pluginFixture = async () => {
const root = await createTestTempDirectory("rea-plugin-release-origin-");
await mkdir(join(root, "scripts"));
await mkdir(join(root, "dist/mcp"), { recursive: true });
const entry = join(root, "scripts/plugin-mcp-launcher.mjs");
await copyFile(launcher, entry);
await writeFile(
join(root, "package.json"),
JSON.stringify({
name: "rea-agents",
version: "6.4.0",
type: "module",
}),
);
await writeFile(join(root, "dist/mcp/main.js"), "");
await writeFile(
join(root, "scripts/rea.mjs"),
'process.stdout.write("bundled-runtime");',
);
return { root, entry };
};
it
.skipIf(process.platform === "win32")
.each(["directory", "worktree file", "exported source"])(
"uses the registry pin even when a %s checkout has same-version dist",
async (kind) => {
const { root, entry } = await pluginFixture();
await mkdir(join(root, "src/application"), { recursive: true });
await copyFile(
fileURLToPath(
new URL(
"../../../src/application/PackageRunnerCommand.mjs",
import.meta.url,
),
),
join(root, "src/application/PackageRunnerCommand.mjs"),
);
if (kind === "directory") await mkdir(join(root, ".git"));
else if (kind === "worktree file")
await writeFile(join(root, ".git"), "gitdir: /fixture/worktree");
const bin = await fakeRunner(
"process.stdout.write(JSON.stringify(process.argv.slice(2)));",
);
const { child, closed } = start(bin, "6.4.0", entry);
child.stdin.end();
expect(await closed).toMatchObject({
code: 0,
stdout: '["-y","rea-agents@6.4.0","mcp"]',
stderr: "",
});
},
);
it("uses the matching installed runtime without another package-runner install", async () => {
const { root, entry } = await pluginFixture();
const { child, closed } = start(root, "6.4.0", entry);
child.stdin.end();
expect(await closed).toMatchObject({
code: 0,
stdout: "bundled-runtime",
stderr: "",
});
});
@@ -12,6 +12,8 @@ import ghidra.program.model.listing.FlowOverride;
import ghidra.program.model.listing.Function;
import ghidra.program.model.listing.Instruction;
import ghidra.program.model.symbol.SourceType;
import ghidra.program.model.symbol.Reference;
import ghidra.program.model.symbol.RefType;
import java.io.File;
import java.lang.reflect.Method;
import java.nio.file.Files;
@@ -20,9 +22,12 @@ import java.util.HexFormat;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.ArrayList;
/** Injects reported analysis flags into imported ELF/Mach-O functions; never executes target code. */
public class ReaNoReturnProbe extends GhidraScript {
private record MemoryCallReference(Address source, Address target, RefType type,
SourceType sourceType, int operandIndex, boolean primary) {}
private final JsonArray checks = new JsonArray();
private final Map<Function, Address> originalEnds = new LinkedHashMap<>();
private final Map<Function, String> originalSignatures = new LinkedHashMap<>();
@@ -85,6 +90,114 @@ public class ReaNoReturnProbe extends GhidraScript {
return instance;
}
private void knownInteriorCall(GhidraScript bridge) throws Exception {
Function caller = function("rea_tls_chain");
Function target = function("rea_tls");
Address interior = target.getEntryPoint().add(1);
require("interior_target_is_owned", currentProgram.getFunctionManager().getFunctionAt(interior) == null &&
target.equals(currentProgram.getFunctionManager().getFunctionContaining(interior)));
Instruction call = null;
for (Instruction instruction : currentProgram.getListing().getInstructions(caller.getBody(), true)) {
if (instruction.getFlowType().isCall()) { call = instruction; break; }
}
if (call == null) throw new IllegalStateException("Missing real call instruction");
var references = currentProgram.getReferenceManager();
List<Reference> originals = new ArrayList<>();
List<MemoryCallReference> saved = new ArrayList<>();
for (Reference reference : call.getReferencesFrom()) {
if (!reference.getReferenceType().isCall()) continue;
originals.add(reference);
saved.add(new MemoryCallReference(reference.getFromAddress(), reference.getToAddress(),
reference.getReferenceType(), reference.getSource(), reference.getOperandIndex(), reference.isPrimary()));
}
require("original_call_reference_present", !originals.isEmpty());
Reference changed = null;
try {
for (Reference reference : originals) references.delete(reference);
changed = references.addMemoryReference(call.getAddress(), interior,
RefType.UNCONDITIONAL_CALL, SourceType.USER_DEFINED, 0);
require("interior_target_omitted_by_callee_getter", !caller.getCalledFunctions(monitor).contains(target));
JsonObject query = new JsonObject();
query.addProperty("document", currentProgram.getName());
query.addProperty("procedure", caller.getName());
query.addProperty("direction", "outgoing");
Method read = bridge.getClass().getDeclaredMethod("procedureReferences", JsonObject.class);
read.setAccessible(true);
JsonObject output = (JsonObject) read.invoke(bridge, query);
require("interior_reference_keeps_known_procedure", output.getAsJsonArray("references").asList().stream().anyMatch(
edge -> !edge.getAsJsonObject().get("target_procedure").isJsonNull() &&
edge.getAsJsonObject().getAsJsonObject("target_procedure").get("name").getAsString().equals(target.getName())));
require("known_reference_is_not_also_unresolved", output.getAsJsonArray("unresolved_calls").isEmpty());
query.remove("direction");
Method callees = bridge.getClass().getDeclaredMethod("procedureCalls", JsonObject.class, boolean.class);
callees.setAccessible(true);
JsonObject inventory = (JsonObject) callees.invoke(bridge, query, false);
require("omitted_callee_inventory_stays_partial", inventory.getAsJsonObject("relationship_coverage").get("status").getAsString().equals("partial"));
require("omitted_callee_direction_is_explicit",
inventory.getAsJsonObject("relationship_coverage").getAsJsonArray("limited_directions").toString().equals("[\"callees\"]"));
require("interior_caller_omitted_by_entry_getter", !target.getCallingFunctions(monitor).contains(caller));
query.addProperty("procedure", target.getName());
JsonObject incoming = (JsonObject) callees.invoke(bridge, query, true);
require("off_entry_caller_inventory_stays_partial",
incoming.getAsJsonObject("relationship_coverage").get("status").getAsString().equals("partial"));
require("off_entry_caller_direction_is_explicit",
incoming.getAsJsonObject("relationship_coverage").getAsJsonArray("limited_directions").toString().equals("[\"callers\"]"));
JsonObject dossierQuery = new JsonObject();
dossierQuery.addProperty("procedure", target.getName());
Method analyze = bridge.getClass().getDeclaredMethod("analyzeFunctionObservation", JsonObject.class);
analyze.setAccessible(true);
Object observation = analyze.invoke(bridge, dossierQuery);
Method coverage = observation.getClass().getDeclaredMethod("relationshipCoverage");
coverage.setAccessible(true);
require("dossier_preserves_caller_only_gap",
((JsonObject) coverage.invoke(observation)).getAsJsonArray("limited_directions").toString().equals("[\"callers\"]"));
} finally {
if (changed != null) references.delete(changed);
for (MemoryCallReference reference : saved) {
Reference restored = references.addMemoryReference(reference.source(), reference.target(),
reference.type(), reference.sourceType(), reference.operandIndex());
references.setPrimary(restored, reference.primary());
}
}
}
private void nonCallInstructionReference(GhidraScript bridge) throws Exception {
Function caller = function("rea_tls_chain");
Function target = function("rea_memcpy");
Instruction source = null;
for (Instruction instruction : currentProgram.getListing().getInstructions(caller.getBody(), true)) {
if (!instruction.getFlowType().isCall()) { source = instruction; break; }
}
if (source == null) throw new IllegalStateException("Missing non-call instruction");
var references = currentProgram.getReferenceManager();
Reference added = references.addMemoryReference(source.getAddress(), target.getEntryPoint().add(1),
RefType.UNCONDITIONAL_CALL, SourceType.USER_DEFINED, -1);
try {
require("non_call_reference_target_omitted", !caller.getCalledFunctions(monitor).contains(target));
JsonObject query = new JsonObject();
query.addProperty("document", currentProgram.getName());
query.addProperty("procedure", caller.getName());
Method read = bridge.getClass().getDeclaredMethod("procedureCalls", JsonObject.class, boolean.class);
read.setAccessible(true);
JsonObject output = (JsonObject) read.invoke(bridge, query, false);
require("non_call_instruction_reference_stays_partial",
output.getAsJsonObject("relationship_coverage").get("status").getAsString().equals("partial"));
query.addProperty("direction", "outgoing");
Method edges = bridge.getClass().getDeclaredMethod("procedureReferences", JsonObject.class);
edges.setAccessible(true);
JsonObject known = (JsonObject) edges.invoke(bridge, query);
require("non_call_known_reference_is_not_unresolved", known.getAsJsonArray("unresolved_calls").isEmpty());
references.delete(added);
Address unknown = currentProgram.getMemory().getMaxAddress().add(0x1000);
added = references.addMemoryReference(source.getAddress(), unknown,
RefType.UNCONDITIONAL_CALL, SourceType.USER_DEFINED, -1);
JsonObject unresolved = (JsonObject) edges.invoke(bridge, query);
require("non_call_unknown_reference_keeps_site", unresolved.getAsJsonArray("unresolved_calls").size() == 1);
} finally {
references.delete(added);
}
}
@Override
public void run() throws Exception {
String[] args = getScriptArgs();
@@ -164,6 +277,8 @@ public class ReaNoReturnProbe extends GhidraScript {
Object output = pseudocode.invoke(bridge, query);
require("recovered_pseudocode", output instanceof JsonObject value &&
value.get("value").getAsString().contains("rea_after_call"));
nonCallInstructionReference(bridge);
knownInteriorCall(bridge);
} finally {
((ghidra.app.decompiler.DecompInterface) field.get(bridge)).dispose();
}
+11 -253
View File
@@ -1,28 +1,13 @@
import { access, readFile } from "node:fs/promises";
import { readFile } from "node:fs/promises";
import { GenericJson } from "release-please/build/src/updaters/generic-json.js";
import { Version } from "release-please/build/src/version.js";
import { describe, expect, it } from "vitest";
import { SUPPORTED_NODE_VERSION_RANGE } from "../../src/domain/runtimeVersion.js";
import { MCP_STARTUP_POLICY } from "../../src/mcpStartupPolicy.js";
const readJson = async (path: string): Promise<unknown> =>
JSON.parse(await readFile(path, "utf8")) as unknown;
// Each file lists a fragment of every example Release Please must keep
// current, so a lost marker is caught as well as a stale version.
const versionedDocumentation = [
{
path: "docs/installation.md",
examples: ["doctor --provider ghidra --json", '"command": "npx"'],
},
] as const;
const pluginManifests = [
".claude-plugin/plugin.json",
".codex-plugin/plugin.json",
] as const;
const pluginServerConfig = ".mcp.json";
const versionedDocumentation = ["docs/installation.md"] as const;
describe("release configuration", () => {
it("keeps doctor and setup runtime support aligned with package engines", async () => {
@@ -39,7 +24,7 @@ describe("release configuration", () => {
packages: {
".": {
"extra-files": [
...versionedDocumentation.map(({ path }) => ({
...versionedDocumentation.map((path) => ({
type: "generic",
path,
})),
@@ -57,35 +42,6 @@ describe("release configuration", () => {
path: "server.json",
jsonpath: "$.packages[0].version",
},
...pluginManifests.flatMap((path) => [
{
type: "json",
path,
jsonpath: "$.version",
},
...(path.startsWith(".claude")
? [
{
type: "json",
path,
jsonpath: "$.mcpServers.rea.args[1]",
},
]
: []),
]),
{
type: "json",
path: pluginServerConfig,
jsonpath: "$.mcpServers.rea.args[1]",
},
...[
".claude-plugin/marketplace.json",
".agents/plugins/marketplace.json",
].map((path) => ({
type: "json",
path,
jsonpath: "$.plugins[0].source.version",
})),
],
},
},
@@ -93,27 +49,18 @@ describe("release configuration", () => {
});
it.each(versionedDocumentation)(
"marks every versioned package example in $path",
async ({ path, examples }) => {
"marks the versioned package example in %s",
async (path) => {
const content = await readFile(path, "utf8");
const packageJson = (await readJson("package.json")) as {
version: string;
};
const versionBlocks = [
...content.matchAll(
/<!-- x-release-please-start-version -->[\s\S]*?<!-- x-release-please-end -->/gu,
),
].map(([block]) => block);
const versionBlock =
/<!-- x-release-please-start-version -->[\s\S]*?<!-- x-release-please-end -->/u.exec(
content,
)?.[0];
for (const example of examples)
expect(
versionBlocks.filter((block) => block.includes(example)),
example,
).toHaveLength(1);
for (const block of versionBlocks)
expect(block.match(/rea-agents@[^\s"`]+/gu)).toEqual([
`rea-agents@${packageJson.version}`,
]);
expect(versionBlock).toContain(`rea-agents@${packageJson.version}`);
},
);
@@ -124,13 +71,10 @@ describe("release configuration", () => {
expect(await readJson(".release-please-manifest.json")).toEqual({
".": packageJson.version,
});
expect(await readJson("npm-shrinkwrap.json")).toMatchObject({
expect(await readJson("package-lock.json")).toMatchObject({
version: packageJson.version,
packages: { "": { version: packageJson.version } },
});
await expect(access("package-lock.json")).rejects.toMatchObject({
code: "ENOENT",
});
const changelog = await readFile("CHANGELOG.md", "utf8");
expect(/^## \[([^\]]+)\]/mu.exec(changelog)?.[1]).toBe(packageJson.version);
});
@@ -176,189 +120,3 @@ describe("release configuration", () => {
});
});
});
describe("plugin manifests", () => {
it("distributes both plugins with generated canonical skills from the exact npm release", async () => {
const pkg = (await readJson("package.json")) as {
version: string;
files: string[];
};
expect(pkg.files).toEqual(
expect.arrayContaining([
...pluginManifests,
pluginServerConfig,
"skills",
]),
);
for (const path of pluginManifests)
expect(await readJson(path)).toMatchObject({ skills: "./skills" });
for (const path of [
".claude-plugin/marketplace.json",
".agents/plugins/marketplace.json",
])
expect(await readJson(path)).toMatchObject({
plugins: [
{
source: {
source: "npm",
package: "rea-agents",
version: pkg.version,
},
},
],
});
});
it.each(pluginManifests)(
"pins the plugin manifest %s and Codex's server config path",
async (path) => {
const packageJson = (await readJson("package.json")) as {
name: string;
version: string;
};
expect(await readJson(path)).toMatchObject({
version: packageJson.version,
...(path.startsWith(".codex")
? { mcpServers: `./${pluginServerConfig}` }
: {}),
});
},
);
});
describe("plugin release pins", () => {
it("advances every plugin pin through Release Please JSON extra-files", async () => {
const packageJson = (await readJson("package.json")) as {
version: string;
};
const currentVersion = Version.parse(packageJson.version);
const nextVersion = `${currentVersion.major}.${currentVersion.minor + 1}.0`;
const version = Version.parse(nextVersion);
const config = (await readJson("release-please-config.json")) as {
packages: {
".": {
"extra-files": Array<{
type: string;
path: string;
jsonpath?: string;
}>;
};
};
};
const updated = new Map<string, string>();
for (const extraFile of config.packages["."]["extra-files"]) {
if (extraFile.type !== "json" || extraFile.jsonpath === undefined)
continue;
const content =
updated.get(extraFile.path) ?? (await readFile(extraFile.path, "utf8"));
updated.set(
extraFile.path,
new GenericJson(extraFile.jsonpath, version).updateContent(content),
);
}
const updatedJson = new Map<string, unknown>(
[...updated].map(
([path, content]) => [path, JSON.parse(content) as unknown] as const,
),
);
expect(updatedJson.get(".claude-plugin/plugin.json")).toMatchObject({
version: nextVersion,
mcpServers: {
rea: {
args: [
"${CLAUDE_PLUGIN_ROOT}/scripts/plugin-mcp-launcher.mjs",
nextVersion,
],
},
},
});
expect(updatedJson.get(".codex-plugin/plugin.json")).toMatchObject({
version: nextVersion,
});
expect(updatedJson.get(pluginServerConfig)).toMatchObject({
mcpServers: {
rea: {
args: ["./scripts/plugin-mcp-launcher.mjs", nextVersion],
startup_timeout_sec: 30,
},
},
});
for (const path of [
".claude-plugin/marketplace.json",
".agents/plugins/marketplace.json",
])
expect(updatedJson.get(path)).toMatchObject({
plugins: [
{
source: {
source: "npm",
package: "rea-agents",
version: nextVersion,
},
},
],
});
});
it("pins the plugin server launch from the shared MCP declaration", async () => {
const packageJson = (await readJson("package.json")) as {
version: string;
};
expect(await readJson(pluginServerConfig)).toMatchObject({
mcpServers: {
rea: {
command: "node",
args: ["./scripts/plugin-mcp-launcher.mjs", packageJson.version],
cwd: ".",
},
},
});
});
it.each(pluginManifests)(
"stores %s in the form Release Please rewrites it to",
async (path) => {
// Release Please reserializes JSON extra-files, so any other layout
// would change formatting in every release PR.
const content = await readFile(path, "utf8");
expect(content).toBe(`${JSON.stringify(JSON.parse(content), null, 2)}
`);
},
);
it("declares the interface fields Codex plugin validation requires", async () => {
const manifest = (await readJson(".codex-plugin/plugin.json")) as {
interface: Record<string, unknown>;
};
for (const field of [
"displayName",
"shortDescription",
"longDescription",
"developerName",
"category",
])
expect(manifest.interface[field]).toEqual(expect.stringMatching(/\S/u));
expect(manifest.interface.capabilities).toEqual(
expect.arrayContaining([expect.stringMatching(/\S/u)]),
);
expect(manifest.interface.defaultPrompt).toEqual(
expect.arrayContaining([expect.stringMatching(/\S/u)]),
);
});
it("carries the owned Codex startup timeout in the companion server entry", async () => {
expect(await readJson(pluginServerConfig)).toMatchObject({
mcpServers: {
rea: {
startup_timeout_sec: MCP_STARTUP_POLICY.codexStartupTimeoutSeconds,
},
},
});
});
});
@@ -34,6 +34,10 @@ it
const procRoot = join(directory, "proc");
await mkdir(procRoot);
const alternateMode = mode === "alternate launcher";
let orphanPid: number | undefined;
const isolatedProcessList = async () => ({
stdout: alternateMode || orphanPid === undefined ? "" : "Xvfb",
});
if (alternateMode) {
await copyFile("/usr/bin/sleep", launcher);
await chmod(launcher, 0o700);
@@ -47,6 +51,7 @@ it
await assertCleanLinuxHopperRuntime(
alternateMode ? [launcher] : [],
procRoot,
isolatedProcessList,
);
const command = alternateMode ? alternate : process.execPath;
const arguments_ = alternateMode ? ["30"] : ["-e", childCode];
@@ -69,7 +74,6 @@ it
const parentPid = parent.child.pid;
if (parentPid === undefined)
throw new Error("Fixture parent did not spawn");
let orphanPid: number | undefined;
try {
const { stdout } = await parent;
const { pid } = JSON.parse(stdout) as { pid: number };
@@ -115,6 +119,7 @@ it
assertCleanLinuxHopperRuntime(
alternateMode ? [launcher] : [],
procRoot,
isolatedProcessList,
),
).rejects.toThrow("Clean Linux runner retained a Hopper demo process");
// Observation must not terminate the fixture, even though it resembles Xvfb.
+1 -2
View File
@@ -1,11 +1,10 @@
{
"extends": "./tsconfig.json",
"compilerOptions": {
"allowJs": true,
"rootDir": "src",
"types": ["node"]
},
"include": ["src/**/*.ts", "src/**/*.mjs"],
"include": ["src/**/*.ts"],
"exclude": [
"src/**/*.fixture.ts",
"src/**/*.test.ts",
+19 -19
View File
@@ -9,12 +9,12 @@
"dependsOn": ["metadata:check"],
"inputs": [
"package.json",
"npm-shrinkwrap.json",
"package-lock.json",
"scripts/check-dependency-install.mjs",
"scripts/clean-build-output.mjs",
"scripts/bundle-runtime.mjs",
"scripts/generate-package-metadata.mjs",
"src/**/*.{ts,mjs,mts}",
"src/**/*.ts",
"!src/**/*.test.ts",
"!src/**/*.fixture.ts",
"tsconfig.json",
@@ -25,8 +25,8 @@
"typecheck": {
"inputs": [
"package.json",
"npm-shrinkwrap.json",
"src/**/*.{ts,mjs,mts}",
"package-lock.json",
"src/**/*.ts",
"tests/**/*.ts",
"tsconfig.json",
"tsconfig.build.json",
@@ -39,7 +39,7 @@
"metadata:check": {
"inputs": [
"package.json",
"npm-shrinkwrap.json",
"package-lock.json",
"scripts/generate-package-metadata.mjs",
"scripts/lib/generated-file.mjs",
".agents/skills/reverse-engineer-anything/SKILL.md",
@@ -52,7 +52,7 @@
".gitignore",
".oxlintrc.json",
"package.json",
"npm-shrinkwrap.json",
"package-lock.json",
"scripts/**/*.{cjs,cts,js,jsx,mjs,mts,ts,tsx}",
"src/**/*.{cjs,cts,js,jsx,mjs,mts,ts,tsx}",
"tests/**/*.{cjs,cts,js,jsx,mjs,mts,ts,tsx}",
@@ -73,7 +73,7 @@
".gitignore",
".oxlintrc.json",
"package.json",
"npm-shrinkwrap.json",
"package-lock.json",
"scripts/**/*.{cjs,cts,js,jsx,mjs,mts,ts,tsx}",
"src/**/*.{cjs,cts,js,jsx,mjs,mts,ts,tsx}",
"tests/**/*.{cjs,cts,js,jsx,mjs,mts,ts,tsx}",
@@ -94,9 +94,9 @@
".gitignore",
"knip.json",
"package.json",
"npm-shrinkwrap.json",
"package-lock.json",
"scripts/**/*.{cjs,cts,js,jsx,mjs,mts,ts,tsx}",
"src/**/*.{ts,mjs,mts}",
"src/**/*.ts",
"tests/**/*.ts",
"tsconfig.json",
"tsconfig.build.json",
@@ -109,7 +109,7 @@
".gitignore",
".jscpd.json",
"package.json",
"npm-shrinkwrap.json",
"package-lock.json",
"scripts/ci/static-check-report.mjs",
"src/**",
"tests/**"
@@ -120,7 +120,7 @@
"inputs": [
"bridge/**/*.{java,py}",
"scripts/**/*.{cjs,cts,js,jsx,mjs,mts,ts,tsx}",
"src/**/*.{ts,mjs,mts}",
"src/**/*.ts",
"tests/**/*.ts"
]
},
@@ -141,7 +141,7 @@
],
"inputs": [
"package.json",
"npm-shrinkwrap.json",
"package-lock.json",
"scripts/generate-*.mjs",
"scripts/check-doc-facts.mjs",
"scripts/lib/*.mjs",
@@ -172,7 +172,7 @@
"dependsOn": ["compile"],
"inputs": [
"package.json",
"npm-shrinkwrap.json",
"package-lock.json",
"scripts/generate-mcp-tool-catalog.mjs",
"scripts/lib/generated-file.mjs"
],
@@ -182,7 +182,7 @@
"dependsOn": ["compile"],
"inputs": [
"package.json",
"npm-shrinkwrap.json",
"package-lock.json",
"scripts/generate-skill-metadata.mjs",
"scripts/lib/generated-file.mjs",
".agents/skills/reverse-engineer-anything/**"
@@ -193,7 +193,7 @@
"dependsOn": ["compile"],
"inputs": [
"package.json",
"npm-shrinkwrap.json",
"package-lock.json",
"scripts/generate-product-catalog.mjs",
"scripts/lib/**/*.mjs",
".oxfmtrc.json"
@@ -204,7 +204,7 @@
"dependsOn": ["artifacts:skills"],
"inputs": [
"package.json",
"npm-shrinkwrap.json",
"package-lock.json",
"scripts/generate-completion-ledger.mjs",
"scripts/lib/generated-file.mjs",
"scripts/verify/managed/**",
@@ -219,7 +219,7 @@
"dependsOn": ["compile"],
"inputs": [
"package.json",
"npm-shrinkwrap.json",
"package-lock.json",
"scripts/generate-error-schema.mjs",
"scripts/lib/generated-file.mjs",
"scripts/lib/format-generated-file.mjs",
@@ -231,7 +231,7 @@
"dependsOn": ["artifacts:mcp-catalog"],
"inputs": [
"package.json",
"npm-shrinkwrap.json",
"package-lock.json",
"scripts/generate-mcp-tool-catalog.mjs",
"scripts/lib/generated-file.mjs",
{
@@ -244,7 +244,7 @@
"dependsOn": ["artifacts:managed-evidence"],
"inputs": [
"package.json",
"npm-shrinkwrap.json",
"package-lock.json",
"scripts/generate-completion-ledger.mjs",
"scripts/lib/generated-file.mjs",
"scripts/verify/managed/**",