Compare commits

...
Author SHA1 Message Date
Kaoru0822andClaude Opus 5.5 110151e523 fix(capture): keep URLs that WHATWG rejects instead of redacting userinfo
File URLs cannot carry userinfo (an "@" in a file host fails the parse),
and a special authority whose host or port WHATWG rejects fails the whole
URL. Neither has credentials, so the bridge now keeps their source text
and byte sidecar. Checked against Node's URL over 30,000 spellings: no
under-redaction, over-redaction 20,540 -> 5,865 (remaining: empty
userinfo, IPv4 range, IDNA, non-special and baseless references).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 00:26:29 +00:00
Kaoru0822 b93e585d68 Merge remote-tracking branch 'origin/main' into fix/issue-2671 2026-10-14 00:22:25 +00:00
Xiangfang ChenandGitHub e049ef70f7 feat(setup): add WorkBuddy client support (#3041)
* feat(setup): add WorkBuddy client support

Register rea in the global ~/.workbuddy/mcp.json through a new workbuddy
entry in SUPPORTED_CLIENT_DEFINITIONS (json dialect, ~/.workbuddy marker,
~/.workbuddy/skills skill directory; WorkBuddy AI shares the file).
Document the client, extend the unreleased-client note, and cover
registration and uninstall with boundary tests.

* docs(setup): record verified WorkBuddy tool calls

Real-client verification completed: WorkBuddy loads the registration and
real REA tool calls execute through the registered server.
2026-10-14 08:12:33 +08:00
bbf0638e88 fix(javascript): report EventSource and sendBeacon URLs as network endpoints (#3044)
Co-authored-by: xpeng5278-web <330743750+xpeng5278-web@users.noreply.github.com>
2026-10-14 08:10:04 +08:00
242e2b0001 fix(ghidra): preserve function facet and call coverage (#3002)
* fix(ghidra): preserve observed function facet and call coverage

Declare successful query availability in the Java producer, including explicit unavailable pseudocode reasons. Carry observed static call-set coverage through the parsed provider result and Evidence without inventing facts for legacy omissions. Use Ghidra indexed comment addresses over the function body and entry, retaining cancellation checks and sparse collection.

Closes #2951

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>

* fix(ghidra): retain unresolved call coverage in observations

Share decoded unresolved-call facts between dossiers, callee coverage and reference observations. Combine terminal and unresolved observations in one instruction traversal and retain site records only when requested. Keep missing procedure identities incomplete and validate unresolved locations at the provider boundary.

Bind snapshots to the corrected call coverage profile so older complete claims cannot replay under the new semantics. Add real public MCP callback trace/comparison regressions.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>

* fix(ghidra): separate reference targets from callee coverage

Reuse containing-function ownership for resolved reference identities and independently compare those identities with the actual returned callee set. Keep known interior routes out of unresolved site records while retaining partial inventory coverage.

Advance the coverage profile to v3 and reject interim v2 cache bindings. Add a real Ghidra producer regression with restored reference metadata; the previous bridge fails the known-route assertion.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>

* fix(ghidra): measure incoming call coverage across function bodies

Compare incoming reference source identities across the owned function body and entry with the actual returned caller set. Stream caller-only checks and reuse dossier reference observations. Combine caller/callee coverage from observed counts without reinterpreting serialized metadata, and share the body scope owner with comments and references.

Bind snapshots to function-body coverage semantics v4 and preserve the real off-entry caller regression.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>

* fix(ghidra): preserve whole-scope directional call coverage

Check body-and-entry call references independently of decoded instruction flow. Reuse collected dossier references and stream inventory-only coverage checks. Keep caller-only gaps distinct from incomplete outgoing coverage in offline path reconstruction, and invalidate prior snapshot profiles.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>

* fix(mcp): retain compact Evidence field types

Share optional diagnostics across the three tagged relationship coverage states instead of advertising duplicate branches. Preserve coverage validation while keeping compact Evidence properties within the existing reference expansion budget. Verify invalid direction cardinalities against the actual SDK-advertised schema.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>

---------

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-14 07:52:14 +08:00
20e4f83097 fix(ghidra): fail doctor when the JVM os.arch selects an unvalidated decompiler directory (#2792)
* fix(ghidra): fail doctor when the JVM os.arch selects an unvalidated decompiler directory

probeJava now also reads the JVM os.arch property. When it maps to a
different native decompiler directory than the one validated from the
host architecture (for example an x86_64 JDK under Rosetta on Apple
silicon selecting os/mac_x86_64 while only mac_arm_64 was built), the
native_decompiler check fails with a remediation pointing at a
matching-arch JDK instead of reporting a healthy installation that
fails every analysis. Unknown os.arch values keep the previous
behavior.

* fix(ghidra): tighten JVM arch remediation and cover os.arch extraction

Address review findings on the JVM os.arch decompiler-directory check:
- The remediation no longer suggests building the decompiler for the
  JVM-selected directory, which could never clear the check; it only
  advises pointing JAVA_HOME at a matching-architecture JDK.
- Add a test running the real Java probe against captured
  -XshowSettings:properties output, asserting os.arch reaches the
  installation diagnostics.
- Document the JDK-architecture prerequisite in the Ghidra setup guide.

* fix(ghidra): refuse unrecognized JVM native architectures

* docs(ghidra): align doctor prerequisites with JVM architecture admission

* fix(ghidra): reject unreported JVM architecture

* test(ghidra): separate JVM and executable admission cases

* fix(ghidra): defer decompiler arch validation when Java is absent

* fix(ghidra): reject JVM native architecture override

* fix(ghidra): validate JVM platform overrides

* fix(ghidra): validate JVM os.name and helper-environment platform overrides

* test(ci): isolate report fixtures and declare PowerPC Java bounds

* test(ghidra): provide observed JVM platform in PowerPC fixture

* fix(ghidra): parse quoted JVM environment options before admission

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-14 07:31:11 +08:00
97aff28201 fix(electron): reject non-executable paths and contain launch rejections (#2705)
* fix(electron): reject non-executable paths and contain launch rejections

An executable_path that runs but never prints the inspector line made
Playwright's electron.launch() leave three waits unawaited. They rejected
unhandled and Node ended the CLI or the MCP server (#2269).

Check the resolved executable_path is a regular file, and on non-Windows
hosts that it passes X_OK, before any launch. Wrap only the launch call
in a scoped unhandledRejection listener that absorbs rejections whose
stack is in playwright-core/lib and re-throws anything else.

Tested on Linux with Node 22.23.1, 24.18.0 and 26.11.1.

Refs #2269

🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊

* fix(electron): absorb only this launch's orphaned rejections

The listener around the launch dropped every rejection with a
playwright-core stack, so an unrelated Playwright failure during the
launch window would vanish. Hold those rejections instead. When the
launch settles, drop only the ones whose message is part of the launch's
own error (the orphans of a failed launch). Raise every other one again
through the unhandledRejection path, so the process still fails. A launch
that succeeds raises all of them. Rejections without a Playwright stack
still fail at once.

One setImmediate turn before the decision lets rejections queued by the
settling tick arrive first. It is a single turn, not a timer. A shared
set of absorbed errors stops overlapping launches from raising each
other's orphans. The listener is removed on every path.

Add child-process tests that drive the built provider through its launch
seam, since Vitest's worker has its own unhandledRejection listener.

🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊

* test(electron): distinguish launch success from capture ownership

* test(electron): assert cleanup ownership failures around overlapping launches

* fix(comparison): recheck historical calls-only changes (#2223)

* fix(comparison): preserve incomplete function call evidence

Keep missing, unknown, partial and unresolved call relationships unknown in function comparisons and reconstruction claims. Aggregate IDA caller and callee coverage without overwriting uncertainty.

Validate with 144 focused tests, typecheck/lint, docs checks and test discovery. Real providers were not verified; the full headless IDA session suite requires unavailable Windows native controls.

* fix(comparison): recheck historical calls-only changes

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>

* fix(android): make DEX descriptor pattern portable

* fix(ci): isolate Linux Hopper process fixture inventory

* fix(browser): avoid replaying observed launch rejections

* style: format comparison and IDA sources

* test(electron): allow cold macOS ownership setup

* test(electron): allow cold macOS setup in the early-exit path test

The test that runs an executable which exits at once builds the Swift
ownership helper twice on macOS: in the CLI process, and again in the
MCP server process on its first tool call that prepares. A cold Swift
cache can spend up to 120 seconds on a build. The test had only the
30 second default.

Give the test 180 seconds, as e1579ec31 did for
electronLaunchRejections.test.ts and 65a809a0d did for
webSourceLocationCli.test.ts. cli.run has no deadline of its own, so
it needs no timeoutMs.

🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊

* test(ci): isolate report fixtures and declare PowerPC Java bounds

---------

Co-authored-by: Claude Opus 5.5 <opus@fox.invalid>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Co-authored-by: Lucien <164857476+dakjdakd@users.noreply.github.com>
2026-10-14 07:30:14 +08:00
641a8395ee fix(cli): refuse a JavaScript integrity policy the binary analyze lane cannot honor (#2808)
* fix(cli): refuse an integrity policy the binary analyze lane cannot honor

`analyze` declares --integrity-policy for its JavaScript lane and forwards it
only there. On a binary target the flag was accepted and then dropped:
inspect_artifact-style integrity handling does not exist on this path, and
binary_overview is a strictObject with no input at all, so the request could
never take effect and the run proceeded as if the caller had not asked.

The same branch already refuses source_paths on a binary target with a typed
AnalysisInputError, and the sibling `inspect` command refuses the flag outright,
so the refusal is the established handling here. The policy option carries a
default, so the check keys off a value that differs from the owner schema's own
default rather than the flag's presence; otherwise every binary analyze would be
refused. The default is read from the schema instead of being restated.

Verified with 5 integrity policy CLI cases, check:fast and
verify:test-discovery (1023 files) on macOS arm64, and by running the built CLI
for the no-flag, explicit-default and record-and-continue cases. The regression
case failed with the command reaching the binary lane before the change.

* fix(cli): reject explicit default integrity policies for binaries

* fix(cli): keep an omitted integrity policy absent on the binary lane

`javascriptApplicationOptions.shape.integrityPolicy` fills an omission with its
default, and `.optional()` keeps reporting that parsed value, so
`options.integrityPolicy !== undefined` was true for every `analyze <target>`
call: ordinary binary analysis was rejected before the target was even read.

List `z.undefined()` first in a union so an omitted option stays absent while
the owner still validates a supplied value, and reapply the owner's metadata so
help and `--schema` keep describing the option.

Validation: tests/boundary/cli/javascriptIntegrityPolicyCli.test.ts reports
1 failed | 5 passed without this change (the case "still routes a binary
analyze target without an integrity policy" returned invalid_request for
integrity_policy instead of reaching the target) and 6 passed with it;
`npm run check:fast` passes on macOS 26.5.1 arm64 with Node.js 24.18.0.

* fix(cli): omit integrity defaults from binary option schema

* test(ci): isolate report fixtures and declare PowerPC Java bounds

---------

Co-authored-by: 吴佳浩 <wujiahao@supremind.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-14 07:28:00 +08:00
morluto 929aa67336 chore(main): merge pinned CI checkpoint 2895a18 2026-10-14 03:12:47 +08:00
morluto 4e8475e965 test(ci): isolate alternate launcher process fixture 2026-10-14 03:02:01 +08:00
morluto 557148ed3f fix(ci): declare process inventory callback 2026-10-14 02:25:19 +08:00
morluto f839046776 chore(merge): refresh PR #2767 with latest main 2026-10-14 02:23:57 +08:00
Morluto 281c4b8ec5 test(ci): isolate Hopper process inventory fixtures 2026-10-14 02:15:33 +08:00
Morluto a0da18d7fa test(capture): cover empty URL userinfo markers 2026-10-14 02:14:51 +08:00
Morluto 2c6e074cac fix(capture): remove empty URL userinfo markers 2026-10-14 02:14:51 +08:00
morluto 29c871e51b chore(merge): refresh PR #2767 with main 2026-10-14 02:10:19 +08:00
REA MaintenanceandKaoru0822 c0018ca403 fix(ci): isolate Linux Hopper process fixture inventory 2026-10-14 01:23:30 +08:00
morlutoandKaoru0822 ad16d250e3 fix(android): make DEX descriptor pattern portable 2026-10-14 01:23:25 +08:00
Lucienandmorluto af2f55cda9 fix(comparison): recheck historical calls-only changes (#2223)
* fix(comparison): preserve incomplete function call evidence

Keep missing, unknown, partial and unresolved call relationships unknown in function comparisons and reconstruction claims. Aggregate IDA caller and callee coverage without overwriting uncertainty.

Validate with 144 focused tests, typecheck/lint, docs checks and test discovery. Real providers were not verified; the full headless IDA session suite requires unavailable Windows native controls.

* fix(comparison): recheck historical calls-only changes

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-14 01:23:19 +08:00
morluto 4d5d1f5daa fix(capture): preserve file-relative path evidence after separators 2026-10-13 23:02:00 +08:00
morluto 0aff997b31 fix(capture): distinguish native paths and URL base authorities 2026-10-13 21:35:04 +08:00
morluto e8361b7715 fix(capture): parse URL controls and mixed network authorities 2026-10-13 20:31:54 +08:00
morluto 72e931f3a3 chore(ci): retain contributor updates alongside verification fixes 2026-10-13 20:21:10 +08:00
morluto 0266e6552b fix(capture): preserve same-scheme relative redirect paths 2026-10-13 20:19:00 +08:00
Kaoru0822 7d972a7e85 Merge remote-tracking branch 'origin/main' into fix/issue-2671 2026-10-13 12:04:14 +00:00
Kaoru0822andClaude Opus 5.5 e1fb40460d fix(capture): skip special-scheme leading slashes before userinfo
Ending the authority at a backslash exposed a credential: for
`https://\user:pass@host/path`, WHATWG parsing skips every leading "/"
or "\" after a special scheme (and in scheme-relative references), so
`user:pass` is userinfo, but the new boundary stopped at the first
backslash and found no "@". The same applied to `https:\\user:pass@host`
and `https:user:pass@host`.

Start the authority after that whole slash run for non-file special
schemes and scheme-relative references. File URLs keep their "//"
boundary because they carry no userinfo. Load the adapter in tests with
fresh mitmproxy stand-ins scoped to its import instead of leaving them
in sys.modules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 11:25:05 +00:00
Kaoru0822andClaude Opus 5.5 553cd6e358 fix(capture): end special-scheme URL authorities at a backslash
The native mitmproxy adapter removed transport userinfo with
`^(...//)[^/?#]*@`, so the authority ran to the last "@" before "/", "?"
or "#". For `https://example.test\part@tail` it treated
`example.test\part` as userinfo, reported `https://tail` as the
Location, and withdrew the original bytes as a transport credential.
WHATWG URL parsing, used by the shared TypeScript owner
(recordedTransportUrl.ts) and by HAR captures, ends a special-scheme or
scheme-relative authority at a backslash, so that "@" is ordinary path
text.

Locate the authority with the same boundary rule before removing
userinfo. Bytes are handled as Latin-1 text so unrelated bytes stay
identical. Userinfo before the backslash is still removed and redacted.
Add source unit tests for the adapter projection to test:python.

Fixes #2671

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 11:06:25 +00:00
50 changed files with 2909 additions and 274 deletions
+282 -97
View File
@@ -81,6 +81,7 @@ import ghidra.program.model.listing.Function;
import ghidra.program.model.listing.FunctionIterator;
import ghidra.program.model.listing.Instruction;
import ghidra.program.model.listing.InstructionIterator;
import ghidra.program.model.listing.Listing;
import ghidra.program.model.listing.Variable;
import ghidra.program.model.mem.MemoryBlock;
import ghidra.program.model.mem.MemoryBlockSourceInfo;
@@ -405,7 +406,9 @@ public final class ReaGhidraBridge extends HeadlessScript {
case "xrefs" -> xrefs(request.params);
case "analyze_function" -> {
FunctionAnalysisObservation observation = analyzeFunctionObservation(request.params);
yield functionObservation(observation.dossier(), observation.limitations());
JsonObject result = functionObservation(observation.dossier(), observation.limitations());
result.add("relationship_coverage", observation.relationshipCoverage());
yield result;
}
case "set_address_name", "set_addresses_names" -> {
if (!descriptor.transport.equals("unix-socket"))
@@ -1016,12 +1019,19 @@ public final class ReaGhidraBridge extends HeadlessScript {
for (Function item : ordered) {
result.add(canonicalAddress(item.getEntryPoint()));
}
return functionObservation(
result,
noReturnCallLimitations(
if (callers) {
JsonObject observation = functionObservation(result, noReturnCallLimitations(
currentProgram.getListing().getInstructions(function.getBody(), true)
)
));
observation.add("relationship_coverage", callRelationshipCoverage(null,
unrepresentedIncomingCalls(function, observed)));
monitor.checkCancelled();
return observation;
}
CallBoundaryObservation calls = callBoundaryObservation(
function, currentProgram.getListing().getInstructions(function.getBody(), true), observed, false, null
);
return callRelationshipObservation(result, calls);
}
private JsonObject procedureInfo(JsonObject params) throws Exception {
@@ -1071,18 +1081,10 @@ public final class ReaGhidraBridge extends HeadlessScript {
result.addProperty("direction", direction);
result.add("references", edges);
result.addProperty("reference_kinds_available", true);
JsonArray unresolvedCalls = new JsonArray();
if (direction.equals("outgoing")) for (Instruction instruction : scanInstructions(function, Integer.MAX_VALUE).instructions) {
if (!instruction.getFlowType().isCall()) continue;
boolean resolved = false;
for (Reference reference : instruction.getReferencesFrom()) if (reference.getReferenceType().isCall()) resolved = true;
if (!resolved) {
JsonObject site = new JsonObject();
site.addProperty("address", canonicalAddress(instruction.getAddress()));
site.addProperty("reason", "Instruction decoder identifies a call but ReferenceManager supplies no call destination");
unresolvedCalls.add(site);
}
}
JsonArray unresolvedCalls = direction.equals("outgoing")
? callBoundaryObservation(
function, currentProgram.getListing().getInstructions(function.getBody(), true), null, true, references
).unresolvedCalls() : new JsonArray();
result.add("unresolved_calls", unresolvedCalls);
return result;
}
@@ -1490,16 +1492,28 @@ public final class ReaGhidraBridge extends HeadlessScript {
InstructionScan scan = scanInstructions(function, Integer.MAX_VALUE);
DecompileResults decompilation = decompile(function);
String pseudocode = decompiledText(decompilation);
List<Reference> incomingReferences = collectReferences(function, "incoming")
.stream()
JsonObject availability = new JsonObject();
availability.add("pseudocode", facetAvailability(
pseudocode != null,
function.isExternal() ? "Ghidra does not decompile external functions."
: function.getBody().isEmpty() ? "Ghidra has no function body to decompile."
: "Ghidra completed decompilation without returning pseudocode."
));
if (pseudocode == null) {
pseudocode = "";
}
List<Reference> bodyIncoming = collectReferences(function, "incoming");
List<Reference> incomingReferences = bodyIncoming.stream()
.filter(reference -> !function.getBody().contains(reference.getFromAddress()))
.toList();
List<Reference> outgoingReferences = collectReferences(function, "outgoing");
JsonArray incoming = referenceEdges(incomingReferences);
JsonArray outgoing = referenceEdges(outgoingReferences);
JsonArray comments = comments(scan.instructions);
JsonArray callers = procedureIdentities(function.getCallingFunctions(monitor));
JsonArray callees = procedureIdentities(function.getCalledFunctions(monitor));
JsonArray comments = comments(function);
Set<Function> callerFunctions = function.getCallingFunctions(monitor);
JsonArray callers = procedureIdentities(callerFunctions);
Set<Function> calleeFunctions = function.getCalledFunctions(monitor);
JsonArray callees = procedureIdentities(calleeFunctions);
JsonArray referencedStrings = referencedStrings(outgoingReferences);
JsonArray referencedNames = referencedNames(outgoingReferences);
JsonArray blocks = basicBlockValues(function);
@@ -1511,7 +1525,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
procedure.add("locals", functionLocals(function));
JsonObject result = new JsonObject();
result.add("procedure", procedure);
result.addProperty("pseudocode", pseudocode == null ? "" : pseudocode);
result.addProperty("pseudocode", pseudocode);
result.add("assembly", assembly);
result.add("comments", comments);
result.add("callers", callers);
@@ -1520,20 +1534,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
result.add("outgoing_references", outgoing);
result.add("referenced_strings", referencedStrings);
result.add("referenced_names", referencedNames);
JsonObject facets = new JsonObject();
JsonObject pseudocodeAvailability = new JsonObject();
pseudocodeAvailability.addProperty("available", pseudocode != null);
if (pseudocode == null) {
pseudocodeAvailability.addProperty("reason", "Ghidra did not supply pseudocode.");
}
facets.add("pseudocode", pseudocodeAvailability);
for (String facet : List.of("assembly", "comments", "callers", "callees",
"references", "strings_names", "cfg")) {
JsonObject available = new JsonObject();
available.addProperty("available", true);
facets.add(facet, available);
}
result.add("facet_availability", facets);
boolean swift = swiftProgram();
boolean aarch64 = currentProgram.getLanguageID().getIdAsString().startsWith("AARCH64:");
SwiftStringLiteralDecode swiftLiterals = swift && aarch64 ? swiftStringLiterals(function) : null;
@@ -1547,6 +1547,15 @@ public final class ReaGhidraBridge extends HeadlessScript {
nativeApiBoundary(function, decompilation)
);
result.add("native_value_flow", nativeValueFlow(decompilation));
// Successful full queries distinguish collected empty sets from missing facts.
// Availability does not claim that Ghidra resolved every runtime relationship.
for (String facet : List.of("comments", "callers", "callees", "references", "strings_names", "cfg")) {
availability.add(facet, facetAvailability(true, null));
}
availability.add("assembly", facetAvailability(
!scan.truncated(), "The function instruction scan was truncated."
));
result.add("facet_availability", availability);
JsonArray limitations = new JsonArray();
limitations.add(
"Unresolved computed or indirect flows without target addresses are not represented as reference edges."
@@ -1574,8 +1583,10 @@ public final class ReaGhidraBridge extends HeadlessScript {
"Swift String literal decoding is implemented for AArch64 register pairs only; no Swift literals were decoded for this architecture."
);
}
List<String> boundaryLimitations = noReturnCallLimitations(scan.instructions);
for (String limitation : boundaryLimitations) {
CallBoundaryObservation calls = callBoundaryObservation(
function, scan.instructions, calleeFunctions, true, outgoingReferences);
result.add("unresolved_calls", calls.unresolvedCalls());
for (String limitation : calls.limitations()) {
limitations.add(limitation);
}
if (decompilation != null && decompilation.getErrorMessage() != null &&
@@ -1583,7 +1594,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
limitations.add("Ghidra decompiler diagnostic: " + decompilation.getErrorMessage());
}
result.add("limitations", limitations);
return new FunctionAnalysisObservation(result, boundaryLimitations);
monitor.checkCancelled();
int missingCallers = 0;
for (Reference reference : bodyIncoming) {
monitor.checkCancelled();
if (unrepresentedCaller(reference, callerFunctions)) missingCallers++;
}
return new FunctionAnalysisObservation(result, calls.limitations(),
callRelationshipCoverage(calls, missingCallers));
}
private JsonObject nativeValueFlow(DecompileResults decompilation) throws Exception {
@@ -3287,54 +3305,202 @@ public final class ReaGhidraBridge extends HeadlessScript {
return result;
}
private static JsonObject facetAvailability(boolean available, String reason) {
JsonObject result = new JsonObject();
result.addProperty("available", available);
if (!available) result.addProperty("reason", reason);
return result;
}
private JsonObject callRelationshipObservation(JsonElement value, CallBoundaryObservation calls)
throws Exception {
monitor.checkCancelled();
JsonObject result = functionObservation(value, calls.limitations());
result.add("relationship_coverage", callRelationshipCoverage(calls, null));
return result;
}
private static JsonObject callRelationshipCoverage(
CallBoundaryObservation calls, Integer missingCallers) {
JsonObject coverage = completeResolvedCallCoverage();
Integer omitted = calls == null ? null : calls.unrepresentedCalleeCalls();
JsonArray directions = new JsonArray();
if (missingCallers != null && missingCallers > 0) directions.add("callers");
String callerLimitation = missingCallers != null && missingCallers > 0
? "Ghidra's caller inventory omits sources of " + missingCallers +
" call reference(s) into this function's body." : null;
if ((calls != null && omitted == null) || (calls == null && missingCallers == null)) {
coverage.addProperty("status", "unknown");
String reason = calls == null ? "No call inventory was examined."
: "The callee inventory was not examined alongside decoded call targets.";
coverage.addProperty("reason", reason + (callerLimitation == null ? "" : " " + callerLimitation));
if (calls == null && missingCallers == null) directions.add("callers");
directions.add("callees");
coverage.add("limited_directions", directions);
return coverage;
}
JsonArray limitations = new JsonArray();
if (callerLimitation != null) limitations.add(callerLimitation);
if (omitted != null && omitted > 0) {
limitations.add("Ghidra's callee inventory omits targets of " + omitted +
" call reference(s) or decoded call(s) without destinations.");
directions.add("callees");
}
if (!limitations.isEmpty()) {
coverage.addProperty("status", "partial");
coverage.add("limitations", limitations);
coverage.add("limited_directions", directions);
}
return coverage;
}
private static JsonObject completeResolvedCallCoverage() {
JsonObject coverage = new JsonObject();
coverage.addProperty("status", "complete");
JsonArray diagnostics = new JsonArray();
diagnostics.add(
"Only resolved Ghidra call references are returned; unresolved computed or indirect calls remain unknown, while function classifications distinguish thunks and externals."
);
coverage.add("diagnostics", diagnostics);
return coverage;
}
private boolean unrepresentedCaller(Reference reference, Set<Function> callers) {
if (!reference.getReferenceType().isCall()) return false;
Function caller = containingFunction(reference.getFromAddress());
return caller == null || !callers.contains(caller);
}
private int unrepresentedIncomingCalls(Function function, Set<Function> callers)
throws Exception {
int[] omitted = { 0 };
visitReferences(function, "incoming", reference -> {
if (unrepresentedCaller(reference, callers)) omitted[0]++;
});
return omitted[0];
}
private List<String> noReturnCallLimitations(Iterable<Instruction> instructions)
throws Exception {
List<String> limitations = new ArrayList<>();
for (Instruction instruction : instructions) {
monitor.checkCancelled();
if (!instruction.getFlowType().isCall() || !instruction.getFlowType().isTerminal()) {
continue;
}
Address[] destinations = instruction.getFlows();
StringBuilder limitation = new StringBuilder();
limitation.append("Ghidra Listing reports a terminal call at ")
.append(canonicalAddress(instruction.getAddress()));
if (destinations.length == 0) {
limitation.append(" with no resolved callee; its no-return basis is unknown.");
}
else {
limitation.append(" to ");
for (int index = 0; index < destinations.length; index++) {
if (index > 0) limitation.append(", ");
Address destination = destinations[index];
limitation.append(canonicalAddress(destination));
Function target = currentProgram.getFunctionManager().getFunctionAt(destination);
if (target == null) {
limitation.append(" (callee identity unknown)");
}
else {
limitation.append(" ").append(procedureName(target));
limitation.append(" (external=").append(target.isExternal());
limitation.append(", thunk=").append(target.isThunk());
limitation.append(", hasNoReturn=").append(target.hasNoReturn()).append(")");
}
}
limitation.append("; fallthrough is excluded by Ghidra's flow model.");
Function target = destinations.length == 1
? currentProgram.getFunctionManager().getFunctionAt(destinations[0])
: null;
if (target != null && target.hasNoReturn()) {
limitation.append(" This records Ghidra's FunctionManager flag, not an independent verification that the callee cannot return.");
}
else {
limitation.append(" The reason for the terminal flow is not established by this observation.");
}
}
limitations.add(limitation.toString());
String limitation = terminalCallLimitation(instruction);
if (limitation != null) limitations.add(limitation);
}
return limitations;
}
private CallBoundaryObservation callBoundaryObservation(
Function function, Iterable<Instruction> instructions, Set<Function> returnedCallees,
boolean includeUnresolvedSites, List<Reference> observedReferences)
throws Exception {
List<String> limitations = new ArrayList<>();
CallBoundaryReferences calls = new CallBoundaryReferences(returnedCallees, includeUnresolvedSites);
if (observedReferences == null) visitReferences(function, "outgoing", calls::observe);
else for (Reference reference : observedReferences) {
monitor.checkCancelled();
calls.observe(reference);
}
for (Instruction instruction : instructions) {
monitor.checkCancelled();
String limitation = terminalCallLimitation(instruction);
if (limitation != null) limitations.add(limitation);
if (!instruction.getFlowType().isCall()) continue;
boolean destination = false;
for (Reference reference : instruction.getReferencesFrom()) {
if (!reference.getReferenceType().isCall()) continue;
destination = true;
break;
}
if (!destination) {
calls.unrepresented++;
calls.unresolved(instruction.getAddress(),
"Instruction decoder identifies a call but ReferenceManager supplies no call destination");
}
}
return new CallBoundaryObservation(List.copyOf(limitations),
returnedCallees == null ? null : calls.unrepresented, calls.unresolvedCalls());
}
private final class CallBoundaryReferences {
private final Set<Function> returnedCallees;
private final TreeMap<Address, String> unresolved;
private int unrepresented;
CallBoundaryReferences(Set<Function> returnedCallees, boolean includeUnresolvedSites) {
this.returnedCallees = returnedCallees;
unresolved = includeUnresolvedSites ? new TreeMap<>() : null;
}
void observe(Reference reference) {
if (!reference.getReferenceType().isCall()) return;
// Reference identity follows the same containing-function owner as referenceEdge.
Function target = containingFunction(reference.getToAddress());
if (returnedCallees != null && (target == null || !returnedCallees.contains(target)))
unrepresented++;
if (target == null) unresolved(reference.getFromAddress(),
"A call destination has no known containing Ghidra procedure");
}
void unresolved(Address address, String reason) {
if (unresolved != null) unresolved.put(address, reason);
}
JsonArray unresolvedCalls() {
if (unresolved == null) return null;
JsonArray result = new JsonArray();
for (var item : unresolved.entrySet()) {
JsonObject site = new JsonObject();
site.addProperty("address", canonicalAddress(item.getKey()));
site.addProperty("reason", item.getValue());
result.add(site);
}
return result;
}
}
private String terminalCallLimitation(Instruction instruction) {
if (!instruction.getFlowType().isCall() || !instruction.getFlowType().isTerminal())
return null;
Address[] destinations = instruction.getFlows();
StringBuilder limitation = new StringBuilder();
limitation.append("Ghidra Listing reports a terminal call at ")
.append(canonicalAddress(instruction.getAddress()));
if (destinations.length == 0) {
limitation.append(" with no resolved callee; its no-return basis is unknown.");
}
else {
limitation.append(" to ");
for (int index = 0; index < destinations.length; index++) {
if (index > 0) limitation.append(", ");
Address destination = destinations[index];
limitation.append(canonicalAddress(destination));
Function target = currentProgram.getFunctionManager().getFunctionAt(destination);
if (target == null) {
limitation.append(" (callee identity unknown)");
}
else {
limitation.append(" ").append(procedureName(target));
limitation.append(" (external=").append(target.isExternal());
limitation.append(", thunk=").append(target.isThunk());
limitation.append(", hasNoReturn=").append(target.hasNoReturn()).append(")");
}
}
limitation.append("; fallthrough is excluded by Ghidra's flow model.");
Function target = destinations.length == 1
? currentProgram.getFunctionManager().getFunctionAt(destinations[0])
: null;
if (target != null && target.hasNoReturn()) {
limitation.append(" This records Ghidra's FunctionManager flag, not an independent verification that the callee cannot return.");
}
else {
limitation.append(" The reason for the terminal flow is not established by this observation.");
}
}
return limitation.toString();
}
private static String renderAssembly(List<Instruction> instructions) {
return String.join("\n", renderAssemblyLines(instructions));
}
@@ -3358,15 +3524,27 @@ public final class ReaGhidraBridge extends HeadlessScript {
private List<Reference> collectReferences(
Function function,
String direction) throws Exception {
TreeMap<String, Reference> observed = new TreeMap<>();
visitReferences(function, direction, reference -> addReference(observed, reference));
List<Reference> result = new ArrayList<>(observed.values());
result.sort(REFERENCE_ORDER);
return result;
}
@FunctionalInterface
private interface ReferenceVisitor {
void accept(Reference reference) throws Exception;
}
private void visitReferences(Function function, String direction, ReferenceVisitor visitor)
throws Exception {
// References can target instruction interiors or originate in embedded
// data. Use actual body ownership, including an external's exact entry.
AddressSet scope = new AddressSet(function.getBody());
scope.add(function.getEntryPoint());
AddressSet scope = functionObservationScope(function);
boolean outgoing = direction.equals("outgoing");
AddressIterator addresses = outgoing
? currentProgram.getReferenceManager().getReferenceSourceIterator(scope, true)
: currentProgram.getReferenceManager().getReferenceDestinationIterator(scope, true);
TreeMap<String, Reference> observed = new TreeMap<>();
while (addresses.hasNext()) {
monitor.checkCancelled();
Address address = addresses.next();
@@ -3374,7 +3552,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
for (Reference reference : currentProgram.getReferenceManager()
.getReferencesFrom(address)) {
monitor.checkCancelled();
addReference(observed, reference);
if (!reference.isEntryPointReference()) visitor.accept(reference);
}
}
else {
@@ -3382,19 +3560,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
.getReferencesTo(address);
while (iterator.hasNext()) {
monitor.checkCancelled();
addReference(observed, iterator.next());
Reference reference = iterator.next();
if (!reference.isEntryPointReference()) visitor.accept(reference);
}
}
}
List<Reference> result = new ArrayList<>(observed.values());
result.sort(REFERENCE_ORDER);
return result;
}
private static void addReference(TreeMap<String, Reference> observed, Reference reference) {
if (reference.isEntryPointReference()) {
return;
}
String key = canonicalAddress(reference.getFromAddress()) + "\u0000" +
canonicalAddress(reference.getToAddress()) + "\u0000" +
reference.getReferenceType().getName() + "\u0000" +
@@ -3403,6 +3576,12 @@ public final class ReaGhidraBridge extends HeadlessScript {
observed.putIfAbsent(key, reference);
}
private static AddressSet functionObservationScope(Function function) {
AddressSet scope = new AddressSet(function.getBody());
scope.add(function.getEntryPoint());
return scope;
}
private JsonArray referenceEdges(List<Reference> references) {
JsonArray result = new JsonArray();
for (Reference reference : references) {
@@ -3556,23 +3735,28 @@ public final class ReaGhidraBridge extends HeadlessScript {
return result;
}
private static JsonArray comments(List<Instruction> instructions) {
private JsonArray comments(Function function) throws Exception {
Listing listing = currentProgram.getListing();
AddressSet scope = functionObservationScope(function);
AddressIterator addresses = listing.getCommentAddressIterator(scope, true);
TreeMap<String, JsonObject> observed = new TreeMap<>();
for (Instruction instruction : instructions) {
while (addresses.hasNext()) {
monitor.checkCancelled();
Address address = addresses.next();
for (CommentType type : CommentType.values()) {
String text = instruction.getComment(type);
String text = listing.getComment(type, address);
if (text == null || text.isEmpty()) {
continue;
}
String kind = type == CommentType.EOL ? "inline" : "comment";
JsonObject item = new JsonObject();
item.addProperty("address", canonicalAddress(instruction.getAddress()));
item.addProperty("address", canonicalAddress(address));
item.addProperty("kind", kind);
item.addProperty("text", text);
String key = canonicalAddress(instruction.getAddress()) + "\u0000" +
String key = canonicalAddress(address) + "\u0000" +
kind + "\u0000" + type.name() + "\u0000" + text;
observed.put(key, item);
}
}
}
JsonArray result = new JsonArray();
observed.values().forEach(result::add);
@@ -4508,7 +4692,8 @@ public final class ReaGhidraBridge extends HeadlessScript {
private record InventoryItem(Address address, String value, JsonObject facts) {}
private record AddressNameRequest(String inputAddress, Address address, String name) {}
private record FunctionEntry(Function function, InventoryItem item) {}
private record FunctionAnalysisObservation(JsonObject dossier, List<String> limitations) {}
private record FunctionAnalysisObservation(JsonObject dossier, List<String> limitations, JsonObject relationshipCoverage) {}
private record CallBoundaryObservation(List<String> limitations, Integer unrepresentedCalleeCalls, JsonArray unresolvedCalls) {}
private record SwiftStringLiteralDecode(JsonArray literals, JsonArray omissions) {}
private record SwiftLiteralLayout(
String representation,
+133 -12
View File
@@ -11,6 +11,7 @@ import math
import re
import resource
from pathlib import Path
from urllib.parse import unquote_to_bytes
from mitmproxy import ctx, version
from mitmproxy.io import tnetstring
@@ -41,6 +42,113 @@ class BoundedReader:
return self.handle.read(count)
SPECIAL_SCHEMES = {"ftp", "file", "http", "https", "ws", "wss"}
SCHEME_PREFIX = r"^[\x00-\x20]*([a-z][a-z0-9+.-]*):"
NETWORK_PREFIX = r"^[\x00-\x20]*[/\\]{2}"
URL_IGNORABLE_CONTROLS = "\t\n\r"
URL_CONTROL_TRANSLATION = {ord(char): None for char in URL_IGNORABLE_CONTROLS}
FORBIDDEN_DOMAIN_CODE_POINTS = {chr(code) for code in range(0x21)} | set("#%/:<>?@[\\]^|\x7f")
def authority_start(text, base_scheme=None):
"""Return where an authority begins and whether it is special, or None.
Follows WHATWG URL parsing as src/domain/recordedTransportUrl.ts does:
special schemes (and scheme-relative references) skip every leading "/"
or "\\" before the authority, so `https://\\user:pass@host` still has
userinfo. File URLs never carry userinfo: an "@" in a file host makes
WHATWG reject the whole URL, so there are no credentials to remove.
"""
scheme = re.match(SCHEME_PREFIX, text, flags=re.I)
name = scheme.group(1).lower() if scheme is not None else None
if name == "file" or (name is None and base_scheme == "file"):
return None
if name is not None and name in SPECIAL_SCHEMES:
# A same-scheme Location without two slash-equivalent separators is
# relative to the recorded request, so its @ belongs to the path.
following = text[scheme.end():scheme.end() + 2]
if name == base_scheme and (len(following) < 2 or any(char not in "/\\" for char in following)):
return None
start = scheme.end()
else:
if scheme is not None:
if text[scheme.end():scheme.end() + 2] != "//":
return None
return scheme.end() + 2, False
# A non-special base treats backslashes as authority data and only
# recognizes two forward slashes as a network reference.
if base_scheme is not None and base_scheme not in SPECIAL_SCHEMES:
prefix = re.match(r"^[\x00-\x20]*//", text)
return None if prefix is None else (prefix.end(), False)
prefix = re.match(NETWORK_PREFIX, text)
if prefix is None:
return None
start = prefix.end()
while start < len(text) and text[start] in "/\\":
start += 1
return start, True
def special_host_parses(host_port):
"""Whether WHATWG could accept a special authority's host and port.
A rejected host fails the whole URL, so it has no credentials to remove.
This mirrors the forbidden-code-point, empty-host and port rules; IDNA,
IPv4 range and IPv6 syntax failures are not modeled and still redact.
"""
if host_port.startswith("["):
host_end = host_port.find("]") + 1
if host_end == 0:
return False
else:
colon = host_port.find(":")
host_end = len(host_port) if colon < 0 else colon
host = unquote_to_bytes(host_port[:host_end]).decode("utf-8", errors="replace")
if host == "" or any(char in FORBIDDEN_DOMAIN_CODE_POINTS for char in host):
return False
port = host_port[host_end:]
if port == "":
return True
digits = port[1:]
return port[0] == ":" and (digits == "" or (digits.isascii() and digits.isdigit() and int(digits) <= 65535))
def without_userinfo(value, base_scheme=None):
"""Remove an authority's userinfo using the WHATWG authority boundary.
Special schemes and scheme-relative references also end the authority
at a backslash, so later path text containing "@" is not userinfo.
"""
binary = isinstance(value, bytes)
text = value.decode("latin-1") if binary else value
# WHATWG removes these controls before interpreting scheme/separators.
# Keep that parse view separate from the exact source spelling and bytes.
parsed_text = text.translate(URL_CONTROL_TRANSLATION)
located = authority_start(parsed_text, base_scheme)
if located is None:
return value
start, special = located
terminators = "/?#\\" if special else "/?#"
end = start
while end < len(parsed_text) and parsed_text[end] not in terminators:
end += 1
at = parsed_text.rfind("@", start, end)
if at < start or (special and not special_host_parses(parsed_text[at + 1 : end])):
return value
parsed_offset = 0
raw_start = 0
for raw_offset, char in enumerate(text):
if char in URL_IGNORABLE_CONTROLS:
continue
if parsed_offset == start:
raw_start = raw_offset
if parsed_offset == at:
safe = text[:raw_start] + text[raw_offset + 1 :]
return safe.encode("latin-1") if binary else safe
parsed_offset += 1
raise CaptureFailure("decoder", "Normalized URL authority could not be bound to its source.")
def project_record(state, secrets):
binaries, numbers, redactions = [], [], []
byte_secrets = []
@@ -58,16 +166,23 @@ def project_record(state, secrets):
return None
return text
def transport_value(value, path, transport_url):
def transport_value(value, path, transport_url, base_scheme):
if not (transport_url or re.fullmatch(r"/(?:backup/)*request/(?:path|authority)", path)):
return value, False
binary = isinstance(value, bytes)
pattern = rb"^([\x00-\x20]*(?:[a-z][a-z0-9+.-]*:)?//)[^/?#]*@" if binary else r"^([\x00-\x20]*(?:[a-z][a-z0-9+.-]*:)?//)[^/?#]*@"
replacement = rb"\1" if binary else r"\1"
separator = b"@" if binary else "@"
safe_url = re.sub(pattern, replacement, value, flags=re.I)
if path.endswith("/authority") and separator in safe_url:
safe_url = safe_url.rsplit(separator, 1)[1]
if transport_url:
safe_url = without_userinfo(value, base_scheme)
elif path.endswith("/authority"):
safe_url = value.rsplit(separator, 1)[-1]
else:
# Native request.path is origin-form unless it explicitly carries
# a scheme (absolute-form). Leading separators are path evidence,
# not a header URL reference's network authority.
text = value.decode("latin-1") if binary else value
safe_url = without_userinfo(value) if re.match(
SCHEME_PREFIX, text.translate(URL_CONTROL_TRANSLATION), flags=re.I
) else value
changed = safe_url != value
if changed:
redactions.append({"pointer": path, "reason": "transport-credential"})
@@ -80,7 +195,7 @@ def project_record(state, secrets):
del numbers[before[1]:]
del redactions[before[2]:]
def visit(value, path, depth=0, credential=False, transport_url=False):
def visit(value, path, depth=0, credential=False, transport_url=False, base_scheme=None):
if depth > 64:
raise CaptureFailure("input-limit", "Native capture exceeds the 64-level evidence nesting budget.", path)
if credential:
@@ -90,7 +205,7 @@ def project_record(state, secrets):
binaries.append({"pointer": path, "representation": "producer-bytes", "state": "redacted", "content_base64": None, "bytes": None, "sha256": None})
return None
if isinstance(value, bytes):
safe_bytes, credential_url = transport_value(value, path, transport_url)
safe_bytes, credential_url = transport_value(value, path, transport_url, base_scheme)
content_base64 = base64.b64encode(value).decode("ascii")
digest = hashlib.sha256(value).hexdigest()
hidden = any(secret in value or secret in safe_bytes for secret in byte_secrets) or any(secret in content_base64 or secret in digest or secret in str(len(value)) for secret in secrets)
@@ -107,7 +222,7 @@ def project_record(state, secrets):
return None
return safe_text
if isinstance(value, str):
safe_text, _ = transport_value(value, path, transport_url)
safe_text, _ = transport_value(value, path, transport_url, base_scheme)
return redact(safe_text, path, value)
if isinstance(value, bool) or value is None:
return value
@@ -125,10 +240,16 @@ def project_record(state, secrets):
for index, field in enumerate(value):
if not isinstance(field, (list, tuple)) or len(field) != 2 or not isinstance(field[0], bytes):
raise CaptureFailure("format", "Native header/trailer is not an ordered byte pair.", pointer(path, index))
result.append([visit(field[0], pointer(pointer(path, index), 0), depth + 2), visit(field[1], pointer(pointer(path, index), 1), depth + 2, field[0].lower() in CREDENTIALS, field[0].lower() in {b"location", b"referer", b"origin"})])
result.append([visit(field[0], pointer(pointer(path, index), 0), depth + 2), visit(field[1], pointer(pointer(path, index), 1), depth + 2, field[0].lower() in CREDENTIALS, field[0].lower() in {b"location", b"referer", b"origin"}, base_scheme if field[0].lower() == b"location" else None)])
return result
return [visit(item, pointer(path, index), depth + 1) for index, item in enumerate(value)]
return [visit(item, pointer(path, index), depth + 1, base_scheme=base_scheme) for index, item in enumerate(value)]
if isinstance(value, dict):
if path == "" or re.fullmatch(r"/backup(?:/backup)*", path):
request = value.get("request")
scheme = request.get("scheme") if isinstance(request, dict) else None
if isinstance(scheme, bytes):
scheme = scheme.decode("latin-1")
base_scheme = scheme.lower() if isinstance(scheme, str) and re.fullmatch(r"[a-z][a-z0-9+.-]*", scheme, flags=re.I) else None
result = {}
for key, item in value.items():
if not isinstance(key, str):
@@ -139,7 +260,7 @@ def project_record(state, secrets):
validate_omitted(item, pointer(path, key), depth + 1)
redactions.append({"pointer": path, "reason": "explicit-sensitive-value", "scope": "property-name"})
continue
result[key] = visit(item, pointer(path, key), depth + 1)
result[key] = visit(item, pointer(path, key), depth + 1, base_scheme=base_scheme)
return result
raise CaptureFailure("decoder", "Native parser returned an unexpected value type: " + type(value).__name__ + ".", path)
+214
View File
@@ -0,0 +1,214 @@
"""Source-owned tests for the mitmproxy adapter's record projection."""
import importlib.util
import os
import sys
import types
import unittest
from unittest import mock
def load_capture():
"""Load the adapter with fresh mitmproxy stand-ins scoped to its import."""
fake_mitmproxy = types.ModuleType("mitmproxy")
fake_mitmproxy.ctx = types.SimpleNamespace()
fake_mitmproxy.version = types.SimpleNamespace(VERSION="12.2.3")
fake_io = types.ModuleType("mitmproxy.io")
fake_io.tnetstring = types.ModuleType("mitmproxy.io.tnetstring")
stand_ins = {
"mitmproxy": fake_mitmproxy,
"mitmproxy.io": fake_io,
"mitmproxy.io.tnetstring": fake_io.tnetstring,
}
spec = importlib.util.spec_from_file_location(
"rea_mitmproxy_capture", os.path.join(os.path.dirname(__file__), "capture.py")
)
module = importlib.util.module_from_spec(spec)
with mock.patch.dict(sys.modules, stand_ins):
spec.loader.exec_module(module)
return module
capture = load_capture()
def location(value, scheme=None):
state = {"response": {"headers": [[b"Location", value]]}}
if scheme is not None:
state["request"] = {"scheme": scheme}
record = capture.project_record(state, [])
sidecar = next(
field for field in record["binary_fields"] if field["pointer"] == "/response/headers/0/1"
)
return record["reported"]["response"]["headers"][0][1], sidecar, record["redactions"]
class TransportUserinfoTests(unittest.TestCase):
def test_ignorable_controls_cannot_hide_authority_credentials(self):
for control in [b"\t", b"\n", b"\r"]:
for value in [b"https://" + control + b"/user:pass@host/path", b"ht" + control + b"tps://user:" + control + b"pass@host/path"]:
with self.subTest(value=value):
text, sidecar, redactions = location(value, b"https")
self.assertNotIn("user", text)
self.assertNotIn("pass", text)
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(len(redactions), 1)
def test_mixed_network_separators_remove_credentials(self):
for prefix in [b"\\\\", b"/\\", b"\\/"]:
with self.subTest(prefix=prefix):
text, sidecar, redactions = location(prefix + b"user:pass@host/path", b"https")
self.assertEqual(text, prefix.decode("ascii") + "host/path")
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(len(redactions), 1)
def test_non_special_base_keeps_backslashes_inside_network_authorities(self):
text, sidecar, redactions = location(b"//user\\name:pass@host/x", b"foo")
self.assertEqual(text, "//host/x")
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(len(redactions), 1)
text, sidecar, redactions = location(b"/\\invoice@customer", b"foo")
self.assertEqual(text, "/\\invoice@customer")
self.assertEqual(sidecar["state"], "retained")
self.assertEqual(redactions, [])
def test_native_request_paths_preserve_origin_form_and_redact_absolute_form(self):
for value in [b"/\\invoice@customer", b"//invoice@customer", b"\\\\invoice@customer"]:
with self.subTest(value=value):
record = capture.project_record({"request": {"path": value}}, [])
self.assertEqual(record["reported"]["request"]["path"], value.decode("ascii"))
self.assertEqual(record["binary_fields"][0]["state"], "retained")
self.assertEqual(record["redactions"], [])
record = capture.project_record({"request": {"path": b"https://user:pass@host/x"}}, [])
self.assertEqual(record["reported"]["request"]["path"], "https://host/x")
self.assertEqual(record["binary_fields"][0]["state"], "redacted")
def test_same_scheme_locations_preserve_relative_path_text_and_bytes(self):
for value in [b"https:invoice@customer", b"https:/invoice@customer", b"https:\\invoice@customer"]:
with self.subTest(value=value):
text, sidecar, redactions = location(value, b"https")
self.assertEqual(text, value.decode("ascii"))
self.assertEqual(sidecar["state"], "retained")
self.assertEqual(redactions, [])
def test_location_authorities_still_remove_credentials_with_a_base_scheme(self):
for value, scheme, expected in [
(b"https://user:pass@host/path", b"https", "https://host/path"),
(b"https:\\\\user:pass@host/path", b"https", "https:\\\\host/path"),
(b"https:user:pass@host/path", b"http", "https:host/path"),
]:
with self.subTest(value=value, scheme=scheme):
text, sidecar, redactions = location(value, scheme)
self.assertEqual(text, expected)
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(len(redactions), 1)
def test_backup_location_uses_its_own_request_scheme(self):
record = capture.project_record({
"request": {"scheme": b"http"},
"backup": {
"request": {"scheme": b"https"},
"response": {"headers": [[b"Location", b"https:invoice@customer"]]},
},
}, [])
self.assertEqual(record["reported"]["backup"]["response"]["headers"][0][1], "https:invoice@customer")
self.assertEqual(record["redactions"], [])
def test_backslash_ends_a_special_scheme_authority(self):
text, sidecar, redactions = location(b"https://example.test\\part@tail")
self.assertEqual(text, "https://example.test\\part@tail")
self.assertEqual(sidecar["state"], "retained")
self.assertIsNotNone(sidecar["content_base64"])
self.assertEqual(redactions, [])
def test_userinfo_before_a_backslash_is_still_removed(self):
text, sidecar, redactions = location(b"https://user:pw@example.test\\path")
self.assertEqual(text, "https://example.test\\path")
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(
redactions,
[{"pointer": "/response/headers/0/1", "reason": "transport-credential"}],
)
def test_ordinary_userinfo_is_removed(self):
text, _sidecar, _redactions = location(b"https://user:pw@example.test/a@b")
self.assertEqual(text, "https://example.test/a@b")
def test_empty_userinfo_marker_is_removed(self):
self.assertEqual(
capture.without_userinfo("https://@host"), "https://host"
)
def test_special_schemes_skip_leading_slashes_before_userinfo(self):
for value, expected in [
(b"https://\\user:pass@host/path", b"https://\\host/path"),
(b"https:\\\\user:pass@host/path", b"https:\\\\host/path"),
(b"https:user:pass@host", b"https:host"),
(b"//\\user:pass@host", b"//\\host"),
]:
with self.subTest(value=value):
self.assertEqual(capture.without_userinfo(value), expected)
text, sidecar, redactions = location(b"https://\\user:pass@host/path")
self.assertEqual(text, "https://\\host/path")
self.assertEqual(sidecar["state"], "redacted")
self.assertEqual(len(redactions), 1)
def test_file_base_keeps_paths_after_the_fixed_network_boundary(self):
for value in [b"///u@h", b"////u@h"]:
with self.subTest(value=value):
text, sidecar, redactions = location(value, b"file")
self.assertEqual(text, value.decode("ascii"))
self.assertEqual(sidecar["state"], "retained")
self.assertIsNotNone(sidecar["content_base64"])
self.assertEqual(redactions, [])
def test_file_urls_keep_path_text(self):
self.assertEqual(capture.without_userinfo("file:///a@b"), "file:///a@b")
def test_file_urls_reject_userinfo_and_keep_their_source(self):
# WHATWG rejects an "@" in a file host, so nothing parses as credentials.
self.assertEqual(
capture.without_userinfo("file://user:pass@host/x"), "file://user:pass@host/x"
)
for value in [b"\\\\user:pass@host/x", b"/\\user:pass@host/x", b"//user:pass@host/x"]:
with self.subTest(value=value):
text, sidecar, redactions = location(value, b"file")
self.assertEqual(text, value.decode("ascii"))
self.assertEqual(sidecar["state"], "retained")
self.assertEqual(redactions, [])
def test_rejected_special_hosts_keep_their_source(self):
# A rejected host or port fails the whole URL, so it has no userinfo.
for value in [
"https:user@%",
"https://user@",
"https://user:pass@h^st/x",
"https://user@ho st/x",
"https://user@host:99999/x",
"https://user@host:12a",
"https://user@[::1",
]:
with self.subTest(value=value):
self.assertEqual(capture.without_userinfo(value), value)
text, sidecar, redactions = location(b"https://user:pass@h^st/x", b"https")
self.assertEqual(text, "https://user:pass@h^st/x")
self.assertEqual(sidecar["state"], "retained")
self.assertEqual(redactions, [])
def test_accepted_special_hosts_still_remove_userinfo(self):
for value, expected in [
("https:user@%41", "https:%41"),
("https://user@[::1]:8443/x", "https://[::1]:8443/x"),
("https://user@host:/x", "https://host:/x"),
("wss://user:pass@host:65535", "wss://host:65535"),
]:
with self.subTest(value=value):
self.assertEqual(capture.without_userinfo(value), expected)
def test_non_special_schemes_keep_backslashes_in_the_authority(self):
self.assertEqual(capture.without_userinfo("foo://a\\b@host/x"), "foo://host/x")
self.assertEqual(capture.without_userinfo(b"//a\\b@host"), b"//a\\b@host")
if __name__ == "__main__":
unittest.main()
+15
View File
@@ -265,6 +265,21 @@ The user-level MCP paths follow the editor's VS Code layout. Trae's official
documentation describes the raw JSON editor without specifying those paths;
loading this registration in Trae remains unverified.
## WorkBuddy
For [WorkBuddy](https://www.workbuddy.cn/docs/workbuddy/From-Beginner-to-Expert-Guide/Function-Description/MCP-Guide), setup writes `mcpServers.rea` into the
global `~/.workbuddy/mcp.json` (a standard `mcpServers` JSON file at the user
home root on every platform). Entries carry `command`, `args`, and `env`;
stdio is the default transport. WorkBuddy discovers personal skills from
`~/.workbuddy/skills` (`%userprofile%\.workbuddy\skills` on Windows), so
setup installs its skill copy there.
The global MCP path is a plain JSON file; setup preserves every other server
and top-level key when it upserts `rea`. WorkBuddy also reads a project-level
`<project>/.workbuddy/mcp.json`, which setup does not touch: it registers the
user-level file only. Loading this registration and real REA tool calls
through WorkBuddy are verified.
## ZCode
For [ZCode](https://zcode.z.ai/en/docs/mcp-services), setup registers `rea` in the nested `mcp.servers` table of the
+15 -3
View File
@@ -288,11 +288,12 @@ hosted connector; see [its configuration](client-configuration.md#other-client-s
| Qoder | `qoder` |
| Factory Droid | `droid` |
| Trae | `trae` |
| WorkBuddy | `workbuddy` |
| ZCode | `zcode` |
| DeepSeek Harness | `dsh` |
`dsh` and `kimi` require a release containing them. The registry check above does
not cover them.
`dsh`, `kimi`, and `workbuddy` require a release containing them. The registry
check above does not cover them.
See [client configuration and troubleshooting](client-configuration.md) for
configuration paths, profiles, trust policies, and schema-limit workarounds.
@@ -493,7 +494,13 @@ copying license secrets into logs, or killing unrelated Hopper processes.
REA connects to an existing Ghidra installation on Linux x64/arm64, macOS x64/arm64,
or experimental Windows x64 P0.
It accepts Ghidra 12.1.x and the 64-bit full JDK declared by that installation's
`application.java.min` and `application.java.max`. Current 12.1 releases require
`application.java.min` and `application.java.max`. The JDK's reported platform must also
map to the same native decompiler directory as the host: a JDK running under
emulation (for example x86_64 under Rosetta on Apple silicon) or reporting a
non-host `os.name` fails the doctor
even when it is an otherwise accepted 64-bit full JDK. Select a matching JDK
through `JAVA_HOME`; adding another decompiler in the JVM-selected directory
does not satisfy this host-validated admission rule. Current 12.1 releases require
JDK 21 or newer and set no maximum. The bridge is verified with Ghidra 12.1.4
and JDK 21. Each installation must include the native decompiler for the host
architecture in `Ghidra/Features/Decompiler/os/<platform>/` or the corresponding
@@ -556,6 +563,11 @@ registrations. See [Windows Ghidra P0](windows-ghidra-p0.md) for provider diagno
Doctor validates the platform, architecture, Ghidra 12.1.x application version,
`support/analyzeHeadless` or `support/analyzeHeadless.bat`, the installation's
Java major range, 64-bit JDK bitness, and the presence of `javac`/`javac.exe`.
It also checks that the reported JVM `os.arch` and `os.name` map to the same native decompiler
directory validated for the host; unrecognized reported architectures or a
non-host operating system are refused. LaunchSupport overrides of either
property, including through helper-environment JVM option variables, are
rejected at launch.
When Java is found through `PATH`, setup records its observed JDK home so GUI
MCP clients do not depend on an incidental shell path. Setup shows every exact
environment entry in its plan, writes only after approval, and never downloads,
+1 -1
View File
@@ -138,7 +138,7 @@
"test": "npm run test:with-build",
"test:with-build": "npm run test:prepare && vitest run",
"test:run": "vitest run",
"test:python": "python3 -m unittest bridge/native/test_rea_lldb_tracer.py -v && python3 -m unittest discover -s experiments/analysis-efficiency -p 'test_*.py' -v",
"test:python": "python3 -m unittest bridge/native/test_rea_lldb_tracer.py bridge/mitmproxy/test_capture.py -v && python3 -m unittest discover -s experiments/analysis-efficiency -p 'test_*.py' -v",
"test:local": "node scripts/run-development-tests.mjs local",
"test:fast": "node scripts/ci/test-group.mjs source",
"test:built": "node scripts/ci/test-group.mjs built",
@@ -1,4 +1,5 @@
export function assertCleanLinuxHopperRuntime(
launcherPaths?: readonly string[],
procRoot?: string,
listProcesses?: () => Promise<{ stdout: string }>,
): Promise<void>;
+2 -1
View File
@@ -17,8 +17,9 @@ export const assertCleanLinuxHopperRuntime = async (
: [process.env.HOPPER_LAUNCHER_PATH]),
],
procRoot = "/proc",
listProcesses = () => execFileAsync("ps", ["-ax", "-o", "command="]),
) => {
const { stdout } = await execFileAsync("ps", ["-ax", "-o", "command="]);
const { stdout } = await listProcesses();
if (/\/opt\/hopper\/bin\/Hopper|\bXvfb\b|hopper-demo-x11\.py/u.test(stdout))
throw new Error("Clean Linux runner retained a Hopper demo process");
const launchers = [];
+82
View File
@@ -215,6 +215,27 @@ async function verifyNativeValueAdapters(
const functionEvidence = parseEvidence(analyzed.structuredContent);
assert.equal(functionEvidence.operation, "analyze_function");
assert.equal(functionEvidence.provider.id, "ghidra");
assert.deepEqual(
functionEvidence.normalized_result.facet_availability,
Object.fromEntries(
[
"pseudocode",
"assembly",
"comments",
"callers",
"callees",
"references",
"strings_names",
"cfg",
].map((facet) => [facet, { available: true }]),
),
);
assert.equal(functionEvidence.relationship_coverage?.status, "complete");
assert.ok(
functionEvidence.relationship_coverage.diagnostics.some((diagnostic) =>
diagnostic.includes("indirect calls remain unknown"),
),
);
const functionProfile = analysisProfileSchema.parse(
functionEvidence.analysis_profile,
);
@@ -247,6 +268,7 @@ async function verifyNativeValueAdapters(
assert.ok(
comparisonEvidence.evidence_links.includes(functionEvidence.evidence_id),
);
await verifyUnresolvedCallEvidence(client);
const bundle = await client.callTool({
name: "get_evidence_bundle",
arguments: {},
@@ -307,3 +329,63 @@ async function verifyNativeValueAdapters(
unknowns: cliGraph.unknowns.length,
};
}
async function verifyUnresolvedCallEvidence(client) {
const inventory = await client.callTool({
name: "list_procedures",
arguments: {},
});
assert.notEqual(inventory.isError, true, JSON.stringify(inventory));
const procedures = parseEvidence(
inventory.structuredContent,
).normalized_result;
const indirect = procedures.find((item) =>
item.value.endsWith("rea_ghidra_inventory_indirect"),
);
const leaf = procedures.find((item) =>
item.value.endsWith("rea_ghidra_inventory_leaf"),
);
assert.ok(
indirect && leaf,
"Source fixture lacks the unresolved callback and its possible target",
);
const analyzed = await client.callTool(
{ name: "analyze_function", arguments: { procedure: indirect.address } },
{ timeout: 180000 },
);
assert.notEqual(analyzed.isError, true, JSON.stringify(analyzed));
const evidence = parseEvidence(analyzed.structuredContent);
assert.equal(evidence.relationship_coverage?.status, "partial");
assert.ok(evidence.normalized_result.unresolved_calls.length > 0);
const traced = await client.callTool(
{
name: "trace_call_path",
arguments: {
start: indirect.address,
goal: leaf.address,
direction: "forward",
},
},
{ timeout: 180000 },
);
assert.notEqual(traced.isError, true, JSON.stringify(traced));
const trace = parseEvidence(traced.structuredContent).normalized_result;
assert.equal(trace.goal_status, "unknown");
assert.equal(trace.truncated, true);
assert.ok(trace.residual_unknowns.length > 0);
const compared = await client.callTool(
{
name: "compare_functions",
arguments: {
left: analyzed.structuredContent,
right: analyzed.structuredContent,
},
},
{ timeout: 180000 },
);
assert.notEqual(compared.isError, true, JSON.stringify(compared));
assert.equal(
parseEvidence(compared.structuredContent).normalized_result.status,
"unknown",
);
}
+1 -1
View File
@@ -136,7 +136,7 @@ export async function verifyGhidraBoundaries(
assert.deepEqual(externalDossier.callees, []);
assert.deepEqual(externalDossier.facet_availability.pseudocode, {
available: false,
reason: "Ghidra did not supply pseudocode.",
reason: "Ghidra does not decompile external functions.",
});
for (const facet of [
"assembly",
+8
View File
@@ -606,6 +606,14 @@ export const SUPPORTED_CLIENT_DEFINITIONS = [
markerPath: traeUserDirectory,
format: "json",
},
{
name: "workbuddy",
displayName: "WorkBuddy",
skillPath: [".workbuddy", "skills"],
configPath: [".workbuddy", "mcp.json"],
markerPath: [".workbuddy"],
format: "json",
},
{
name: "grok_build",
displayName: "Grok Build",
@@ -1,5 +1,6 @@
import { dirname } from "node:path";
import { realpath } from "node:fs/promises";
import { constants } from "node:fs";
import { access, realpath, stat } from "node:fs/promises";
import type { ExecutionOptions } from "../AnalysisProvider.js";
import type {
@@ -43,11 +44,35 @@ const resolveInputPath = async (
}
};
/** Resolve the executable and require a launchable regular file before any spawn. */
const resolveExecutablePath = async (
selected: string,
): ReturnType<typeof resolveInputPath> => {
const resolved = await resolveInputPath("executable_path", selected);
if (!resolved.ok) return resolved;
try {
if (!(await stat(resolved.value)).isFile())
throw new Error("not a regular file");
if (process.platform !== "win32")
await access(resolved.value, constants.X_OK);
return resolved;
} catch (cause: unknown) {
return err({
issue: {
path: ["executable_path"],
reason: "invalid_value",
message: `executable_path ${selected} is not an executable file: ${cause instanceof Error ? cause.message : String(cause)}`,
},
cause,
});
}
};
const canonicalizeInput = async (
input: ElectronActiveObservationInput,
): Promise<Result<CanonicalElectronActiveObservationInput, AnalysisError>> => {
const [executable, application, root] = await Promise.all([
resolveInputPath("executable_path", input.executable_path),
resolveExecutablePath(input.executable_path),
resolveInputPath("application_path", input.application_path),
input.application_root === undefined
? Promise.resolve(ok(undefined))
@@ -96,13 +96,16 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati
await systemProcessOwnershipHost.captureBaseline?.(options.signal);
if (options.signal?.aborted)
throw new BrowserObservationError(OPERATION, "cancelled");
application = await this.launch({
const launchOptions: Parameters<typeof electron.launch>[0] = {
executablePath: paths.executable,
cwd: paths.root,
env: safeElectronEnvironment(this.environment, runId),
args: ["-r", hookPath, paths.application, ...input.args],
timeout: Math.max(1, startupDeadline - performance.now()),
});
};
application = await containLaunchRejections(() =>
this.launch(launchOptions),
);
const leaderPid = application.process().pid;
if (
typeof leaderPid !== "number" ||
@@ -377,6 +380,88 @@ const safeElectronEnvironment = (
["REA_PROCESS_RUN_ID", runId] as const,
]);
const PLAYWRIGHT_INTERNAL_FRAME = /playwright-core[\\/]lib[\\/]/;
interface LaunchRejectionScope {
failure?: unknown;
}
const activeLaunches = new Set<LaunchRejectionScope>();
const heldLaunchRejections = new Map<
Promise<unknown>,
{
readonly reason: Error;
readonly candidates: Set<LaunchRejectionScope>;
readonly hadExternalListener: boolean;
}
>();
const holdLaunchRejection = (
reason: unknown,
promise: Promise<unknown>,
): void => {
if (
!(reason instanceof Error) ||
!PLAYWRIGHT_INTERNAL_FRAME.test(reason.stack ?? "")
)
throw reason;
if (!heldLaunchRejections.has(promise))
heldLaunchRejections.set(promise, {
reason,
candidates: new Set(activeLaunches),
hadExternalListener: process
.listeners("unhandledRejection")
.some((listener) => listener !== holdLaunchRejection),
});
};
const forwardLaunchRejection = (
reason: Error,
hadExternalListener: boolean,
): void => {
// Listeners present at the original event already observed it. When the
// holding listener was the only listener, restore Node's fatal behavior.
if (!hadExternalListener)
process.nextTick(() => {
throw reason;
});
};
/** Playwright leaves rejected launch promises unhandled; absorb only those that repeat this launch's failure. */
const containLaunchRejections = async <T>(
launch: () => Promise<T>,
): Promise<T> => {
const scope: LaunchRejectionScope = {};
if (activeLaunches.size === 0)
process.on("unhandledRejection", holdLaunchRejection);
activeLaunches.add(scope);
try {
return await launch();
} catch (cause: unknown) {
scope.failure = cause;
throw cause;
} finally {
await new Promise<void>((resolve) => setImmediate(resolve));
activeLaunches.delete(scope);
if (activeLaunches.size === 0)
process.off("unhandledRejection", holdLaunchRejection);
for (const [
promise,
{ reason, candidates, hadExternalListener },
] of heldLaunchRejections) {
if (!candidates.delete(scope)) continue;
if (
scope.failure instanceof Error &&
reason.message !== "" &&
scope.failure.message.includes(reason.message)
)
heldLaunchRejections.delete(promise);
else if (candidates.size === 0) {
heldLaunchRejections.delete(promise);
forwardLaunchRejection(reason, hadExternalListener);
}
}
}
};
const providerError = (cause: unknown): AnalysisError =>
cause instanceof AnalysisError
? cause
+34 -2
View File
@@ -45,6 +45,24 @@ const registerOverviewCommands = (
.describe("Load and update a local analysis snapshot"),
...nativeOpenOptionFields,
});
// `analyze` declares the JavaScript integrity policy for its JavaScript
// lane. Preserve whether the caller supplied it until the target lane is
// known, then apply the JavaScript default only in that lane. The owner
// field fills an omission with that default, and an `.optional()` wrapper
// keeps reporting it, so every binary target would look like an explicit
// policy. Remove the owner's default before making the CLI option optional;
// reapply its metadata so help and `--schema` keep describing the option.
const ownerIntegrityPolicy =
javascriptApplicationOptions.shape.integrityPolicy;
const defaultIntegrityPolicy = ownerIntegrityPolicy.parse(undefined);
const policyMetadata = ownerIntegrityPolicy.meta();
const omittableIntegrityPolicy = ownerIntegrityPolicy
.removeDefault()
.optional();
const laneIntegrityPolicy =
policyMetadata === undefined
? omittableIntegrityPolicy
: omittableIntegrityPolicy.meta(policyMetadata);
cli.command(CLI_COMMANDS.analyze, {
description: "Get an overview of an app",
args: z.object({
@@ -54,7 +72,7 @@ const registerOverviewCommands = (
.describe("App, program, or analysis database path"),
}),
options: overviewOptions.extend({
integrityPolicy: javascriptApplicationOptions.shape.integrityPolicy,
integrityPolicy: laneIntegrityPolicy,
sourcePath: javascriptApplicationOptions.shape.sourcePath,
}),
run: async ({ args, options }) => {
@@ -67,7 +85,8 @@ const registerOverviewCommands = (
runCliJavaScriptApplicationAnalysis(
{
input_path: resolve(args.path),
integrity_policy: options.integrityPolicy,
integrity_policy:
options.integrityPolicy ?? defaultIntegrityPolicy,
...(options.sourcePath === undefined
? {}
: { source_paths: options.sourcePath }),
@@ -90,6 +109,19 @@ const registerOverviewCommands = (
},
],
);
if (options.integrityPolicy !== undefined)
throw new AnalysisInputError(
"analyze_javascript_application",
undefined,
[
{
path: ["integrity_policy"],
reason: "invalid_value",
message:
"An integrity policy requires the static JavaScript ASAR/directory analysis lane; binary_overview accepts no input.",
},
],
);
return runDirectAnalysis(
args.path,
"binary_overview",
+36
View File
@@ -337,6 +337,42 @@ describe("call path topology and capacity", () => {
});
});
describe("directional call relationship coverage", () => {
it.each([
{
directions: ["callers"] as ["callers"],
status: "not_found",
exhaustive: true,
},
{
directions: ["callees"] as ["callees"],
status: "unknown",
exhaustive: false,
},
{
directions: ["callers", "callees"] as ["callers", "callees"],
status: "unknown",
exhaustive: false,
},
])(
"uses $directions coverage for the forward search",
({ directions, status, exhaustive }) => {
const limitation = "The inventory omitted a reference.";
const relationshipCoverage: RelationshipCoverage = {
status: "partial",
limitations: [limitation],
limited_directions: directions,
};
const result = run([
observe("0x1000", ["0x2000"], "rea-workflow", { relationshipCoverage }),
observe("0x2000", [], "rea-workflow", { relationshipCoverage }),
]);
expect(result).toMatchObject({ status, search_scope: { exhaustive } });
expect(result.limitations).toContain(limitation);
},
);
});
describe("explicit call relationship coverage", () => {
it("accumulates 150,000 partial reasons without argument-stack exhaustion", () => {
const result = run([
+2 -2
View File
@@ -10,7 +10,7 @@ import {
parseFunctionEvidence,
type FunctionSnapshot,
} from "./functionDossierEvidence.js";
import { isCallerOnlyUnknownCoverage } from "./functionCallCoverage.js";
import { isCallerOnlyIncompleteCoverage } from "./functionCallCoverage.js";
import { compareUnicodeCodePoints } from "./unicodeCodePointOrder.js";
/** Minimal directed caller-to-callee adjacency for call-path search. */
@@ -268,7 +268,7 @@ const inspectSearch = ({
coverage === null
? snapshot.limitations.length > 0
: coverage.status !== "complete" &&
!isCallerOnlyUnknownCoverage(coverage)
!isCallerOnlyIncompleteCoverage(coverage)
)
incompleteExpandedCoverage = true;
} else if (node === goal) {
+33 -7
View File
@@ -2,7 +2,7 @@ import { describe, expect, it } from "vitest";
import {
combineFunctionRelationshipCoverage,
isCallerOnlyUnknownCoverage,
isCallerOnlyIncompleteCoverage,
} from "./functionCallCoverage.js";
import type { RelationshipCoverage } from "./relationshipCoverage.js";
@@ -22,14 +22,14 @@ const cappedCallees: RelationshipCoverage = {
};
const complete: RelationshipCoverage = { status: "complete" };
describe("isCallerOnlyUnknownCoverage", () => {
describe("isCallerOnlyIncompleteCoverage", () => {
it("detects unknown callers beside complete callees", () => {
const combined = combineFunctionRelationshipCoverage(
unknownCallers,
complete,
);
expect(combined.status).toBe("unknown");
expect(isCallerOnlyUnknownCoverage(combined)).toBe(true);
expect(isCallerOnlyIncompleteCoverage(combined)).toBe(true);
});
it("rejects unknown callees beside complete callers", () => {
@@ -37,7 +37,7 @@ describe("isCallerOnlyUnknownCoverage", () => {
complete,
unknownCallees,
);
expect(isCallerOnlyUnknownCoverage(combined)).toBe(false);
expect(isCallerOnlyIncompleteCoverage(combined)).toBe(false);
});
it("rejects unknown coverage on both sides", () => {
@@ -45,7 +45,7 @@ describe("isCallerOnlyUnknownCoverage", () => {
unknownCallers,
unknownCallees,
);
expect(isCallerOnlyUnknownCoverage(combined)).toBe(false);
expect(isCallerOnlyIncompleteCoverage(combined)).toBe(false);
});
it("rejects unknown callers beside IDA-capped callees", () => {
@@ -54,11 +54,37 @@ describe("isCallerOnlyUnknownCoverage", () => {
cappedCallees,
);
expect(combined.status).toBe("unknown");
expect(isCallerOnlyUnknownCoverage(combined)).toBe(false);
expect(isCallerOnlyIncompleteCoverage(combined)).toBe(false);
});
it("rejects complete coverage", () => {
const combined = combineFunctionRelationshipCoverage(complete, complete);
expect(isCallerOnlyUnknownCoverage(combined)).toBe(false);
expect(isCallerOnlyIncompleteCoverage(combined)).toBe(false);
});
it("retains caller-only partial coverage without relying on limitation wording", () => {
const combined = combineFunctionRelationshipCoverage(
{ status: "partial", limitations: ["Missing incoming references."] },
complete,
);
expect(combined).toMatchObject({ limited_directions: ["callers"] });
expect(isCallerOnlyIncompleteCoverage(combined)).toBe(true);
});
it("uses explicit directions even when explanation text contains direction labels", () => {
expect(
isCallerOnlyIncompleteCoverage({
status: "unknown",
reason: "Callers: unavailable. Callees: diagnostic text.",
limited_directions: ["callers"],
}),
).toBe(true);
expect(
isCallerOnlyIncompleteCoverage({
status: "unknown",
reason: "Callers: unavailable.",
limited_directions: ["callees"],
}),
).toBe(false);
});
});
+27 -10
View File
@@ -22,6 +22,16 @@ export const combineFunctionRelationshipCoverage = (
),
);
const details = diagnostics.length > 0 ? { diagnostics } : {};
const callersLimited = callers?.status !== "complete";
const calleesLimited = callees?.status !== "complete";
const limited_directions: Extract<
RelationshipCoverage,
{ status: "partial" }
>["limited_directions"] = callersLimited
? calleesLimited
? ["callers", "callees"]
: ["callers"]
: ["callees"];
const reasons = reads.flatMap(({ direction, coverage }) => {
const label =
direction === "callers" ? callerCoveragePrefix : calleeCoveragePrefix;
@@ -43,30 +53,37 @@ export const combineFunctionRelationshipCoverage = (
return {
status: "unknown",
reason: uniqueSorted(reasons).join(" "),
limited_directions,
...details,
};
if (reasons.length > 0)
return {
status: "partial",
limitations: uniqueSorted(reasons),
limited_directions,
...details,
};
return { status: "complete", ...details };
};
/**
* Whether combined relationship coverage is unknown only on the caller side.
* The combine helper above prefixes each direction's reasons, so a reason
* that carries the caller prefix and no callee prefix records observed
* callees with unobservable callers. Forward (callee-edge) searches stay
* exhaustive under that caveat; the caller reason remains a limitation.
* Forward searches depend on callee coverage. Preserve the historical
* unknown-reason encoding when reading observations without explicit directions.
*/
export const isCallerOnlyUnknownCoverage = (
export const isCallerOnlyIncompleteCoverage = (
coverage: RelationshipCoverage,
): boolean =>
coverage.status === "unknown" &&
coverage.reason.startsWith(callerCoveragePrefix) &&
!coverage.reason.includes(calleeCoveragePrefix);
): boolean => {
if (coverage.status === "complete") return false;
if (coverage.limited_directions !== undefined)
return !coverage.limited_directions.some(
(direction) => direction === "callees",
);
return (
coverage.status === "unknown" &&
coverage.reason.startsWith(callerCoveragePrefix) &&
!coverage.reason.includes(calleeCoveragePrefix)
);
};
/** Explain why two resolved call sets cannot establish call-set equality. */
export const functionCallCoverageLimitations = (
@@ -0,0 +1,40 @@
import { describe, expect, it } from "vitest";
import { analyzeJavaScriptStaticSource } from "./javascriptStaticAnalysis.js";
const endpoints = (source: string) =>
analyzeJavaScriptStaticSource(source).endpoints.map(
({ kind, value, mechanism }) => ({ kind, value, mechanism }),
);
describe("server-sent event endpoints", () => {
it("reports EventSource URLs like WebSocket URLs", () => {
expect(
endpoints(`
new WebSocket("wss://chat.example/ws");
new EventSource("/api/chat/stream");
new window.EventSource("https://chat.example/events", { withCredentials: true });
`),
).toEqual([
{
kind: "network",
value: "wss://chat.example/ws",
mechanism: "call:WebSocket",
},
{
kind: "network",
value: "/api/chat/stream",
mechanism: "call:EventSource",
},
{
kind: "network",
value: "https://chat.example/events",
mechanism: "call:window.EventSource",
},
]);
});
it("keeps a dynamic EventSource URL unreported", () => {
expect(endpoints("new EventSource(url);")).toEqual([]);
});
});
@@ -0,0 +1,34 @@
import { describe, expect, it } from "vitest";
import { analyzeJavaScriptStaticSource } from "./javascriptStaticAnalysis.js";
const endpoints = (source: string) =>
analyzeJavaScriptStaticSource(source).endpoints.map(
({ kind, value, mechanism }) => ({ kind, value, mechanism }),
);
describe("beacon endpoints", () => {
it("reports navigator.sendBeacon URLs", () => {
expect(
endpoints(`
navigator.sendBeacon("/api/telemetry", payload);
window.navigator.sendBeacon("https://metrics.example/collect");
`),
).toEqual([
{
kind: "network",
value: "/api/telemetry",
mechanism: "call:navigator.sendBeacon",
},
{
kind: "network",
value: "https://metrics.example/collect",
mechanism: "call:window.navigator.sendBeacon",
},
]);
});
it("keeps a dynamic beacon URL unreported", () => {
expect(endpoints("navigator.sendBeacon(url, payload);")).toEqual([]);
});
});
@@ -554,7 +554,13 @@ export const endpointArgument = (
receiverFact?: JavaScriptOpenReceiverFact,
): string | undefined => {
if (receiverFact === "window") return undefined;
if (name === "fetch" || name.endsWith(".fetch") || name === "WebSocket")
if (
name === "fetch" ||
name.endsWith(".fetch") ||
name === "WebSocket" ||
name === "EventSource" ||
name.endsWith(".EventSource")
)
return stringValue(args[0]);
if (name.endsWith(".open"))
return xhrOpenUrl(name, args[0], args[1], receiverFact);
@@ -574,6 +580,9 @@ export const endpointArgument = (
? undefined
: value;
}
// navigator.sendBeacon(url, data) queues a POST to its first argument.
if (name === "sendBeacon" || name.endsWith(".sendBeacon"))
return stringValue(args[0]);
if (name.endsWith("loadURL")) return stringValue(args[0]);
return undefined;
};
+11
View File
@@ -0,0 +1,11 @@
import { expect, it } from "vitest";
import { parseRecordedTransportUrl } from "./recordedTransportUrl.js";
it("recognizes an empty userinfo marker at the authority start", () => {
expect(parseRecordedTransportUrl("https://@host")).toMatchObject({
origin: "https://host",
path: "/",
credentials: { start: 8, end: 9 },
});
});
+17 -6
View File
@@ -1,24 +1,35 @@
import { z } from "zod";
const diagnostics = z.array(z.string().min(1)).min(1);
const completeCoverage = z.strictObject({ status: z.literal("complete") });
const diagnostics = z.array(z.string().min(1)).min(1).exactOptional();
const completeCoverage = z.strictObject({
status: z.literal("complete"),
diagnostics,
});
const limitedDirections = z.union([
z.tuple([z.literal("callers")]).check(z.minLength(1), z.maxLength(1)),
z.tuple([z.literal("callees")]).check(z.minLength(1), z.maxLength(1)),
z
.tuple([z.literal("callers"), z.literal("callees")])
.check(z.minLength(2), z.maxLength(2)),
]);
const partialCoverage = z.strictObject({
status: z.literal("partial"),
limitations: z.array(z.string().min(1)).min(1),
limited_directions: limitedDirections.exactOptional(),
diagnostics,
});
const unknownCoverage = z.strictObject({
status: z.literal("unknown"),
reason: z.string().min(1),
limited_directions: limitedDirections.exactOptional(),
diagnostics,
});
/** Explicit coverage state for a provider's observed caller/callee set. */
export const relationshipCoverageSchema = z.union([
export const relationshipCoverageSchema = z.discriminatedUnion("status", [
completeCoverage,
completeCoverage.extend({ diagnostics }),
partialCoverage,
partialCoverage.extend({ diagnostics }),
unknownCoverage,
unknownCoverage.extend({ diagnostics }),
]);
export type RelationshipCoverage = z.infer<typeof relationshipCoverageSchema>;
+4 -2
View File
@@ -40,6 +40,8 @@ import {
NATIVE_AOT_PROFILE_CONTRACT_REVISION,
} from "../domain/native/nativeAotPe.js";
const callRelationshipCoverage = "function-body-call-coverage-v5";
/**
* The processor language REA pins for this target, or undefined when Ghidra may
* select from the header. The committed profile and the session readback both
@@ -125,7 +127,7 @@ export const resolveGhidraAnalysisProfile = async (
load_image_observations: "source-mappings-entry-context-v2",
function_body_evidence: "complete-inclusive-ranges-v1",
function_references: "complete-body-and-entry-reference-manager-v2",
call_relationship_coverage: "resolved-call-reference-diagnostics-v1",
call_relationship_coverage: callRelationshipCoverage,
function_dossier_coverage:
"observed-facets-and-call-relationships-v1",
string_inventory_evidence: "defined-data-coverage-v1",
@@ -222,7 +224,7 @@ export const resolveGhidraAnalysisProfile = async (
load_image_observations: "source-mappings-entry-context-v2",
function_body_evidence: "complete-inclusive-ranges-v1",
function_references: "complete-body-and-entry-reference-manager-v2",
call_relationship_coverage: "resolved-call-reference-diagnostics-v1",
call_relationship_coverage: callRelationshipCoverage,
function_dossier_coverage: "observed-facets-and-call-relationships-v1",
string_inventory_evidence: "defined-data-coverage-v1",
location_resolution: "explicit-address-exact-entry-symbol-first-v3",
+6 -1
View File
@@ -15,10 +15,13 @@ import { projectGhidraDoctorInspection } from "./GhidraDoctor.js";
const INSTALL = "/opt/ghidra_12.1.4_PUBLIC";
const installationHost = (
overrides: Partial<GhidraInstallationHost> = {},
platform: NodeJS.Platform = "linux",
): GhidraInstallationHost => ({
readText: () => "application.version=12.1.4\napplication.java.min=21\n",
executable: () => true,
probeJava: () => ({
osArch: "amd64",
osName: platform === "win32" ? "Windows 11" : "Linux",
version: "21.0.11",
major: 21,
home: "/usr/lib/jvm/jdk-21",
@@ -45,7 +48,7 @@ const inspection = (
? {}
: { installDir: options.installDir }),
},
installationHost(overrides),
installationHost(overrides, options.platform ?? "linux"),
);
const doctorHost = (
@@ -129,6 +132,8 @@ describe("Ghidra doctor integration", () => {
{ installDir: INSTALL },
{
probeJava: () => ({
osArch: "amd64",
osName: "Linux",
version: "17.0.12",
major: 17,
home: "/usr/lib/jvm/jdk-17",
+113
View File
@@ -43,7 +43,120 @@ const parseFunctionValueFixture = (
return parsed.ok ? { ok: true as const, value: parsed.value.value } : parsed;
};
describe("Ghidra unresolved call locations", () => {
it("rejects a noncanonical unresolved call site rather than retaining unverifiable location evidence", () => {
expect(
parseGhidraFunctionResult("analyze_function", {
value: {
...functionDossierSchema.parse(ghidraFunctionDossier()),
unresolved_calls: [
{ address: "not-an-address", reason: "No call destination." },
],
},
limitations: [],
relationship_coverage: {
status: "partial",
limitations: ["An unresolved call remains."],
},
}),
).toMatchObject({ ok: false, error: { _tag: "AnalysisOutputError" } });
});
});
describe("Ghidra function-analysis result values", () => {
it("retains unavailable pseudocode instead of treating its empty representation as observed source", () => {
const availability = {
pseudocode: {
available: false,
reason: "Ghidra does not decompile external functions.",
},
comments: { available: true },
};
const parsed = parseGhidraFunctionResult("analyze_function", {
value: {
...functionDossierSchema.parse(ghidraFunctionDossier()),
pseudocode: "",
facet_availability: availability,
},
limitations: [],
});
expect(parsed).toMatchObject({
ok: true,
value: { value: { pseudocode: "", facet_availability: availability } },
});
});
it.each([
{ status: "complete", diagnostics: ["Resolved static references only."] },
{
status: "partial",
limitations: ["Only the first instruction range was examined."],
},
{ status: "unknown", reason: "Relationship inventory was unavailable." },
{
status: "partial",
limitations: ["Missing callers."],
limited_directions: ["callers"],
},
{
status: "unknown",
reason: "Missing callees.",
limited_directions: ["callees"],
},
])("preserves observed $status relationship coverage", (coverage) => {
const parsed = parseGhidraFunctionResult("analyze_function", {
value: ghidraFunctionDossier(),
limitations: [],
relationship_coverage: coverage,
});
expect(parsed).toMatchObject({
ok: true,
value: { relationshipCoverage: coverage },
});
});
it("preserves omitted relationship coverage for legacy observations", () => {
const parsed = parseGhidraFunctionResult("analyze_function", {
value: ghidraFunctionDossier(),
limitations: [],
});
if (!parsed.ok) throw parsed.error;
expect(parsed.value).not.toHaveProperty("relationshipCoverage");
});
});
describe("Ghidra relationship metadata validation", () => {
it.each([
{ status: "partial", limitations: [] },
{ status: "unknown", reason: "" },
{ status: "complete", limitations: ["Truncated."] },
{ status: "partial", limitations: ["Truncated."], limited_directions: [] },
{
status: "partial",
limitations: ["Truncated."],
limited_directions: ["callers", "callers"],
},
{ status: "unknown", reason: "Missing.", limited_directions: ["incoming"] },
{ status: "complete", limited_directions: ["callers"] },
])("rejects malformed relationship coverage %j", (coverage) => {
expect(
parseGhidraFunctionResult("analyze_function", {
value: ghidraFunctionDossier(),
limitations: [],
relationship_coverage: coverage,
}),
).toMatchObject({ ok: false, error: { _tag: "AnalysisOutputError" } });
});
it("rejects relationship metadata on unrelated function operations", () => {
expect(
parseGhidraFunctionResult("procedure_assembly", {
value: "ret",
limitations: [],
relationship_coverage: { status: "complete" },
}),
).toMatchObject({ ok: false, error: { _tag: "AnalysisOutputError" } });
});
it("preserves file-offset resolution in Evidence locations and rejects inexact offsets", () => {
const selection = {
file_offset: 0x210,
+16 -1
View File
@@ -27,6 +27,10 @@ import {
import { nativeApiBoundarySchema } from "../domain/native/nativeApiBoundary.js";
import { nativeValueFlowSchema } from "../domain/native/nativeValueFlow.js";
import type { EvidenceLocation } from "../domain/evidence.js";
import {
relationshipCoverageSchema,
type RelationshipCoverage,
} from "../domain/relationshipCoverage.js";
import { err, ok, type Result } from "../domain/result.js";
import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js";
import {
@@ -307,6 +311,7 @@ const ghidraFunctionDossier = functionDossierSchema
...value.comments.map(({ address }) => address),
...value.callers.map(({ address }) => address),
...value.callees.map(({ address }) => address),
...(value.unresolved_calls ?? []).map(({ address }) => address),
...value.incoming_references.flatMap(referenceAddresses),
...value.outgoing_references.flatMap(referenceAddresses),
...value.referenced_strings.flatMap(({ address, source_address }) => [
@@ -419,6 +424,7 @@ const functionBoundaryObservationOperations: ReadonlySet<GhidraFunctionOperation
const functionObservationEnvelope = z.strictObject({
value: z.unknown(),
limitations: z.array(z.string()),
relationship_coverage: relationshipCoverageSchema.exactOptional(),
selection: z
.strictObject({
file_offset: z.number().int().min(0).max(Number.MAX_SAFE_INTEGER),
@@ -432,6 +438,7 @@ export interface ParsedGhidraFunctionResult {
readonly value: JsonValue;
readonly limitations: readonly string[];
readonly locations?: readonly EvidenceLocation[];
readonly relationshipCoverage?: RelationshipCoverage;
}
/** Require exact, bounded Java-bridge function output before creating Evidence. */
@@ -442,6 +449,7 @@ export const parseGhidraFunctionResult = (
let observation: {
readonly value: unknown;
readonly limitations: readonly string[];
readonly relationship_coverage?: RelationshipCoverage;
readonly selection?: z.infer<
typeof functionObservationEnvelope
>["selection"];
@@ -450,13 +458,17 @@ export const parseGhidraFunctionResult = (
const envelope = functionObservationEnvelope.safeParse(value);
if (
!envelope.success ||
(operation !== "analyze_function" &&
operation !== "procedure_callers" &&
operation !== "procedure_callees" &&
envelope.data.relationship_coverage !== undefined) ||
(operation !== "procedure_pseudo_code" &&
envelope.data.selection !== undefined)
)
return err(
new AnalysisOutputError(
operation,
"Ghidra bridge omitted the function observation envelope",
"Ghidra bridge returned an invalid function observation envelope for this operation",
envelope.success ? undefined : { cause: envelope.error },
),
);
@@ -469,6 +481,9 @@ export const parseGhidraFunctionResult = (
? ok({
value: jsonValueSchema.parse(parsed.data),
limitations: observation.limitations,
...(observation.relationship_coverage === undefined
? {}
: { relationshipCoverage: observation.relationship_coverage }),
...(observation.selection === undefined
? {}
: {
+2
View File
@@ -12,5 +12,7 @@ export const ghidraProfileInstallationHost: GhidraInstallationHost = {
home: "/jdk-21",
bits: 64,
runtime: "jdk",
osArch: "amd64",
osName: "Linux",
}),
};
+238 -3
View File
@@ -1,8 +1,10 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import { spawnSync } from "node:child_process";
import { elf, fat, thinMach } from "../domain/binaryTarget.fixture.js";
import { projectGhidraDoctorInspection } from "./GhidraDoctor.js";
import {
ghidraInstallationDiagnostics,
ghidraJavaEnvironment,
inspectGhidraInstallation,
type GhidraExecutableHeaderProbe,
@@ -16,6 +18,14 @@ import {
ghidraJavaRequirement,
} from "./GhidraInstallationPolicy.js";
// The real Java probe shells out via spawnSync; stub it so one test can run
// the real probeJava against captured `-XshowSettings:properties` output.
// Every other test overrides probeJava, so the stub never fires for them.
vi.mock("node:child_process", async (importOriginal) => {
const original = await importOriginal<typeof import("node:child_process")>();
return { ...original, spawnSync: vi.fn() };
});
const INSTALL = "/opt/ghidra";
const PROPERTIES = `${INSTALL}/Ghidra/application.properties`;
const HEADLESS = `${INSTALL}/support/analyzeHeadless`;
@@ -32,6 +42,8 @@ const headerProbe = (bytes: Buffer): GhidraExecutableHeaderProbe => ({
});
const JAVA: GhidraJavaObservation = {
osArch: "amd64",
osName: "Linux",
version: "21.0.11",
major: SUPPORTED_GHIDRA_JAVA_MAJOR,
home: "/usr/lib/jvm/jdk-21",
@@ -174,6 +186,11 @@ describe("Ghidra installation inspection", () => {
{ environment: {}, installDir: INSTALL, platform, architecture },
host({
executable: (path) => path === HEADLESS || path === decompiler,
probeJava: () => ({
...JAVA,
osArch: architecture === "arm64" ? "aarch64" : "amd64",
osName: platform === "darwin" ? "Mac OS X" : "Linux",
}),
}),
),
).toMatchObject({
@@ -202,6 +219,7 @@ describe("Ghidra installation inspection", () => {
return {
...JAVA,
home: "C:\\Java\\jdk-21",
osName: "Windows 11",
};
},
};
@@ -721,7 +739,10 @@ describe("Ghidra FAT native decompiler admission", () => {
platform: "darwin",
architecture: "arm64",
},
host({ executableHeader: () => headerProbe(bytes) }),
host({
probeJava: () => ({ ...JAVA, osArch: "aarch64", osName: "Mac OS X" }),
executableHeader: () => headerProbe(bytes),
}),
);
expect(result.status).toBe(status);
expect(
@@ -739,7 +760,10 @@ describe("Ghidra FAT native decompiler admission", () => {
platform: "darwin",
architecture: "arm64",
},
host({ executableHeader: () => headerProbe(bytes) }),
host({
probeJava: () => ({ ...JAVA, osArch: "aarch64", osName: "Mac OS X" }),
executableHeader: () => headerProbe(bytes),
}),
);
expect(result.status).toBe("available");
expect(
@@ -788,7 +812,14 @@ describe("Ghidra native decompiler admission", () => {
])("distinguishes $name", ({ options, executableHeader }) => {
const result = inspectGhidraInstallation(
options,
host({ executableHeader }),
host({
probeJava: () => ({
...JAVA,
osArch: options.architecture === "arm64" ? "aarch64" : "amd64",
osName: options.platform === "darwin" ? "Mac OS X" : "Linux",
}),
executableHeader,
}),
);
expect(result.status).toBe("unavailable");
expect(
@@ -849,6 +880,7 @@ describe("Ghidra native decompiler admission", () => {
architecture: "arm64",
},
host({
probeJava: () => ({ ...JAVA, osArch: "aarch64", osName: "Mac OS X" }),
executableHeader: (path: string) =>
path === MAC_DECOMPILER ? headerProbe(bytes) : undefined,
}),
@@ -872,6 +904,7 @@ describe("Ghidra native decompiler admission", () => {
architecture: "arm64",
},
host({
probeJava: () => ({ ...JAVA, osArch: "aarch64", osName: "Mac OS X" }),
executableHeader: (path: string) =>
path === MAC_DECOMPILER ? headerProbe(elf(2, 1, 62)) : undefined,
}),
@@ -889,3 +922,205 @@ describe("Ghidra native decompiler admission", () => {
});
});
});
describe("Ghidra JVM native directory admission", () => {
it("rejects a JDK whose os.arch selects a different native decompiler directory", () => {
const result = inspectGhidraInstallation(
{
environment: {},
installDir: INSTALL,
platform: "darwin",
architecture: "arm64",
},
host({
probeJava: () => ({ ...JAVA, osArch: "x86_64", osName: "Mac OS X" }),
}),
);
expect(result.status).toBe("unavailable");
if (result.status !== "unavailable") return;
expect(
result.checks.find(({ name }) => name === "native_decompiler"),
).toMatchObject({
status: "failed",
code: "executable_missing",
detail: expect.stringContaining(
"JVM reports os.arch=x86_64, so Ghidra selects its native decompiler from os/mac_x86_64",
),
remediation: expect.stringContaining("JAVA_HOME"),
});
});
it("accepts a JDK whose os.arch selects the validated native decompiler directory", () => {
const result = inspectGhidraInstallation(
{
environment: {},
installDir: INSTALL,
platform: "darwin",
architecture: "arm64",
},
host({
probeJava: () => ({ ...JAVA, osArch: "aarch64", osName: "Mac OS X" }),
}),
);
expect(result.status).toBe("available");
});
it("rejects a successful Java probe that omits os.arch", () => {
const { osArch: _observedArchitecture, ...withoutArchitecture } = JAVA;
const result = inspectGhidraInstallation(
{
environment: {},
installDir: INSTALL,
platform: "darwin",
architecture: "arm64",
},
host({ probeJava: () => withoutArchitecture }),
);
expect(result.status).toBe("unavailable");
expect(
result.checks.find(({ name }) => name === "native_decompiler"),
).toMatchObject({
status: "failed",
detail: expect.stringContaining("did not report os.arch"),
});
expect(projectGhidraDoctorInspection(result).available).toBe(false);
});
it.each(["riscv64", "sparcv9", ""])(
"refuses an unrecognized JVM os.arch (%s) instead of assuming alignment",
(osArch) => {
const result = inspectGhidraInstallation(
{
environment: {},
installDir: INSTALL,
platform: "darwin",
architecture: "arm64",
},
host({
probeJava: () => ({ ...JAVA, osArch }),
}),
);
expect(result.status).toBe("unavailable");
if (result.status !== "unavailable") return;
expect(
result.checks.find(({ name }) => name === "native_decompiler"),
).toMatchObject({
status: "failed",
detail: expect.stringContaining("unrecognized os.arch="),
});
},
);
it("rejects a JDK whose os.name selects a different host OS", () => {
const result = inspectGhidraInstallation(
{
environment: {},
installDir: INSTALL,
platform: "darwin",
architecture: "arm64",
},
host({
probeJava: () => ({
...JAVA,
osArch: "aarch64",
osName: "Windows 11",
}),
}),
);
expect(result.status).toBe("unavailable");
if (result.status !== "unavailable") return;
expect(
result.checks.find(({ name }) => name === "native_decompiler"),
).toMatchObject({
status: "failed",
code: "executable_missing",
detail: expect.stringContaining("JVM reports os.name=Windows 11"),
remediation: expect.stringContaining("JAVA_HOME"),
});
});
});
describe("Ghidra JVM OS admission", () => {
it("rejects a successful Java probe that omits os.name", () => {
const { osName: _observedOsName, ...withoutOsName } = JAVA;
const result = inspectGhidraInstallation(
{
environment: {},
installDir: INSTALL,
platform: "linux",
architecture: "x64",
},
host({ probeJava: () => withoutOsName }),
);
expect(result.status).toBe("unavailable");
expect(
result.checks.find(({ name }) => name === "native_decompiler"),
).toMatchObject({
status: "failed",
detail: expect.stringContaining("did not report os.name"),
});
expect(projectGhidraDoctorInspection(result).available).toBe(false);
});
it.each(["SunOS", "FreeBSD", ""])(
"refuses an unrecognized JVM os.name (%s) instead of assuming alignment",
(osName) => {
const result = inspectGhidraInstallation(
{
environment: {},
installDir: INSTALL,
platform: "linux",
architecture: "x64",
},
host({
probeJava: () => ({ ...JAVA, osName }),
}),
);
expect(result.status).toBe("unavailable");
if (result.status !== "unavailable") return;
expect(
result.checks.find(({ name }) => name === "native_decompiler"),
).toMatchObject({
status: "failed",
detail: expect.stringContaining("os.name="),
});
},
);
});
describe("Ghidra Java probe os.arch extraction", () => {
it("extracts os.arch through the real Java probe into the installation diagnostics", () => {
vi.mocked(spawnSync).mockReturnValue({
status: 0,
stdout: "",
stderr: [
"Property settings:",
" file.encoding = UTF-8",
" java.home = /usr/lib/jvm/jdk-21",
" java.version = 21.0.11",
" java.vm.name = OpenJDK 64-Bit Server VM",
" os.arch = x86_64",
" os.name = Linux",
" sun.arch.data.model = 64",
"",
].join("\n"),
} as never);
try {
// No host override: the real system host runs the real probeJava
// against the stubbed spawnSync.
const inspection = inspectGhidraInstallation({
environment: {},
installDir: INSTALL,
javaHome: "/usr/lib/jvm/jdk-21",
platform: "linux",
architecture: "x64",
});
expect(ghidraInstallationDiagnostics(inspection)).toMatchObject({
java_os_arch: "x86_64",
java_os_name: "Linux",
});
} finally {
vi.mocked(spawnSync).mockReset();
}
});
});
+126
View File
@@ -203,6 +203,17 @@ interface GhidraInstallationObservation {
readonly javaCommand: string;
/** Observed java.vm.name; omitted when the probe cannot establish VM identity. */
readonly javaVmName?: string;
/**
* Observed JVM os.arch; omitted when the probe cannot establish it. Ghidra
* selects its native decompiler directory from this value.
*/
readonly javaOsArch?: string;
/**
* Observed JVM os.name; omitted when the probe cannot establish it. Ghidra
* selects its native decompiler OS directory from the JVM OS/architecture
* pair, so a non-host value selects an unvalidated directory.
*/
readonly javaOsName?: string;
readonly checks: readonly GhidraInstallationCheck[];
}
@@ -244,6 +255,17 @@ export interface GhidraJavaObservation {
readonly bits: number;
readonly runtime: "jdk" | "jre";
readonly vmName?: string;
/**
* Observed JVM os.arch property, which Ghidra uses to select its native
* decompiler directory. Omitted when the probe cannot establish it.
*/
readonly osArch?: string;
/**
* Observed JVM os.name property, which Ghidra uses with os.arch to select
* its native decompiler OS directory. Omitted when the probe cannot
* establish it.
*/
readonly osName?: string;
}
/** A bounded executable-header probe: a small prefix plus the whole file size. */
@@ -324,6 +346,12 @@ export const inspectGhidraInstallation = (
...(coordinates.java?.vmName === undefined
? {}
: { javaVmName: coordinates.java.vmName }),
...(coordinates.java?.osArch === undefined
? {}
: { javaOsArch: coordinates.java.osArch }),
...(coordinates.java?.osName === undefined
? {}
: { javaOsName: coordinates.java.osName }),
checks,
};
if (failed !== undefined)
@@ -555,6 +583,8 @@ export const ghidraInstallationDiagnostics = (
java_home: inspection.javaHome,
java_version: inspection.javaVersion,
java_vm_name: inspection.javaVmName ?? null,
java_os_arch: inspection.javaOsArch ?? null,
java_os_name: inspection.javaOsName ?? null,
platform: inspection.platform,
architecture: inspection.architecture,
checks: inspection.checks.map((check) => ({
@@ -632,6 +662,61 @@ const nativeDecompilerCheck = ({
architecture,
host.executableHeader?.(path),
);
// Ghidra resolves its native decompiler directory from the JVM's os.arch,
// not the host process architecture. A JDK running under emulation (for
// example x86_64 under Rosetta on Apple silicon) selects a directory that
// was never validated, so every analysis fails after a passing doctor.
// Ghidra resolves the OS directory from the JVM os.name/os.arch pair, so a
// JVM reporting a non-host os.name (for example through -Dos.name=Windows)
// selects a different OS directory even when os.arch matches.
const osArch = coordinates.java?.osArch;
const osName = coordinates.java?.osName;
const selectedDirectory = jvmOsArchNativeDirectory(platform, osArch);
const validatedDirectory = NATIVE_PLATFORMS[platform]?.[architecture];
// Without a JVM observation the java check reports the missing runtime;
// the decompiler directory cannot be selected, so this check defers to it.
if (coordinates.java !== undefined) {
if (selectedDirectory === undefined)
return installationCheck({
name: "native_decompiler",
passed: false,
code: "executable_missing",
detail:
osArch === undefined
? "The Java probe did not report os.arch; the JVM-selected native decompiler directory is unknown and cannot be matched to the validated host directory"
: `JVM reports unrecognized os.arch=${osArch}; its selected native decompiler directory is unknown and cannot be matched to the validated host directory`,
remediation: `Point JAVA_HOME at a JDK whose reported architecture maps to the validated Ghidra decompiler (${validatedDirectory ?? "unsupported host"}). REA does not build or install native tools.`,
});
if (
validatedDirectory !== undefined &&
selectedDirectory !== validatedDirectory &&
path !== null
)
return installationCheck({
name: "native_decompiler",
passed: false,
code: "executable_missing",
detail: `JVM reports os.arch=${osArch}, so Ghidra selects its native decompiler from os/${selectedDirectory}, but the validated decompiler targets ${validatedDirectory} (host ${platform}/${architecture})`,
remediation: `Point JAVA_HOME at a JDK whose architecture matches the built decompiler (${validatedDirectory}). REA does not build or install native tools.`,
});
if (osName === undefined)
return installationCheck({
name: "native_decompiler",
passed: false,
code: "executable_missing",
detail:
"The Java probe did not report os.name; the JVM-selected native decompiler OS directory is unknown and cannot be matched to the validated host platform",
remediation: `Point JAVA_HOME at a JDK whose reported operating system matches the host platform (${platform}). REA does not build or install native tools.`,
});
if (!jvmOsNameMatchesPlatform(platform, osName) && path !== null)
return installationCheck({
name: "native_decompiler",
passed: false,
code: "executable_missing",
detail: `JVM reports os.name=${osName}, which does not match the validated host platform ${platform} (host ${platform}/${architecture})`,
remediation: `Point JAVA_HOME at a JDK whose reported operating system matches the host platform (${platform}). REA does not build or install native tools.`,
});
}
return installationCheck({
name: "native_decompiler",
passed: path !== null && compatibility?.status !== "incompatible",
@@ -646,6 +731,43 @@ const nativeDecompilerCheck = ({
});
};
/**
* Whether a JVM os.name value selects the host platform's native OS directory.
* Unknown values never align with the validated host directory; admission
* must not treat them as matching.
*/
const jvmOsNameMatchesPlatform = (
platform: NodeJS.Platform,
osName: string,
): boolean => {
const normalized = osName.toLowerCase();
if (platform === "win32") return normalized.startsWith("windows");
if (platform === "darwin")
return normalized.startsWith("mac") || normalized.startsWith("darwin");
if (platform === "linux") return normalized.startsWith("linux");
return false;
};
/**
* Map a JVM os.arch value to the Ghidra native directory that JVM selects.
* Unknown values stay unknown; admission must not treat them as aligned with
* the validated host directory.
*/
const jvmOsArchNativeDirectory = (
platform: NodeJS.Platform,
osArch: string | undefined,
): string | undefined => {
if (osArch === undefined) return undefined;
const normalized = osArch.toLowerCase();
const node: NodeJS.Architecture | undefined =
normalized === "x86_64" || normalized === "amd64" || normalized === "x64"
? "x64"
: normalized === "aarch64" || normalized === "arm64"
? "arm64"
: undefined;
return node === undefined ? undefined : NATIVE_PLATFORMS[platform]?.[node];
};
const versionCheck = (
properties: GhidraApplicationProperties | undefined,
): GhidraInstallationCheck => {
@@ -781,6 +903,8 @@ const systemGhidraInstallationHost = (): GhidraInstallationHost => ({
const version = javaProperty(output, "java.version");
const home = javaProperty(output, "java.home");
const vmName = javaProperty(output, "java.vm.name");
const osArch = javaProperty(output, "os.arch");
const osName = javaProperty(output, "os.name");
const bits = Number.parseInt(
javaProperty(output, "sun.arch.data.model") ?? "0",
10,
@@ -791,6 +915,8 @@ const systemGhidraInstallationHost = (): GhidraInstallationHost => ({
major: javaMajor(version),
home,
...(vmName === null ? {} : { vmName }),
...(osArch === null ? {} : { osArch }),
...(osName === null ? {} : { osName }),
bits,
runtime: executablePath(
(process.platform === "win32" ? win32 : posix).join(
+62 -2
View File
@@ -2,11 +2,43 @@ import { snapshotEnvironment } from "../process/snapshotEnvironment.js";
import { dirname, join } from "node:path";
import { execFileOutput } from "../process/ExecFileOutput.js";
/** JVM option environments remove quotes, including quotes inside an option. */
const jvmEnvironmentArguments = (value: string): readonly string[] => {
const arguments_: string[] = [];
let argument = "";
let quote: "'" | '"' | undefined;
for (const character of value) {
if (quote !== undefined) {
if (character === quote) quote = undefined;
else argument += character;
} else if (character === "'" || character === '"') quote = character;
else if (/\s/u.test(character)) {
if (argument !== "") arguments_.push(argument);
argument = "";
} else argument += character;
}
if (quote !== undefined)
throw new Error(
"Invalid quoted Ghidra LaunchSupport JVM environment option",
);
if (argument !== "") arguments_.push(argument);
return arguments_;
};
const rejectJvmPlatformOverrides = (arguments_: readonly string[]): void => {
if (
arguments_.some((argument) => /^-Dos\.(?:arch|name)(?:=|$)/u.test(argument))
)
throw new Error(
"Ghidra LaunchSupport may not override os.arch or os.name; the native decompiler was validated for the inspected JVM platform.",
);
};
/** Launch coordinates obtained from Ghidra's own release configuration helper. */
export const ghidraJavaLaunch = async (options: {
readonly platform: NodeJS.Platform;
readonly analyzeHeadlessPath: string;
readonly javaHome: string;
readonly javaHome?: string;
readonly javaVmName?: string;
readonly homeRoot: string;
readonly tempRoot: string;
@@ -21,7 +53,17 @@ export const ghidraJavaLaunch = async (options: {
}> => {
const supportRoot = dirname(options.analyzeHeadlessPath);
const installRoot = dirname(supportRoot);
const command = join(options.javaHome, "bin", "java");
const selectedJavaHome = options.javaHome ?? options.environment.JAVA_HOME;
const command =
selectedJavaHome === undefined
? options.platform === "win32"
? "java.exe"
: "java"
: join(
selectedJavaHome,
"bin",
options.platform === "win32" ? "java.exe" : "java",
);
const isolatedProperties = [
`-Duser.home=${options.homeRoot}`,
`-Djava.io.tmpdir=${options.tempRoot}`,
@@ -48,6 +90,9 @@ export const ghidraJavaLaunch = async (options: {
[...supportArguments, "-envvars"],
{
env: environment,
// Wait for the helper's exit on abort so a cancelled or timed-out
// startup never leaves the probe running without an owning process.
stopSignal: "SIGTERM",
...(options.signal === undefined ? {} : { signal: options.signal }),
},
);
@@ -65,11 +110,25 @@ export const ghidraJavaLaunch = async (options: {
if ((environment[key] ?? "") === "")
environment[key] = line.slice(separator + 1);
}
// The merged environment becomes the analysis JVM's environment, and the
// JVM applies these options variables automatically. An injected
// -Dos.arch/-Dos.name there selects an unvalidated native directory without
// appearing in vmArguments, so reject the effective values as well.
for (const key of [
"JAVA_TOOL_OPTIONS",
"_JAVA_OPTIONS",
"JDK_JAVA_OPTIONS",
] as const) {
rejectJvmPlatformOverrides(jvmEnvironmentArguments(environment[key] ?? ""));
}
const configuredArguments = await execFileOutput(
command,
[...supportArguments, "-vmargs"],
{
env: environment,
// Wait for the helper's exit on abort so a cancelled or timed-out
// startup never leaves the probe running without an owning process.
stopSignal: "SIGTERM",
...(options.signal === undefined ? {} : { signal: options.signal }),
},
);
@@ -83,6 +142,7 @@ export const ghidraJavaLaunch = async (options: {
)
)
throw new Error("Invalid Ghidra LaunchSupport JVM argument");
rejectJvmPlatformOverrides(vmArguments);
return {
command,
environment,
+41 -36
View File
@@ -172,6 +172,8 @@ export interface GhidraHeadlessLauncherOptions {
readonly mipsSoftFloat?: true;
/** Spawn seam for provider-boundary lifecycle tests. */
readonly spawnProcess?: typeof spawnOwnedProviderProcess;
/** LaunchSupport probe seam for Java-free launcher fixtures. */
readonly javaLaunch?: typeof ghidraJavaLaunch;
}
/** Launch Ghidra without copying scripts into or modifying its installation. */
@@ -303,43 +305,46 @@ export class GhidraHeadlessLauncher implements GhidraLauncher {
throw new GhidraLaunchError(
"POSIX Ghidra requires its inspected JDK home",
);
const inspectedJavaLaunch = await (
this.options.javaLaunch ?? ghidraJavaLaunch
)({
platform,
analyzeHeadlessPath: this.options.analyzeHeadlessPath,
...(this.options.javaHome === undefined
? {}
: { javaHome: this.options.javaHome }),
...(this.options.javaVmName === undefined
? {}
: { javaVmName: this.options.javaVmName }),
homeRoot: paths.homeRoot,
tempRoot: paths.tempRoot,
arguments: headlessArguments,
environment,
onStderr: (mode, stderr) => {
if (stderr === "") return;
const bytes = Buffer.byteLength(stderr);
const redacted = redactCapturedTransportCredential(
{ text: stderr, bytes, observedBytes: bytes },
session.token,
"[REDACTED]",
);
const bounded = boundedUtf8Text(
redacted,
GHIDRA_LAUNCH_COMMAND_OUTPUT_BYTES,
);
launchSupport.push({
mode,
stderr: bounded.text,
stderr_bytes: bytes,
stderr_truncated: bounded.truncated,
});
},
...(options.signal === undefined ? {} : { signal: options.signal }),
});
const command =
platform !== "win32" && this.options.javaHome !== undefined
? await ghidraJavaLaunch({
platform,
analyzeHeadlessPath: this.options.analyzeHeadlessPath,
javaHome: this.options.javaHome,
...(this.options.javaVmName === undefined
? {}
: { javaVmName: this.options.javaVmName }),
homeRoot: paths.homeRoot,
tempRoot: paths.tempRoot,
arguments: headlessArguments,
environment,
onStderr: (mode, stderr) => {
if (stderr === "") return;
const bytes = Buffer.byteLength(stderr);
const redacted = redactCapturedTransportCredential(
{ text: stderr, bytes, observedBytes: bytes },
session.token,
"[REDACTED]",
);
const bounded = boundedUtf8Text(
redacted,
GHIDRA_LAUNCH_COMMAND_OUTPUT_BYTES,
);
launchSupport.push({
mode,
stderr: bounded.text,
stderr_bytes: bytes,
stderr_truncated: bounded.truncated,
});
},
...(options.signal === undefined
? {}
: { signal: options.signal }),
})
: { ...scriptCommand, environment };
platform === "win32"
? { ...scriptCommand, environment }
: inspectedJavaLaunch;
started = await (this.options.spawnProcess ?? spawnOwnedProviderProcess)({
command: command.command,
arguments: command.arguments,
+28 -11
View File
@@ -37,12 +37,16 @@ import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import { ProviderStartupDeadline } from "../process/ProviderDeadline.js";
const INSTALL = "/opt/ghidra_12.1.4_PUBLIC";
const installationHost = (): GhidraInstallationHost => ({
platform: "linux",
const installationHost = (
platform: NodeJS.Platform = "linux",
): GhidraInstallationHost => ({
platform,
architecture: "x64",
readText: () => "application.version=12.1.4\napplication.java.min=21\n",
executable: () => true,
probeJava: () => ({
osArch: "amd64",
osName: platform === "win32" ? "Windows 11" : "Linux",
version: "21.0.11",
major: 21,
home: "/usr/lib/jvm/jdk-21",
@@ -102,7 +106,7 @@ describe("Ghidra provider", () => {
},
},
});
const windows = provider({ ...installationHost(), platform: "win32" });
const windows = provider(installationHost("win32"));
expect(windows.inspectTargetSupport(target).status).toBe("unsupported");
});
it("discovers the exact installation once without launching Ghidra", () => {
@@ -112,6 +116,8 @@ describe("Ghidra provider", () => {
probeJava: () => {
probeCount += 1;
return {
osArch: "amd64",
osName: "Linux",
version: "21.0.11",
major: 21,
home: "/usr/lib/jvm/jdk-21",
@@ -193,7 +199,7 @@ describe("Ghidra provider", () => {
describe("Ghidra raw-binary profile", () => {
it("admits raw 6502 on Windows and binds its 16-bit language without mutation authority", async () => {
const ghidra = provider({ ...installationHost(), platform: "win32" });
const ghidra = provider(installationHost("win32"));
const target: BinaryTarget = {
path: "/tmp/6502.bin",
sha256: "a".repeat(64),
@@ -260,7 +266,7 @@ describe("Ghidra raw-binary profile", () => {
expect(shifted.value.profile?.digest).not.toBe(
resolved.value.profile.digest,
);
const windows = provider({ ...installationHost(), platform: "win32" });
const windows = provider(installationHost("win32"));
expect(windows.inspectTargetSupport(target).status).toBe("unsupported");
});
});
@@ -302,7 +308,7 @@ describe("Ghidra Mach-O slice support", () => {
describe("Ghidra platform support", () => {
it("commits the PE role and managed classification to profile identity", async () => {
const ghidra = provider({ ...installationHost(), platform: "win32" });
const ghidra = provider(installationHost("win32"));
const application = peTarget("x86_64");
const library = {
...application,
@@ -343,7 +349,7 @@ describe("Ghidra platform support", () => {
});
it("keeps Windows annotation mutation unavailable independently of native controls", () => {
const ghidra = provider({ ...installationHost(), platform: "win32" });
const ghidra = provider(installationHost("win32"));
expect(
ghidra
.capabilities()
@@ -363,7 +369,7 @@ describe("Ghidra platform support", () => {
reason: WINDOWS_NATIVE_AUTHORITY_UNAVAILABLE_REASON,
});
onTestFinished(() => native.mockRestore());
const ghidra = provider({ ...installationHost(), platform: "win32" });
const ghidra = provider(installationHost("win32"));
const nativeApplication = peTarget("x86_64");
for (const architecture of ["x86", "x86_64"] as const)
@@ -454,7 +460,7 @@ describe("Ghidra Windows PE fixture admission", () => {
await promisify(execFile)(process.execPath, [
join(root, "scripts/create-ghidra-windows-fixture.mjs"),
]);
const ghidra = provider({ ...installationHost(), platform: "win32" });
const ghidra = provider(installationHost("win32"));
for (const architecture of ["x86", "x86_64"] as const) {
for (const dll of [false, true]) {
const path = join(
@@ -627,7 +633,7 @@ describe("Ghidra client projection", () => {
expect(resolved.value.profile.parameters).toMatchObject({
language_id: "auto-from-header",
compiler_spec_id: "auto-default",
call_relationship_coverage: "resolved-call-reference-diagnostics-v1",
call_relationship_coverage: "function-body-call-coverage-v5",
function_dossier_coverage: "observed-facets-and-call-relationships-v1",
});
@@ -864,7 +870,16 @@ describe("Ghidra relationship coverage", () => {
Promise.resolve(
ok(
operation === "procedure_callees"
? { value: ["0x2000"], limitations: [] }
? {
value: ["0x2000"],
limitations: [],
relationship_coverage: {
status: "complete",
diagnostics: [
"Only resolved Ghidra call references are returned; unresolved computed or indirect calls remain unknown, while function classifications distinguish thunks and externals.",
],
},
}
: { value: [], limitations: [] },
),
),
@@ -1818,6 +1833,8 @@ describe("compatible Ghidra builds", () => {
readText: () =>
"application.version=12.1.2\napplication.java.min=21\napplication.java.max=\n",
probeJava: () => ({
osArch: "amd64",
osName: "Linux",
version: "27",
major: 27,
home: "/usr/lib/jvm/java-27-openjdk",
-17
View File
@@ -2,7 +2,6 @@ import type {
CapabilityDescriptor,
ProviderIdentity,
} from "../application/AnalysisProvider.js";
import type { RelationshipCoverage } from "../domain/relationshipCoverage.js";
import {
hasWindowsNativeAuthority,
windowsNativeAuthorityUnavailableReason,
@@ -44,22 +43,6 @@ const stringInventoryLimitation =
const searchLimitation =
"Regex mode uses Java Pattern semantics. Matcher or compiler stack exhaustion returns a resource constraint without closing the session; literal mode avoids regex recursion and returns complete matching values.";
const resolvedCallReferenceLimitation =
"Only resolved Ghidra call references are returned; unresolved computed or indirect calls remain unknown, while function classifications distinguish thunks and externals.";
/** Relationship-only caveats belong to each observed caller/callee result. */
export const relationshipCoverageFor = (
operation: string,
): RelationshipCoverage | undefined =>
operation === "procedure_callers" ||
operation === "procedure_callees" ||
operation === "analyze_function"
? {
status: "complete",
diagnostics: [resolvedCallReferenceLimitation],
}
: undefined;
/** Limitation text for one admitted operation, including the common base. */
export const limitationsFor = (operation: string): readonly string[] => {
const common = [
+5 -2
View File
@@ -34,6 +34,7 @@ import {
} from "../domain/analysisErrorCore.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import type { EvidenceLocation } from "../domain/evidence.js";
import type { RelationshipCoverage } from "../domain/relationshipCoverage.js";
import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js";
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
import { err, ok, type Result } from "../domain/result.js";
@@ -86,7 +87,6 @@ import {
healthLimitations,
windowsP0Limitations,
limitationsFor,
relationshipCoverageFor,
} from "./GhidraProviderCapabilities.js";
import type { GhidraSessionError } from "./GhidraSessionError.js";
import {
@@ -733,8 +733,8 @@ export const createGhidraProviderClient = (input: {
rawResult: JsonValue,
observationLimitations: readonly string[] = [],
locations: readonly EvidenceLocation[] = [],
relationshipCoverage?: RelationshipCoverage,
) => {
const relationshipCoverage = relationshipCoverageFor(operation);
return createAnalysisExecution(normalized, committedProfile.provider, {
rawResult,
locations,
@@ -838,12 +838,14 @@ export const createGhidraProviderClient = (input: {
let normalized: JsonValue;
let observationLimitations: readonly string[] = [];
let observationLocations: readonly EvidenceLocation[] = [];
let relationshipCoverage: RelationshipCoverage | undefined;
if (isGhidraFunctionOperation(operation)) {
const result = parseGhidraFunctionResult(operation, called.value);
if (!result.ok) return result;
normalized = result.value.value;
observationLimitations = result.value.limitations;
observationLocations = result.value.locations ?? [];
relationshipCoverage = result.value.relationshipCoverage;
} else {
const result = parseGhidraInventoryResult(operation, called.value);
if (!result.ok) return result;
@@ -939,6 +941,7 @@ export const createGhidraProviderClient = (input: {
called.value,
observationLimitations,
observationLocations,
relationshipCoverage,
),
);
},
+123 -37
View File
@@ -11,6 +11,7 @@ import { silentLogger } from "../logger.js";
import { GhidraProvider } from "./GhidraProvider.js";
import { GhidraSessionError } from "./GhidraSessionError.js";
import { GHIDRA_SESSION_CAPABILITIES } from "./GhidraSessionValues.js";
import type { RelationshipCoverage } from "../domain/relationshipCoverage.js";
const TARGET: BinaryTarget = {
path: "/tmp/ghidra-cache-fixture",
@@ -23,7 +24,7 @@ const TARGET: BinaryTarget = {
// Exercise the real provider descriptors, profile, input/output projection and
// session cache; only the installed Ghidra process is replaced at its seam.
const fixture = (version = "12.1.4") => {
const fixture = (version = "12.1.4", coverage?: RelationshipCoverage) => {
const config = parseConfig({ GHIDRA_INSTALL_DIR: "/opt/ghidra" });
if (!config.ok) throw config.error;
const calls: string[] = [];
@@ -43,6 +44,8 @@ const fixture = (version = "12.1.4") => {
`application.version=${version}\napplication.java.min=21\n`,
executable: () => true,
probeJava: () => ({
osArch: "amd64",
osName: "Linux",
version: "21.0.11",
major: 21,
home: "/usr/lib/jvm/jdk-21",
@@ -111,7 +114,13 @@ const fixture = (version = "12.1.4") => {
return ok(
operation === "procedure_pseudo_code" ||
operation === "analyze_function"
? { value, limitations: [] }
? {
value,
limitations: [],
...(operation === "analyze_function" && coverage !== undefined
? { relationship_coverage: coverage }
: {}),
}
: value,
);
},
@@ -131,42 +140,119 @@ const open = async (session: ReturnType<typeof fixture>["session"]) => {
expect((await session.openResolvedTarget(preview.value)).ok).toBe(true);
};
describe("Ghidra snapshot cache", () => {
it("refuses snapshots whose profile predates typed incoming references", async () => {
const initial = fixture();
await open(initial.session);
const snapshot = initial.session.exportAnalysisSnapshot();
if (!snapshot.ok) throw snapshot.error;
await initial.session.close();
const profile = snapshot.value.binding.analysis_profile;
expect(profile.parameters.instruction_references).toBe(
"exact-address-incoming-v1",
);
const legacy = {
...snapshot.value,
binding: {
...snapshot.value.binding,
analysis_profile: createAnalysisProfile(
profile.provider,
Object.fromEntries(
Object.entries(profile.parameters).filter(
([key]) => key !== "instruction_references",
),
),
),
},
};
const replay = fixture();
const imported = replay.session.importAnalysisSnapshot(legacy);
if (!imported.ok) throw imported.error;
expect(await replay.session.previewTarget(TARGET)).toMatchObject({
ok: false,
error: { _tag: "EvidenceIntegrityError" },
});
expect(replay.calls).toEqual([]);
await replay.session.close();
});
describe("Ghidra relationship coverage replay", () => {
it.each([
undefined,
{ status: "complete", diagnostics: ["Resolved static references only."] },
{ status: "partial", limitations: ["Instruction coverage was truncated."] },
{ status: "unknown", reason: "Relationship inventory was unavailable." },
{
status: "partial",
limitations: ["Missing callers."],
limited_directions: ["callers"],
},
] satisfies (RelationshipCoverage | undefined)[])(
"preserves relationship coverage %j through live execution, cache and snapshot replay",
async (coverage) => {
const initial = fixture("12.1.4", coverage);
await open(initial.session);
const parameters = { procedure: "0x1000" };
for (let i = 0; i < 2; i++) {
const execution = await initial.session.execute(
"analyze_function",
parameters,
);
if (!execution.ok) throw execution.error;
expect(execution.value.relationshipCoverage).toEqual(coverage);
}
expect(initial.calls).toEqual(["analyze_function"]);
const snapshot = initial.session.exportAnalysisSnapshot();
if (!snapshot.ok) throw snapshot.error;
await initial.session.close();
const replay = fixture();
expect(replay.session.importAnalysisSnapshot(snapshot.value).ok).toBe(
true,
);
await open(replay.session);
const execution = await replay.session.execute(
"analyze_function",
parameters,
);
if (!execution.ok) throw execution.error;
expect(execution.value.relationshipCoverage ?? undefined).toEqual(
coverage,
);
expect(replay.calls).toEqual([]);
await replay.session.close();
},
);
});
describe("Ghidra snapshot profile admission", () => {
it.each([
{
parameter: "instruction_references",
current: "exact-address-incoming-v1",
legacy: undefined,
},
{
parameter: "call_relationship_coverage",
current: "function-body-call-coverage-v5",
legacy: "resolved-call-reference-diagnostics-v1",
},
{
parameter: "call_relationship_coverage",
current: "function-body-call-coverage-v5",
legacy: "decoded-call-target-coverage-v2",
},
{
parameter: "call_relationship_coverage",
current: "function-body-call-coverage-v5",
legacy: "decoded-call-target-coverage-v3",
},
{
parameter: "call_relationship_coverage",
current: "function-body-call-coverage-v5",
legacy: "function-body-call-coverage-v4",
},
])(
"refuses snapshots with outdated $parameter semantics",
async ({ parameter, current, legacy: prior }) => {
const initial = fixture();
await open(initial.session);
const snapshot = initial.session.exportAnalysisSnapshot();
if (!snapshot.ok) throw snapshot.error;
await initial.session.close();
const profile = snapshot.value.binding.analysis_profile;
expect(profile.parameters[parameter]).toBe(current);
const legacy = {
...snapshot.value,
binding: {
...snapshot.value.binding,
analysis_profile: createAnalysisProfile(profile.provider, {
...Object.fromEntries(
Object.entries(profile.parameters).filter(
([key]) => key !== parameter,
),
),
...(prior === undefined ? {} : { [parameter]: prior }),
}),
},
};
const replay = fixture();
const imported = replay.session.importAnalysisSnapshot(legacy);
if (!imported.ok) throw imported.error;
expect(await replay.session.previewTarget(TARGET)).toMatchObject({
ok: false,
error: { _tag: "EvidenceIntegrityError" },
});
expect(replay.calls).toEqual([]);
await replay.session.close();
},
);
});
describe("Ghidra snapshot cache", () => {
it.each([
{ operation: "analyze_function", parameters: { procedure: "0x1000" } },
{
@@ -1,4 +1,4 @@
import { writeFile } from "node:fs/promises";
import { chmod, mkdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { expect } from "vitest";
@@ -68,6 +68,130 @@ cliTest(
},
);
const isWindows = process.platform === "win32";
const errorBody = (result: {
readonly content: readonly { readonly type: string }[];
}) => {
const content = result.content.find((item) => item.type === "text");
if (content?.type !== "text" || !("text" in content))
throw new Error("Missing error response");
return JSON.parse(String(content.text));
};
const createElectronApplication = async (root: string) => {
const application = join(root, "app");
await mkdir(application);
await writeFile(
join(application, "package.json"),
JSON.stringify({ name: "fixture", main: "main.js" }),
);
await writeFile(join(application, "main.js"), "");
return application;
};
cliTest.skipIf(isWindows)(
"rejects a non-executable Electron executable_path before launch through CLI and MCP",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-electron-not-executable-");
const application = await createElectronApplication(root);
const executable = join(root, "not-an-executable.txt");
await writeFile(executable, "plain text");
await chmod(executable, 0o644);
const request = {
executable_path: executable,
application_path: application,
actions: [],
};
const requestPath = join(root, "input.json");
await writeFile(requestPath, JSON.stringify(request));
const response = await cli.run({
arguments: ["capture-electron-scenario", requestPath, "--format", "json"],
environment: { REA_LOG_LEVEL: "silent" },
});
expect(response.exitCode).toBe(1);
expect(response.json).toMatchObject({
code: "invalid_request",
details: {
issues: [{ path: ["executable_path"], reason: "invalid_value" }],
},
});
const { call } = await connectLocalToolsMcp();
const result = await call("capture_electron_scenario", request);
expect(result.isError).toBe(true);
expect(errorBody(result)).toMatchObject({
error: {
code: "invalid_request",
details: {
issues: [{ path: ["executable_path"], reason: "invalid_value" }],
},
},
});
const session = await call("binary_session", {});
expect(session.isError, JSON.stringify(session)).not.toBe(true);
},
);
cliTest.skipIf(isWindows)(
"returns a typed failure when an executable exits before Electron starts and keeps the MCP server and its Evidence",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-electron-early-exit-");
const application = await createElectronApplication(root);
const executable = join(root, "exits-immediately");
await writeFile(executable, "#!/bin/sh\nexit 0\n");
await chmod(executable, 0o755);
const request = {
executable_path: executable,
application_path: application,
actions: [],
};
const requestPath = join(root, "input.json");
await writeFile(requestPath, JSON.stringify(request));
const response = await cli.run({
arguments: ["capture-electron-scenario", requestPath, "--format", "json"],
environment: { REA_LOG_LEVEL: "silent" },
});
expect(response.exitCode).toBe(1);
expect(response.json).toMatchObject({
code: "execution_failure",
details: { operation: "capture_electron_scenario" },
});
expect(response.stdout).toContain("Process failed to launch");
const { call } = await connectLocalToolsMcp();
const analysis = await call("analyze_javascript_application", {
input_path: application,
});
expect(analysis.isError, JSON.stringify(analysis)).not.toBe(true);
const retained = async () => {
const bundle = await call("get_evidence_bundle", {});
expect(bundle.isError, JSON.stringify(bundle)).not.toBe(true);
const structured = bundle.structuredContent as {
result: { records: { evidence_id: string }[] };
};
return structured.result.records.map((record) => record.evidence_id);
};
const before = await retained();
expect(before).toHaveLength(1);
const result = await call("capture_electron_scenario", request);
expect(result.isError).toBe(true);
expect(errorBody(result)).toMatchObject({
error: {
code: "execution_failure",
details: { operation: "capture_electron_scenario" },
},
});
const session = await call("binary_session", {});
expect(session.isError, JSON.stringify(session)).not.toBe(true);
expect(await retained()).toEqual(before);
},
180_000,
);
cliTest(
"rejects a file URL path field without resolving an invented local path",
async ({ cli }) => {
@@ -15,6 +15,19 @@ const run = async (arguments_: readonly string[]): Promise<unknown> => {
return JSON.parse(stdout);
};
const runFailure = async (arguments_: readonly string[]): Promise<unknown> => {
try {
await run(arguments_);
} catch (cause: unknown) {
if (typeof cause !== "object" || cause === null || !("stdout" in cause))
throw cause;
const stdout = cause.stdout;
if (typeof stdout !== "string") throw cause;
return JSON.parse(stdout);
}
throw new Error("Expected the CLI command to fail");
};
it.each(["analyze-javascript-application", "analyze"])(
"%s carries the integrity policy into shared JavaScript analysis",
async (command) => {
@@ -46,3 +59,48 @@ it("rejects the JavaScript integrity policy on binary-only inspection", async ()
stdout: expect.stringContaining("Unknown flag: --integrityPolicy"),
});
});
it.each(["fail", "record-and-continue"] as const)(
"rejects the explicitly selected %s integrity policy on a binary analyze target",
async (policy) => {
const failure = await runFailure([
"analyze",
"unused.bin",
"--integrity-policy",
policy,
]);
expect(failure).toMatchObject({
code: "invalid_request",
details: {
issues: [
{
path: ["integrity_policy"],
reason: "invalid_value",
message:
"An integrity policy requires the static JavaScript ASAR/directory analysis lane; binary_overview accepts no input.",
},
],
},
});
},
);
it("still routes a binary analyze target without an integrity policy", async () => {
await expect(run(["analyze", "unused.bin"])).rejects.toMatchObject({
code: 1,
stdout: expect.stringContaining("target does not exist"),
});
});
it("advertises the optional integrity policy without an undefined union", async () => {
const schema = (await run(["analyze", "--schema"])) as {
options?: { properties?: { integrityPolicy?: unknown } };
};
const policy = schema.options?.properties?.integrityPolicy;
expect(policy).toMatchObject({
type: "string",
enum: ["fail", "record-and-continue"],
description: "Behavior when declared application integrity does not match",
});
expect(JSON.stringify(policy)).not.toContain("undefined");
});
@@ -67,6 +67,7 @@ describe("client configuration filesystem lifecycle", () => {
"kimi",
"vscode",
"trae",
"workbuddy",
"grok_build",
"omp",
"pi",
@@ -55,6 +55,8 @@ const host: GhidraInstallationHost = {
readText: () => "application.version=12.1.3\napplication.java.min=21\n",
executable: () => true,
probeJava: () => ({
osArch: "amd64",
osName: "Linux",
version: "21.0.12",
major: 21,
home: "/jdk-21",
@@ -0,0 +1,159 @@
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { readClientRegistrationStatuses } from "../../../src/application/ClientRegistrationStatus.js";
import { npxRegistrationCommand } from "../../../src/application/ClientRegistrationIdentity.js";
import { configureClientConfiguration } from "../../../src/application/SetupClientConfiguration.js";
import { skillDestinations } from "../../../src/application/SetupSkill.js";
import { supportedClients } from "../../../src/application/SupportedClients.js";
import { systemUninstallHost } from "../../../src/application/Uninstall.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { clearClientLocationEnvironment } from "../../fixtures/clientEnvironment.js";
/**
* The registration command setup writes on the host running the tests. Windows
* wraps it in `cmd.exe /d /c`, so the expected entry follows the host too.
*/
const command = npxRegistrationCommand(process.platform);
beforeEach(clearClientLocationEnvironment);
afterEach(() => vi.unstubAllEnvs());
const workbuddyPath = (
home: string,
platform: NodeJS.Platform = process.platform,
env: Parameters<typeof supportedClients>[2] = {},
) =>
supportedClients(home, platform, env).find(
({ name }) => name === "workbuddy",
);
describe("WorkBuddy configuration paths", () => {
it("resolves the global mcp.json under the home root", () => {
expect(workbuddyPath("/home/a")?.configPath).toBe(
join("/home/a", ".workbuddy", "mcp.json"),
);
expect(workbuddyPath("/Users/a")?.markerPath).toBe(
join("/Users/a", ".workbuddy"),
);
expect(workbuddyPath("C:\\Users\\a", "win32")?.configPath).toBe(
join("C:\\Users\\a", ".workbuddy", "mcp.json"),
);
});
it("resolves the registration format as the plain mcpServers JSON dialect", () => {
expect(workbuddyPath("/home/a")?.format).toBe("json");
});
it("resolves the personal skills directory under the home root", () => {
expect(
skillDestinations("/home/a", ["workbuddy"]).map(({ path }) => path),
).toEqual([
join("/home/a", ".workbuddy", "skills", "reverse-engineer-anything"),
]);
expect(
skillDestinations("C:\\Users\\a", ["workbuddy"], {}, "win32").map(
({ path }) => path,
),
).toEqual([
join("C:\\Users\\a", ".workbuddy", "skills", "reverse-engineer-anything"),
]);
});
});
describe("WorkBuddy JSON registration", () => {
const workbuddyClient = async () => {
const home = await createTestTempDirectory("rea-workbuddy-");
const client = workbuddyPath(home, process.platform);
if (client?.markerPath === undefined)
throw new Error("missing WorkBuddy client");
await mkdir(client.markerPath, { recursive: true });
return { home, client };
};
it("upserts rea into mcpServers while preserving unrelated settings", async () => {
const { home, client } = await workbuddyClient();
await writeFile(
client.configPath,
`{
"mcpServers": {
"other": { "command": "other-server" }
}
}`,
);
expect(
(await configureClientConfiguration(client, {}, command)).status,
).toBe("configured");
const parsed = JSON.parse(await readFile(client.configPath, "utf8"));
expect(parsed.mcpServers.other).toEqual({ command: "other-server" });
expect(parsed.mcpServers.rea).toEqual({
command: command[0],
args: command.slice(1),
});
expect(await configureClientConfiguration(client, {}, command)).toEqual({
status: "unchanged",
});
expect(
(
await readClientRegistrationStatuses(home, undefined, {
platform: process.platform,
environment: {},
})
).find(({ client }) => client === "workbuddy"),
).toMatchObject({ state: "aligned", command });
});
it("creates the mcpServers table in an existing mcp.json without servers", async () => {
const { client } = await workbuddyClient();
await writeFile(
client.configPath,
JSON.stringify({ unrelated: { setting: true } }, null, 2),
);
expect(
(await configureClientConfiguration(client, {}, command)).status,
).toBe("configured");
const parsed = JSON.parse(await readFile(client.configPath, "utf8"));
expect(parsed.unrelated).toEqual({ setting: true });
expect(parsed.mcpServers.rea).toEqual({
command: command[0],
args: command.slice(1),
});
});
it("registers into an empty mcp.json", async () => {
const { client } = await workbuddyClient();
await writeFile(client.configPath, "");
expect(
(await configureClientConfiguration(client, {}, command)).status,
).toBe("configured");
const parsed = JSON.parse(await readFile(client.configPath, "utf8"));
expect(parsed.mcpServers.rea).toEqual({
command: command[0],
args: command.slice(1),
});
});
it("removes only the rea entry on uninstall", async () => {
const { home, client } = await workbuddyClient();
await writeFile(
client.configPath,
JSON.stringify(
{ mcpServers: { other: { command: "other-server" } } },
null,
2,
),
);
await configureClientConfiguration(client, {}, command);
expect((await systemUninstallHost(home).removeClient(client)).status).toBe(
"removed",
);
const parsed = JSON.parse(await readFile(client.configPath, "utf8"));
expect(parsed.mcpServers.other).toEqual({ command: "other-server" });
expect(parsed.mcpServers.rea).toBeUndefined();
});
});
@@ -0,0 +1,138 @@
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
import { Ajv2020 } from "ajv/dist/2020.js";
import { expect, it } from "vitest";
import { FUNCTION_COMPARISON_EXAMPLE } from "../../../src/contracts/functionComparisonExample.js";
import { createEvidence } from "../../../src/domain/evidence.js";
import { functionDossierSchema } from "../../../src/domain/hopperValues.js";
import type { RelationshipCoverage } from "../../../src/domain/relationshipCoverage.js";
import { createServer } from "../../../src/server/createServer.js";
import { createTestBinarySession } from "../../fixtures/binarySession.js";
const observe = (relationshipCoverage: RelationshipCoverage) => {
const template = FUNCTION_COMPARISON_EXAMPLE.left;
if (template.subject === null)
throw new Error("Expected artifact-bound fixture");
const dossier = functionDossierSchema.parse(template.normalized_result);
return createEvidence(
{
path: template.subject.local_path,
sha256: template.subject.digest.sha256,
format: template.subject.format,
},
template.provider,
{
operation: "analyze_function",
parameters: template.parameters,
result: { ...dossier, callees: [], unresolved_calls: [] },
confidence: template.confidence,
authority: template.authority,
relationshipCoverage,
},
);
};
it("preserves directional gaps through advertised schemas and public path reconstruction", async () => {
const session = createTestBinarySession(() => {
throw new Error("Offline call paths must not launch a provider");
});
const server = createServer({ kind: "session", session });
const client = new Client({ name: "function-call-directions", version: "1" });
const [clientTransport, serverTransport] =
InMemoryTransport.createLinkedPair();
try {
await server.connect(serverTransport);
await client.connect(clientTransport);
const advertised = (await client.listTools()).tools.find(
({ name }) => name === "build_call_path",
);
if (advertised === undefined)
throw new Error("Missing build_call_path contract");
const ajv = new Ajv2020({ strict: false, validateFormats: false });
expect(ajv.validateSchema(advertised.inputSchema)).toBe(true);
const validate = ajv.compile(advertised.inputSchema);
for (const { directions, status, exhaustive } of [
{
directions: ["callers"] as ["callers"],
status: "not_found",
exhaustive: true,
},
{
directions: ["callees"] as ["callees"],
status: "unknown",
exhaustive: false,
},
{
directions: ["callers", "callees"] as ["callers", "callees"],
status: "unknown",
exhaustive: false,
},
]) {
const limitation =
"An observed call reference is absent from the inventory.";
const coverage: RelationshipCoverage = {
status: "partial",
limitations: [limitation],
limited_directions: directions,
};
const observation = observe(coverage);
expect(session.recordEvidence(observation).ok).toBe(true);
const input = {
functions: [observation],
start: { address: "0x1000" },
goal: { address: "0x9999" },
};
expect(validate(input), JSON.stringify(validate.errors)).toBe(true);
for (const limited_directions of [
[],
["callers", "callers"],
["callees", "callees"],
["callers", "callees", "callers"],
["incoming"],
])
expect(
validate({
...input,
functions: [
{
...observation,
relationship_coverage: {
...coverage,
limited_directions,
},
},
],
}),
).toBe(false);
const result = await client.callTool({
name: "build_call_path",
arguments: input,
});
if (result.isError) throw new Error(JSON.stringify(result));
expect(result.structuredContent).toMatchObject({
normalized_result: {
status,
search_scope: { exhaustive },
limitations: expect.arrayContaining([limitation]),
},
});
const comparison = await client.callTool({
name: "compare_functions",
arguments: { left: observation, right: observation },
});
if (comparison.isError) throw new Error(JSON.stringify(comparison));
expect(comparison.structuredContent).toMatchObject({
normalized_result: {
dimensions: expect.arrayContaining([
expect.objectContaining({ dimension: "calls", status: "unknown" }),
]),
},
});
// Each scenario owns its Evidence and residual-unknown ledger.
const closed = await session.close();
if (!closed.ok) throw closed.error;
}
} finally {
await Promise.allSettled([session.close(), client.close(), server.close()]);
}
});
+21 -11
View File
@@ -15,9 +15,17 @@ import {
type GhidraFunctionOperation,
} from "../../../src/ghidra/GhidraFunctionValues.js";
it.each([true, false])(
"compares observed empty Ghidra facets while preserving pseudocode availability (%s)",
async (pseudocodeAvailable) => {
it.each([
{ pseudocodeAvailable: true, reportsCoverage: true },
{ pseudocodeAvailable: false, reportsCoverage: true },
{ pseudocodeAvailable: true, reportsCoverage: false },
{ pseudocodeAvailable: false, reportsCoverage: false },
])(
"compares observed empty Ghidra facets with pseudocode=$pseudocodeAvailable and coverage=$reportsCoverage",
async ({ pseudocodeAvailable, reportsCoverage }) => {
const coverage = reportsCoverage
? { status: "complete" as const }
: undefined;
const dossier = functionDossierSchema.parse(ghidraFunctionDossier());
Object.assign(dossier, {
comments: [],
@@ -47,7 +55,13 @@ it.each([true, false])(
if (operation !== "analyze_function")
throw new Error(`Unexpected operation: ${operation}`);
return Promise.resolve(
ok({ value: jsonValueSchema.parse(dossier), limitations: [] }),
ok({
value: jsonValueSchema.parse(dossier),
limitations: [],
...(coverage === undefined
? {}
: { relationship_coverage: coverage }),
}),
);
},
);
@@ -61,9 +75,7 @@ it.each([true, false])(
harness.session,
analyzed.structuredContent,
);
expect(evidence.relationship_coverage).toMatchObject({
status: "complete",
});
expect(evidence.relationship_coverage).toEqual(coverage ?? null);
expect(evidence.normalized_result).toEqual(dossier);
const reply = await harness.mcp.callTool({
name: "compare_functions",
@@ -86,7 +98,7 @@ it.each([true, false])(
pseudocode: pseudocodeAvailable ? "unchanged" : "unknown",
assembly: "unchanged",
comments: "unchanged",
calls: "unchanged",
calls: reportsCoverage ? "unchanged" : "unknown",
references: "unchanged",
strings_names: "unchanged",
cfg: "unchanged",
@@ -96,9 +108,7 @@ it.each([true, false])(
evidence.parameters,
);
if (!replay.ok) throw replay.error;
expect(replay.value.relationshipCoverage).toEqual(
evidence.relationship_coverage,
);
expect(replay.value.relationshipCoverage).toEqual(coverage);
expect(harness.calls).toEqual(["analyze_function"]);
} finally {
await harness.close();
+2
View File
@@ -106,6 +106,8 @@ export const ghidraInstallationHost = (): GhidraInstallationHost => ({
readText: () => "application.version=12.1.4\napplication.java.min=21\n",
executable: () => true,
probeJava: () => ({
osArch: "amd64",
osName: "Linux",
version: "21.0.11",
major: 21,
home: "/usr/lib/jvm/jdk-21",
+2
View File
@@ -102,6 +102,8 @@ const installationHost: GhidraInstallationHost = {
readText: () => "application.version=12.1.4\napplication.java.min=21\n",
executable: () => true,
probeJava: () => ({
osArch: "amd64",
osName: "Linux",
version: "21.0.11",
major: 21,
home: "/fixture/jdk-21",
@@ -0,0 +1,323 @@
import { chmod, mkdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { expect, vi } from "vitest";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import {
processTest,
type TestProcesses,
} from "../../support/process/processFixture.js";
const script = `
import { PlaywrightElectronActiveProvider } from "./dist/browser/PlaywrightElectronActiveProvider.js";
import { electronActiveObservationInputSchema } from "./dist/domain/javascript/electronActiveObservation.js";
const [scenario, executable, root] = JSON.parse(process.argv[1]);
let observedRejections = 0;
if (scenario === "oneFailureWithManySuccesses" || scenario === "externalListenerSeesUnrelatedOnce")
process.on("unhandledRejection", () => { observedRejections += 1; });
const orphanStack = (message, file) =>
message + "\\n at waitForLine (/app/node_modules/playwright-core/lib/server/" + file + ":247:21)";
const playwrightError = (message, file = "electron/electron.js") => {
const error = new Error(message);
error.stack = "Error: " + orphanStack(message, file);
return error;
};
const rejectLater = (error, ms) => setTimeout(() => void Promise.reject(error), ms);
const fakeApplication = { process: () => ({ pid: undefined }), close: async () => undefined };
const launchFailure = (message) =>
new Error("electron.launch: " + message + "\\nCall log:\\n - <launching>");
const after = (ms, action) => new Promise((resolve, reject) => setTimeout(() => {
try { resolve(action()); } catch (error) { reject(error); }
}, ms));
const orphans = (message) => {
for (let index = 0; index < 3; index += 1) void Promise.reject(playwrightError(message));
};
let during = -1;
const launches = {
realLaunch: undefined,
realWithUnrelatedPlaywrightRejection: async (options) => {
void Promise.reject(playwrightError("unrelated playwright failure", "page.js"));
return (await import("playwright-core"))._electron.launch(options);
},
failureWithOrphans: async () => {
during = process.listenerCount("unhandledRejection");
orphans("Process failed to launch!");
await after(10, () => { throw launchFailure("Process failed to launch!"); });
},
failureInOneTick: async () => {
orphans("Process failed to launch!");
throw launchFailure("Process failed to launch!");
},
failureWithUnrelatedPlaywrightRejection: async () => {
orphans("Process failed to launch!");
void Promise.reject(playwrightError("Target page, context or browser has been closed", "page.js"));
await after(10, () => { throw launchFailure("Process failed to launch!"); });
},
externalListenerSeesUnrelatedOnce: async () => {
void Promise.reject(playwrightError("unrelated playwright failure", "page.js"));
await after(10, () => { throw launchFailure("Process failed to launch!"); });
},
pendingWithUnrelatedRejection: () => {
setInterval(() => {}, 1000);
rejectLater(new Error("unrelated application failure"), 20);
return new Promise(() => {});
},
success: async () => {
during = process.listenerCount("unhandledRejection");
return fakeApplication;
},
successWithPlaywrightRejection: async () => {
void Promise.reject(playwrightError("stray playwright failure", "page.js"));
await after(10, () => fakeApplication);
return fakeApplication;
},
};
const input = (executablePath) => ({
...electronActiveObservationInputSchema.parse({
executable_path: executablePath,
application_path: root,
}),
application_root: root,
});
const capture = (launch, executablePath = executable) =>
new PlaywrightElectronActiveProvider({}, launch).capture(input(executablePath));
const before = process.listenerCount("unhandledRejection");
let results;
if (scenario === "concurrentFailures" || scenario === "failureThenSuccess" || scenario === "successThenFailure" || scenario === "oneFailureWithManySuccesses") {
const failing = (message, orphanAt, settleAt) => async () => {
during = Math.max(during, process.listenerCount("unhandledRejection"));
await after(orphanAt, () => orphans(message));
await after(settleAt - orphanAt, () => { throw launchFailure(message); });
};
const succeeding = (settleAt) => () => {
during = Math.max(during, process.listenerCount("unhandledRejection"));
return after(settleAt, () => fakeApplication);
};
const pair = {
concurrentFailures: [failing("alpha failed", 10, 30), failing("beta failed", 20, 50)],
failureThenSuccess: [failing("alpha failed", 10, 20), succeeding(40)],
successThenFailure: [failing("alpha failed", 20, 40), succeeding(30)],
oneFailureWithManySuccesses: [failing("alpha failed", 10, 80), ...[20, 30, 40, 50].map(succeeding)],
}[scenario];
results = await Promise.all(pair.map((launch) => capture(launch)));
} else {
results = [await capture(launches[scenario] ?? undefined)];
}
await new Promise((resolve) => setTimeout(resolve, 50));
console.log(JSON.stringify({
before,
during,
after: process.listenerCount("unhandledRejection"),
observedRejections,
reasons: results.map((result) => result.ok ? null : result.error.reason ?? null),
ok: results.map((result) => result.ok),
causeReasons: results.map((result) => result.ok ? null : result.error.cause?.reason ?? null),
cleanupResources: results.map((result) => result.ok ? null : result.error.cleanupResources ?? null),
causes: results.map((result) => result.ok ? null : String(result.error.cause?.message).split("\\n")[0]),
}));
`;
// A cold macOS Swift toolchain may spend up to 120 seconds building the
// ownership reader before any child launch reaches its assertion.
vi.setConfig({ testTimeout: 180_000 });
type Outcome = {
readonly before: number;
readonly during: number;
readonly after: number;
readonly ok: readonly boolean[];
readonly causes: readonly (string | null)[];
readonly observedRejections: number;
readonly reasons: readonly (string | null)[];
readonly causeReasons: readonly (string | null)[];
readonly cleanupResources: readonly (readonly string[] | null)[];
};
const run = async (
processes: TestProcesses,
scenario: string,
executable = "/not/launched",
root = "/",
) =>
processes.run(
process.execPath,
[
"--input-type=module",
"--eval",
script,
JSON.stringify([scenario, executable, root]),
],
{ timeoutMs: 150_000 },
);
const outcome = (result: { readonly stdout: string }): Outcome =>
JSON.parse(result.stdout.trim().split("\n").at(-1) ?? "") as Outcome;
const earlyExit = async () => {
const root = await createTestTempDirectory("rea-launch-rejections-");
const executable = join(root, "exits-immediately");
await mkdir(join(root, "app"));
await writeFile(executable, "#!/bin/sh\nexit 0\n");
await chmod(executable, 0o755);
return { executable, root: join(root, "app") };
};
processTest(
"absorbs the orphaned rejections of a failed launch and removes its listener",
async ({ processes }) => {
const result = await run(processes, "failureWithOrphans");
expect(result.exitCode, result.stderr).toBe(0);
expect(outcome(result)).toMatchObject({
during: outcome(result).before + 1,
after: outcome(result).before,
ok: [false],
causes: ["electron.launch: Process failed to launch!"],
});
},
);
processTest(
"absorbs orphans raised in the same tick as the launch failure",
async ({ processes }) => {
const result = await run(processes, "failureInOneTick");
expect(result.exitCode, result.stderr).toBe(0);
expect(outcome(result)).toMatchObject({ ok: [false] });
},
);
processTest.skipIf(process.platform === "win32")(
"absorbs the orphans of a real Playwright launch that exits at once",
async ({ processes }) => {
const { executable, root } = await earlyExit();
const result = await run(processes, "realLaunch", executable, root);
expect(result.exitCode, result.stderr).toBe(0);
expect(outcome(result)).toMatchObject({
ok: [false],
causes: ["electron.launch: Process failed to launch!"],
});
expect(outcome(result).after).toBe(outcome(result).before);
},
);
processTest.skipIf(process.platform === "win32")(
"still fails on an unrelated Playwright rejection during a real failing launch",
async ({ processes }) => {
const { executable, root } = await earlyExit();
const result = await run(
processes,
"realWithUnrelatedPlaywrightRejection",
executable,
root,
);
expect(result.exitCode, result.stderr).toBe(1);
expect(result.stderr).toContain("unrelated playwright failure");
expect(result.stdout).toBe("");
},
);
processTest(
"does not replay an unrelated orphan to a listener that already received it",
async ({ processes }) => {
const result = await run(processes, "externalListenerSeesUnrelatedOnce");
expect(result.exitCode, result.stderr).toBe(0);
expect(outcome(result)).toMatchObject({
observedRejections: 1,
ok: [false],
causes: ["electron.launch: Process failed to launch!"],
});
},
);
processTest(
"still fails on an unrelated Playwright rejection beside a failing launch's own orphans",
async ({ processes }) => {
const result = await run(
processes,
"failureWithUnrelatedPlaywrightRejection",
);
expect(result.exitCode, result.stderr).toBe(1);
expect(result.stderr).toContain(
"Target page, context or browser has been closed",
);
expect(result.stdout).toBe("");
},
);
processTest(
"still fails at once on an unrelated rejection while a launch is pending",
async ({ processes }) => {
const result = await run(processes, "pendingWithUnrelatedRejection");
expect(result.exitCode, result.stderr).toBe(1);
expect(result.stderr).toContain("unrelated application failure");
expect(result.stdout).toBe("");
},
);
processTest(
"removes its listener after a successful launch",
async ({ processes }) => {
const result = await run(processes, "success");
expect(result.exitCode, result.stderr).toBe(0);
expect(outcome(result)).toMatchObject({
during: outcome(result).before + 1,
after: outcome(result).before,
});
},
);
processTest(
"re-raises a Playwright rejection when the launch succeeds",
async ({ processes }) => {
const result = await run(processes, "successWithPlaywrightRejection");
expect(result.exitCode, result.stderr).toBe(1);
expect(result.stderr).toContain("stray playwright failure");
expect(result.stdout).toBe("");
},
);
processTest.for([
"concurrentFailures",
"failureThenSuccess",
"successThenFailure",
])(
"keeps overlapping launches from re-raising each other's orphans (%s)",
async (scenario, { processes }) => {
const result = await run(processes, scenario);
expect(result.exitCode, result.stderr).toBe(0);
expect(outcome(result).after).toBe(outcome(result).before);
},
);
processTest(
"holds each orphan once across many overlapping successful launches",
async ({ processes }) => {
const result = await run(processes, "oneFailureWithManySuccesses");
expect(result.exitCode, result.stderr).toBe(0);
const observed = outcome(result);
expect(observed.during).toBe(observed.before + 1);
expect(observed.after).toBe(observed.before);
expect(observed.observedRejections).toBe(3);
expect(observed.ok).toEqual([false, false, false, false, false]);
expect(observed.causes[0]).toBe("electron.launch: alpha failed");
expect(observed.reasons).toEqual([
null,
"cleanup_failed",
"cleanup_failed",
"cleanup_failed",
"cleanup_failed",
]);
expect(observed.causeReasons).toEqual([
null,
...Array.from({ length: 4 }, () => "process_ownership_unavailable"),
]);
expect(observed.cleanupResources).toEqual([
[],
...Array.from({ length: 4 }, () => ["owned_process_group"]),
]);
},
);
@@ -50,9 +50,16 @@ if (mode === "env-failure" || (probe === "-vmargs" && mode === "vm-failure")) {
process.stderr.write("failed " + probe + "\\n");
process.exitCode = 3;
} else if (probe === "-envvars") {
process.stdout.write(mode === "env-malformed" ? "invalid environment\\n" : "GHIDRA_MAXMEM=1G\\n");
process.stdout.write(mode === "env-malformed" ? "invalid environment\\n" : mode.startsWith("env-injection:") ? mode.slice("env-injection:".length) + "\\n" : "GHIDRA_MAXMEM=1G\\n");
} else {
process.stdout.write(mode === "vm-malformed" ? "invalid JVM argument\\n" : "-Dfile.encoding=UTF8\\n");
const vmArguments = mode === "vm-malformed"
? "invalid JVM argument\\n"
: mode === "os-arch-override"
? "-Dos.arch=x86_64\\n"
: mode === "os-name-override"
? "-Dos.name=Windows\\n"
: "-Dfile.encoding=UTF8\\n";
process.stdout.write(vmArguments);
}
`,
{ mode: 0o755 },
@@ -215,6 +222,57 @@ describe.skipIf(process.platform === "win32")(
},
);
it.each(["os-arch-override", "os-name-override"])(
"rejects LaunchSupport %s before starting analysis",
async (mode) => {
const { session, launcher, analysisArguments } = await fixture(mode);
const launched = await launcher.launch(session);
expect(launched.ok).toBe(false);
expect(analysisArguments).toEqual([]);
if (launched.ok)
throw new Error("Expected a JVM platform override failure");
expect(launched.error.cause).toMatchObject({
message: expect.stringContaining(
"may not override os.arch or os.name",
),
});
},
);
it.each([
["JAVA_TOOL_OPTIONS", "-Dos.arch=x86_64"],
["JAVA_TOOL_OPTIONS", '"-Dos.arch=x86_64"'],
["_JAVA_OPTIONS", "'-Dos.name=Windows'"],
["JDK_JAVA_OPTIONS", '-D"os.name"=Windows'],
["JDK_JAVA_OPTIONS", "-D'os.arch'=x86_64"],
])(
"rejects helper-environment platform injection from %s=%s before analysis",
async (key, value) => {
const { session, launcher, analysisArguments } = await fixture(
`env-injection:${key}=${value}`,
);
const launched = await launcher.launch(session);
expect(launched.ok).toBe(false);
expect(analysisArguments).toEqual([]);
if (launched.ok)
throw new Error("Expected a JVM platform override failure");
expect(launched.error.cause).toMatchObject({
message: expect.stringContaining(
"may not override os.arch or os.name",
),
});
},
);
it("admits a quoted non-platform option without mistaking its value for a platform argument", async () => {
const { session, launcher, analysisArguments } = await fixture(
'env-injection:JAVA_TOOL_OPTIONS=-Ddescription="text -Dos.arch=x86_64"',
);
const launched = await launcher.launch(session);
if (!launched.ok) throw launched.error;
expect(analysisArguments).toHaveLength(1);
});
it("bounds retained UTF-8 warnings and reports original byte counts", async () => {
const { session, launcher } = await fixture("large");
const launched = await launcher.launch(session);
@@ -290,6 +348,13 @@ describe.skipIf(process.platform === "win32")(
"application.version=12.1.4\napplication.java.min=21\n",
executable: () => true,
probeJava: () => ({
osArch: process.arch === "arm64" ? "aarch64" : "amd64",
osName:
process.platform === "darwin"
? "Mac OS X"
: process.platform === "win32"
? "Windows 11"
: "Linux",
version: "21.0.12",
major: 21,
home: "/fixture/jdk",
@@ -524,6 +524,13 @@ describe("Ghidra Windows launcher runtime", () => {
),
javaHome: "C:\\Java\\jdk-21",
bridgeScriptPath: join(parent, "ReaGhidraBridge.java"),
// The preflight validates the inspected JVM's LaunchSupport output;
// these fixtures assert batch-spawn behavior without a working JDK.
javaLaunch: async () => ({
command: "C:\\Java\\jdk-21\\bin\\java.exe",
arguments: [],
environment: {},
}),
spawnProcess,
});
const launched = await launcher.launch({
@@ -569,6 +576,13 @@ describe("Ghidra Windows launcher runtime", () => {
platform: "win32",
javaHome,
bridgeScriptPath: "/package/bridge/ReaGhidraBridge.java",
// The preflight validates the inspected JVM's LaunchSupport output;
// this fixture captures the batch launcher without a working JDK.
javaLaunch: async () => ({
command: `${javaHome}\\bin\\java.exe`,
arguments: [],
environment: {},
}),
});
const launched = await launcher.launch({
runtimeRoot,
@@ -12,6 +12,8 @@ import ghidra.program.model.listing.FlowOverride;
import ghidra.program.model.listing.Function;
import ghidra.program.model.listing.Instruction;
import ghidra.program.model.symbol.SourceType;
import ghidra.program.model.symbol.Reference;
import ghidra.program.model.symbol.RefType;
import java.io.File;
import java.lang.reflect.Method;
import java.nio.file.Files;
@@ -20,9 +22,12 @@ import java.util.HexFormat;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.ArrayList;
/** Injects reported analysis flags into imported ELF/Mach-O functions; never executes target code. */
public class ReaNoReturnProbe extends GhidraScript {
private record MemoryCallReference(Address source, Address target, RefType type,
SourceType sourceType, int operandIndex, boolean primary) {}
private final JsonArray checks = new JsonArray();
private final Map<Function, Address> originalEnds = new LinkedHashMap<>();
private final Map<Function, String> originalSignatures = new LinkedHashMap<>();
@@ -85,6 +90,114 @@ public class ReaNoReturnProbe extends GhidraScript {
return instance;
}
private void knownInteriorCall(GhidraScript bridge) throws Exception {
Function caller = function("rea_tls_chain");
Function target = function("rea_tls");
Address interior = target.getEntryPoint().add(1);
require("interior_target_is_owned", currentProgram.getFunctionManager().getFunctionAt(interior) == null &&
target.equals(currentProgram.getFunctionManager().getFunctionContaining(interior)));
Instruction call = null;
for (Instruction instruction : currentProgram.getListing().getInstructions(caller.getBody(), true)) {
if (instruction.getFlowType().isCall()) { call = instruction; break; }
}
if (call == null) throw new IllegalStateException("Missing real call instruction");
var references = currentProgram.getReferenceManager();
List<Reference> originals = new ArrayList<>();
List<MemoryCallReference> saved = new ArrayList<>();
for (Reference reference : call.getReferencesFrom()) {
if (!reference.getReferenceType().isCall()) continue;
originals.add(reference);
saved.add(new MemoryCallReference(reference.getFromAddress(), reference.getToAddress(),
reference.getReferenceType(), reference.getSource(), reference.getOperandIndex(), reference.isPrimary()));
}
require("original_call_reference_present", !originals.isEmpty());
Reference changed = null;
try {
for (Reference reference : originals) references.delete(reference);
changed = references.addMemoryReference(call.getAddress(), interior,
RefType.UNCONDITIONAL_CALL, SourceType.USER_DEFINED, 0);
require("interior_target_omitted_by_callee_getter", !caller.getCalledFunctions(monitor).contains(target));
JsonObject query = new JsonObject();
query.addProperty("document", currentProgram.getName());
query.addProperty("procedure", caller.getName());
query.addProperty("direction", "outgoing");
Method read = bridge.getClass().getDeclaredMethod("procedureReferences", JsonObject.class);
read.setAccessible(true);
JsonObject output = (JsonObject) read.invoke(bridge, query);
require("interior_reference_keeps_known_procedure", output.getAsJsonArray("references").asList().stream().anyMatch(
edge -> !edge.getAsJsonObject().get("target_procedure").isJsonNull() &&
edge.getAsJsonObject().getAsJsonObject("target_procedure").get("name").getAsString().equals(target.getName())));
require("known_reference_is_not_also_unresolved", output.getAsJsonArray("unresolved_calls").isEmpty());
query.remove("direction");
Method callees = bridge.getClass().getDeclaredMethod("procedureCalls", JsonObject.class, boolean.class);
callees.setAccessible(true);
JsonObject inventory = (JsonObject) callees.invoke(bridge, query, false);
require("omitted_callee_inventory_stays_partial", inventory.getAsJsonObject("relationship_coverage").get("status").getAsString().equals("partial"));
require("omitted_callee_direction_is_explicit",
inventory.getAsJsonObject("relationship_coverage").getAsJsonArray("limited_directions").toString().equals("[\"callees\"]"));
require("interior_caller_omitted_by_entry_getter", !target.getCallingFunctions(monitor).contains(caller));
query.addProperty("procedure", target.getName());
JsonObject incoming = (JsonObject) callees.invoke(bridge, query, true);
require("off_entry_caller_inventory_stays_partial",
incoming.getAsJsonObject("relationship_coverage").get("status").getAsString().equals("partial"));
require("off_entry_caller_direction_is_explicit",
incoming.getAsJsonObject("relationship_coverage").getAsJsonArray("limited_directions").toString().equals("[\"callers\"]"));
JsonObject dossierQuery = new JsonObject();
dossierQuery.addProperty("procedure", target.getName());
Method analyze = bridge.getClass().getDeclaredMethod("analyzeFunctionObservation", JsonObject.class);
analyze.setAccessible(true);
Object observation = analyze.invoke(bridge, dossierQuery);
Method coverage = observation.getClass().getDeclaredMethod("relationshipCoverage");
coverage.setAccessible(true);
require("dossier_preserves_caller_only_gap",
((JsonObject) coverage.invoke(observation)).getAsJsonArray("limited_directions").toString().equals("[\"callers\"]"));
} finally {
if (changed != null) references.delete(changed);
for (MemoryCallReference reference : saved) {
Reference restored = references.addMemoryReference(reference.source(), reference.target(),
reference.type(), reference.sourceType(), reference.operandIndex());
references.setPrimary(restored, reference.primary());
}
}
}
private void nonCallInstructionReference(GhidraScript bridge) throws Exception {
Function caller = function("rea_tls_chain");
Function target = function("rea_memcpy");
Instruction source = null;
for (Instruction instruction : currentProgram.getListing().getInstructions(caller.getBody(), true)) {
if (!instruction.getFlowType().isCall()) { source = instruction; break; }
}
if (source == null) throw new IllegalStateException("Missing non-call instruction");
var references = currentProgram.getReferenceManager();
Reference added = references.addMemoryReference(source.getAddress(), target.getEntryPoint().add(1),
RefType.UNCONDITIONAL_CALL, SourceType.USER_DEFINED, -1);
try {
require("non_call_reference_target_omitted", !caller.getCalledFunctions(monitor).contains(target));
JsonObject query = new JsonObject();
query.addProperty("document", currentProgram.getName());
query.addProperty("procedure", caller.getName());
Method read = bridge.getClass().getDeclaredMethod("procedureCalls", JsonObject.class, boolean.class);
read.setAccessible(true);
JsonObject output = (JsonObject) read.invoke(bridge, query, false);
require("non_call_instruction_reference_stays_partial",
output.getAsJsonObject("relationship_coverage").get("status").getAsString().equals("partial"));
query.addProperty("direction", "outgoing");
Method edges = bridge.getClass().getDeclaredMethod("procedureReferences", JsonObject.class);
edges.setAccessible(true);
JsonObject known = (JsonObject) edges.invoke(bridge, query);
require("non_call_known_reference_is_not_unresolved", known.getAsJsonArray("unresolved_calls").isEmpty());
references.delete(added);
Address unknown = currentProgram.getMemory().getMaxAddress().add(0x1000);
added = references.addMemoryReference(source.getAddress(), unknown,
RefType.UNCONDITIONAL_CALL, SourceType.USER_DEFINED, -1);
JsonObject unresolved = (JsonObject) edges.invoke(bridge, query);
require("non_call_unknown_reference_keeps_site", unresolved.getAsJsonArray("unresolved_calls").size() == 1);
} finally {
references.delete(added);
}
}
@Override
public void run() throws Exception {
String[] args = getScriptArgs();
@@ -164,6 +277,8 @@ public class ReaNoReturnProbe extends GhidraScript {
Object output = pseudocode.invoke(bridge, query);
require("recovered_pseudocode", output instanceof JsonObject value &&
value.get("value").getAsString().contains("rea_after_call"));
nonCallInstructionReference(bridge);
knownInteriorCall(bridge);
} finally {
((ghidra.app.decompiler.DecompInterface) field.get(bridge)).dispose();
}
@@ -34,6 +34,10 @@ it
const procRoot = join(directory, "proc");
await mkdir(procRoot);
const alternateMode = mode === "alternate launcher";
let orphanPid: number | undefined;
const isolatedProcessList = async () => ({
stdout: alternateMode || orphanPid === undefined ? "" : "Xvfb",
});
if (alternateMode) {
await copyFile("/usr/bin/sleep", launcher);
await chmod(launcher, 0o700);
@@ -47,6 +51,7 @@ it
await assertCleanLinuxHopperRuntime(
alternateMode ? [launcher] : [],
procRoot,
isolatedProcessList,
);
const command = alternateMode ? alternate : process.execPath;
const arguments_ = alternateMode ? ["30"] : ["-e", childCode];
@@ -69,7 +74,6 @@ it
const parentPid = parent.child.pid;
if (parentPid === undefined)
throw new Error("Fixture parent did not spawn");
let orphanPid: number | undefined;
try {
const { stdout } = await parent;
const { pid } = JSON.parse(stdout) as { pid: number };
@@ -115,6 +119,7 @@ it
assertCleanLinuxHopperRuntime(
alternateMode ? [launcher] : [],
procRoot,
isolatedProcessList,
),
).rejects.toThrow("Clean Linux runner retained a Hopper demo process");
// Observation must not terminate the fixture, even though it resembles Xvfb.