Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Kaoru0822
|
f3cd103b31 | ||
|
Danish Rafie EkaputraGitHubKaoru0822factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> |
a6c3d3a365 | ||
|
|
d82ad1faf0 | ||
|
JANG BYUNGKUNandGitHub
|
2dc89f81be | ||
|
morluto
|
30128a03cf | ||
|
Xiangfang ChenandGitHub
|
e049ef70f7 | ||
|
|
bbf0638e88 | ||
|
|
242e2b0001 | ||
|
|
20e4f83097 | ||
|
|
97aff28201 | ||
|
|
641a8395ee | ||
|
morluto
|
94e360c11b | ||
|
morluto
|
7b9ab778a1 | ||
|
morluto
|
a92b960f6f | ||
|
|
da49ea2692 | ||
|
|
f993d33028 |
@@ -2,7 +2,7 @@
|
||||
name: reverse-engineer-anything
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
metadata:
|
||||
version: "41"
|
||||
version: "42"
|
||||
---
|
||||
|
||||
# REA
|
||||
@@ -104,6 +104,10 @@ explicit path or endpoint and do not need it.
|
||||
consult the NativeAOT workflow in `references/native-and-artifacts.md` for
|
||||
optional metadata recovery and its supported host/layout boundary.
|
||||
- User-owned browser page already open: `list_browser_targets`.
|
||||
- Browser action the agent must perform (click, type, navigate):
|
||||
`capture_browser_scenario` with declared actions. See
|
||||
[runtime observation](references/runtime-observation.md) for connect/launch
|
||||
and the focused follow-up browser tools.
|
||||
- Fiddler Everywhere traffic or a native `.saz` capture: read the
|
||||
[Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging).
|
||||
Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`;
|
||||
@@ -323,7 +327,8 @@ as complete while required questions remain open.
|
||||
[references/javascript-applications.md](references/javascript-applications.md)
|
||||
- Android APK declarations, classes, methods, and static references:
|
||||
[references/android-applications.md](references/android-applications.md)
|
||||
- Passive browser/Electron observation and static/runtime reconciliation:
|
||||
- Browser/Electron observation (passive and scenario) and static/runtime
|
||||
reconciliation:
|
||||
[references/runtime-observation.md](references/runtime-observation.md)
|
||||
- Evidence paging, comparisons, residual unknowns, and verification:
|
||||
[references/evidence-workflows.md](references/evidence-workflows.md)
|
||||
|
||||
@@ -103,6 +103,11 @@ architectures, build_metadata, load_commands, dependencies, imports, exports,
|
||||
segments and provenance. `source_exhaustive: false` means unreturned symbols
|
||||
may exist.
|
||||
|
||||
`decode_interface_builder` graphs are often several megabytes and can exceed the
|
||||
MCP response budget. Select `detail: "summary"` for every decoded document and
|
||||
node/edge counts, then page `view: {kind: "interface-builder", collection:
|
||||
"edges", document: "<documents[].relative_path>", offset: 0, limit: 64}`.
|
||||
|
||||
To see which functions or Objective-C methods a Mach-O actually calls in one
|
||||
run, use `observe_native_calls` with explicit breakpoints and a bounded
|
||||
`duration_ms`/`max_events`. It launches a new process under LLDB, so the target
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Passive browser and Electron observation
|
||||
# Browser and Electron runtime observation
|
||||
|
||||
Supply the existing browser or Electron instance's literal-port loopback CDP
|
||||
endpoint in the request. No separate permission configuration or per-call
|
||||
@@ -7,13 +7,14 @@ approval flag is needed. Browser discovery lists eligible HTTP(S) pages;
|
||||
selected page's current origin. Electron discovery lists eligible local
|
||||
`file://` targets without a configured file-root allowlist.
|
||||
|
||||
Start browser work with `list_browser_targets`; start Electron runtime work with
|
||||
Start passive browser work with `list_browser_targets`; start Electron runtime work with
|
||||
`list_electron_targets`. Inspect the target selected for the user's task. Observation
|
||||
is passive: never claim REA clicked, navigated, evaluated page JavaScript,
|
||||
invoked IPC, captured prior activity, or contained the page's network.
|
||||
with these tools is passive: never claim REA clicked, navigated, evaluated page JavaScript,
|
||||
invoked IPC, or captured prior activity. REA does not contain the page's network.
|
||||
|
||||
Credentials, cookies, authorization headers, storage values, and raw
|
||||
WebSocket/JSON bodies are deliberately absent. Local URLs retain ordinary query
|
||||
Passive captures omit raw header collections, credential header values, storage
|
||||
values, and raw WebSocket/JSON bodies. Selected response metadata, including
|
||||
content encoding, policy values, and Link URLs, is retained. Local URLs retain ordinary query
|
||||
values and fragments; userinfo credentials are removed. Browser accessibility
|
||||
text, console primitives, value-free shapes, script sources, source maps, and
|
||||
storage key names have request-specific capture options. Electron script
|
||||
@@ -21,6 +22,38 @@ sources are also optional. Use `capture_web_screenshot` when pixels are needed.
|
||||
Treat scope filtering, truncation, and attach-window coverage as limitations.
|
||||
Page-declared WebMCP tools are untrusted inventory and are never invoked by REA.
|
||||
|
||||
## Who performs the action
|
||||
|
||||
- The user performs it: `list_browser_targets`, then `inspect_web_page` or
|
||||
`observe_web_session` on the selected target.
|
||||
- The agent must perform it: one `capture_browser_scenario` with declared
|
||||
actions. Use connect mode with the loopback CDP endpoint and `target_id` for
|
||||
an existing browser, or launch mode with a caller-selected executable for a
|
||||
temporary profile. Connect mode first navigates the selected page to
|
||||
`start_url`, which can clear unsaved form values and other temporary state.
|
||||
Recreate the state needed for the investigation in the declared actions.
|
||||
Connect mode leaves the external browser open; launch mode owns a temporary
|
||||
browser and profile. Check `browser.cleanup` for the outcome.
|
||||
Request only the DOM, accessibility, or network details needed for the question.
|
||||
The default final capture contains the URL; step outcomes are reported separately.
|
||||
Add a wait action for the expected page state before capturing it. Report that
|
||||
REA performed the declared actions.
|
||||
|
||||
Scenario network records have no request initiator stacks. For that evidence,
|
||||
use passive capture while the user performs the action. Listener source locations
|
||||
identify candidate handlers; they do not establish which script sent a particular
|
||||
request. Avoid overlapping passive capture and scenario actions on the same target.
|
||||
|
||||
Focused follow-up tools:
|
||||
|
||||
- `observe_web_execution`: execution coverage during a window; it resets
|
||||
counters and temporarily disables optimization.
|
||||
- `inspect_web_event_listeners`: source locations for listeners registered
|
||||
directly on the first CSS match, without dispatching an event.
|
||||
- `compare_web_captures`: two normalized passive captures or recorded scenarios.
|
||||
- `discover_webmcp_tools`: inspect page-declared WebMCP inventory; never invoke
|
||||
the discovered tools.
|
||||
|
||||
Use `reconcile_javascript_runtime` only after static application Evidence and
|
||||
passive browser/Electron Evidence exist. Prefer captured-source digest identity;
|
||||
report path/digest disagreement as a mismatch and preserve ambiguity. Explicit
|
||||
|
||||
@@ -81,6 +81,7 @@ import ghidra.program.model.listing.Function;
|
||||
import ghidra.program.model.listing.FunctionIterator;
|
||||
import ghidra.program.model.listing.Instruction;
|
||||
import ghidra.program.model.listing.InstructionIterator;
|
||||
import ghidra.program.model.listing.Listing;
|
||||
import ghidra.program.model.listing.Variable;
|
||||
import ghidra.program.model.mem.MemoryBlock;
|
||||
import ghidra.program.model.mem.MemoryBlockSourceInfo;
|
||||
@@ -405,7 +406,9 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
case "xrefs" -> xrefs(request.params);
|
||||
case "analyze_function" -> {
|
||||
FunctionAnalysisObservation observation = analyzeFunctionObservation(request.params);
|
||||
yield functionObservation(observation.dossier(), observation.limitations());
|
||||
JsonObject result = functionObservation(observation.dossier(), observation.limitations());
|
||||
result.add("relationship_coverage", observation.relationshipCoverage());
|
||||
yield result;
|
||||
}
|
||||
case "set_address_name", "set_addresses_names" -> {
|
||||
if (!descriptor.transport.equals("unix-socket"))
|
||||
@@ -1016,12 +1019,19 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
for (Function item : ordered) {
|
||||
result.add(canonicalAddress(item.getEntryPoint()));
|
||||
}
|
||||
return functionObservation(
|
||||
result,
|
||||
noReturnCallLimitations(
|
||||
if (callers) {
|
||||
JsonObject observation = functionObservation(result, noReturnCallLimitations(
|
||||
currentProgram.getListing().getInstructions(function.getBody(), true)
|
||||
)
|
||||
));
|
||||
observation.add("relationship_coverage", callRelationshipCoverage(null,
|
||||
unrepresentedIncomingCalls(function, observed)));
|
||||
monitor.checkCancelled();
|
||||
return observation;
|
||||
}
|
||||
CallBoundaryObservation calls = callBoundaryObservation(
|
||||
function, currentProgram.getListing().getInstructions(function.getBody(), true), observed, false, null
|
||||
);
|
||||
return callRelationshipObservation(result, calls);
|
||||
}
|
||||
|
||||
private JsonObject procedureInfo(JsonObject params) throws Exception {
|
||||
@@ -1071,18 +1081,10 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
result.addProperty("direction", direction);
|
||||
result.add("references", edges);
|
||||
result.addProperty("reference_kinds_available", true);
|
||||
JsonArray unresolvedCalls = new JsonArray();
|
||||
if (direction.equals("outgoing")) for (Instruction instruction : scanInstructions(function, Integer.MAX_VALUE).instructions) {
|
||||
if (!instruction.getFlowType().isCall()) continue;
|
||||
boolean resolved = false;
|
||||
for (Reference reference : instruction.getReferencesFrom()) if (reference.getReferenceType().isCall()) resolved = true;
|
||||
if (!resolved) {
|
||||
JsonObject site = new JsonObject();
|
||||
site.addProperty("address", canonicalAddress(instruction.getAddress()));
|
||||
site.addProperty("reason", "Instruction decoder identifies a call but ReferenceManager supplies no call destination");
|
||||
unresolvedCalls.add(site);
|
||||
}
|
||||
}
|
||||
JsonArray unresolvedCalls = direction.equals("outgoing")
|
||||
? callBoundaryObservation(
|
||||
function, currentProgram.getListing().getInstructions(function.getBody(), true), null, true, references
|
||||
).unresolvedCalls() : new JsonArray();
|
||||
result.add("unresolved_calls", unresolvedCalls);
|
||||
return result;
|
||||
}
|
||||
@@ -1490,16 +1492,28 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
InstructionScan scan = scanInstructions(function, Integer.MAX_VALUE);
|
||||
DecompileResults decompilation = decompile(function);
|
||||
String pseudocode = decompiledText(decompilation);
|
||||
List<Reference> incomingReferences = collectReferences(function, "incoming")
|
||||
.stream()
|
||||
JsonObject availability = new JsonObject();
|
||||
availability.add("pseudocode", facetAvailability(
|
||||
pseudocode != null,
|
||||
function.isExternal() ? "Ghidra does not decompile external functions."
|
||||
: function.getBody().isEmpty() ? "Ghidra has no function body to decompile."
|
||||
: "Ghidra completed decompilation without returning pseudocode."
|
||||
));
|
||||
if (pseudocode == null) {
|
||||
pseudocode = "";
|
||||
}
|
||||
List<Reference> bodyIncoming = collectReferences(function, "incoming");
|
||||
List<Reference> incomingReferences = bodyIncoming.stream()
|
||||
.filter(reference -> !function.getBody().contains(reference.getFromAddress()))
|
||||
.toList();
|
||||
List<Reference> outgoingReferences = collectReferences(function, "outgoing");
|
||||
JsonArray incoming = referenceEdges(incomingReferences);
|
||||
JsonArray outgoing = referenceEdges(outgoingReferences);
|
||||
JsonArray comments = comments(scan.instructions);
|
||||
JsonArray callers = procedureIdentities(function.getCallingFunctions(monitor));
|
||||
JsonArray callees = procedureIdentities(function.getCalledFunctions(monitor));
|
||||
JsonArray comments = comments(function);
|
||||
Set<Function> callerFunctions = function.getCallingFunctions(monitor);
|
||||
JsonArray callers = procedureIdentities(callerFunctions);
|
||||
Set<Function> calleeFunctions = function.getCalledFunctions(monitor);
|
||||
JsonArray callees = procedureIdentities(calleeFunctions);
|
||||
JsonArray referencedStrings = referencedStrings(outgoingReferences);
|
||||
JsonArray referencedNames = referencedNames(outgoingReferences);
|
||||
JsonArray blocks = basicBlockValues(function);
|
||||
@@ -1511,7 +1525,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
procedure.add("locals", functionLocals(function));
|
||||
JsonObject result = new JsonObject();
|
||||
result.add("procedure", procedure);
|
||||
result.addProperty("pseudocode", pseudocode == null ? "" : pseudocode);
|
||||
result.addProperty("pseudocode", pseudocode);
|
||||
result.add("assembly", assembly);
|
||||
result.add("comments", comments);
|
||||
result.add("callers", callers);
|
||||
@@ -1520,20 +1534,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
result.add("outgoing_references", outgoing);
|
||||
result.add("referenced_strings", referencedStrings);
|
||||
result.add("referenced_names", referencedNames);
|
||||
JsonObject facets = new JsonObject();
|
||||
JsonObject pseudocodeAvailability = new JsonObject();
|
||||
pseudocodeAvailability.addProperty("available", pseudocode != null);
|
||||
if (pseudocode == null) {
|
||||
pseudocodeAvailability.addProperty("reason", "Ghidra did not supply pseudocode.");
|
||||
}
|
||||
facets.add("pseudocode", pseudocodeAvailability);
|
||||
for (String facet : List.of("assembly", "comments", "callers", "callees",
|
||||
"references", "strings_names", "cfg")) {
|
||||
JsonObject available = new JsonObject();
|
||||
available.addProperty("available", true);
|
||||
facets.add(facet, available);
|
||||
}
|
||||
result.add("facet_availability", facets);
|
||||
boolean swift = swiftProgram();
|
||||
boolean aarch64 = currentProgram.getLanguageID().getIdAsString().startsWith("AARCH64:");
|
||||
SwiftStringLiteralDecode swiftLiterals = swift && aarch64 ? swiftStringLiterals(function) : null;
|
||||
@@ -1547,6 +1547,15 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
nativeApiBoundary(function, decompilation)
|
||||
);
|
||||
result.add("native_value_flow", nativeValueFlow(decompilation));
|
||||
// Successful full queries distinguish collected empty sets from missing facts.
|
||||
// Availability does not claim that Ghidra resolved every runtime relationship.
|
||||
for (String facet : List.of("comments", "callers", "callees", "references", "strings_names", "cfg")) {
|
||||
availability.add(facet, facetAvailability(true, null));
|
||||
}
|
||||
availability.add("assembly", facetAvailability(
|
||||
!scan.truncated(), "The function instruction scan was truncated."
|
||||
));
|
||||
result.add("facet_availability", availability);
|
||||
JsonArray limitations = new JsonArray();
|
||||
limitations.add(
|
||||
"Unresolved computed or indirect flows without target addresses are not represented as reference edges."
|
||||
@@ -1574,8 +1583,10 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
"Swift String literal decoding is implemented for AArch64 register pairs only; no Swift literals were decoded for this architecture."
|
||||
);
|
||||
}
|
||||
List<String> boundaryLimitations = noReturnCallLimitations(scan.instructions);
|
||||
for (String limitation : boundaryLimitations) {
|
||||
CallBoundaryObservation calls = callBoundaryObservation(
|
||||
function, scan.instructions, calleeFunctions, true, outgoingReferences);
|
||||
result.add("unresolved_calls", calls.unresolvedCalls());
|
||||
for (String limitation : calls.limitations()) {
|
||||
limitations.add(limitation);
|
||||
}
|
||||
if (decompilation != null && decompilation.getErrorMessage() != null &&
|
||||
@@ -1583,7 +1594,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
limitations.add("Ghidra decompiler diagnostic: " + decompilation.getErrorMessage());
|
||||
}
|
||||
result.add("limitations", limitations);
|
||||
return new FunctionAnalysisObservation(result, boundaryLimitations);
|
||||
monitor.checkCancelled();
|
||||
int missingCallers = 0;
|
||||
for (Reference reference : bodyIncoming) {
|
||||
monitor.checkCancelled();
|
||||
if (unrepresentedCaller(reference, callerFunctions)) missingCallers++;
|
||||
}
|
||||
return new FunctionAnalysisObservation(result, calls.limitations(),
|
||||
callRelationshipCoverage(calls, missingCallers));
|
||||
}
|
||||
|
||||
private JsonObject nativeValueFlow(DecompileResults decompilation) throws Exception {
|
||||
@@ -3287,54 +3305,202 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return result;
|
||||
}
|
||||
|
||||
private static JsonObject facetAvailability(boolean available, String reason) {
|
||||
JsonObject result = new JsonObject();
|
||||
result.addProperty("available", available);
|
||||
if (!available) result.addProperty("reason", reason);
|
||||
return result;
|
||||
}
|
||||
|
||||
private JsonObject callRelationshipObservation(JsonElement value, CallBoundaryObservation calls)
|
||||
throws Exception {
|
||||
monitor.checkCancelled();
|
||||
JsonObject result = functionObservation(value, calls.limitations());
|
||||
result.add("relationship_coverage", callRelationshipCoverage(calls, null));
|
||||
return result;
|
||||
}
|
||||
|
||||
private static JsonObject callRelationshipCoverage(
|
||||
CallBoundaryObservation calls, Integer missingCallers) {
|
||||
JsonObject coverage = completeResolvedCallCoverage();
|
||||
Integer omitted = calls == null ? null : calls.unrepresentedCalleeCalls();
|
||||
JsonArray directions = new JsonArray();
|
||||
if (missingCallers != null && missingCallers > 0) directions.add("callers");
|
||||
String callerLimitation = missingCallers != null && missingCallers > 0
|
||||
? "Ghidra's caller inventory omits sources of " + missingCallers +
|
||||
" call reference(s) into this function's body." : null;
|
||||
if ((calls != null && omitted == null) || (calls == null && missingCallers == null)) {
|
||||
coverage.addProperty("status", "unknown");
|
||||
String reason = calls == null ? "No call inventory was examined."
|
||||
: "The callee inventory was not examined alongside decoded call targets.";
|
||||
coverage.addProperty("reason", reason + (callerLimitation == null ? "" : " " + callerLimitation));
|
||||
if (calls == null && missingCallers == null) directions.add("callers");
|
||||
directions.add("callees");
|
||||
coverage.add("limited_directions", directions);
|
||||
return coverage;
|
||||
}
|
||||
JsonArray limitations = new JsonArray();
|
||||
if (callerLimitation != null) limitations.add(callerLimitation);
|
||||
if (omitted != null && omitted > 0) {
|
||||
limitations.add("Ghidra's callee inventory omits targets of " + omitted +
|
||||
" call reference(s) or decoded call(s) without destinations.");
|
||||
directions.add("callees");
|
||||
}
|
||||
if (!limitations.isEmpty()) {
|
||||
coverage.addProperty("status", "partial");
|
||||
coverage.add("limitations", limitations);
|
||||
coverage.add("limited_directions", directions);
|
||||
}
|
||||
return coverage;
|
||||
}
|
||||
|
||||
private static JsonObject completeResolvedCallCoverage() {
|
||||
JsonObject coverage = new JsonObject();
|
||||
coverage.addProperty("status", "complete");
|
||||
JsonArray diagnostics = new JsonArray();
|
||||
diagnostics.add(
|
||||
"Only resolved Ghidra call references are returned; unresolved computed or indirect calls remain unknown, while function classifications distinguish thunks and externals."
|
||||
);
|
||||
coverage.add("diagnostics", diagnostics);
|
||||
return coverage;
|
||||
}
|
||||
|
||||
private boolean unrepresentedCaller(Reference reference, Set<Function> callers) {
|
||||
if (!reference.getReferenceType().isCall()) return false;
|
||||
Function caller = containingFunction(reference.getFromAddress());
|
||||
return caller == null || !callers.contains(caller);
|
||||
}
|
||||
|
||||
private int unrepresentedIncomingCalls(Function function, Set<Function> callers)
|
||||
throws Exception {
|
||||
int[] omitted = { 0 };
|
||||
visitReferences(function, "incoming", reference -> {
|
||||
if (unrepresentedCaller(reference, callers)) omitted[0]++;
|
||||
});
|
||||
return omitted[0];
|
||||
}
|
||||
|
||||
private List<String> noReturnCallLimitations(Iterable<Instruction> instructions)
|
||||
throws Exception {
|
||||
List<String> limitations = new ArrayList<>();
|
||||
for (Instruction instruction : instructions) {
|
||||
monitor.checkCancelled();
|
||||
if (!instruction.getFlowType().isCall() || !instruction.getFlowType().isTerminal()) {
|
||||
continue;
|
||||
}
|
||||
Address[] destinations = instruction.getFlows();
|
||||
StringBuilder limitation = new StringBuilder();
|
||||
limitation.append("Ghidra Listing reports a terminal call at ")
|
||||
.append(canonicalAddress(instruction.getAddress()));
|
||||
if (destinations.length == 0) {
|
||||
limitation.append(" with no resolved callee; its no-return basis is unknown.");
|
||||
}
|
||||
else {
|
||||
limitation.append(" to ");
|
||||
for (int index = 0; index < destinations.length; index++) {
|
||||
if (index > 0) limitation.append(", ");
|
||||
Address destination = destinations[index];
|
||||
limitation.append(canonicalAddress(destination));
|
||||
Function target = currentProgram.getFunctionManager().getFunctionAt(destination);
|
||||
if (target == null) {
|
||||
limitation.append(" (callee identity unknown)");
|
||||
}
|
||||
else {
|
||||
limitation.append(" ").append(procedureName(target));
|
||||
limitation.append(" (external=").append(target.isExternal());
|
||||
limitation.append(", thunk=").append(target.isThunk());
|
||||
limitation.append(", hasNoReturn=").append(target.hasNoReturn()).append(")");
|
||||
}
|
||||
}
|
||||
limitation.append("; fallthrough is excluded by Ghidra's flow model.");
|
||||
Function target = destinations.length == 1
|
||||
? currentProgram.getFunctionManager().getFunctionAt(destinations[0])
|
||||
: null;
|
||||
if (target != null && target.hasNoReturn()) {
|
||||
limitation.append(" This records Ghidra's FunctionManager flag, not an independent verification that the callee cannot return.");
|
||||
}
|
||||
else {
|
||||
limitation.append(" The reason for the terminal flow is not established by this observation.");
|
||||
}
|
||||
}
|
||||
limitations.add(limitation.toString());
|
||||
String limitation = terminalCallLimitation(instruction);
|
||||
if (limitation != null) limitations.add(limitation);
|
||||
}
|
||||
return limitations;
|
||||
}
|
||||
|
||||
private CallBoundaryObservation callBoundaryObservation(
|
||||
Function function, Iterable<Instruction> instructions, Set<Function> returnedCallees,
|
||||
boolean includeUnresolvedSites, List<Reference> observedReferences)
|
||||
throws Exception {
|
||||
List<String> limitations = new ArrayList<>();
|
||||
CallBoundaryReferences calls = new CallBoundaryReferences(returnedCallees, includeUnresolvedSites);
|
||||
if (observedReferences == null) visitReferences(function, "outgoing", calls::observe);
|
||||
else for (Reference reference : observedReferences) {
|
||||
monitor.checkCancelled();
|
||||
calls.observe(reference);
|
||||
}
|
||||
for (Instruction instruction : instructions) {
|
||||
monitor.checkCancelled();
|
||||
String limitation = terminalCallLimitation(instruction);
|
||||
if (limitation != null) limitations.add(limitation);
|
||||
if (!instruction.getFlowType().isCall()) continue;
|
||||
boolean destination = false;
|
||||
for (Reference reference : instruction.getReferencesFrom()) {
|
||||
if (!reference.getReferenceType().isCall()) continue;
|
||||
destination = true;
|
||||
break;
|
||||
}
|
||||
if (!destination) {
|
||||
calls.unrepresented++;
|
||||
calls.unresolved(instruction.getAddress(),
|
||||
"Instruction decoder identifies a call but ReferenceManager supplies no call destination");
|
||||
}
|
||||
}
|
||||
return new CallBoundaryObservation(List.copyOf(limitations),
|
||||
returnedCallees == null ? null : calls.unrepresented, calls.unresolvedCalls());
|
||||
}
|
||||
|
||||
private final class CallBoundaryReferences {
|
||||
private final Set<Function> returnedCallees;
|
||||
private final TreeMap<Address, String> unresolved;
|
||||
private int unrepresented;
|
||||
|
||||
CallBoundaryReferences(Set<Function> returnedCallees, boolean includeUnresolvedSites) {
|
||||
this.returnedCallees = returnedCallees;
|
||||
unresolved = includeUnresolvedSites ? new TreeMap<>() : null;
|
||||
}
|
||||
|
||||
void observe(Reference reference) {
|
||||
if (!reference.getReferenceType().isCall()) return;
|
||||
// Reference identity follows the same containing-function owner as referenceEdge.
|
||||
Function target = containingFunction(reference.getToAddress());
|
||||
if (returnedCallees != null && (target == null || !returnedCallees.contains(target)))
|
||||
unrepresented++;
|
||||
if (target == null) unresolved(reference.getFromAddress(),
|
||||
"A call destination has no known containing Ghidra procedure");
|
||||
}
|
||||
|
||||
void unresolved(Address address, String reason) {
|
||||
if (unresolved != null) unresolved.put(address, reason);
|
||||
}
|
||||
|
||||
JsonArray unresolvedCalls() {
|
||||
if (unresolved == null) return null;
|
||||
JsonArray result = new JsonArray();
|
||||
for (var item : unresolved.entrySet()) {
|
||||
JsonObject site = new JsonObject();
|
||||
site.addProperty("address", canonicalAddress(item.getKey()));
|
||||
site.addProperty("reason", item.getValue());
|
||||
result.add(site);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
}
|
||||
|
||||
private String terminalCallLimitation(Instruction instruction) {
|
||||
if (!instruction.getFlowType().isCall() || !instruction.getFlowType().isTerminal())
|
||||
return null;
|
||||
Address[] destinations = instruction.getFlows();
|
||||
StringBuilder limitation = new StringBuilder();
|
||||
limitation.append("Ghidra Listing reports a terminal call at ")
|
||||
.append(canonicalAddress(instruction.getAddress()));
|
||||
if (destinations.length == 0) {
|
||||
limitation.append(" with no resolved callee; its no-return basis is unknown.");
|
||||
}
|
||||
else {
|
||||
limitation.append(" to ");
|
||||
for (int index = 0; index < destinations.length; index++) {
|
||||
if (index > 0) limitation.append(", ");
|
||||
Address destination = destinations[index];
|
||||
limitation.append(canonicalAddress(destination));
|
||||
Function target = currentProgram.getFunctionManager().getFunctionAt(destination);
|
||||
if (target == null) {
|
||||
limitation.append(" (callee identity unknown)");
|
||||
}
|
||||
else {
|
||||
limitation.append(" ").append(procedureName(target));
|
||||
limitation.append(" (external=").append(target.isExternal());
|
||||
limitation.append(", thunk=").append(target.isThunk());
|
||||
limitation.append(", hasNoReturn=").append(target.hasNoReturn()).append(")");
|
||||
}
|
||||
}
|
||||
limitation.append("; fallthrough is excluded by Ghidra's flow model.");
|
||||
Function target = destinations.length == 1
|
||||
? currentProgram.getFunctionManager().getFunctionAt(destinations[0])
|
||||
: null;
|
||||
if (target != null && target.hasNoReturn()) {
|
||||
limitation.append(" This records Ghidra's FunctionManager flag, not an independent verification that the callee cannot return.");
|
||||
}
|
||||
else {
|
||||
limitation.append(" The reason for the terminal flow is not established by this observation.");
|
||||
}
|
||||
}
|
||||
return limitation.toString();
|
||||
}
|
||||
|
||||
private static String renderAssembly(List<Instruction> instructions) {
|
||||
return String.join("\n", renderAssemblyLines(instructions));
|
||||
}
|
||||
@@ -3358,15 +3524,27 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
private List<Reference> collectReferences(
|
||||
Function function,
|
||||
String direction) throws Exception {
|
||||
TreeMap<String, Reference> observed = new TreeMap<>();
|
||||
visitReferences(function, direction, reference -> addReference(observed, reference));
|
||||
List<Reference> result = new ArrayList<>(observed.values());
|
||||
result.sort(REFERENCE_ORDER);
|
||||
return result;
|
||||
}
|
||||
|
||||
@FunctionalInterface
|
||||
private interface ReferenceVisitor {
|
||||
void accept(Reference reference) throws Exception;
|
||||
}
|
||||
|
||||
private void visitReferences(Function function, String direction, ReferenceVisitor visitor)
|
||||
throws Exception {
|
||||
// References can target instruction interiors or originate in embedded
|
||||
// data. Use actual body ownership, including an external's exact entry.
|
||||
AddressSet scope = new AddressSet(function.getBody());
|
||||
scope.add(function.getEntryPoint());
|
||||
AddressSet scope = functionObservationScope(function);
|
||||
boolean outgoing = direction.equals("outgoing");
|
||||
AddressIterator addresses = outgoing
|
||||
? currentProgram.getReferenceManager().getReferenceSourceIterator(scope, true)
|
||||
: currentProgram.getReferenceManager().getReferenceDestinationIterator(scope, true);
|
||||
TreeMap<String, Reference> observed = new TreeMap<>();
|
||||
while (addresses.hasNext()) {
|
||||
monitor.checkCancelled();
|
||||
Address address = addresses.next();
|
||||
@@ -3374,7 +3552,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
for (Reference reference : currentProgram.getReferenceManager()
|
||||
.getReferencesFrom(address)) {
|
||||
monitor.checkCancelled();
|
||||
addReference(observed, reference);
|
||||
if (!reference.isEntryPointReference()) visitor.accept(reference);
|
||||
}
|
||||
}
|
||||
else {
|
||||
@@ -3382,19 +3560,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
.getReferencesTo(address);
|
||||
while (iterator.hasNext()) {
|
||||
monitor.checkCancelled();
|
||||
addReference(observed, iterator.next());
|
||||
Reference reference = iterator.next();
|
||||
if (!reference.isEntryPointReference()) visitor.accept(reference);
|
||||
}
|
||||
}
|
||||
}
|
||||
List<Reference> result = new ArrayList<>(observed.values());
|
||||
result.sort(REFERENCE_ORDER);
|
||||
return result;
|
||||
}
|
||||
|
||||
private static void addReference(TreeMap<String, Reference> observed, Reference reference) {
|
||||
if (reference.isEntryPointReference()) {
|
||||
return;
|
||||
}
|
||||
String key = canonicalAddress(reference.getFromAddress()) + "\u0000" +
|
||||
canonicalAddress(reference.getToAddress()) + "\u0000" +
|
||||
reference.getReferenceType().getName() + "\u0000" +
|
||||
@@ -3403,6 +3576,12 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
observed.putIfAbsent(key, reference);
|
||||
}
|
||||
|
||||
private static AddressSet functionObservationScope(Function function) {
|
||||
AddressSet scope = new AddressSet(function.getBody());
|
||||
scope.add(function.getEntryPoint());
|
||||
return scope;
|
||||
}
|
||||
|
||||
private JsonArray referenceEdges(List<Reference> references) {
|
||||
JsonArray result = new JsonArray();
|
||||
for (Reference reference : references) {
|
||||
@@ -3556,23 +3735,28 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return result;
|
||||
}
|
||||
|
||||
private static JsonArray comments(List<Instruction> instructions) {
|
||||
private JsonArray comments(Function function) throws Exception {
|
||||
Listing listing = currentProgram.getListing();
|
||||
AddressSet scope = functionObservationScope(function);
|
||||
AddressIterator addresses = listing.getCommentAddressIterator(scope, true);
|
||||
TreeMap<String, JsonObject> observed = new TreeMap<>();
|
||||
for (Instruction instruction : instructions) {
|
||||
while (addresses.hasNext()) {
|
||||
monitor.checkCancelled();
|
||||
Address address = addresses.next();
|
||||
for (CommentType type : CommentType.values()) {
|
||||
String text = instruction.getComment(type);
|
||||
String text = listing.getComment(type, address);
|
||||
if (text == null || text.isEmpty()) {
|
||||
continue;
|
||||
}
|
||||
String kind = type == CommentType.EOL ? "inline" : "comment";
|
||||
JsonObject item = new JsonObject();
|
||||
item.addProperty("address", canonicalAddress(instruction.getAddress()));
|
||||
item.addProperty("address", canonicalAddress(address));
|
||||
item.addProperty("kind", kind);
|
||||
item.addProperty("text", text);
|
||||
String key = canonicalAddress(instruction.getAddress()) + "\u0000" +
|
||||
String key = canonicalAddress(address) + "\u0000" +
|
||||
kind + "\u0000" + type.name() + "\u0000" + text;
|
||||
observed.put(key, item);
|
||||
}
|
||||
}
|
||||
}
|
||||
JsonArray result = new JsonArray();
|
||||
observed.values().forEach(result::add);
|
||||
@@ -4508,7 +4692,8 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
private record InventoryItem(Address address, String value, JsonObject facts) {}
|
||||
private record AddressNameRequest(String inputAddress, Address address, String name) {}
|
||||
private record FunctionEntry(Function function, InventoryItem item) {}
|
||||
private record FunctionAnalysisObservation(JsonObject dossier, List<String> limitations) {}
|
||||
private record FunctionAnalysisObservation(JsonObject dossier, List<String> limitations, JsonObject relationshipCoverage) {}
|
||||
private record CallBoundaryObservation(List<String> limitations, Integer unrepresentedCalleeCalls, JsonArray unresolvedCalls) {}
|
||||
private record SwiftStringLiteralDecode(JsonArray literals, JsonArray omissions) {}
|
||||
private record SwiftLiteralLayout(
|
||||
String representation,
|
||||
|
||||
@@ -265,6 +265,21 @@ The user-level MCP paths follow the editor's VS Code layout. Trae's official
|
||||
documentation describes the raw JSON editor without specifying those paths;
|
||||
loading this registration in Trae remains unverified.
|
||||
|
||||
## WorkBuddy
|
||||
|
||||
For [WorkBuddy](https://www.workbuddy.cn/docs/workbuddy/From-Beginner-to-Expert-Guide/Function-Description/MCP-Guide), setup writes `mcpServers.rea` into the
|
||||
global `~/.workbuddy/mcp.json` (a standard `mcpServers` JSON file at the user
|
||||
home root on every platform). Entries carry `command`, `args`, and `env`;
|
||||
stdio is the default transport. WorkBuddy discovers personal skills from
|
||||
`~/.workbuddy/skills` (`%userprofile%\.workbuddy\skills` on Windows), so
|
||||
setup installs its skill copy there.
|
||||
|
||||
The global MCP path is a plain JSON file; setup preserves every other server
|
||||
and top-level key when it upserts `rea`. WorkBuddy also reads a project-level
|
||||
`<project>/.workbuddy/mcp.json`, which setup does not touch: it registers the
|
||||
user-level file only. Loading this registration and real REA tool calls
|
||||
through WorkBuddy are verified.
|
||||
|
||||
## ZCode
|
||||
|
||||
For [ZCode](https://zcode.z.ai/en/docs/mcp-services), setup registers `rea` in the nested `mcp.servers` table of the
|
||||
|
||||
+15
-3
@@ -288,11 +288,12 @@ hosted connector; see [its configuration](client-configuration.md#other-client-s
|
||||
| Qoder | `qoder` |
|
||||
| Factory Droid | `droid` |
|
||||
| Trae | `trae` |
|
||||
| WorkBuddy | `workbuddy` |
|
||||
| ZCode | `zcode` |
|
||||
| DeepSeek Harness | `dsh` |
|
||||
|
||||
`dsh` and `kimi` require a release containing them. The registry check above does
|
||||
not cover them.
|
||||
`dsh`, `kimi`, and `workbuddy` require a release containing them. The registry
|
||||
check above does not cover them.
|
||||
|
||||
See [client configuration and troubleshooting](client-configuration.md) for
|
||||
configuration paths, profiles, trust policies, and schema-limit workarounds.
|
||||
@@ -493,7 +494,13 @@ copying license secrets into logs, or killing unrelated Hopper processes.
|
||||
REA connects to an existing Ghidra installation on Linux x64/arm64, macOS x64/arm64,
|
||||
or experimental Windows x64 P0.
|
||||
It accepts Ghidra 12.1.x and the 64-bit full JDK declared by that installation's
|
||||
`application.java.min` and `application.java.max`. Current 12.1 releases require
|
||||
`application.java.min` and `application.java.max`. The JDK's reported platform must also
|
||||
map to the same native decompiler directory as the host: a JDK running under
|
||||
emulation (for example x86_64 under Rosetta on Apple silicon) or reporting a
|
||||
non-host `os.name` fails the doctor
|
||||
even when it is an otherwise accepted 64-bit full JDK. Select a matching JDK
|
||||
through `JAVA_HOME`; adding another decompiler in the JVM-selected directory
|
||||
does not satisfy this host-validated admission rule. Current 12.1 releases require
|
||||
JDK 21 or newer and set no maximum. The bridge is verified with Ghidra 12.1.4
|
||||
and JDK 21. Each installation must include the native decompiler for the host
|
||||
architecture in `Ghidra/Features/Decompiler/os/<platform>/` or the corresponding
|
||||
@@ -556,6 +563,11 @@ registrations. See [Windows Ghidra P0](windows-ghidra-p0.md) for provider diagno
|
||||
Doctor validates the platform, architecture, Ghidra 12.1.x application version,
|
||||
`support/analyzeHeadless` or `support/analyzeHeadless.bat`, the installation's
|
||||
Java major range, 64-bit JDK bitness, and the presence of `javac`/`javac.exe`.
|
||||
It also checks that the reported JVM `os.arch` and `os.name` map to the same native decompiler
|
||||
directory validated for the host; unrecognized reported architectures or a
|
||||
non-host operating system are refused. LaunchSupport overrides of either
|
||||
property, including through helper-environment JVM option variables, are
|
||||
rejected at launch.
|
||||
When Java is found through `PATH`, setup records its observed JDK home so GUI
|
||||
MCP clients do not depend on an incidental shell path. Setup shows every exact
|
||||
environment entry in its plan, writes only after approval, and never downloads,
|
||||
|
||||
+13
-1
@@ -193,7 +193,8 @@ after target content was read.
|
||||
## Retained application Evidence inputs
|
||||
|
||||
`analyze_javascript_application`, `inspect_binary_layout`, `inspect_artifact`,
|
||||
`inspect_wasm_artifact`, and `inspect_macho` accept `detail`:
|
||||
`inspect_wasm_artifact`, `inspect_macho`, and `decode_interface_builder` accept
|
||||
`detail`:
|
||||
`complete` (default) returns the complete analysis Evidence; `summary` records
|
||||
that complete Evidence in the session and returns the derived
|
||||
`inspect_analysis_view` summary Evidence instead. Its
|
||||
@@ -276,6 +277,17 @@ symbols (Electron Framework: 2.5 MB of imports and exports, 3.3 KB as a
|
||||
summary), so select `detail: "summary"` on the first call. Delivery detail
|
||||
never reaches the native tools or changes the source Evidence ID.
|
||||
|
||||
Completed `decode_interface_builder` Evidence supports a summary and
|
||||
`view: {kind: "interface-builder", collection: "nodes", offset: 0, limit: 64}`.
|
||||
Collections are `documents`, `nodes`, `edges`, and `coverage`; for nodes and
|
||||
edges an optional `document` keeps rows whose evidence names that exact decoded
|
||||
`relative_path`, and an undecoded path is refused. The summary keeps every
|
||||
decoded document, the provider, node-kind, edge-relation and coverage-status
|
||||
counts, and whether the decoded graph is incomplete. Each page repeats `source_truncated`. A complete
|
||||
app graph often exceeds the default MCP response budget (XLD.app: 6.7 MB), so
|
||||
select `detail: "summary"` on the first call. Delivery detail never reaches the
|
||||
decoder or changes the source Evidence ID.
|
||||
|
||||
`trace_application_feature`, `trace_javascript_semantics`,
|
||||
`compare_application_versions`, `compare_source_to_bundle`, and
|
||||
`compare_javascript_export_shapes` accept complete inline application Evidence
|
||||
|
||||
@@ -319,7 +319,16 @@ complete observation.
|
||||
digest are preserved; key interpretation follows the pinned plist decoder.
|
||||
- `decode_interface_builder` / `rea decode-interface-builder <app>` reads
|
||||
compiled storyboard and nib resources into objects, outlets, actions,
|
||||
connections and controller/class names. `verify:interface-builder` compiles
|
||||
connections and controller/class names. A full app graph is often several
|
||||
megabytes (XLD.app: 6.7 MB, about 2M tokens, which also exceeds the default
|
||||
MCP response budget), so MCP callers should pass `detail: "summary"`. The
|
||||
summary keeps every decoded document and node/edge/coverage counts and
|
||||
retains the complete Evidence; page rows with `inspect_analysis_view`
|
||||
`view: {kind: "interface-builder", collection: "nodes", document:
|
||||
"Contents/Resources/Base.lproj/MainMenu.nib", offset: 0, limit: 64}`.
|
||||
`document` selects rows whose evidence names that exact
|
||||
`documents[].relative_path` (nodes and edges only); `source_truncated` keeps
|
||||
graph incompleteness visible on every page. `verify:interface-builder` compiles
|
||||
a source-owned AppKit XIB with `ibtool`. Storyboards require an installed
|
||||
iOS platform; unsupported archive forms remain explicit.
|
||||
XML plist archives accept UTF-8 and BOM-marked UTF-16 in either byte order.
|
||||
|
||||
+22
-25
@@ -20,7 +20,7 @@
|
||||
|
||||
**[웹사이트](https://rea.tools/ko/) · [가이드](https://rea.tools/ko/guides/) · [사례](https://rea.tools/ko/showcase/)**
|
||||
|
||||
[빠른 시작](#빠른-시작) · [REA의 작동 방식](#rea의-작동-방식) · [분석할 수 있는 대상](#분석할-수-있는-대상) · [사례](#사례) · [자주 묻는 질문](#자주-묻는-질문) · [GNU objdump 및 Rizin 도구](../reverse-engineering-tools.md) · [문서](#문서)
|
||||
[빠른 시작](#빠른-시작) · [REA의 작동 방식](#rea의-작동-방식) · [분석할 수 있는 대상](#분석할-수-있는-대상) · [사례](#사례) · [자주 묻는 질문](#자주-묻는-질문) · [문서](#문서)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
@@ -56,7 +56,7 @@
|
||||
|
||||
REA는 에이전트를 분석 도구에 연결해 네이티브 바이너리, JavaScript와 Electron 앱, .NET 어셈블리, 웹사이트를 검사할 수 있게 합니다. 터미널에서도 같은 도구를 사용할 수 있습니다. 분석은 로컬에서 실행되며, 결과에는 각 결론의 근거와 한계가 포함됩니다.
|
||||
|
||||
설정 과정은 에이전트에 REA를 등록하고 해당 버전에 맞는 워크플로 지침을 설치합니다. 네이티브 분석에는 기존 Hopper 또는 Ghidra 설치를 사용할 수 있습니다. 승인하면 설정 과정에서 Hopper를 설치할 수도 있습니다. 정적 JavaScript 분석에는 두 엔진 모두 필요하지 않습니다.
|
||||
설정 과정은 에이전트에 REA를 등록하고 해당 버전에 맞는 워크플로 지침을 설치합니다. 네이티브 분석에는 기존 Hopper, Ghidra 또는 IDA 설치를 사용할 수 있습니다. 승인하면 설정 과정에서 Hopper를 설치할 수도 있습니다. 정적 JavaScript 분석에는 네이티브 분석 엔진이 필요하지 않습니다.
|
||||
|
||||
> **[REA 웹사이트](https://rea.tools/ko/)**에서 설정 방법, 그림으로 설명한 가이드, 실제 사례를 확인하세요.
|
||||
|
||||
@@ -105,12 +105,14 @@ rea --help
|
||||
|
||||
REA는 빠르게 바뀌며 새 릴리스에는 버그 수정이 자주 포함됩니다. 설치된 버전을 최신으로 유지하세요.
|
||||
|
||||
npm으로 설치한 CLI에서는 다음 명령을 사용합니다.
|
||||
`npm install --global` 또는 `bun add --global`로 설치한 CLI에서는 다음 명령을 사용합니다.
|
||||
|
||||
```bash
|
||||
rea update
|
||||
```
|
||||
|
||||
Bun 전역 업데이트에는 Bun 1.2.17 이상이 필요합니다. 이전 버전의 Bun을 사용한다면 `rea update`를 실행하기 전에 Bun을 직접 업그레이드하세요.
|
||||
|
||||
에이전트 등록과 스킬을 갱신하려면 업데이트 과정에서 출력된 설정 명령을 실행하세요.
|
||||
|
||||
`npx`를 사용한다면 다음 명령으로 에이전트 설정을 갱신하세요.
|
||||
@@ -119,7 +121,7 @@ rea update
|
||||
npx rea-agents@latest setup
|
||||
```
|
||||
|
||||
설정 변경 사항을 검토하고 에이전트를 다시 시작하세요. 일회성 CLI 작업에는 `npx rea-agents@latest` 뒤에 필요한 명령을 붙여 실행하세요.
|
||||
설정 변경 사항을 검토하고 에이전트를 다시 시작하세요. 일회성 CLI 작업에는 `npx rea-agents@latest` 뒤에 필요한 명령을 붙여 실행하세요. Bun을 사용한다면 위 명령의 `npx rea-agents@latest`를 `bunx rea-agents@latest`로 바꿔 실행하세요.
|
||||
|
||||
## REA의 작동 방식
|
||||
|
||||
@@ -139,14 +141,16 @@ REA에는 Node.js 22.x(>=22.19), 24.x(>=24.11) 또는 26+와 npm이 필요합니
|
||||
| --------------------- | ----------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 네이티브 바이너리 | 의사 코드, 어셈블리, 문자열, 심볼, 호출, 참조 | Hopper, Ghidra 또는 IDA; [네이티브 분석](https://rea.tools/ko/guides/native/) |
|
||||
| 오프라인 ELF 레이아웃 | 섹션, 세그먼트, 원본 심볼/재배치 정보, 정적 분석으로 얻은 보안 완화 기법 후보 | Linux x64에서 호출자가 제공하는 pwntools; [바이너리 진단](../binary-diagnostics.md) |
|
||||
| Go 바이너리 | 내장된 컴파일러 버전, 모듈, `replace` 지시어, 빌드 설정 | REA 내장 ELF/PE/thin Mach-O 정적 검사; Go 설치 불필요 |
|
||||
| Rust 바이너리 | cargo-auditable이 내장한 크레이트 이름, 버전, 소스, 의존성 관계 | REA 내장 ELF/PE/thin Mach-O 정적 검사; [Rust 바이너리](../rust-binaries.md) |
|
||||
| WASM 아티팩트 | 유효성 검사, 섹션, import/export의 정확한 WAT 형식, 디코딩된 WAT | 호출자 제공 WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM 바이트코드 | 디스패치 선택자, 바이트 오프셋, 추론한 인자와 상태 변경 가능 여부 | Linux x64의 로컬 원시 바이트/16진수 입력 파일; [오프라인 EVM 가이드](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| SQLite 데이터베이스 | 테이블, 열, 인덱스, 뷰, 트리거, 선택한 일반 테이블의 행 | 로컬 데이터베이스/WAL 스냅샷; Node.js 24.15 이상의 24.x 또는 26 이상(네이티브 SQLite 한도 적용) |
|
||||
| 기록된 Linux 크래시 | 원시 note 레코드, 기록된 모든 스레드의 레지스터/시그널, 선택적 매핑 후보 | Linux x64에서 호출자가 제공하는 pwntools; GDB/pwndbg는 선택 사항; [기록된 크래시](../recorded-crashes.md) |
|
||||
| JavaScript / Electron | 모듈, 임포트, 소스 맵, 라우트, IPC, 네이티브 애드온 관계 | Node.js와 npm; [애플리케이션 분석](https://rea.tools/ko/guides/javascript/) |
|
||||
| 웹사이트 | 페이지 구조, 스크립트, 네트워크 관찰 결과, 요청한 스크린샷 | Chrome 계열 브라우저; [브라우저 분석](https://rea.tools/ko/guides/browser/) |
|
||||
| 저장된 네트워크 캡처 | 요청, 응답, 접근 가능한 페이로드, 소스 위치 | HAR; mitmproxy 네이티브 캡처에는 Linux의 mitmdump; [캡처 가이드](../web-network-captures.md) |
|
||||
| 패킷 캡처 | 엔드포인트, DNS, TLS SNI, 프로토콜 분포, HTTP 요청 | 호출자 제공 tshark 및 capinfos; [pcap 가이드](../pcap-analysis.md) |
|
||||
| .NET 어셈블리 | 메타데이터, CIL 명령, 선언된 네이티브 의존성, 빌드 비교 | 정적 검사; [관리 코드 가이드](../managed-code-analysis.md) |
|
||||
| 저장된 네트워크 캡처 | 요청, 응답, 접근 가능한 페이로드, 소스 위치 | HAR 또는 Fiddler Everywhere SAZ; mitmproxy 네이티브 캡처에는 Linux의 mitmdump; [캡처 가이드](../web-network-captures.md) |
|
||||
| .NET 어셈블리 | 메타데이터, CIL 명령, 선언된 네이티브 의존성, 빌드 비교 | 별도 도구 불필요(정적 검사만); [관리 코드 가이드](../managed-code-analysis.md) |
|
||||
| Android APK | 매니페스트 선언, 클래스, 디컴파일된 메서드, 참조 | Linux/macOS/Windows x64의 헤드리스 JADX와 전체 JDK; [Android 가이드](../android-analysis.md) |
|
||||
| Android 기기 | 기기, 패키지, 프로세스, 로그, 덤프, 화면 캡처, APK/파일 전송 | 호출자 제공 adb(에뮬레이터 또는 실제 기기); [ADB 가이드](../adb-device-analysis.md) |
|
||||
| JEB 프로젝트 | 엔진 기반 프로젝트 유닛, 아티팩트 해시, 타입/메서드 의사코드 | 호출자가 시작하고 MCP를 제공하는 JEB 클라이언트(REA_JEB_MCP_URL로 선택); [JEB 가이드](../jeb-analysis.md) |
|
||||
@@ -154,15 +158,18 @@ REA에는 Node.js 22.x(>=22.19), 24.x(>=24.11) 또는 26+와 npm이 필요합니
|
||||
| Flutter 앱 | Dart 스냅샷 해시, ABI별 라이브러리 해시, 엔진 빌드 출처 | 없음 (정적 APK 파싱); [Flutter 가이드](../flutter-analysis.md) |
|
||||
| 펌웨어 | 영역, 추출 결과, 네이티브 분석으로 전달할 정보 | Linux의 Binwalk / Unblob; [펌웨어 가이드](../firmware-analysis.md) |
|
||||
| 기록된 패킷 | 선택 필드, 반복 값, 필터 및 디코딩 규칙 | Linux: TShark 4.x + prlimit; [캡처 가이드](../packet-captures.md) |
|
||||
| 패킷 요약 | 엔드포인트, DNS, TLS SNI, 프로토콜 분포, HTTP 요청 | 호출자 제공 tshark 및 capinfos(Linux에서 검증됨); [pcap 가이드](../pcap-analysis.md) |
|
||||
| 기록된 논리 신호 | 디코더 주석, 샘플 범위, 진단 정보 | Linux: sigrok-cli 0.7.2 + prlimit; [신호 기록 가이드](../signal-recordings.md) |
|
||||
| 패키지 및 리소스 | 파일 목록, 다이제스트, plist, Apple 번들 구조, 추출한 리소스 | [아티팩트 및 JavaScript 가이드](../javascript-artifact-reconstruction.md), [Apple 애플리케이션](../apple-application-analysis.md) |
|
||||
| 프로세스 동작 | 터미널 출력, 상호작용, 종료 및 파일 시스템 관찰 결과, 실행 비교 | 네이티브 PTY를 지원하는 Linux/macOS; [프로세스 캡처](../process-capture.md) |
|
||||
|
||||
정적 JavaScript 및 .NET 검사는 제공된 파일을 읽으며 애플리케이션을 실행하지 않습니다. 런타임 캡처는 사용자의 권한으로 선택한 대상을 실행하거나 대상과 상호작용합니다. 각 런타임 가이드에서 그 영향을 설명합니다.
|
||||
|
||||
오프라인 네이티브 바이너리 검사에는 호출자가 제공하는 GNU objdump와 Rizin도 사용할 수 있습니다. 제공자 설정, CLI 명령, Evidence의 한계는 [리버스 엔지니어링 도구 가이드](../reverse-engineering-tools.md)를 참고하세요. REA는 objdump와 Rizin을 설치하거나 업데이트하지 않습니다.
|
||||
|
||||
<a id="choosing-a-deep-analysis-provider"></a>
|
||||
|
||||
지원하는 네이티브 형식과 호스트는 제공자마다 다릅니다. [Hopper 및 Ghidra 설정](../installation.md#hopper), [IDA 가이드](../ida-provider.md), [실험적 Windows Ghidra 지원](../windows-ghidra-p0.md)을 참고하세요. Ghidra는 [16비트 DOS 분석](../ghidra-dos.md)도 지원합니다. 제공자 선택은 [CLI 가이드](../cli.md#choose-a-provider)를 참고하세요. 최신 npm 릴리스 이후 추가된 기능은 [릴리스에서의 사용 가능 여부](../installation.md#released-package-and-main)를 확인하세요.
|
||||
지원하는 네이티브 형식과 호스트는 제공자마다 다릅니다. [Hopper 및 Ghidra 설정](../installation.md#hopper), [IDA 가이드](../ida-provider.md)(선택 사항인 네이티브 MCP 브리지 포함), [실험적 Windows Ghidra 지원](../windows-ghidra-p0.md)을 참고하세요. Ghidra는 [16비트 DOS 분석](../ghidra-dos.md)도 지원합니다. 큰 바이너리를 분석할 때는 `REA_GHIDRA_STARTUP_TIMEOUT_MS` 또는 `REA_HOPPER_STARTUP_TIMEOUT_MS`로 제공자의 시작 제한 시간을 늘리세요. 제공자 선택은 [CLI 가이드](../cli.md#choose-a-provider)를 참고하세요. 최신 npm 릴리스 이후 추가된 기능은 [릴리스에서의 사용 가능 여부](../installation.md#released-package-and-main)를 확인하세요.
|
||||
|
||||
## 사례
|
||||
|
||||
@@ -209,7 +216,7 @@ REA로 흥미로운 대상을 분석했다면 알려 주세요. [issue](https://
|
||||
<details>
|
||||
<summary><strong>Hopper를 먼저 실행해야 하나요?</strong></summary>
|
||||
|
||||
REA는 작업에 필요할 때 Hopper를 실행합니다. macOS에서는 처음 실행할 때 데모 모드를 선택하거나 라이선스를 활성화하라는 대화 상자가 나타날 수 있습니다. [Hopper 실행 및 문제 해결](../installation.md#launcher-paths-and-troubleshooting)을 참고하세요.
|
||||
REA는 작업에 필요할 때 Hopper를 실행합니다. macOS에서는 처음 실행할 때 데모 모드를 선택하거나 라이선스를 활성화하라는 대화 상자가 나타날 수 있습니다. 첫 CLI/MCP 작업이나 무인 작업을 실행하기 전에 같은 사용자 세션에서 이 대화 상자를 완료하세요. 필요하면 Hopper를 직접 여세요. [Hopper 실행 및 문제 해결](../installation.md#launcher-paths-and-troubleshooting)을 참고하세요.
|
||||
|
||||
</details>
|
||||
|
||||
@@ -239,7 +246,7 @@ REA는 대상을 로컬에서 분석합니다. 에이전트는 도구 결과를
|
||||
|
||||
먼저 업데이트하세요. 최근 릴리스에서 이미 수정되었을 수 있습니다.
|
||||
|
||||
npm으로 설치한 CLI에서는 다음 명령을 사용합니다.
|
||||
`npm install --global` 또는 `bun add --global`로 설치한 CLI에서는 다음 명령을 사용합니다.
|
||||
|
||||
```bash
|
||||
rea update
|
||||
@@ -286,11 +293,9 @@ REA를 사용하고, 버그를 보고하고, 기능을 요청하고, 빌드를
|
||||
|
||||
책임 있는 사용에 관한 안내는 [DISCLAIMER.md](../../DISCLAIMER.md)를 참고하세요.
|
||||
|
||||
REA는 법을 준수하는 연구, 교육, 상호운용성 확보, 소프트웨어 보존, 그리고 본인이 소유하거나 조사 권한을 가진 소프트웨어의 분석을 위한 도구입니다.
|
||||
불법적이거나 허가되지 않은 사용에 대한 프로젝트의 입장은 모든 REA 버전, 배포 채널, 플랫폼에 적용되며, 각 사본에는 해당 라이선스가 그대로 유지됩니다.
|
||||
|
||||
The project's position on illegal or unauthorized use covers all REA
|
||||
versions, distribution channels, and platforms; each copy retains its
|
||||
applicable licenses.
|
||||
REA는 법을 준수하는 연구, 교육, 상호운용성 확보, 소프트웨어 보존, 그리고 본인이 소유하거나 조사 권한을 가진 소프트웨어의 분석을 위한 도구입니다.
|
||||
|
||||
## 기여
|
||||
|
||||
@@ -302,14 +307,6 @@ REA에 도움을 보태 주세요! [issue를 등록](https://github.com/morluto/
|
||||
|
||||
[MIT](../../LICENSE)
|
||||
|
||||
REA-authored code is licensed under MIT. Third-party material retains its own
|
||||
licenses and copyright notices; see [Third-party notices](../../THIRD_PARTY_NOTICES.md)
|
||||
for the dependency inventory, preserved license texts, and external-tool terms.
|
||||
REA가 작성한 코드는 MIT 라이선스를 따릅니다. 제3자 자료에는 각자의 라이선스와 저작권 고지가 그대로 유지됩니다. 의존성 목록, 보존된 라이선스 원문, 외부 도구 약관은 [제3자 고지](../../THIRD_PARTY_NOTICES.md)를 참고하세요.
|
||||
|
||||
**You must legally obtain and be authorized to use commercial tools**, including
|
||||
IDA Pro / Hex-Rays decompilers, Hopper, and JEB, under their applicable licenses.
|
||||
REA grants no tool license, key, or activation entitlement. Trials and demos
|
||||
remain subject to vendor terms, including user, feature, automation, and output
|
||||
sharing restrictions. Cracked copies, stolen keys, unauthorized key sharing,
|
||||
and bypassing license restrictions are not authorized. This guidance does not
|
||||
amend MIT or third-party licenses.
|
||||
IDA Pro / Hex-Rays 디컴파일러, Hopper, JEB 등 상용 도구는 해당 라이선스에 따라 **합법적으로 취득하고 사용 권한을 갖춰야 합니다.** REA는 어떤 도구의 라이선스, 키, 활성화 권한도 부여하지 않습니다. 체험판과 데모는 사용자, 기능, 자동화, 출력 공유 제한을 포함한 공급업체 약관을 따릅니다. 크랙 버전, 도난 키, 무단 키 공유, 라이선스 제한 우회는 허용되지 않습니다. 이 안내는 MIT 라이선스나 제3자 라이선스를 변경하지 않습니다.
|
||||
|
||||
@@ -19,10 +19,6 @@ Follow-up: [SQLite existing-control measurements](sqlite-existing-controls.md)
|
||||
tests QGIS's shipped GeoPackage through the existing column selection, preserving
|
||||
full selected-table schema and typed cells before considering another API.
|
||||
|
||||
Follow-up: [small real PCAP recordings](pcap-small-captures.md) compares existing
|
||||
complete delivery with summary/pages. Preserve the negative SDK/model input
|
||||
results: compact captures do not justify another production mechanism.
|
||||
|
||||
Follow-up: [native PE/DOS entry views and real model verification](native-entry-views.md)
|
||||
records the post-merge native continuation, including a small-function case where
|
||||
four selected replies cost more bytes than the complete dossier.
|
||||
|
||||
@@ -180,7 +180,7 @@ def main():
|
||||
'inspect_android_package', 'search_android_classes',
|
||||
'inspect_android_class', 'inspect_android_method',
|
||||
'inspect_firmware_regions', 'inspect_managed_artifact',
|
||||
'inspect_sqlite_database', 'inspect_pcap_capture',
|
||||
'inspect_sqlite_database',
|
||||
'inspect_managed_members', 'inspect_managed_native_boundaries',
|
||||
'inspect_artifact', 'inspect_wasm_artifact',
|
||||
'inspect_evm_interface', 'get_evidence_bundle', 'export_evidence_bundle',
|
||||
|
||||
@@ -70,44 +70,6 @@ for (const [side, answer] of [
|
||||
value: answer.evidence_ids,
|
||||
});
|
||||
}
|
||||
// PCAP delivery observations preserve each actual receipt's identity, limitations
|
||||
// and view coverage. Summary/page scopes differ from complete source scope;
|
||||
// retain them separately rather than equating page exhaustion with capture
|
||||
// coverage. Repeated equivalent deliveries must also match every observation
|
||||
// field except the receipt and parent IDs, which identify their own invocations.
|
||||
if (
|
||||
["wireshark-http.cap", "wireshark-http_redirects.pcapng"].includes(
|
||||
left.artifact,
|
||||
)
|
||||
) {
|
||||
for (const [side, answer] of [
|
||||
["left", a],
|
||||
["right", b],
|
||||
]) {
|
||||
if (
|
||||
!Array.isArray(answer.delivery_observations) ||
|
||||
!answer.delivery_observations.length
|
||||
)
|
||||
throw new Error("Missing original PCAP delivery observations");
|
||||
runSpecific.push({
|
||||
side,
|
||||
path: ["delivery_observations"],
|
||||
value: answer.delivery_observations,
|
||||
});
|
||||
if (left.delivery === right.delivery) {
|
||||
answer.delivery_observations = answer.delivery_observations.map(
|
||||
(observation) =>
|
||||
Object.fromEntries(
|
||||
Object.entries(observation).filter(
|
||||
([key]) => key !== "evidence_id" && key !== "parent_evidence_id",
|
||||
),
|
||||
),
|
||||
);
|
||||
} else {
|
||||
delete answer.delivery_observations;
|
||||
}
|
||||
}
|
||||
}
|
||||
// Binwalk assigns a new opaque region UUID in each invocation. Preserve both
|
||||
// original lookup values in this report; compare the observed region facts
|
||||
// separately. The product and original replies retain those IDs unchanged.
|
||||
|
||||
@@ -263,34 +263,6 @@
|
||||
"url": "https://raw.githubusercontent.com/qgis/QGIS/e1223b8e9e71fc50ee995ed556f711c54ccc101b/resources/data/world_map.gpkg",
|
||||
"derivation": "unmodified official resource file; streaming Git blob and SHA-256 verification"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "wireshark-http.cap",
|
||||
"file": "http.cap",
|
||||
"bytes": 25803,
|
||||
"sha256": "25a72bdf10339f2c29916920c8b9501d294923108de8f29b19aba7cc001ab60d",
|
||||
"provenance": {
|
||||
"project": "Wireshark public sample captures",
|
||||
"source_page": "https://wiki.wireshark.org/SampleCaptures",
|
||||
"url": "https://wiki.wireshark.org/uploads/27707187aeb30df68e70c8fb9d614981/http.cap",
|
||||
"resolved_url": "https://gitlab.com/wireshark/wireshark/-/wikis/uploads/27707187aeb30df68e70c8fb9d614981/http.cap",
|
||||
"upstream_expected_digest": null,
|
||||
"derivation": "Unmodified capture published by official Wireshark wiki. Local digest pins downloaded bytes; no independently published upstream digest is claimed."
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "wireshark-http_redirects.pcapng",
|
||||
"file": "http_redirects.pcapng",
|
||||
"bytes": 47804,
|
||||
"sha256": "ff432a6b71d06cd20720f7763e584014edd0a43c3b55981c682d1b7640207abc",
|
||||
"provenance": {
|
||||
"project": "Wireshark public sample captures",
|
||||
"source_page": "https://wiki.wireshark.org/SampleCaptures",
|
||||
"url": "https://wiki.wireshark.org/uploads/__moin_import__/attachments/SampleCaptures/http_redirects.pcapng",
|
||||
"resolved_url": "https://gitlab.com/wireshark/wireshark/-/wikis/uploads/__moin_import__/attachments/SampleCaptures/http_redirects.pcapng",
|
||||
"upstream_expected_digest": null,
|
||||
"derivation": "Unmodified capture published by official Wireshark wiki. Local digest pins downloaded bytes; no independently published upstream digest is claimed."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -523,207 +523,7 @@ const inspectEvmTarget = async () => {
|
||||
};
|
||||
};
|
||||
|
||||
const PCAP_EVIDENCE_ID = /^ev_[a-f0-9]{64}$/u;
|
||||
const pcapObservation = (evidence, requestedView, parseEvidence) => {
|
||||
// Authenticate with the selected runtime's owner, preserving the original
|
||||
// envelope for measurement and provenance rather than reconstructing it.
|
||||
parseEvidence(evidence);
|
||||
const evidenceId = evidence.evidence_id;
|
||||
const value = normalized(evidence);
|
||||
for (const key of ["limitations", "unknowns"])
|
||||
if (
|
||||
Object.hasOwn(value, key) &&
|
||||
(!Array.isArray(value[key]) ||
|
||||
!value[key].every((item) => typeof item === "string"))
|
||||
)
|
||||
throw new Error("PCAP normalized provenance lacks a valid string list");
|
||||
const target =
|
||||
value.kind === "summary"
|
||||
? value.summary?.target
|
||||
: value.kind === "pcap"
|
||||
? value.artifact
|
||||
: value.target;
|
||||
if (
|
||||
evidence.subject?.digest?.sha256 !== inputSha ||
|
||||
evidence.subject?.local_path !== values.path ||
|
||||
target?.sha256 !== inputSha ||
|
||||
target?.path !== values.path ||
|
||||
(value.kind === "summary" &&
|
||||
(value.artifact?.sha256 !== inputSha ||
|
||||
value.artifact?.path !== values.path))
|
||||
)
|
||||
throw new Error("PCAP Evidence does not identify the selected capture");
|
||||
// Summary delivery delegates to a retained view, whose Evidence operation is
|
||||
// inspect_analysis_view even when inspect_pcap_capture was requested.
|
||||
const operation = requestedView
|
||||
? "inspect_analysis_view"
|
||||
: "inspect_pcap_capture";
|
||||
if (evidence.operation !== operation)
|
||||
throw new Error(
|
||||
"PCAP Evidence operation differs from the requested delivery",
|
||||
);
|
||||
if (
|
||||
requestedView &&
|
||||
(typeof value.parent_evidence_id !== "string" ||
|
||||
!PCAP_EVIDENCE_ID.test(value.parent_evidence_id) ||
|
||||
value.parent_evidence_id === evidenceId ||
|
||||
!Array.isArray(evidence.evidence_links) ||
|
||||
!evidence.evidence_links.includes(value.parent_evidence_id))
|
||||
)
|
||||
throw new Error("PCAP retained parent lacks a distinct Evidence link");
|
||||
if (
|
||||
requestedView &&
|
||||
value.parent_digest !== value.parent_evidence_id.slice(3)
|
||||
)
|
||||
throw new Error("PCAP retained parent digest differs from its identity");
|
||||
const parameters = requestedView
|
||||
? { parent_evidence_id: value.parent_evidence_id, view: requestedView }
|
||||
: { path: values.path };
|
||||
if (
|
||||
!isDeepStrictEqual(evidence.parameters, parameters) ||
|
||||
(requestedView && !isDeepStrictEqual(value.view, requestedView))
|
||||
)
|
||||
throw new Error("PCAP Evidence parameters or view differ from the request");
|
||||
return {
|
||||
evidence_id: evidenceId,
|
||||
limitations: evidence.limitations,
|
||||
...Object.fromEntries(
|
||||
[
|
||||
"kind",
|
||||
"view",
|
||||
"parent_evidence_id",
|
||||
"coverage",
|
||||
"limitations",
|
||||
"unknowns",
|
||||
]
|
||||
.filter((key) => Object.hasOwn(value, key) && key !== "limitations")
|
||||
.map((key) => [key, value[key]]),
|
||||
),
|
||||
...(Object.hasOwn(value, "limitations")
|
||||
? { normalized_limitations: value.limitations }
|
||||
: {}),
|
||||
};
|
||||
};
|
||||
const inspectPcapTarget = async () => {
|
||||
const { parseEvidence } = await import(
|
||||
pathToFileURL(resolve(values["runtime-root"], "dist/domain/evidence.js"))
|
||||
.href
|
||||
);
|
||||
const input = {
|
||||
path: values.path,
|
||||
...(values.delivery === "summary" ? { detail: "summary" } : {}),
|
||||
};
|
||||
const definition = catalogDefinitions.get("inspect_pcap_capture");
|
||||
const ajv = new Ajv2020({ strict: false, validateFormats: false });
|
||||
if (
|
||||
!definition ||
|
||||
!ajv.validateSchema(definition.inputSchema) ||
|
||||
!ajv.validate(definition.inputSchema, input)
|
||||
)
|
||||
throw new Error("Selected PCAP request is not advertised by this runtime");
|
||||
const inspected = await call("inspect_pcap_capture", input);
|
||||
const n = normalized(inspected);
|
||||
const observations = [
|
||||
pcapObservation(
|
||||
inspected,
|
||||
values.delivery === "summary" ? { kind: "summary" } : undefined,
|
||||
parseEvidence,
|
||||
),
|
||||
];
|
||||
let capture, uris;
|
||||
if (values.delivery === "summary") {
|
||||
if (n.kind !== "summary" || n.parent_operation !== "inspect_pcap_capture")
|
||||
throw new Error("Runtime returned no requested PCAP summary");
|
||||
if (
|
||||
!isDeepStrictEqual(n.coverage, {
|
||||
status: "complete-within-view",
|
||||
examined: 1,
|
||||
total: 1,
|
||||
next_offset: null,
|
||||
exhausted: true,
|
||||
})
|
||||
)
|
||||
throw new Error("PCAP summary coverage differs from its complete view");
|
||||
capture = n.summary;
|
||||
const parentEvidenceId = observations[0].parent_evidence_id;
|
||||
const view = { kind: "pcap", facet: "http-uris", offset: 0, limit: 2 };
|
||||
const selected = await call("inspect_analysis_view", {
|
||||
source: { kind: "retained-evidence", evidence_id: parentEvidenceId },
|
||||
view,
|
||||
});
|
||||
const page = normalized(selected);
|
||||
const observation = pcapObservation(selected, view, parseEvidence);
|
||||
const total = capture.retained_counts.http_request_uris;
|
||||
const examined = Math.min(view.limit, total);
|
||||
const expectedCoverage = {
|
||||
status: "page",
|
||||
examined,
|
||||
total,
|
||||
next_offset: examined < total ? examined : null,
|
||||
exhausted: examined >= total,
|
||||
};
|
||||
if (
|
||||
page.parent_evidence_id !== parentEvidenceId ||
|
||||
page.parent_operation !== "inspect_pcap_capture" ||
|
||||
page.kind !== "pcap" ||
|
||||
page.page.facet !== "http-uris" ||
|
||||
page.source_coverage !== capture.coverage ||
|
||||
!isDeepStrictEqual(page.coverage, expectedCoverage)
|
||||
)
|
||||
throw new Error(
|
||||
"PCAP page identity, scope or coverage differs from its parent",
|
||||
);
|
||||
uris = page.page.items;
|
||||
if (
|
||||
observations.some((item) => item.evidence_id === observation.evidence_id)
|
||||
)
|
||||
throw new Error("PCAP retained read reused an Evidence identity");
|
||||
observations.push(observation);
|
||||
} else {
|
||||
const {
|
||||
endpoints,
|
||||
dns_queries,
|
||||
tls_sni_values,
|
||||
protocol_distribution,
|
||||
http_request_uris,
|
||||
unattributed_dns_addresses,
|
||||
diagnostics,
|
||||
...metadata
|
||||
} = n;
|
||||
capture = {
|
||||
...metadata,
|
||||
retained_counts: {
|
||||
endpoints: endpoints.length,
|
||||
dns_queries: dns_queries.length,
|
||||
tls_sni_values: tls_sni_values.length,
|
||||
protocol_distribution: protocol_distribution.length,
|
||||
http_request_uris: http_request_uris.length,
|
||||
unattributed_dns_addresses: unattributed_dns_addresses.length,
|
||||
},
|
||||
diagnostics_code_units:
|
||||
diagnostics === undefined
|
||||
? null
|
||||
: {
|
||||
tshark: diagnostics.tshark_stderr.length,
|
||||
capinfos: diagnostics.capinfos_stderr.length,
|
||||
},
|
||||
};
|
||||
uris = http_request_uris.slice(0, 2);
|
||||
}
|
||||
if (uris.length !== Math.min(2, capture.retained_counts.http_request_uris))
|
||||
throw new Error("Runtime did not return the requested retained URI prefix");
|
||||
return {
|
||||
artifact_sha256: targetSha(inspected),
|
||||
capture,
|
||||
http_request_uris_first2: uris,
|
||||
delivery_observations: observations,
|
||||
evidence_ids: observations.map((item) => item.evidence_id),
|
||||
};
|
||||
};
|
||||
|
||||
const workflows = {
|
||||
"wireshark-http.cap": inspectPcapTarget,
|
||||
"wireshark-http_redirects.pcapng": inspectPcapTarget,
|
||||
"qgis-world-map.gpkg": async () => {
|
||||
const columns = ["fid", "NAME", "ISO_A3"];
|
||||
const input = {
|
||||
|
||||
@@ -1,291 +0,0 @@
|
||||
# Small real PCAP recordings: keep complete delivery
|
||||
|
||||
Continues [#2016](https://github.com/morluto/rea/issues/2016) after the QGIS
|
||||
SQLite round. Measured main is pinned to
|
||||
`3ae826ceba6f25b08bce50bcb51001209bc16b57`, with 23 intervening commits.
|
||||
These include bounded PCAP summaries/pages, authenticated artifact subjects,
|
||||
unknown-snapshot reuse and shared transcript grading. This round measures existing
|
||||
PCAP delivery choices; it adds no production, API, default, cache or skill change.
|
||||
|
||||
## Corpus and question
|
||||
|
||||
Use two unmodified recordings published on the official
|
||||
[Wireshark sample-capture page](https://wiki.wireshark.org/SampleCaptures):
|
||||
|
||||
| Recording | Bytes | SHA-256 |
|
||||
| --------------------------------------- | -----: | ------------------------------------------------------------------ |
|
||||
| HTTP request/response, `http.cap` | 25,803 | `25a72bdf10339f2c29916920c8b9501d294923108de8f29b19aba7cc001ab60d` |
|
||||
| HTTP redirects, `http_redirects.pcapng` | 47,804 | `ff432a6b71d06cd20720f7763e584014edd0a43c3b55981c682d1b7640207abc` |
|
||||
|
||||
`corpus.json` preserves original/resolved URLs. Local digests pin downloaded
|
||||
bytes; no independently published upstream digest or immutable source commit is
|
||||
claimed. These are upstream saved application-protocol recordings, not generated
|
||||
REA fixtures. Recorded clients and current deployments are unknown. No new
|
||||
traffic is captured and no recorded target/URL is executed or fetched.
|
||||
|
||||
The question asks for complete returned identity/client/scalar metadata, every
|
||||
observed count and source coverage, six retained inventory counts, explicit nulls,
|
||||
diagnostic UTF-16 lengths, and the first two retained HTTP URIs in original order.
|
||||
Preserve each used receipt's limitations and identity. Summary/page scope and
|
||||
unknowns remain distinct from source coverage; exhaustion does not establish
|
||||
network completeness. Other inventory lists and diagnostic text are unrequested.
|
||||
|
||||
Compare default `{path}` with existing `{path, detail: "summary"}` plus a retained
|
||||
`http-uris` page, offset 0, limit 2. Ordinary selected-field
|
||||
`inspect_packet_capture` remains a separate boundary, not a substitute claim
|
||||
about complete inventories. `questions/pcap-http.txt` contains the fixed question.
|
||||
|
||||
## SDK measurements
|
||||
|
||||
Frozen dist/bridge file-table SHA-256:
|
||||
`db050545e94dc305a48042e06df96ac6c64b21ee6c5c4707c787d5e46df79e47`.
|
||||
Linux x64, Node 24.18.0, SDK client/server 2.3.1, installed Wireshark 4.0.17.
|
||||
Explicit commands are `/usr/bin/tshark`, `/usr/bin/capinfos`, `/usr/bin/prlimit`;
|
||||
private receipts retain executable hashes. Each workflow starts a fresh server
|
||||
and uses actual named advertised definitions/schema validation. Host page caches
|
||||
are not flushed. Both tools read the owned snapshot.
|
||||
|
||||
Eight trials run complete/summary/summary/complete per recording. Both pairs
|
||||
match selected capture facts, URI prefixes, settings and original source bytes.
|
||||
Each delivery's exact limitations, identities and coverage remain separately
|
||||
recorded, rather than equating source coverage with page exhaustion.
|
||||
|
||||
| Trial | Reply bytes | SDK seconds | Payload CPU seconds | Kernel peak bytes |
|
||||
| -------------------- | ----------: | ----------: | ------------------: | ----------------: |
|
||||
| HTTP complete-1 | 7,601 | 4.349 | 5.336 | 611,778,560 |
|
||||
| HTTP summary-1 | 17,540 | 4.972 | 5.973 | 601,661,440 |
|
||||
| HTTP summary-2 | 17,540 | 5.101 | 6.092 | 611,438,592 |
|
||||
| HTTP complete-2 | 7,601 | 4.463 | 5.503 | 617,148,416 |
|
||||
| Redirects complete-1 | 7,535 | 4.670 | 5.641 | 610,627,584 |
|
||||
| Redirects summary-1 | 17,240 | 5.099 | 6.104 | 615,522,304 |
|
||||
| Redirects summary-2 | 17,240 | 5.060 | 6.056 | 610,385,920 |
|
||||
| Redirects complete-2 | 7,535 | 4.466 | 5.458 | 608,141,312 |
|
||||
|
||||
Bytes are compact client-result JSON summed over one complete or two summary/page
|
||||
calls, not model tokens or raw wire bytes. Summary/page expands replies to more
|
||||
than twice the complete result on both recordings. All four SDK pairs are slower
|
||||
and use more payload CPU; memory differences are small/inconsistent. CPU includes
|
||||
driver/catalog work outside the SDK workflow timer.
|
||||
|
||||
Reproduce with `measure.mjs --artifact wireshark-http.cap` or
|
||||
`wireshark-http_redirects.pcapng`, pinned path/runtime/revision, explicit provider
|
||||
JSON, `--schemas selected`, and `--delivery complete|summary`. Use `compare.mjs`
|
||||
on completed output directories. Original replies, catalogs, comparison receipts
|
||||
and exact measured driver source stay private. Repeated deliveries in the same
|
||||
mode also compare the ordered receipt observations, including limitations, view
|
||||
parameters, coverage and unknowns; only receipt/parent IDs are invocation-specific.
|
||||
Comparator regressions run with `npm run test:local -- tests/evaluation/analysisEfficiencyComparison.test.ts`
|
||||
using synthetic measurement directories; they do not establish provider behavior.
|
||||
|
||||
## Actual model tasks
|
||||
|
||||
Four actual Codex CLI 0.161.0 tasks use requested `gpt-5.6-terra` high, the same
|
||||
redirects question/runtime/artifact/provider settings, direct mode, two exact
|
||||
enabled tools and equal context/discovery capture flags. The requested alias is
|
||||
not verified server-resolved model identity. Equal cohort instructions select
|
||||
complete or summary plus one URI page. The experiment driver applies its existing
|
||||
private per-tool approval policy to `inspect_pcap_capture`; global configuration
|
||||
and REA authority are unchanged.
|
||||
|
||||
| Trial | Input | Cached | Uncached | Output | CLI wall seconds | MCP calls | Facts |
|
||||
| ---------- | -----: | -----: | -------: | -----: | ---------------: | --------: | ----- |
|
||||
| complete-1 | 38,947 | 9,984 | 28,963 | 2,119 | 69.469 | 1 | pass |
|
||||
| summary-1 | 58,779 | 38,144 | 20,635 | 3,241 | 141.304 | 2 | pass |
|
||||
| summary-2 | 58,584 | 38,144 | 20,440 | 4,261 | 50.916 | 2 | pass |
|
||||
| complete-2 | 38,917 | 23,040 | 15,877 | 2,813 | 106.896 | 1 | pass |
|
||||
|
||||
Summary/page total input **increases 50.92% / 50.54%** and output increases
|
||||
52.95% / 51.48%. Uncached input is mixed: 28.75% lower / 28.74% higher. Time is
|
||||
mixed: 103.41% slower / 52.37% faster. No cause is inferred for caching or timing
|
||||
variation. Cached input is included in total; each CLI reports one final aggregate
|
||||
usage. Output includes 1,149 / 1,162 / 2,196 / 1,837 reasoning tokens.
|
||||
|
||||
All calls succeed without extras/retries. Events contain only the selected MCP
|
||||
tools and agent messages. All source-digest and private-profile cleanup checks
|
||||
pass. Model payload CPU seconds/kernel peaks: 7.818/791,060,480;
|
||||
9.793/752,705,536; 7.685/757,891,072; 7.999/750,993,408. These do not establish
|
||||
a general CPU/memory gain.
|
||||
|
||||
The first summary initially fails the grader because it uses explicit equivalent
|
||||
labels and a URI object retaining page/source coverage. Its facts are correct.
|
||||
The grading boundary accepts these known forms while checking exact types,
|
||||
null presence, original limitations and extra scopes against actual receipts.
|
||||
Conflicting duplicates and unsupported claims remain failures. Each cited
|
||||
receipt's own ID must be reported; its parent lookup ID is checked separately.
|
||||
Original answers/reviews/grader source remain intact. No model is rerun. A grader
|
||||
review pause separates the pairs; it is not task time or a cold-cache claim.
|
||||
|
||||
## Verification and decision
|
||||
|
||||
An independent stdlib raw PCAP/PCAPNG/Ethernet/IP/TCP/complete-HTTP-header parser
|
||||
agrees on 43/271 packets, per-packet endpoint counts, first URI prefixes and 2/23
|
||||
distinct HTTP URIs. It invokes no Wireshark. DNS associations, protocol-stack
|
||||
dissection, nullable timestamps and live-network semantics are not independently
|
||||
validated.
|
||||
|
||||
Both real CLI complete results match full MCP normalized results, subjects,
|
||||
parameters, IDs and limitations. Portable CLI summary/URI views match MCP retained
|
||||
normalized facts and subjects; source forms/caller parameters remain distinct.
|
||||
Source bytes stay unchanged. Thirty-four context/grading regressions pass, including
|
||||
cited complete metadata/counts/UTF-16 diagnostics and exact ordered response ID
|
||||
lists that reject parent lookups, duplicates and nested representations. URI
|
||||
prefix coverage must come from its cited page, even if an uncited page has the
|
||||
same items. The SDK driver also checks the returned view against the exact
|
||||
requested kind, facet, offset and limit. Five real-answer
|
||||
mutations—omitted null, false page exhaustion, contradictory source scope,
|
||||
unsupported absence claim and non-string receipt identity—fail correctly.
|
||||
Malformed identities produce failed facts rather than crashing the grader.
|
||||
JS syntax, Python compilation and
|
||||
relevant standard lint pass.
|
||||
|
||||
A fresh real SDK summary/URI smoke matches the original selected facts. Three
|
||||
private consumer fault injections alter the returned view's offset, limit or
|
||||
presence after a real SDK reply; the driver rejects each. These establish the
|
||||
experiment consumer's guard, not provider malformed-reply behavior. The initial
|
||||
injection setup loaded the separate CommonJS client and did not inject a fault;
|
||||
that discarded setup attempt remains private alongside the corrected ESM runs.
|
||||
Original measured driver sources, token totals and timing data remain unchanged.
|
||||
|
||||
The driver also requires a distinct Evidence receipt for the retained URI page.
|
||||
A private SDK consumer injection reuses the summary's response ID after a real
|
||||
MCP reply: the preceding driver accepts it, the corrected driver rejects it
|
||||
without writing an answer, and an unmodified real reply still passes. This
|
||||
checks receipt uniqueness before recording observations; it does not establish
|
||||
that a provider actually returned duplicate IDs.
|
||||
|
||||
The page guard checks the complete expected coverage object, including status,
|
||||
continuation and exhaustion. Private actual-reply injections demonstrate that
|
||||
the preceding driver accepts inconsistent coverage, while the corrected driver
|
||||
rejects wrong status, next offset or exhaustion. Unmodified exhausted and
|
||||
continuable URI pages still pass. The grader also requires each cited receipt's
|
||||
subject digest and local path to identify the reference capture, and uses the
|
||||
reference page's verified coverage. Thirty-four grading/context regressions pass,
|
||||
including missing or malformed subjects and faithful but inconsistent coverage;
|
||||
all four original model answers still pass factual regrading. Strict resource
|
||||
grades remain false where terminal whole-scope counters are unknown.
|
||||
|
||||
PCAP observation production now validates both the Evidence subject and
|
||||
normalized target against the selected path/digest for complete, summary and
|
||||
page replies. Retained parents must be declared in each view receipt's Evidence
|
||||
links and differ from that response's own ID. The grader enforces the same link
|
||||
relationship. Private actual-reply injections and normal complete/summary
|
||||
controls exercise this boundary; these do not establish a provider defect.
|
||||
|
||||
The grader preserves each receipt's producing tool while indexing it; conflicting
|
||||
tool/envelope ownership for one ID remains ambiguous, while identical repeated
|
||||
receipts are allowed. Complete delivery requires the capture tool/operation.
|
||||
Summary delivery uses the capture tool but correctly delegates its Evidence
|
||||
operation to `inspect_analysis_view`; the URI page uses that view tool/operation.
|
||||
Both the grader and SDK observation producer validate this actual contract.
|
||||
Wrong-tool/operation regressions fail before the correction and pass afterward;
|
||||
private SDK operation faults are rejected while real complete/summary controls
|
||||
and all four stored model factual grades still pass. No operation is rewritten
|
||||
to make the tool name and delegated Evidence operation identical.
|
||||
|
||||
The receipt index parses the required envelope/nested-view containers before
|
||||
allowing lookup, so missing or malformed normalized results/limitations become
|
||||
failed facts with a written report. Empty observation lists also remain failed
|
||||
facts and never enter citation checks that require a first receipt. Coverage uses type-sensitive JSON comparison
|
||||
to distinguish booleans, integers and floats. Receipt-identity conflict checks
|
||||
use the same type-sensitive rule, so a scalar type change cannot hide in a
|
||||
repeated ID. Existing PCAP scope-alias checks already preserve JSON types.
|
||||
Requested and cited URI views use the same type-sensitive JSON rule, rejecting
|
||||
boolean or floating-point offsets/limits even when an answer faithfully copies
|
||||
them. The observation producer rejects missing, null or malformed response IDs
|
||||
before recording either capture or page receipts. Private real-reply injections
|
||||
exercise these ID failures alongside normal complete/summary controls; they do
|
||||
not establish malformed provider behavior.
|
||||
Completed-call arguments stay with their producing tool and receipt in the
|
||||
grader's identity index. Cited envelope parameters and actual call arguments
|
||||
must match the selected path or retained parent/view, including delegated
|
||||
summary parameters. The observation producer validates both summary and URI
|
||||
views and envelope parameters at their common boundary. Private real-reply
|
||||
faults reproduce missing-parameter/summary-view acceptance before the fix and
|
||||
are rejected afterward; normal complete/summary controls still pass.
|
||||
Negative regressions reproduce
|
||||
prior crashes and false passes; all original real-model factual grades still
|
||||
pass without model reruns. The SDK workflow is unchanged by these grader fixes.
|
||||
|
||||
**Decision: keep existing complete defaults; add no production optimization.**
|
||||
For these compact recordings and this question, summary/page costs more total
|
||||
input in both real-model pairs. No stable speed or CPU/memory benefit justifies
|
||||
another API, automatic delivery switch, cache or plugin layer. Existing pages
|
||||
remain available; their benefit needs evidence for the actual question/inventory.
|
||||
Large captures, other protocols, Windows/macOS/Bun Wireshark and other model
|
||||
workloads remain unverified here.
|
||||
|
||||
Heavy work stays serial with one CPU, 3 GiB early stop/4 GiB hard memory and no
|
||||
swap. Build kernel peak: 2,575,732,736 bytes. Payload cleanup and zero phase limit
|
||||
events are verified; missing terminal whole-scope counters remain unknown, so
|
||||
strict whole-resource grades stay false. No raw capture, trace, credential,
|
||||
runtime or result JSON is committed. No unrelated REA product bug is identified
|
||||
or fixed, and no release is performed.
|
||||
|
||||
Review follow-up binds the cited URI page to its exact requested facet/range,
|
||||
reported values and the summary retained parent. Small synthetic trial receipts
|
||||
exercise the entire grading entrypoint, including faithful but wrong-page/range/
|
||||
parent citations, missing nulls, false scope and malformed identities. These are
|
||||
grader regressions, not real-provider evidence. All four stored actual model
|
||||
answers still pass; no model reruns are required. Transcript readers close their
|
||||
owned file handles after grading.
|
||||
|
||||
The SDK driver and factual grader require summary coverage to describe its one
|
||||
complete view and require each present normalized limitations/unknowns field to
|
||||
be a string array. Missing fields and valid empty arrays remain distinct.
|
||||
Controlled consumer mutations over actual SDK replies reproduce the previously
|
||||
accepted inconsistent summary coverage and verify rejection after the guard;
|
||||
complete and summary controls still pass. These mutations do not establish a
|
||||
malformed-response defect in the provider. The four stored model answers still
|
||||
pass factual regrading without another model run.
|
||||
|
||||
Summary views must preserve both the outer artifact path/digest and the nested
|
||||
summary target, as well as the Evidence subject. The full grader rejects
|
||||
faithful receipts with missing or incorrect outer artifact identities. Private
|
||||
mutations over actual SDK replies reproduce the previous false success and
|
||||
verify rejection after the shared observation guard; normal deliveries and all
|
||||
four original model answers retain their recorded facts.
|
||||
|
||||
Declared retained parent digests must match the digest encoded in the linked
|
||||
parent Evidence ID for both summary and page views. SDK acceptance and factual
|
||||
grading reject mismatches and missing digests while preserving complete
|
||||
deliveries. Before/after private actual-reply mutations verify both view roles;
|
||||
unmodified deliveries and four stored original-model factual regrades pass.
|
||||
|
||||
The SDK observation boundary additionally authenticates each complete, summary
|
||||
and page envelope with the selected runtime’s existing `parseEvidence` owner.
|
||||
It preserves the original envelope, without duplicating semantic-ID hashing.
|
||||
Format-valid swapped-ID mutations over actual SDK replies pass before this
|
||||
guard and fail afterward; unmodified complete/summary controls pass. The model
|
||||
grader tests selected facts against cited transcript receipts; its synthetic
|
||||
fixtures do not establish Evidence authentication. The original real-model
|
||||
receipts are independently authenticated against the pinned runtime.
|
||||
|
||||
The grader separates truthful cited facts from equivalence of the measured
|
||||
workflow. PCAP workflow checks require the planned ordered capture call (complete)
|
||||
or summary capture followed by the URI-page call, with exact arguments and
|
||||
completed status. Extra, reversed, failed or undecodable attempts fail workflow
|
||||
acceptance even when the final cited facts are correct; their actual cost and
|
||||
errors remain recorded. All four original model trials pass both factual and
|
||||
workflow checks without model reruns. Strict resource grades remain unchanged.
|
||||
|
||||
Each planned call must also decode as an Evidence object with its own cited
|
||||
response ID. An unreadable planned reply cannot independently pass workflow
|
||||
checks merely because tool names, arguments and terminal status match. The
|
||||
regression reproduces the previous workflow-only false success; original model
|
||||
factual/workflow grades remain unchanged.
|
||||
|
||||
Workflow citation cardinality must equal the planned call count before binding
|
||||
per-call IDs. A shorter or longer answer list cannot silently shorten Python
|
||||
zip’s expected plan; missing URI calls and extra citations are rejected.
|
||||
Original complete/summary model workflows continue to pass unchanged.
|
||||
|
||||
The six comparator CLI regressions now live in the normally discovered Vitest
|
||||
evaluation suite, so routine local/PR/CI gates execute them. The original six
|
||||
scenarios and CLI assertions are retained; the former manual Node suite is
|
||||
removed. They remain synthetic consumer checks, not provider evidence.
|
||||
|
||||
Failed calls with scalar/list replies or malformed nested resource-error
|
||||
containers are parsed defensively. The grader writes its report, preserves the
|
||||
original failure and any supported cited facts, and rejects workflow acceptance
|
||||
instead of crashing. Genuine structured resource Evidence references remain
|
||||
observable; no malformed producer behavior is inferred from synthetic cases.
|
||||
@@ -1 +0,0 @@
|
||||
Statically inspect the supplied saved Wireshark capture at {{artifact_path}} using only registered REA inspect_pcap_capture and inspect_analysis_view. Report its original client/target identity and every returned scalar metadata/count/coverage field exactly, preserving explicit nulls. Include retained_counts for endpoints, dns_queries, tls_sni_values, protocol_distribution, http_request_uris and unattributed_dns_addresses; derive array lengths only when reading complete Evidence. Include diagnostics_code_units as exact UTF-16 lengths of retained capinfos/tshark stderr, or null if diagnostics are unrecorded. In capture, omit inventory arrays and diagnostic text; do not omit their observed counts or change source coverage. Report the first two retained HTTP URIs in original order, or all if fewer; do not request other inventory pages. Report delivery_observations in dependency order for the Evidence used: evidence_id, outer limitations, and each present normalized kind, view, parent_evidence_id, coverage, unknowns and normalized_limitations. Preserve presence, explicit nulls, scalar types and original limitation order. Distinguish source coverage from URI-page exhaustion. Final JSON fields: artifact_sha256, capture, http_request_uris_first2, delivery_observations, evidence_ids. Include only actual response IDs. Saved packet/URI observations do not establish current deployment, process identity, missing traffic, decrypted payloads or current DNS resolution. Do not use shell, host file tools, network, other agents, traffic acquisition or execute recorded targets. Stop once these selected facts are supported.
|
||||
@@ -67,47 +67,9 @@ def scope_with_verified_counts(value, expected_scope, returned_counts):
|
||||
return scope, checks
|
||||
|
||||
|
||||
def pcap_observation_fields(value):
|
||||
"""Equivalent explicit labels are accepted; conflicting duplicates remain failures."""
|
||||
if not isinstance(value, dict):
|
||||
return value
|
||||
result = dict(value)
|
||||
for label, canonical in {
|
||||
'outer_limitations': 'limitations', 'normalized_kind': 'kind',
|
||||
'normalized_view': 'view', 'normalized_parent_evidence_id': 'parent_evidence_id',
|
||||
'normalized_coverage': 'coverage', 'normalized_unknowns': 'unknowns',
|
||||
'normalized_source_coverage': 'source_coverage',
|
||||
}.items():
|
||||
if label not in result:
|
||||
continue
|
||||
if canonical in result and json.dumps(result[label], sort_keys=True) != json.dumps(result[canonical], sort_keys=True):
|
||||
return {**result, 'conflicting_fields': [label, canonical]}
|
||||
result[canonical] = result.pop(label)
|
||||
return result
|
||||
|
||||
|
||||
def pcap_complete_capture(value):
|
||||
"""Project the selected metadata/counts from the cited complete receipt."""
|
||||
inventories = ['endpoints', 'dns_queries', 'tls_sni_values', 'protocol_distribution',
|
||||
'http_request_uris', 'unattributed_dns_addresses']
|
||||
if not isinstance(value, dict) or not all(isinstance(value.get(key), list) for key in inventories):
|
||||
return None
|
||||
capture = {key: item for key, item in value.items() if key not in inventories + ['diagnostics']}
|
||||
capture['retained_counts'] = {key: len(value[key]) for key in inventories}
|
||||
diagnostics = value.get('diagnostics')
|
||||
if 'diagnostics' not in value:
|
||||
capture['diagnostics_code_units'] = None
|
||||
elif isinstance(diagnostics, dict) and all(isinstance(diagnostics.get(key + '_stderr'), str) for key in ['tshark', 'capinfos']):
|
||||
capture['diagnostics_code_units'] = {key: len(diagnostics[key + '_stderr'].encode('utf-16-le', errors='surrogatepass')) // 2
|
||||
for key in ['tshark', 'capinfos']}
|
||||
else:
|
||||
return None
|
||||
return capture
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument('--domain', choices=['wasm', 'evm', 'android', 'flutter-aot', 'sqlite', 'pcap', 'managed', 'managed-nonempty', 'firmware', 'inventory', 'javascript', 'native', 'browser'], required=True)
|
||||
parser.add_argument('--domain', choices=['wasm', 'evm', 'android', 'flutter-aot', 'sqlite', 'managed', 'managed-nonempty', 'firmware', 'inventory', 'javascript', 'native', 'browser'], required=True)
|
||||
parser.add_argument('--trial', type=Path, required=True)
|
||||
parser.add_argument('--reference', type=Path)
|
||||
parser.add_argument('--resource-log', type=Path, required=True)
|
||||
@@ -121,43 +83,25 @@ def main():
|
||||
reference = read(args.reference / 'answer.json') if args.reference else None
|
||||
observed_ids = set()
|
||||
producer_results = {}
|
||||
pcap_calls = []
|
||||
pcap_attempts = []
|
||||
workflow_checks = {'pcap_exact_call_sequence': False} if args.domain == 'pcap' else {}
|
||||
tool_errors = []
|
||||
with (args.trial / 'events.jsonl').open() as event_lines:
|
||||
for line in event_lines:
|
||||
event = json.loads(line)
|
||||
item = event.get('item', {})
|
||||
if event.get('type') != 'item.completed' or item.get('type') != 'mcp_tool_call':
|
||||
continue
|
||||
result = item.get('result') or {}
|
||||
parsed = decode(result)
|
||||
if args.domain == 'pcap':
|
||||
pcap_attempts.append({'tool': item.get('tool'), 'arguments': item.get('arguments'),
|
||||
'status': item.get('status'),
|
||||
'evidence_id': parsed.get('evidence_id') if isinstance(parsed, dict) else None,
|
||||
'readable_evidence': isinstance(parsed, dict)
|
||||
and isinstance(parsed.get('normalized_result'), dict)
|
||||
and isinstance(parsed.get('evidence_id'), str)
|
||||
and re.fullmatch(r'ev_[a-f0-9]{64}', parsed['evidence_id']) is not None})
|
||||
if item['status'] == 'completed':
|
||||
# Only response text contributes observed IDs, never model arguments.
|
||||
for text in strings(result):
|
||||
observed_ids.update(re.findall(r'ev_[a-f0-9]{64}', text))
|
||||
if parsed is not None:
|
||||
if args.domain == 'pcap':
|
||||
pcap_calls.append((item['tool'], item.get('arguments'), parsed))
|
||||
else:
|
||||
producer_results.setdefault(item['tool'], []).append(parsed)
|
||||
else:
|
||||
tool_errors.append({'tool': item['tool'], 'arguments': item['arguments'], 'response': parsed if parsed is not None else result})
|
||||
error = parsed.get('error') if isinstance(parsed, dict) else None
|
||||
if isinstance(error, dict) and error.get('code') == 'resource_constraint':
|
||||
details = error.get('details')
|
||||
limits = details.get('reported_limits') if isinstance(details, dict) else None
|
||||
if isinstance(limits, dict):
|
||||
observed_ids.update(evidence_ids(limits.get('evidence_reference')))
|
||||
for line in (args.trial / 'events.jsonl').open():
|
||||
event = json.loads(line)
|
||||
item = event.get('item', {})
|
||||
if event.get('type') != 'item.completed' or item.get('type') != 'mcp_tool_call':
|
||||
continue
|
||||
result = item.get('result') or {}
|
||||
parsed = decode(result)
|
||||
if item['status'] == 'completed':
|
||||
# Only response text contributes observed IDs, never model arguments.
|
||||
for text in strings(result):
|
||||
observed_ids.update(re.findall(r'ev_[a-f0-9]{64}', text))
|
||||
if parsed is not None:
|
||||
producer_results.setdefault(item['tool'], []).append(parsed)
|
||||
else:
|
||||
tool_errors.append({'tool': item['tool'], 'arguments': item['arguments'], 'response': parsed if parsed is not None else result})
|
||||
if parsed is not None and parsed.get('error', {}).get('code') == 'resource_constraint':
|
||||
reference_value = parsed['error'].get('details', {}).get('reported_limits', {}).get('evidence_reference')
|
||||
observed_ids.update(evidence_ids(reference_value))
|
||||
reported_ids = evidence_ids(answer.get('evidence_ids'))
|
||||
checks = {
|
||||
'reported_evidence_ids_observed_in_responses': bool(reported_ids) and reported_ids <= observed_ids,
|
||||
@@ -194,187 +138,6 @@ def main():
|
||||
# Missing is not SQL/JSON null; booleans are not numeric counters.
|
||||
check(key, (key in answer, json.dumps(answer.get(key), sort_keys=True)),
|
||||
(True, json.dumps(reference[key], sort_keys=True)))
|
||||
elif domain == 'pcap':
|
||||
check('capture', ('capture' in answer, json.dumps(answer.get('capture'), sort_keys=True)),
|
||||
(True, json.dumps(reference['capture'], sort_keys=True)))
|
||||
prefix = answer.get('http_request_uris_first2')
|
||||
records = {}
|
||||
for tool, arguments, value in pcap_calls:
|
||||
if not isinstance(value, dict) or not isinstance(value.get('evidence_id'), str):
|
||||
continue
|
||||
identity = value['evidence_id']
|
||||
normalized = value.get('normalized_result')
|
||||
supported = (isinstance(normalized, dict) and isinstance(value.get('limitations'), list)
|
||||
and all(isinstance(item, str) for item in value['limitations']))
|
||||
if supported:
|
||||
supported = all(key not in normalized or (isinstance(normalized[key], list)
|
||||
and all(isinstance(item, str) for item in normalized[key]))
|
||||
for key in ['limitations', 'unknowns'])
|
||||
if supported and ('summary' in normalized or normalized.get('kind') == 'summary'):
|
||||
supported = isinstance(normalized.get('summary'), dict)
|
||||
if supported and ('page' in normalized or normalized.get('kind') == 'pcap'):
|
||||
supported = isinstance(normalized.get('page'), dict)
|
||||
entry = (tool, arguments, value) if supported else None
|
||||
if identity not in records:
|
||||
records[identity] = entry
|
||||
elif json.dumps(records[identity], sort_keys=True) != json.dumps(entry, sort_keys=True):
|
||||
# An ID with conflicting producers or envelopes is ambiguous.
|
||||
records[identity] = None
|
||||
observations = answer.get('delivery_observations')
|
||||
canonical_observations = [pcap_observation_fields(value) for value in observations] if isinstance(observations, list) else observations
|
||||
if isinstance(prefix, dict):
|
||||
checks['pcap_prefix_fields_supported'] = set(prefix) <= {'items', 'coverage', 'source_coverage'} and 'items' in prefix
|
||||
if 'source_coverage' in prefix:
|
||||
check('pcap_prefix_source_coverage', prefix['source_coverage'], reference['capture']['coverage'])
|
||||
if 'coverage' in prefix:
|
||||
last = canonical_observations[-1] if isinstance(canonical_observations, list) and canonical_observations else None
|
||||
identity = last.get('evidence_id') if isinstance(last, dict) else None
|
||||
entry = records.get(identity) if isinstance(identity, str) else None
|
||||
record = entry[2] if entry is not None else None
|
||||
page = record['normalized_result'] if record is not None else {}
|
||||
checks['pcap_prefix_coverage_observed'] = (
|
||||
page.get('kind') == 'pcap'
|
||||
and page.get('page', {}).get('facet') == 'http-uris'
|
||||
and json.dumps(page['page'].get('items'), sort_keys=True) == json.dumps(prefix.get('items'), sort_keys=True)
|
||||
and 'coverage' in page
|
||||
and json.dumps(prefix['coverage'], sort_keys=True) == json.dumps(page['coverage'], sort_keys=True)
|
||||
)
|
||||
prefix = prefix.get('items')
|
||||
check('http_request_uris_first2', ('http_request_uris_first2' in answer, json.dumps(prefix, sort_keys=True)),
|
||||
(True, json.dumps(reference['http_request_uris_first2'], sort_keys=True)))
|
||||
expected_observations = []
|
||||
if isinstance(canonical_observations, list):
|
||||
for observation in canonical_observations:
|
||||
identity = observation.get('evidence_id') if isinstance(observation, dict) else None
|
||||
entry = records.get(identity) if isinstance(identity, str) else None
|
||||
record = entry[2] if entry is not None else None
|
||||
if record is None:
|
||||
expected_observations.append(None)
|
||||
continue
|
||||
value = record['normalized_result']
|
||||
expected = {'evidence_id': record['evidence_id'], 'limitations': record['limitations']}
|
||||
for key in ['kind', 'view', 'parent_evidence_id', 'coverage', 'unknowns']:
|
||||
if key in value:
|
||||
expected[key] = value[key]
|
||||
if 'limitations' in value:
|
||||
expected['normalized_limitations'] = value['limitations']
|
||||
if 'source_coverage' in observation and 'source_coverage' in value:
|
||||
expected['source_coverage'] = value['source_coverage']
|
||||
expected_observations.append(expected)
|
||||
checks['pcap_delivery_observations_nonempty'] = isinstance(observations, list) and bool(observations)
|
||||
check('pcap_original_delivery_observations', json.dumps(canonical_observations, sort_keys=True),
|
||||
json.dumps(expected_observations, sort_keys=True))
|
||||
if isinstance(canonical_observations, list) and canonical_observations and all(isinstance(x, dict) for x in canonical_observations):
|
||||
check('pcap_requested_delivery_chain', [x.get('kind', 'complete') for x in canonical_observations],
|
||||
[x.get('kind', 'complete') for x in reference['delivery_observations']])
|
||||
check('pcap_requested_views', json.dumps([x.get('view') for x in canonical_observations], sort_keys=True),
|
||||
json.dumps([x.get('view') for x in reference['delivery_observations']], sort_keys=True))
|
||||
if len(reference['delivery_observations']) == 2:
|
||||
check('pcap_summary_coverage', json.dumps(canonical_observations[0].get('coverage'), sort_keys=True),
|
||||
json.dumps({'status': 'complete-within-view', 'examined': 1, 'total': 1,
|
||||
'next_offset': None, 'exhausted': True}, sort_keys=True))
|
||||
identities = [x.get('evidence_id') for x in canonical_observations]
|
||||
checks['pcap_observations_have_reported_ids'] = (
|
||||
isinstance(answer.get('evidence_ids'), list)
|
||||
and answer['evidence_ids'] == identities
|
||||
and all(isinstance(value, str) and re.fullmatch(r'ev_[a-f0-9]{64}', value) for value in identities)
|
||||
and len(set(identities)) == len(identities)
|
||||
)
|
||||
entries = [records.get(identity) if isinstance(identity, str) else None for identity in identities]
|
||||
cited = [entry[2] if entry is not None else None for entry in entries]
|
||||
expected_tools = ['inspect_pcap_capture'] + (['inspect_analysis_view'] if len(reference['delivery_observations']) == 2 else [])
|
||||
check('pcap_cited_producer_tools', [entry[0] if entry is not None else None for entry in entries], expected_tools)
|
||||
# Summary delivery delegates to a retained view; its envelope operation
|
||||
# is inspect_analysis_view even though the requested tool is inspect_pcap_capture.
|
||||
expected_operations = ['inspect_pcap_capture'] if len(expected_tools) == 1 else ['inspect_analysis_view'] * 2
|
||||
check('pcap_cited_operations', [record.get('operation') if isinstance(record, dict) else None for record in cited], expected_operations)
|
||||
path = reference['capture']['target']['path']
|
||||
expected_arguments = [{'path': path}]
|
||||
expected_parameters = [{'path': path}]
|
||||
if len(expected_tools) == 2:
|
||||
parent = cited[0]['normalized_result'].get('parent_evidence_id') if isinstance(cited[0], dict) else None
|
||||
views = [observation.get('view') for observation in reference['delivery_observations']]
|
||||
expected_arguments = [{'path': path, 'detail': 'summary'},
|
||||
{'source': {'kind': 'retained-evidence', 'evidence_id': parent}, 'view': views[-1]}]
|
||||
expected_parameters = [{'parent_evidence_id': parent, 'view': view} for view in views]
|
||||
check('pcap_cited_call_arguments', json.dumps([entry[1] if entry is not None else None for entry in entries], sort_keys=True),
|
||||
json.dumps(expected_arguments, sort_keys=True))
|
||||
workflow_checks['pcap_exact_call_sequence'] = (
|
||||
len(identities) == len(expected_tools)
|
||||
and json.dumps(pcap_attempts, sort_keys=True) == json.dumps([
|
||||
{'tool': tool, 'arguments': arguments, 'status': 'completed',
|
||||
'evidence_id': identity, 'readable_evidence': True}
|
||||
for tool, arguments, identity in zip(expected_tools, expected_arguments, identities)
|
||||
], sort_keys=True)
|
||||
)
|
||||
check('pcap_cited_parameters', json.dumps([record.get('parameters') if isinstance(record, dict) else None for record in cited], sort_keys=True),
|
||||
json.dumps(expected_parameters, sort_keys=True))
|
||||
checks['pcap_cited_subjects_match_artifact'] = bool(cited) and all(
|
||||
isinstance(record, dict)
|
||||
and isinstance(record.get('subject'), dict)
|
||||
and isinstance(record['subject'].get('digest'), dict)
|
||||
and record['subject']['digest'].get('sha256') == reference['artifact_sha256']
|
||||
and record['subject'].get('local_path') == reference['capture']['target']['path']
|
||||
for record in cited
|
||||
)
|
||||
checks['pcap_cited_parents_linked'] = bool(cited) and all(isinstance(record, dict) for record in cited) and (
|
||||
len(cited) == 1 or (len(cited) == 2 and all(
|
||||
isinstance(record['normalized_result'].get('parent_evidence_id'), str)
|
||||
and re.fullmatch(r'ev_[a-f0-9]{64}', record['normalized_result']['parent_evidence_id'])
|
||||
and record['normalized_result'].get('parent_digest') == record['normalized_result']['parent_evidence_id'][3:]
|
||||
and record['normalized_result']['parent_evidence_id'] != record['evidence_id']
|
||||
and isinstance(record.get('evidence_links'), list)
|
||||
and record['normalized_result']['parent_evidence_id'] in record['evidence_links']
|
||||
for record in cited
|
||||
))
|
||||
)
|
||||
checks['pcap_cited_source_supports_uri_prefix'] = False
|
||||
checks['pcap_cited_capture'] = False
|
||||
if all(isinstance(record, dict) for record in cited):
|
||||
if len(cited) == 1:
|
||||
source = cited[0]['normalized_result']
|
||||
check('pcap_cited_capture', json.dumps(answer.get('capture'), sort_keys=True),
|
||||
json.dumps(pcap_complete_capture(source), sort_keys=True))
|
||||
checks['pcap_cited_source_supports_uri_prefix'] = (
|
||||
source.get('target') == reference['capture']['target']
|
||||
and isinstance(source.get('http_request_uris'), list)
|
||||
and json.dumps(source['http_request_uris'][:2], sort_keys=True) == json.dumps(prefix, sort_keys=True)
|
||||
)
|
||||
elif len(cited) == 2:
|
||||
source, page = [record['normalized_result'] for record in cited]
|
||||
capture = source.get('summary', {})
|
||||
check('pcap_cited_capture', json.dumps(answer.get('capture'), sort_keys=True),
|
||||
json.dumps(capture, sort_keys=True))
|
||||
selected_page = page.get('page', {})
|
||||
checks['pcap_cited_source_supports_uri_prefix'] = (
|
||||
source.get('kind') == 'summary'
|
||||
and source.get('parent_operation') == 'inspect_pcap_capture'
|
||||
and capture == reference['capture']
|
||||
and source.get('artifact') == {key: capture['target'][key] for key in ['path', 'sha256']}
|
||||
and isinstance(source.get('parent_evidence_id'), str)
|
||||
and page.get('parent_evidence_id') == source['parent_evidence_id']
|
||||
and page.get('parent_operation') == 'inspect_pcap_capture'
|
||||
and json.dumps(page.get('view'), sort_keys=True) == json.dumps({'kind': 'pcap', 'facet': 'http-uris', 'offset': 0, 'limit': 2}, sort_keys=True)
|
||||
and selected_page.get('facet') == 'http-uris'
|
||||
and json.dumps(selected_page.get('items'), sort_keys=True) == json.dumps(prefix, sort_keys=True)
|
||||
and page.get('source_coverage') == capture.get('coverage')
|
||||
and json.dumps(page.get('coverage'), sort_keys=True) == json.dumps(reference['delivery_observations'][-1].get('coverage'), sort_keys=True)
|
||||
and page.get('artifact') == {key: capture['target'][key] for key in ['path', 'sha256']}
|
||||
)
|
||||
else:
|
||||
checks['pcap_requested_delivery_chain'] = False
|
||||
if len(reference['delivery_observations']) == 2:
|
||||
checks['pcap_summary_coverage'] = False
|
||||
checks['pcap_observations_have_reported_ids'] = False
|
||||
checks['pcap_requested_views'] = False
|
||||
checks['pcap_cited_source_supports_uri_prefix'] = False
|
||||
checks['pcap_cited_capture'] = False
|
||||
checks['pcap_cited_subjects_match_artifact'] = False
|
||||
checks['pcap_cited_parents_linked'] = False
|
||||
checks['pcap_cited_producer_tools'] = False
|
||||
checks['pcap_cited_operations'] = False
|
||||
checks['pcap_cited_call_arguments'] = False
|
||||
checks['pcap_cited_parameters'] = False
|
||||
elif domain == 'flutter-aot':
|
||||
for key in ['target', 'abi', 'libapp', 'string_pool', 'coverage', 'limitations']:
|
||||
check(key, answer.get(key), reference[key])
|
||||
@@ -560,7 +323,6 @@ def main():
|
||||
statuses[status] = statuses.get(status, 0) + 1
|
||||
check('network_response_statuses', network['observed_response_statuses'], statuses)
|
||||
factual_checks = dict(checks)
|
||||
checks.update(workflow_checks)
|
||||
scopes = [json.loads(line) for line in args.resource_log.read_text().splitlines() if line.startswith('{')]
|
||||
scope = next(value for value in reversed(scopes) if 'resource_scope' in value)
|
||||
checks['no_resource_abort'] = scope['abort'] is None
|
||||
@@ -580,8 +342,6 @@ def main():
|
||||
'review_source_sha256': hashlib.sha256(Path(__file__).read_bytes()).hexdigest(),
|
||||
'domain': domain, 'checks': checks, 'all_selected_checks_pass': all(checks.values()),
|
||||
'factual_checks': factual_checks, 'all_factual_checks_pass': all(factual_checks.values()),
|
||||
**({'workflow_checks': workflow_checks, 'all_workflow_checks_pass': all(workflow_checks.values())}
|
||||
if domain == 'pcap' else {}),
|
||||
'differences': differences, 'tool_errors': tool_errors, 'summary': summary,
|
||||
'fingerprints': {key: value for key, value in settings.items() if 'sha256' in key or key in ['runtime_revision', 'enabled_tools', 'authority']},
|
||||
'answer_file_sha256': hashlib.sha256((args.trial / 'answer.txt').read_bytes()).hexdigest(),
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
"""Distinct capture/cleanup failures; simulated CLI is not real-model evidence."""
|
||||
|
||||
import importlib.util
|
||||
import copy
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
@@ -39,546 +38,6 @@ class ManagedCoverageTests(unittest.TestCase):
|
||||
self.assertNotEqual(scope, expected)
|
||||
|
||||
|
||||
class PcapCoverageTests(unittest.TestCase):
|
||||
def test_explicit_labels_preserve_original_nulls_and_source_scope(self):
|
||||
canonical = runpy.run_path(str(Path(__file__).with_name('review-model.py')))['pcap_observation_fields']
|
||||
original = {'outer_limitations': [], 'normalized_kind': 'pcap',
|
||||
'normalized_coverage': {'examined': 2, 'total': 23, 'next_offset': 2, 'exhausted': False},
|
||||
'normalized_source_coverage': 'partial', 'normalized_parent_evidence_id': None}
|
||||
result = canonical(original)
|
||||
self.assertEqual(result['limitations'], [])
|
||||
self.assertEqual(result['source_coverage'], 'partial')
|
||||
self.assertEqual(result['coverage']['total'], 23)
|
||||
self.assertIsNone(result['parent_evidence_id'])
|
||||
self.assertIn('outer_limitations', original)
|
||||
self.assertNotIn('parent_evidence_id', canonical({}))
|
||||
|
||||
def test_conflicting_duplicates_and_json_type_substitutions_stay_failures(self):
|
||||
canonical = runpy.run_path(str(Path(__file__).with_name('review-model.py')))['pcap_observation_fields']
|
||||
for wrong in ['partial', None]:
|
||||
result = canonical({'normalized_source_coverage': 'complete', 'source_coverage': wrong})
|
||||
self.assertIn('conflicting_fields', result)
|
||||
self.assertEqual(result['normalized_source_coverage'], 'complete')
|
||||
self.assertEqual(result['source_coverage'], wrong)
|
||||
result = canonical({'normalized_coverage': {'examined': 1}, 'coverage': {'examined': True}})
|
||||
self.assertIn('conflicting_fields', result)
|
||||
self.assertEqual(canonical({'outer_limitations': [], 'limitations': []}), {'limitations': []})
|
||||
|
||||
def test_unsupported_claims_are_retained_for_exact_producer_checking(self):
|
||||
canonical = runpy.run_path(str(Path(__file__).with_name('review-model.py')))['pcap_observation_fields']
|
||||
original = {'normalized_kind': 'pcap', 'unrecorded_traffic_absent': True}
|
||||
self.assertIn('unrecorded_traffic_absent', canonical(original))
|
||||
self.assertIsNone(canonical(None))
|
||||
|
||||
|
||||
class PcapGraderWorkflowTests(unittest.TestCase):
|
||||
"""Synthetic receipts test the grader boundary, not Wireshark or real models."""
|
||||
|
||||
def fixture(self):
|
||||
parent, summary_id, page_id = ['ev_' + value * 64 for value in ['a', 'b', 'c']]
|
||||
capture = {'target': {'path': '/fixture/http.pcapng', 'sha256': 'd' * 64},
|
||||
'capture_start': None, 'coverage': 'complete',
|
||||
'retained_counts': {'http_request_uris': 3}}
|
||||
prefix = ['http://example.test/one', 'http://example.test/two']
|
||||
coverage = {'status': 'page', 'examined': 2, 'total': 3, 'next_offset': 2, 'exhausted': False}
|
||||
subject = {'digest': {'sha256': capture['target']['sha256']}, 'local_path': capture['target']['path']}
|
||||
summary = {'evidence_id': summary_id, 'operation': 'inspect_analysis_view',
|
||||
'subject': copy.deepcopy(subject), 'evidence_links': [parent],
|
||||
'limitations': ['source', 'view'],
|
||||
'normalized_result': {'kind': 'summary', 'view': {'kind': 'summary'},
|
||||
'parent_operation': 'inspect_pcap_capture', 'parent_evidence_id': parent,
|
||||
'summary': capture, 'artifact': copy.deepcopy(capture['target']),
|
||||
'coverage': {'status': 'complete-within-view', 'examined': 1, 'total': 1, 'next_offset': None, 'exhausted': True},
|
||||
'limitations': ['source', 'view'], 'unknowns': ['unrecorded traffic']}}
|
||||
page = {'evidence_id': page_id, 'operation': 'inspect_analysis_view',
|
||||
'subject': copy.deepcopy(subject), 'evidence_links': [parent],
|
||||
'limitations': ['source', 'view'],
|
||||
'normalized_result': {'kind': 'pcap',
|
||||
'view': {'kind': 'pcap', 'facet': 'http-uris', 'offset': 0, 'limit': 2},
|
||||
'parent_operation': 'inspect_pcap_capture', 'parent_evidence_id': parent,
|
||||
'page': {'facet': 'http-uris', 'items': prefix}, 'coverage': coverage,
|
||||
'source_coverage': 'complete', 'artifact': capture['target'],
|
||||
'limitations': ['source', 'view'], 'unknowns': ['unrecorded traffic']}}
|
||||
for record in [summary, page]:
|
||||
record['normalized_result']['parent_digest'] = parent[3:]
|
||||
record['parameters'] = {'parent_evidence_id': parent, 'view': copy.deepcopy(record['normalized_result']['view'])}
|
||||
observations = [self.observation(value) for value in [summary, page]]
|
||||
reference = {'artifact_sha256': 'd' * 64, 'capture': capture,
|
||||
'http_request_uris_first2': prefix, 'delivery_observations': observations}
|
||||
answer = copy.deepcopy({**reference, 'http_request_uris_first2': {
|
||||
'items': prefix, 'coverage': coverage, 'source_coverage': 'complete'},
|
||||
'evidence_ids': [summary_id, page_id]})
|
||||
return answer, [summary, page], copy.deepcopy(reference)
|
||||
|
||||
def observation(self, record):
|
||||
value = record['normalized_result']
|
||||
return {'evidence_id': record['evidence_id'], 'limitations': record['limitations'],
|
||||
'kind': value['kind'], 'view': value['view'],
|
||||
'parent_evidence_id': value['parent_evidence_id'], 'coverage': value['coverage'],
|
||||
'unknowns': value['unknowns'], 'normalized_limitations': value['limitations']}
|
||||
|
||||
def grade(self, answer, records, reference, producer_tools=None, producer_arguments=None, extra_events=None, producer_results=None):
|
||||
review = runpy.run_path(str(Path(__file__).with_name('review-model.py')))
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory);trial = root / 'trial';trial.mkdir()
|
||||
for name, value in [('summary.json', {'completed_mcp_calls': []}),
|
||||
('trial-settings.json', {}), ('cleanup.json', {'profile_removed': True})]:
|
||||
(trial / name).write_text(json.dumps(value))
|
||||
(trial / 'answer.txt').write_text(json.dumps(answer))
|
||||
events = []
|
||||
for index, value in enumerate(records):
|
||||
tool = producer_tools[index] if producer_tools is not None else \
|
||||
('inspect_analysis_view' if value['normalized_result'].get('kind') == 'pcap' else 'inspect_pcap_capture')
|
||||
arguments = {'path': reference['capture']['target']['path']}
|
||||
if tool == 'inspect_analysis_view':
|
||||
normalized = value.get('normalized_result')
|
||||
parent = normalized.get('parent_evidence_id') if isinstance(normalized, dict) else None
|
||||
arguments = {'source': {'kind': 'retained-evidence', 'evidence_id': parent},
|
||||
'view': reference['delivery_observations'][-1].get('view')}
|
||||
elif len(reference['delivery_observations']) == 2:
|
||||
arguments['detail'] = 'summary'
|
||||
if producer_arguments is not None:
|
||||
arguments = producer_arguments[index]
|
||||
events.append({'type': 'item.completed', 'item': {'type': 'mcp_tool_call',
|
||||
'tool': tool, 'arguments': arguments, 'status': 'completed',
|
||||
'result': {'structuredContent': value} if producer_results is None else producer_results[index]}})
|
||||
events.extend(extra_events or [])
|
||||
(trial / 'events.jsonl').write_text(''.join(json.dumps(value) + '\n' for value in events))
|
||||
source = root / 'reference';source.mkdir();(source / 'answer.json').write_text(json.dumps(reference))
|
||||
resource = root / 'resource.log'
|
||||
resource.write_text(json.dumps({'resource_scope': 'simulated-grader-only', 'abort': None,
|
||||
'resource_counters_verified': False, 'resource_events_final': None}) + '\n')
|
||||
output = root / 'review.json'
|
||||
argv = ['review-model.py', '--domain', 'pcap', '--trial', str(trial), '--reference', str(source),
|
||||
'--resource-log', str(resource), '--output', str(output)]
|
||||
with patch.object(sys, 'argv', argv), patch('sys.stdout', io.StringIO()), self.assertRaises(SystemExit) as stopped:
|
||||
review['main']()
|
||||
self.assertEqual(stopped.exception.code, 1) # Unknown resource counters stay false.
|
||||
return json.loads(output.read_text())
|
||||
|
||||
def test_complete_grader_accepts_supported_receipts_and_equivalent_labels(self):
|
||||
answer, records, reference = self.fixture()
|
||||
self.assertTrue(self.grade(answer, records, reference)['all_factual_checks_pass'])
|
||||
for observation in answer['delivery_observations']:
|
||||
observation['outer_limitations'] = observation.pop('limitations')
|
||||
observation['normalized_coverage'] = observation.pop('coverage')
|
||||
self.assertTrue(self.grade(answer, records, reference)['all_factual_checks_pass'])
|
||||
|
||||
def test_complete_grader_rejects_missing_nulls_bad_scope_and_malformed_identity(self):
|
||||
mutations = [lambda x: x['capture'].pop('capture_start'),
|
||||
lambda x: x['http_request_uris_first2']['coverage'].__setitem__('exhausted', True),
|
||||
lambda x: x['delivery_observations'][-1].__setitem__('evidence_id', {'unexpected': 'value'}),
|
||||
lambda x: x['delivery_observations'][-1].__setitem__('unrecorded_traffic_absent', True)]
|
||||
for mutate in mutations:
|
||||
with self.subTest(mutation=mutate):
|
||||
answer, records, reference = self.fixture();mutate(answer)
|
||||
self.assertFalse(self.grade(answer, records, reference)['all_factual_checks_pass'])
|
||||
|
||||
def test_faithful_wrong_receipts_do_not_support_the_reported_uri_prefix(self):
|
||||
for wrong in ['facet', 'range', 'parent']:
|
||||
with self.subTest(wrong=wrong):
|
||||
answer, records, reference = self.fixture()
|
||||
page = copy.deepcopy(records[-1]);page['evidence_id'] = 'ev_' + 'e' * 64
|
||||
value = page['normalized_result']
|
||||
if wrong == 'facet':
|
||||
value['view']['facet'] = 'endpoints'
|
||||
value['page'] = {'facet': 'endpoints', 'items': [{'address': '127.0.0.1'}]}
|
||||
elif wrong == 'range':
|
||||
value['view']['offset'] = 1
|
||||
value['page']['items'] = ['http://example.test/two', 'http://example.test/three']
|
||||
else:
|
||||
value['parent_evidence_id'] = 'ev_' + 'f' * 64
|
||||
records.append(page)
|
||||
answer['delivery_observations'][-1] = self.observation(page)
|
||||
answer['evidence_ids'][-1] = page['evidence_id']
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_source_supports_uri_prefix'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
def test_complete_receipt_metadata_counts_and_diagnostics_support_reported_capture(self):
|
||||
answer, records, reference = self.fixture()
|
||||
capture = reference['capture']
|
||||
capture.update({'packet_count': 3, 'client': {'name': 'fixture', 'version': '1'},
|
||||
'retained_counts': {key: 0 for key in ['endpoints', 'dns_queries', 'tls_sni_values',
|
||||
'protocol_distribution', 'http_request_uris', 'unattributed_dns_addresses']},
|
||||
'diagnostics_code_units': {'tshark': 0, 'capinfos': 2}})
|
||||
capture['retained_counts']['http_request_uris'] = 3
|
||||
source = {key: value for key, value in capture.items() if key not in ['retained_counts', 'diagnostics_code_units']}
|
||||
source.update({key: [] for key in capture['retained_counts']})
|
||||
source['http_request_uris'] = reference['http_request_uris_first2'] + ['http://example.test/three']
|
||||
source['diagnostics'] = {'tshark_stderr': '', 'capinfos_stderr': '\U0001f4a5'}
|
||||
receipt = {'evidence_id': 'ev_' + 'a' * 64, 'operation': 'inspect_pcap_capture',
|
||||
'parameters': {'path': source['target']['path']},
|
||||
'subject': copy.deepcopy(records[0]['subject']), 'evidence_links': [],
|
||||
'limitations': ['source'], 'normalized_result': source}
|
||||
reference['delivery_observations'] = [{'evidence_id': receipt['evidence_id'],
|
||||
'limitations': receipt['limitations'], 'coverage': source['coverage']}]
|
||||
answer = copy.deepcopy({**reference, 'evidence_ids': [receipt['evidence_id']]})
|
||||
self.assertTrue(self.grade(answer, [receipt], reference)['all_factual_checks_pass'])
|
||||
for parameters in [None, {}, {'path': '/fixture/other.pcapng'}]:
|
||||
changed = copy.deepcopy(receipt);changed['parameters'] = parameters
|
||||
report = self.grade(answer, [changed], reference)
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_parameters'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
self.assertFalse(self.grade(answer, [receipt], reference,
|
||||
producer_arguments=[{'path': '/fixture/other.pcapng'}])['all_factual_checks_pass'])
|
||||
self.assertFalse(self.grade(answer, [receipt], reference, ['inspect_analysis_view'])['all_factual_checks_pass'])
|
||||
wrong_operation = copy.deepcopy(receipt);wrong_operation['operation'] = 'inspect_analysis_view'
|
||||
self.assertFalse(self.grade(answer, [wrong_operation], reference)['all_factual_checks_pass'])
|
||||
for subject in [None, {'digest': {'sha256': 'e' * 64}, 'local_path': source['target']['path']}]:
|
||||
changed = copy.deepcopy(receipt);changed['subject'] = subject
|
||||
report = self.grade(answer, [changed], reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_cited_capture'])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_subjects_match_artifact'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
mutations = [lambda x: x.__setitem__('packet_count', 4),
|
||||
lambda x: x['client'].__setitem__('version', '2'),
|
||||
lambda x: x.__setitem__('capture_start', 'changed'),
|
||||
lambda x: x.__setitem__('coverage', 'partial'),
|
||||
lambda x: x['endpoints'].append({'address': '127.0.0.1'}),
|
||||
lambda x: x['diagnostics'].__setitem__('capinfos_stderr', '')]
|
||||
for mutate in mutations:
|
||||
changed = copy.deepcopy(receipt);mutate(changed['normalized_result'])
|
||||
report = self.grade(answer, [changed], reference)
|
||||
self.assertTrue(report['factual_checks']['capture'])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_capture'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
receipt['normalized_result'].pop('diagnostics')
|
||||
reference['capture']['diagnostics_code_units'] = None
|
||||
answer['capture']['diagnostics_code_units'] = None
|
||||
self.assertTrue(self.grade(answer, [receipt], reference)['all_factual_checks_pass'])
|
||||
|
||||
def test_cited_receipt_subjects_must_identify_the_reference_capture(self):
|
||||
for index in [0, 1]:
|
||||
for subject in [None, [], 'artifact', {}, {'digest': None},
|
||||
{'digest': {'sha256': 'e' * 64}, 'local_path': '/fixture/http.pcapng'},
|
||||
{'digest': {'sha256': 'd' * 64}, 'local_path': '/fixture/other.pcapng'}]:
|
||||
with self.subTest(index=index, subject=subject):
|
||||
answer, records, reference = self.fixture()
|
||||
records[index]['subject'] = subject
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_subjects_match_artifact'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
answer, records, reference = self.fixture();records[index].pop('subject')
|
||||
self.assertFalse(self.grade(answer, records, reference)['all_factual_checks_pass'])
|
||||
|
||||
def test_faithful_inconsistent_page_coverage_does_not_support_the_prefix(self):
|
||||
for field, wrong in [('status', 'complete-within-view'), ('next_offset', None), ('exhausted', True)]:
|
||||
with self.subTest(field=field):
|
||||
answer, records, reference = self.fixture()
|
||||
records[-1]['normalized_result']['coverage'][field] = wrong
|
||||
answer['delivery_observations'][-1] = self.observation(records[-1])
|
||||
answer['http_request_uris_first2']['coverage'][field] = wrong
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertTrue(report['factual_checks']['pcap_prefix_coverage_observed'])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_source_supports_uri_prefix'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
def test_cited_coverage_preserves_exact_json_scalar_types(self):
|
||||
for field, wrong in [('exhausted', 0), ('examined', 2.0), ('total', 3.0), ('next_offset', 2.0)]:
|
||||
with self.subTest(field=field):
|
||||
answer, records, reference = self.fixture()
|
||||
records[-1]['normalized_result']['coverage'][field] = wrong
|
||||
answer['delivery_observations'][-1] = self.observation(records[-1])
|
||||
answer['http_request_uris_first2']['coverage'][field] = wrong
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertTrue(report['factual_checks']['pcap_prefix_coverage_observed'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
def test_cited_views_preserve_exact_json_scalar_types(self):
|
||||
for field, wrong in [('offset', False), ('offset', 0.0), ('limit', 2.0), ('limit', True)]:
|
||||
with self.subTest(field=field, wrong=wrong):
|
||||
answer, records, reference = self.fixture()
|
||||
records[-1]['normalized_result']['view'][field] = wrong
|
||||
answer['delivery_observations'][-1] = self.observation(records[-1])
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertFalse(report['factual_checks']['pcap_requested_views'])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_source_supports_uri_prefix'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
def test_empty_delivery_observations_write_failed_facts(self):
|
||||
answer, records, reference = self.fixture()
|
||||
answer['delivery_observations'] = []
|
||||
answer['evidence_ids'] = []
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertFalse(report['factual_checks']['pcap_delivery_observations_nonempty'])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_call_arguments'])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_parameters'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
def test_cited_view_parameters_and_call_arguments_bind_actual_requests(self):
|
||||
for index in [0, 1]:
|
||||
for parameters in [None, {}, {'parent_evidence_id': 'ev_' + 'f' * 64,
|
||||
'view': {'kind': 'summary'}}]:
|
||||
with self.subTest(index=index, parameters=parameters):
|
||||
answer, records, reference = self.fixture()
|
||||
records[index]['parameters'] = parameters
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_parameters'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
for wrong in [None, {}, {'source': {'kind': 'retained-evidence', 'evidence_id': 'ev_' + 'f' * 64},
|
||||
'view': {'kind': 'pcap', 'facet': 'http-uris', 'offset': 0, 'limit': 2}},
|
||||
{'source': {'kind': 'retained-evidence', 'evidence_id': 'ev_' + 'a' * 64},
|
||||
'view': {'kind': 'pcap', 'facet': 'http-uris', 'offset': False, 'limit': 2}}]:
|
||||
with self.subTest(arguments=wrong):
|
||||
answer, records, reference = self.fixture()
|
||||
report = self.grade(answer, records, reference,
|
||||
producer_arguments=[{'path': reference['capture']['target']['path'], 'detail': 'summary'}, wrong])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_call_arguments'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
def test_malformed_receipt_envelopes_write_failed_facts_instead_of_crashing(self):
|
||||
tools = ['inspect_pcap_capture', 'inspect_analysis_view']
|
||||
for index in [0, 1]:
|
||||
for field, wrong in [('normalized_result', None), ('normalized_result', []),
|
||||
('normalized_result', 'capture'), ('limitations', None),
|
||||
('limitations', {}), ('limitations', [None]),
|
||||
('limitations', ['source', 1])]:
|
||||
with self.subTest(index=index, field=field, wrong=wrong):
|
||||
answer, records, reference = self.fixture();records[index][field] = wrong
|
||||
if field == 'limitations':
|
||||
answer['delivery_observations'][index]['limitations'] = copy.deepcopy(wrong)
|
||||
report = self.grade(answer, records, reference, tools)
|
||||
self.assertTrue(report['factual_checks']['capture'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
for field in ['normalized_result', 'limitations']:
|
||||
with self.subTest(index=index, missing=field):
|
||||
answer, records, reference = self.fixture();records[index].pop(field)
|
||||
self.assertFalse(self.grade(answer, records, reference, tools)['all_factual_checks_pass'])
|
||||
for index, field in [(0, 'summary'), (1, 'page')]:
|
||||
for wrong in [None, [], 'view']:
|
||||
with self.subTest(index=index, nested=field, wrong=wrong):
|
||||
answer, records, reference = self.fixture();records[index]['normalized_result'][field] = wrong
|
||||
self.assertFalse(self.grade(answer, records, reference, tools)['all_factual_checks_pass'])
|
||||
|
||||
def test_retained_parent_digest_must_match_its_declared_identity(self):
|
||||
for index in [0, 1]:
|
||||
for digest in [None, {}, [], True, 'digest', 'e' * 64]:
|
||||
with self.subTest(index=index, digest=digest):
|
||||
answer, records, reference = self.fixture()
|
||||
records[index]['normalized_result']['parent_digest'] = digest
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
answer, records, reference = self.fixture()
|
||||
records[index]['normalized_result'].pop('parent_digest')
|
||||
self.assertFalse(self.grade(answer, records, reference)['all_factual_checks_pass'])
|
||||
|
||||
def test_summary_outer_artifact_must_identify_the_selected_capture(self):
|
||||
for wrong in [None, {}, [], 'artifact',
|
||||
{'path': '/fixture/other.pcapng', 'sha256': 'd' * 64},
|
||||
{'path': '/fixture/http.pcapng', 'sha256': 'e' * 64}]:
|
||||
with self.subTest(artifact=wrong):
|
||||
answer, records, reference = self.fixture()
|
||||
records[0]['normalized_result']['artifact'] = wrong
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
answer, records, reference = self.fixture()
|
||||
records[0]['normalized_result'].pop('artifact')
|
||||
self.assertFalse(self.grade(answer, records, reference)['all_factual_checks_pass'])
|
||||
|
||||
def test_faithful_summary_coverage_must_describe_the_single_complete_view(self):
|
||||
for field, wrong in [('status', 'page'), ('examined', 0), ('total', 2),
|
||||
('next_offset', 0), ('exhausted', False), ('examined', 1.0)]:
|
||||
with self.subTest(field=field, wrong=wrong):
|
||||
answer, records, reference = self.fixture()
|
||||
records[0]['normalized_result']['coverage'][field] = wrong
|
||||
answer['delivery_observations'][0] = self.observation(records[0])
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
def test_present_normalized_provenance_requires_string_arrays(self):
|
||||
for index in [0, 1]:
|
||||
for field in ['limitations', 'unknowns']:
|
||||
for wrong in [None, {}, 'scope', [None], [1]]:
|
||||
with self.subTest(index=index, field=field, wrong=wrong):
|
||||
answer, records, reference = self.fixture()
|
||||
records[index]['normalized_result'][field] = wrong
|
||||
answer['delivery_observations'][index] = self.observation(records[index])
|
||||
self.assertFalse(self.grade(answer, records, reference)['all_factual_checks_pass'])
|
||||
for empty in [True, False]:
|
||||
answer, records, reference = self.fixture()
|
||||
key = 'normalized_limitations' if field == 'limitations' else field
|
||||
if empty:
|
||||
records[index]['normalized_result'][field] = []
|
||||
answer['delivery_observations'][index][key] = []
|
||||
else:
|
||||
records[index]['normalized_result'].pop(field)
|
||||
answer['delivery_observations'][index].pop(key)
|
||||
self.assertTrue(self.grade(answer, records, reference)['all_factual_checks_pass'])
|
||||
|
||||
def test_missing_or_wrong_kind_does_not_hide_malformed_view_containers(self):
|
||||
for index, container in [(0, 'summary'), (1, 'page')]:
|
||||
for kind in [None, 'unexpected']:
|
||||
for wrong in [None, [], 'view']:
|
||||
with self.subTest(index=index, kind=kind, container=container):
|
||||
answer, records, reference = self.fixture()
|
||||
if kind is None:
|
||||
records[index]['normalized_result'].pop('kind')
|
||||
else:
|
||||
records[index]['normalized_result']['kind'] = kind
|
||||
records[index]['normalized_result'][container] = wrong
|
||||
report = self.grade(answer, records, reference,
|
||||
producer_tools=['inspect_pcap_capture', 'inspect_analysis_view'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
def test_retained_parents_require_declared_distinct_evidence_links(self):
|
||||
for index in [0, 1]:
|
||||
for links in [None, [], ['ev_' + 'f' * 64], {'parent': 'ev_' + 'a' * 64}]:
|
||||
with self.subTest(index=index, links=links):
|
||||
answer, records, reference = self.fixture();records[index]['evidence_links'] = links
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_parents_linked'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
answer, records, reference = self.fixture();records[index].pop('evidence_links')
|
||||
self.assertFalse(self.grade(answer, records, reference)['all_factual_checks_pass'])
|
||||
answer, records, reference = self.fixture()
|
||||
parent = records[index]['evidence_id']
|
||||
for record in records:
|
||||
record['normalized_result']['parent_evidence_id'] = parent;record['evidence_links'] = [parent]
|
||||
answer['delivery_observations'] = [self.observation(record) for record in records]
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertFalse(report['factual_checks']['pcap_cited_parents_linked'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
def test_cited_receipts_preserve_tool_and_delegated_operation_provenance(self):
|
||||
for tools in [['inspect_analysis_view', 'inspect_analysis_view'],
|
||||
['inspect_pcap_capture', 'inspect_pcap_capture'],
|
||||
['unrelated_tool', 'inspect_analysis_view']]:
|
||||
with self.subTest(tools=tools):
|
||||
answer, records, reference = self.fixture()
|
||||
report = self.grade(answer, records, reference, tools)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
for index in [0, 1]:
|
||||
answer, records, reference = self.fixture();records[index]['operation'] = 'inspect_pcap_capture'
|
||||
self.assertFalse(self.grade(answer, records, reference)['all_factual_checks_pass'])
|
||||
|
||||
def test_failed_reply_shapes_preserve_a_written_report_and_partial_facts(self):
|
||||
payloads = ['malformed', [1], None, False, 7, {'error': 'malformed'},
|
||||
{'error': []}, {'error': None}]
|
||||
payloads += [{'error': {'code': 'resource_constraint', 'details': value}}
|
||||
for value in [None, [], 'details']]
|
||||
payloads += [{'error': {'code': 'resource_constraint', 'details': {'reported_limits': value}}}
|
||||
for value in [None, [], 'limits']]
|
||||
for payload in payloads:
|
||||
with self.subTest(payload=payload):
|
||||
answer, records, reference = self.fixture()
|
||||
result = {'structuredContent': payload}
|
||||
event = {'type': 'item.completed', 'item': {
|
||||
'type': 'mcp_tool_call', 'tool': 'inspect_analysis_view', 'status': 'failed',
|
||||
'arguments': {'view': {'kind': 'pcap', 'facet': 'endpoints'}}, 'result': result}}
|
||||
report = self.grade(answer, records, reference, extra_events=[event])
|
||||
self.assertTrue(report['all_factual_checks_pass'])
|
||||
self.assertFalse(report['all_workflow_checks_pass'])
|
||||
self.assertEqual(report['tool_errors'][-1]['response'], payload if payload is not None else result)
|
||||
answer, records, reference = self.fixture()
|
||||
partial_id = 'ev_' + 'e' * 64
|
||||
payload = {'error': {'code': 'resource_constraint', 'details': {
|
||||
'reported_limits': {'evidence_reference': {'evidence_id': partial_id}}}}}
|
||||
event = {'type': 'item.completed', 'item': {
|
||||
'type': 'mcp_tool_call', 'tool': 'inspect_analysis_view', 'status': 'failed',
|
||||
'arguments': {}, 'result': {'structuredContent': payload}}}
|
||||
answer['evidence_ids'].append(partial_id)
|
||||
report = self.grade(answer, records, reference, extra_events=[event])
|
||||
self.assertTrue(report['factual_checks']['reported_evidence_ids_observed_in_responses'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
self.assertFalse(report['all_workflow_checks_pass'])
|
||||
|
||||
def test_workflow_citation_count_must_match_the_planned_calls(self):
|
||||
for count, made_page_call in [(1, False), (1, True), (3, True)]:
|
||||
with self.subTest(count=count, made_page_call=made_page_call):
|
||||
answer, records, reference = self.fixture()
|
||||
if count == 1:
|
||||
answer['delivery_observations'] = answer['delivery_observations'][:1]
|
||||
answer['evidence_ids'] = answer['evidence_ids'][:1]
|
||||
else:
|
||||
extra = copy.deepcopy(answer['delivery_observations'][-1])
|
||||
extra['evidence_id'] = 'ev_' + 'e' * 64
|
||||
answer['delivery_observations'].append(extra)
|
||||
answer['evidence_ids'].append(extra['evidence_id'])
|
||||
report = self.grade(answer, records if made_page_call else records[:1], reference)
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
self.assertFalse(report['all_workflow_checks_pass'])
|
||||
|
||||
def test_unreadable_planned_reply_does_not_pass_workflow_acceptance(self):
|
||||
answer, records, reference = self.fixture()
|
||||
report = self.grade(answer, records, reference,
|
||||
producer_results=[{'structuredContent': records[0]}, {}])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
self.assertFalse(report['all_workflow_checks_pass'])
|
||||
|
||||
def test_extra_calls_are_not_an_equivalent_workflow_even_with_correct_facts(self):
|
||||
for status in ['completed', 'failed']:
|
||||
for with_receipt in [True, False]:
|
||||
with self.subTest(status=status, with_receipt=with_receipt):
|
||||
answer, records, reference = self.fixture()
|
||||
extra = copy.deepcopy(records[-1]);extra['evidence_id'] = 'ev_' + 'e' * 64
|
||||
extra['normalized_result']['view']['facet'] = 'endpoints'
|
||||
event = {'type': 'item.completed', 'item': {
|
||||
'type': 'mcp_tool_call', 'tool': 'inspect_analysis_view', 'status': status,
|
||||
'arguments': {'source': {'kind': 'retained-evidence', 'evidence_id': 'ev_' + 'a' * 64},
|
||||
'view': extra['normalized_result']['view']},
|
||||
'result': {'structuredContent': extra} if with_receipt else {}}}
|
||||
report = self.grade(answer, records, reference, extra_events=[event])
|
||||
self.assertTrue(report['all_factual_checks_pass'])
|
||||
self.assertFalse(report.get('all_workflow_checks_pass', True))
|
||||
answer, records, reference = self.fixture()
|
||||
report = self.grade(answer, records[::-1], reference)
|
||||
self.assertTrue(report['all_factual_checks_pass'])
|
||||
self.assertFalse(report.get('all_workflow_checks_pass', True))
|
||||
|
||||
def test_repeated_identical_receipts_pass_but_conflicting_id_ownership_fails(self):
|
||||
answer, records, reference = self.fixture();duplicate = copy.deepcopy(records[0])
|
||||
report = self.grade(answer, [duplicate, *records], reference)
|
||||
self.assertTrue(report['all_factual_checks_pass'])
|
||||
self.assertFalse(report['all_workflow_checks_pass'])
|
||||
for duplicate_first in [True, False]:
|
||||
with self.subTest(duplicate_first=duplicate_first):
|
||||
items = [duplicate, *records] if duplicate_first else [*records, duplicate]
|
||||
tools = ['unrelated_tool', 'inspect_pcap_capture', 'inspect_analysis_view'] if duplicate_first else \
|
||||
['inspect_pcap_capture', 'inspect_analysis_view', 'unrelated_tool']
|
||||
self.assertFalse(self.grade(answer, items, reference, tools)['all_factual_checks_pass'])
|
||||
duplicate = copy.deepcopy(records[-1]);duplicate['normalized_result']['coverage']['exhausted'] = 0
|
||||
for items in [[*records, duplicate], [records[0], duplicate, records[-1]]]:
|
||||
self.assertFalse(self.grade(answer, items, reference)['all_factual_checks_pass'])
|
||||
|
||||
def test_reported_ids_are_only_the_ordered_unique_response_receipts(self):
|
||||
answer, records, reference = self.fixture()
|
||||
original = answer['evidence_ids']
|
||||
parent = records[0]['normalized_result']['parent_evidence_id']
|
||||
for ids in [original + [parent], original + [original[-1]], original[::-1],
|
||||
[parent, *original], [original], {'receipts': original},
|
||||
[*original, {'extra': parent}], original + ['not-an-id']]:
|
||||
with self.subTest(ids=ids):
|
||||
changed = copy.deepcopy(answer);changed['evidence_ids'] = ids
|
||||
report = self.grade(changed, records, reference)
|
||||
self.assertFalse(report['factual_checks']['pcap_observations_have_reported_ids'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
def test_prefix_coverage_must_come_from_the_cited_uri_receipt(self):
|
||||
answer, records, reference = self.fixture()
|
||||
extra = copy.deepcopy(records[-1]);extra['evidence_id'] = 'ev_' + 'e' * 64
|
||||
extra['normalized_result']['coverage']['next_offset'] = None
|
||||
extra['normalized_result']['coverage']['exhausted'] = True
|
||||
records.append(extra)
|
||||
answer['http_request_uris_first2']['coverage'] = extra['normalized_result']['coverage']
|
||||
report = self.grade(answer, records, reference)
|
||||
self.assertTrue(report['factual_checks']['pcap_original_delivery_observations'])
|
||||
self.assertTrue(report['factual_checks']['pcap_cited_source_supports_uri_prefix'])
|
||||
self.assertFalse(report['factual_checks']['pcap_prefix_coverage_observed'])
|
||||
self.assertFalse(report['all_factual_checks_pass'])
|
||||
|
||||
|
||||
class ContextCaptureTests(unittest.TestCase):
|
||||
def test_filters_ambient_account_and_nested_usage_state(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
|
||||
@@ -215,6 +215,27 @@ async function verifyNativeValueAdapters(
|
||||
const functionEvidence = parseEvidence(analyzed.structuredContent);
|
||||
assert.equal(functionEvidence.operation, "analyze_function");
|
||||
assert.equal(functionEvidence.provider.id, "ghidra");
|
||||
assert.deepEqual(
|
||||
functionEvidence.normalized_result.facet_availability,
|
||||
Object.fromEntries(
|
||||
[
|
||||
"pseudocode",
|
||||
"assembly",
|
||||
"comments",
|
||||
"callers",
|
||||
"callees",
|
||||
"references",
|
||||
"strings_names",
|
||||
"cfg",
|
||||
].map((facet) => [facet, { available: true }]),
|
||||
),
|
||||
);
|
||||
assert.equal(functionEvidence.relationship_coverage?.status, "complete");
|
||||
assert.ok(
|
||||
functionEvidence.relationship_coverage.diagnostics.some((diagnostic) =>
|
||||
diagnostic.includes("indirect calls remain unknown"),
|
||||
),
|
||||
);
|
||||
const functionProfile = analysisProfileSchema.parse(
|
||||
functionEvidence.analysis_profile,
|
||||
);
|
||||
@@ -247,6 +268,7 @@ async function verifyNativeValueAdapters(
|
||||
assert.ok(
|
||||
comparisonEvidence.evidence_links.includes(functionEvidence.evidence_id),
|
||||
);
|
||||
await verifyUnresolvedCallEvidence(client);
|
||||
const bundle = await client.callTool({
|
||||
name: "get_evidence_bundle",
|
||||
arguments: {},
|
||||
@@ -307,3 +329,63 @@ async function verifyNativeValueAdapters(
|
||||
unknowns: cliGraph.unknowns.length,
|
||||
};
|
||||
}
|
||||
|
||||
async function verifyUnresolvedCallEvidence(client) {
|
||||
const inventory = await client.callTool({
|
||||
name: "list_procedures",
|
||||
arguments: {},
|
||||
});
|
||||
assert.notEqual(inventory.isError, true, JSON.stringify(inventory));
|
||||
const procedures = parseEvidence(
|
||||
inventory.structuredContent,
|
||||
).normalized_result;
|
||||
const indirect = procedures.find((item) =>
|
||||
item.value.endsWith("rea_ghidra_inventory_indirect"),
|
||||
);
|
||||
const leaf = procedures.find((item) =>
|
||||
item.value.endsWith("rea_ghidra_inventory_leaf"),
|
||||
);
|
||||
assert.ok(
|
||||
indirect && leaf,
|
||||
"Source fixture lacks the unresolved callback and its possible target",
|
||||
);
|
||||
const analyzed = await client.callTool(
|
||||
{ name: "analyze_function", arguments: { procedure: indirect.address } },
|
||||
{ timeout: 180000 },
|
||||
);
|
||||
assert.notEqual(analyzed.isError, true, JSON.stringify(analyzed));
|
||||
const evidence = parseEvidence(analyzed.structuredContent);
|
||||
assert.equal(evidence.relationship_coverage?.status, "partial");
|
||||
assert.ok(evidence.normalized_result.unresolved_calls.length > 0);
|
||||
const traced = await client.callTool(
|
||||
{
|
||||
name: "trace_call_path",
|
||||
arguments: {
|
||||
start: indirect.address,
|
||||
goal: leaf.address,
|
||||
direction: "forward",
|
||||
},
|
||||
},
|
||||
{ timeout: 180000 },
|
||||
);
|
||||
assert.notEqual(traced.isError, true, JSON.stringify(traced));
|
||||
const trace = parseEvidence(traced.structuredContent).normalized_result;
|
||||
assert.equal(trace.goal_status, "unknown");
|
||||
assert.equal(trace.truncated, true);
|
||||
assert.ok(trace.residual_unknowns.length > 0);
|
||||
const compared = await client.callTool(
|
||||
{
|
||||
name: "compare_functions",
|
||||
arguments: {
|
||||
left: analyzed.structuredContent,
|
||||
right: analyzed.structuredContent,
|
||||
},
|
||||
},
|
||||
{ timeout: 180000 },
|
||||
);
|
||||
assert.notEqual(compared.isError, true, JSON.stringify(compared));
|
||||
assert.equal(
|
||||
parseEvidence(compared.structuredContent).normalized_result.status,
|
||||
"unknown",
|
||||
);
|
||||
}
|
||||
|
||||
@@ -136,7 +136,7 @@ export async function verifyGhidraBoundaries(
|
||||
assert.deepEqual(externalDossier.callees, []);
|
||||
assert.deepEqual(externalDossier.facet_availability.pseudocode, {
|
||||
available: false,
|
||||
reason: "Ghidra did not supply pseudocode.",
|
||||
reason: "Ghidra does not decompile external functions.",
|
||||
});
|
||||
for (const facet of [
|
||||
"assembly",
|
||||
|
||||
@@ -4,8 +4,8 @@ import {
|
||||
copyFile,
|
||||
mkdir,
|
||||
mkdtemp,
|
||||
readFile,
|
||||
readdir,
|
||||
rmdir,
|
||||
rm,
|
||||
stat,
|
||||
symlink,
|
||||
@@ -54,7 +54,7 @@ try {
|
||||
join(workspace, "dist", "windows", "WindowsNativeLoader.js"),
|
||||
).href;
|
||||
const source = `import assert from 'node:assert/strict';
|
||||
import {existsSync,mkdirSync,readFileSync,readdirSync,renameSync,statSync,writeFileSync} from 'node:fs';
|
||||
import {existsSync,mkdirSync,readdirSync,renameSync,statSync} from 'node:fs';
|
||||
import {join} from 'node:path';
|
||||
import {systemWindowsNativeAuthority} from ${JSON.stringify(loader)};
|
||||
const temp=process.env.TEMP;
|
||||
@@ -64,18 +64,17 @@ assert.equal(first.retryable,true);
|
||||
assert.match(first.reason,/Delete owned object by handle failed.*Win32 5/s);
|
||||
const [name,...extra]=readdirSync(temp);assert.ok(name);assert.equal(extra.length,0);
|
||||
const path=join(temp,name);const identity=statSync(path,{bigint:true});
|
||||
const retainedPath=path+'-retained';renameSync(path,retainedPath);
|
||||
assert.equal(statSync(retainedPath,{bigint:true}).ino,identity.ino);
|
||||
// FILE_SHARE_DELETE permits rename; ownership follows the retained handle.
|
||||
// Ownership follows the retained NTFS handle even when the directory moves.
|
||||
// A replacement at its old pathname must remain caller-owned.
|
||||
const moved=path+'-renamed';renameSync(path,moved);
|
||||
assert.equal(statSync(moved,{bigint:true}).ino,identity.ino);
|
||||
mkdirSync(path);const replacement=statSync(path,{bigint:true});
|
||||
assert.notEqual(replacement.ino,identity.ino);
|
||||
const marker=join(path,'caller-evidence.txt');writeFileSync(marker,'caller-owned replacement');
|
||||
if(process.argv[2]!=='teardown'){
|
||||
const second=systemWindowsNativeAuthority();assert.equal(second.available,true);
|
||||
assert.equal(existsSync(retainedPath),false);
|
||||
assert.equal(existsSync(moved),false);
|
||||
assert.deepEqual(readdirSync(temp),[name]);
|
||||
assert.equal(statSync(path,{bigint:true}).ino,replacement.ino);
|
||||
assert.equal(readFileSync(marker,'utf8'),'caller-owned replacement');
|
||||
assert.equal(systemWindowsNativeAuthority(),second);
|
||||
}
|
||||
console.log(JSON.stringify({mode:process.argv[2],firstFailed:true,retainedOwner:true,inode:identity.ino.toString(),replacementName:name,replacementInode:replacement.ino.toString()}));
|
||||
@@ -100,13 +99,11 @@ console.log(JSON.stringify({mode:process.argv[2],firstFailed:true,retainedOwner:
|
||||
(await stat(replacement, { bigint: true })).ino.toString(),
|
||||
observed.replacementInode,
|
||||
);
|
||||
assert.equal(
|
||||
await readFile(join(replacement, "caller-evidence.txt"), "utf8"),
|
||||
"caller-owned replacement",
|
||||
);
|
||||
await rmdir(replacement);
|
||||
assert.deepEqual(await readdir(temp), []);
|
||||
}
|
||||
process.stdout.write(
|
||||
`${JSON.stringify({ ok: true, controlledDispositionFailure: true, retainedOwner: true, renamePreservesIdentity: true, foreignReplacementPreserved: true, retryRecovers: true, successfulAdmissionCached: true, environmentTeardownCleansProbe: true, issue1701HostCauseReproduced: false })}\n`,
|
||||
`${JSON.stringify({ ok: true, controlledDispositionFailure: true, retainedOwner: true, renamedOwnerCleanup: true, replacementPreserved: true, retryRecovers: true, successfulAdmissionCached: true, environmentTeardownCleansProbe: true, issue1701HostCauseReproduced: false })}\n`,
|
||||
);
|
||||
} finally {
|
||||
await rm(workspace, { recursive: true, force: true });
|
||||
|
||||
@@ -606,6 +606,14 @@ export const SUPPORTED_CLIENT_DEFINITIONS = [
|
||||
markerPath: traeUserDirectory,
|
||||
format: "json",
|
||||
},
|
||||
{
|
||||
name: "workbuddy",
|
||||
displayName: "WorkBuddy",
|
||||
skillPath: [".workbuddy", "skills"],
|
||||
configPath: [".workbuddy", "mcp.json"],
|
||||
markerPath: [".workbuddy"],
|
||||
format: "json",
|
||||
},
|
||||
{
|
||||
name: "grok_build",
|
||||
displayName: "Grok Build",
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { dirname } from "node:path";
|
||||
import { realpath } from "node:fs/promises";
|
||||
import { constants } from "node:fs";
|
||||
import { access, realpath, stat } from "node:fs/promises";
|
||||
|
||||
import type { ExecutionOptions } from "../AnalysisProvider.js";
|
||||
import type {
|
||||
@@ -43,11 +44,35 @@ const resolveInputPath = async (
|
||||
}
|
||||
};
|
||||
|
||||
/** Resolve the executable and require a launchable regular file before any spawn. */
|
||||
const resolveExecutablePath = async (
|
||||
selected: string,
|
||||
): ReturnType<typeof resolveInputPath> => {
|
||||
const resolved = await resolveInputPath("executable_path", selected);
|
||||
if (!resolved.ok) return resolved;
|
||||
try {
|
||||
if (!(await stat(resolved.value)).isFile())
|
||||
throw new Error("not a regular file");
|
||||
if (process.platform !== "win32")
|
||||
await access(resolved.value, constants.X_OK);
|
||||
return resolved;
|
||||
} catch (cause: unknown) {
|
||||
return err({
|
||||
issue: {
|
||||
path: ["executable_path"],
|
||||
reason: "invalid_value",
|
||||
message: `executable_path ${selected} is not an executable file: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
},
|
||||
cause,
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
const canonicalizeInput = async (
|
||||
input: ElectronActiveObservationInput,
|
||||
): Promise<Result<CanonicalElectronActiveObservationInput, AnalysisError>> => {
|
||||
const [executable, application, root] = await Promise.all([
|
||||
resolveInputPath("executable_path", input.executable_path),
|
||||
resolveExecutablePath(input.executable_path),
|
||||
resolveInputPath("application_path", input.application_path),
|
||||
input.application_root === undefined
|
||||
? Promise.resolve(ok(undefined))
|
||||
|
||||
@@ -96,13 +96,16 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati
|
||||
await systemProcessOwnershipHost.captureBaseline?.(options.signal);
|
||||
if (options.signal?.aborted)
|
||||
throw new BrowserObservationError(OPERATION, "cancelled");
|
||||
application = await this.launch({
|
||||
const launchOptions: Parameters<typeof electron.launch>[0] = {
|
||||
executablePath: paths.executable,
|
||||
cwd: paths.root,
|
||||
env: safeElectronEnvironment(this.environment, runId),
|
||||
args: ["-r", hookPath, paths.application, ...input.args],
|
||||
timeout: Math.max(1, startupDeadline - performance.now()),
|
||||
});
|
||||
};
|
||||
application = await containLaunchRejections(() =>
|
||||
this.launch(launchOptions),
|
||||
);
|
||||
const leaderPid = application.process().pid;
|
||||
if (
|
||||
typeof leaderPid !== "number" ||
|
||||
@@ -377,6 +380,88 @@ const safeElectronEnvironment = (
|
||||
["REA_PROCESS_RUN_ID", runId] as const,
|
||||
]);
|
||||
|
||||
const PLAYWRIGHT_INTERNAL_FRAME = /playwright-core[\\/]lib[\\/]/;
|
||||
interface LaunchRejectionScope {
|
||||
failure?: unknown;
|
||||
}
|
||||
const activeLaunches = new Set<LaunchRejectionScope>();
|
||||
const heldLaunchRejections = new Map<
|
||||
Promise<unknown>,
|
||||
{
|
||||
readonly reason: Error;
|
||||
readonly candidates: Set<LaunchRejectionScope>;
|
||||
readonly hadExternalListener: boolean;
|
||||
}
|
||||
>();
|
||||
|
||||
const holdLaunchRejection = (
|
||||
reason: unknown,
|
||||
promise: Promise<unknown>,
|
||||
): void => {
|
||||
if (
|
||||
!(reason instanceof Error) ||
|
||||
!PLAYWRIGHT_INTERNAL_FRAME.test(reason.stack ?? "")
|
||||
)
|
||||
throw reason;
|
||||
if (!heldLaunchRejections.has(promise))
|
||||
heldLaunchRejections.set(promise, {
|
||||
reason,
|
||||
candidates: new Set(activeLaunches),
|
||||
hadExternalListener: process
|
||||
.listeners("unhandledRejection")
|
||||
.some((listener) => listener !== holdLaunchRejection),
|
||||
});
|
||||
};
|
||||
|
||||
const forwardLaunchRejection = (
|
||||
reason: Error,
|
||||
hadExternalListener: boolean,
|
||||
): void => {
|
||||
// Listeners present at the original event already observed it. When the
|
||||
// holding listener was the only listener, restore Node's fatal behavior.
|
||||
if (!hadExternalListener)
|
||||
process.nextTick(() => {
|
||||
throw reason;
|
||||
});
|
||||
};
|
||||
|
||||
/** Playwright leaves rejected launch promises unhandled; absorb only those that repeat this launch's failure. */
|
||||
const containLaunchRejections = async <T>(
|
||||
launch: () => Promise<T>,
|
||||
): Promise<T> => {
|
||||
const scope: LaunchRejectionScope = {};
|
||||
if (activeLaunches.size === 0)
|
||||
process.on("unhandledRejection", holdLaunchRejection);
|
||||
activeLaunches.add(scope);
|
||||
try {
|
||||
return await launch();
|
||||
} catch (cause: unknown) {
|
||||
scope.failure = cause;
|
||||
throw cause;
|
||||
} finally {
|
||||
await new Promise<void>((resolve) => setImmediate(resolve));
|
||||
activeLaunches.delete(scope);
|
||||
if (activeLaunches.size === 0)
|
||||
process.off("unhandledRejection", holdLaunchRejection);
|
||||
for (const [
|
||||
promise,
|
||||
{ reason, candidates, hadExternalListener },
|
||||
] of heldLaunchRejections) {
|
||||
if (!candidates.delete(scope)) continue;
|
||||
if (
|
||||
scope.failure instanceof Error &&
|
||||
reason.message !== "" &&
|
||||
scope.failure.message.includes(reason.message)
|
||||
)
|
||||
heldLaunchRejections.delete(promise);
|
||||
else if (candidates.size === 0) {
|
||||
heldLaunchRejections.delete(promise);
|
||||
forwardLaunchRejection(reason, hadExternalListener);
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
const providerError = (cause: unknown): AnalysisError =>
|
||||
cause instanceof AnalysisError
|
||||
? cause
|
||||
|
||||
@@ -45,6 +45,24 @@ const registerOverviewCommands = (
|
||||
.describe("Load and update a local analysis snapshot"),
|
||||
...nativeOpenOptionFields,
|
||||
});
|
||||
// `analyze` declares the JavaScript integrity policy for its JavaScript
|
||||
// lane. Preserve whether the caller supplied it until the target lane is
|
||||
// known, then apply the JavaScript default only in that lane. The owner
|
||||
// field fills an omission with that default, and an `.optional()` wrapper
|
||||
// keeps reporting it, so every binary target would look like an explicit
|
||||
// policy. Remove the owner's default before making the CLI option optional;
|
||||
// reapply its metadata so help and `--schema` keep describing the option.
|
||||
const ownerIntegrityPolicy =
|
||||
javascriptApplicationOptions.shape.integrityPolicy;
|
||||
const defaultIntegrityPolicy = ownerIntegrityPolicy.parse(undefined);
|
||||
const policyMetadata = ownerIntegrityPolicy.meta();
|
||||
const omittableIntegrityPolicy = ownerIntegrityPolicy
|
||||
.removeDefault()
|
||||
.optional();
|
||||
const laneIntegrityPolicy =
|
||||
policyMetadata === undefined
|
||||
? omittableIntegrityPolicy
|
||||
: omittableIntegrityPolicy.meta(policyMetadata);
|
||||
cli.command(CLI_COMMANDS.analyze, {
|
||||
description: "Get an overview of an app",
|
||||
args: z.object({
|
||||
@@ -54,7 +72,7 @@ const registerOverviewCommands = (
|
||||
.describe("App, program, or analysis database path"),
|
||||
}),
|
||||
options: overviewOptions.extend({
|
||||
integrityPolicy: javascriptApplicationOptions.shape.integrityPolicy,
|
||||
integrityPolicy: laneIntegrityPolicy,
|
||||
sourcePath: javascriptApplicationOptions.shape.sourcePath,
|
||||
}),
|
||||
run: async ({ args, options }) => {
|
||||
@@ -67,7 +85,8 @@ const registerOverviewCommands = (
|
||||
runCliJavaScriptApplicationAnalysis(
|
||||
{
|
||||
input_path: resolve(args.path),
|
||||
integrity_policy: options.integrityPolicy,
|
||||
integrity_policy:
|
||||
options.integrityPolicy ?? defaultIntegrityPolicy,
|
||||
...(options.sourcePath === undefined
|
||||
? {}
|
||||
: { source_paths: options.sourcePath }),
|
||||
@@ -90,6 +109,19 @@ const registerOverviewCommands = (
|
||||
},
|
||||
],
|
||||
);
|
||||
if (options.integrityPolicy !== undefined)
|
||||
throw new AnalysisInputError(
|
||||
"analyze_javascript_application",
|
||||
undefined,
|
||||
[
|
||||
{
|
||||
path: ["integrity_policy"],
|
||||
reason: "invalid_value",
|
||||
message:
|
||||
"An integrity policy requires the static JavaScript ASAR/directory analysis lane; binary_overview accepts no input.",
|
||||
},
|
||||
],
|
||||
);
|
||||
return runDirectAnalysis(
|
||||
args.path,
|
||||
"binary_overview",
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -89,11 +89,17 @@ export const ARTIFACT_TOOL_CONTRACTS = [
|
||||
),
|
||||
artifact(
|
||||
"decode_interface_builder",
|
||||
"Decode compiled storyboard and nib archives inside the active Apple app bundle, including keyed property lists and NIBArchive object tables, into a bounded object, hierarchy, and connection graph. This is static archive parsing; unrecognized archive classes and unresolved code handlers remain explicit unknowns.",
|
||||
"Decode compiled storyboard and nib archives inside the active Apple app bundle, including keyed property lists and NIBArchive object tables, into a bounded object, hierarchy, and connection graph. This is static archive parsing; unrecognized archive classes and unresolved code handlers remain explicit unknowns. A full app graph is often several megabytes; detail summary retains it and returns documents with node, edge and coverage counts, then page rows, optionally one document's, with inspect_analysis_view kind interface-builder.",
|
||||
z.strictObject({
|
||||
max_documents: z.number().int().min(1).max(64).default(64),
|
||||
max_objects: z.number().int().min(1).max(20_000).default(20_000),
|
||||
max_connections: z.number().int().min(1).max(40_000).default(40_000),
|
||||
detail: analysisDetailSchema
|
||||
.unwrap()
|
||||
.optional()
|
||||
.describe(
|
||||
"Omitted or complete returns the complete decoded graph Evidence. summary retains that complete Evidence in this session and returns its inspect_analysis_view summary; use normalized_result.parent_evidence_id for interface-builder pages.",
|
||||
),
|
||||
}),
|
||||
),
|
||||
artifact(
|
||||
|
||||
@@ -2,6 +2,7 @@ import { z } from "zod";
|
||||
import { byteSearchResultSchema } from "../domain/native/bytePattern.js";
|
||||
import {
|
||||
artifactInventorySummaryViewResultSchema,
|
||||
interfaceBuilderSummaryViewResultSchema,
|
||||
machoSummaryViewResultSchema,
|
||||
managedArtifactSummaryViewResultSchema,
|
||||
} from "../domain/analysisView/analysisView.js";
|
||||
@@ -404,7 +405,12 @@ export const artifactOutputSchemas: Readonly<Record<string, z.ZodObject>> = {
|
||||
]),
|
||||
),
|
||||
extract_artifact: evidenceResultOf(artifactExtractionResultSchema),
|
||||
decode_interface_builder: evidenceResultOf(interfaceBuilderAnalysisSchema),
|
||||
decode_interface_builder: evidenceResultOf(
|
||||
z.union([
|
||||
interfaceBuilderAnalysisSchema,
|
||||
interfaceBuilderSummaryViewResultSchema,
|
||||
]),
|
||||
),
|
||||
inspect_keyed_archive: evidenceResultOf(keyedArchiveResultSchema),
|
||||
inspect_asset_catalog: evidenceResultOf(appleAssetCatalogResultSchema),
|
||||
trace_dylib_resolution: evidenceResultOf(dylibResolutionResultSchema),
|
||||
|
||||
@@ -62,6 +62,13 @@ import {
|
||||
machoViewRequestSchema,
|
||||
projectMachoView,
|
||||
} from "./machoView.js";
|
||||
import {
|
||||
interfaceBuilderSummarySchema,
|
||||
interfaceBuilderViewContextSchema,
|
||||
interfaceBuilderViewItemsSchema,
|
||||
interfaceBuilderViewRequestSchema,
|
||||
projectInterfaceBuilderView,
|
||||
} from "./interfaceBuilderView.js";
|
||||
import {
|
||||
projectWebNetworkCaptureView,
|
||||
captureViewRequestSchema,
|
||||
@@ -206,6 +213,7 @@ export const analysisViewRequestSchema = z.discriminatedUnion("kind", [
|
||||
inventoryViewSchema,
|
||||
wasmViewRequestSchema,
|
||||
machoViewRequestSchema,
|
||||
interfaceBuilderViewRequestSchema,
|
||||
evmViewRequestSchema,
|
||||
managedViewRequestSchema,
|
||||
androidViewRequestSchema,
|
||||
@@ -326,6 +334,7 @@ const viewResultBase = {
|
||||
"inventory_artifact",
|
||||
"inspect_wasm_artifact",
|
||||
"inspect_macho",
|
||||
"decode_interface_builder",
|
||||
"inspect_evm_interface",
|
||||
"inspect_managed_artifact",
|
||||
"inspect_web_network_capture",
|
||||
@@ -382,6 +391,13 @@ export const machoSummaryViewResultSchema = z.strictObject({
|
||||
summary: machoSummarySchema,
|
||||
});
|
||||
|
||||
/** Decoded documents and graph counts for Interface Builder's selected delivery. */
|
||||
export const interfaceBuilderSummaryViewResultSchema = z.strictObject({
|
||||
...summaryViewResultBase,
|
||||
parent_operation: z.literal("decode_interface_builder"),
|
||||
summary: interfaceBuilderSummarySchema,
|
||||
});
|
||||
|
||||
/** Exact inferred-interface summary for EVM's caller-selected delivery. */
|
||||
export const evmInterfaceSummaryViewResultSchema = z.strictObject({
|
||||
...summaryViewResultBase,
|
||||
@@ -444,6 +460,7 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [
|
||||
inventorySummarySchema,
|
||||
wasmSummarySchema,
|
||||
machoSummarySchema,
|
||||
interfaceBuilderSummarySchema,
|
||||
evmSummarySchema,
|
||||
managedSummarySchema,
|
||||
captureSummarySchema,
|
||||
@@ -514,6 +531,14 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [
|
||||
...machoViewContextSchema.shape,
|
||||
parent_operation: z.literal("inspect_macho"),
|
||||
}),
|
||||
z.strictObject({
|
||||
kind: z.literal("interface-builder"),
|
||||
view: interfaceBuilderViewRequestSchema,
|
||||
items: interfaceBuilderViewItemsSchema,
|
||||
...viewResultBase,
|
||||
...interfaceBuilderViewContextSchema.shape,
|
||||
parent_operation: z.literal("decode_interface_builder"),
|
||||
}),
|
||||
z.strictObject({
|
||||
kind: z.literal("evm"),
|
||||
view: evmViewRequestSchema,
|
||||
@@ -611,6 +636,10 @@ export type UnsignedAnalysisView =
|
||||
>
|
||||
| Omit<Extract<AnalysisViewResult, { readonly kind: "wasm" }>, "view_digest">
|
||||
| Omit<Extract<AnalysisViewResult, { readonly kind: "macho" }>, "view_digest">
|
||||
| Omit<
|
||||
Extract<AnalysisViewResult, { readonly kind: "interface-builder" }>,
|
||||
"view_digest"
|
||||
>
|
||||
| Omit<Extract<AnalysisViewResult, { readonly kind: "evm" }>, "view_digest">
|
||||
| Omit<Extract<AnalysisViewResult, { readonly kind: "qlog" }>, "view_digest">
|
||||
| Omit<Extract<AnalysisViewResult, { readonly kind: "pcap" }>, "view_digest">
|
||||
@@ -674,7 +703,7 @@ const unsupportedParent = (
|
||||
new AnalysisUnsupportedTargetError(
|
||||
INSPECT_ANALYSIS_VIEW_OPERATION,
|
||||
parentArtifactPath(parent),
|
||||
`Parent Evidence operation ${JSON.stringify(parent.operation)} does not support analysis views. Use completed binary layout, JavaScript application, native function, artifact inventory/inspection, WASM, Mach-O, EVM, managed artifact, Android package, Dart AOT, or web network capture Evidence.`,
|
||||
`Parent Evidence operation ${JSON.stringify(parent.operation)} does not support analysis views. Use completed binary layout, JavaScript application, native function, artifact inventory/inspection, WASM, Mach-O, Interface Builder, EVM, managed artifact, Android package, Dart AOT, or web network capture Evidence.`,
|
||||
);
|
||||
|
||||
const parentArtifactPath = (parent: AnalysisViewParent): string => {
|
||||
@@ -702,6 +731,7 @@ const parentArtifactPath = (parent: AnalysisViewParent): string => {
|
||||
const retainedViewProjectors = new Map<string, typeof projectWasmArtifactView>([
|
||||
["inspect_wasm_artifact", projectWasmArtifactView],
|
||||
["inspect_macho", projectMachoView],
|
||||
["decode_interface_builder", projectInterfaceBuilderView],
|
||||
["inspect_evm_interface", projectEvmInterfaceView],
|
||||
["inspect_managed_artifact", projectManagedArtifactView],
|
||||
["inspect_android_package", projectAndroidPackageView],
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
import type { z } from "zod";
|
||||
import type { AnalysisError } from "../analysisErrorBase.js";
|
||||
import { analysisInputErrorFromIssues } from "../inputIssueProjection.js";
|
||||
import { err, ok, type Result } from "../result.js";
|
||||
import type { AnalysisViewParent } from "./analysisView.js";
|
||||
|
||||
/** Parse a parent's result with its owning schema; issues point into the view source. */
|
||||
export const parseAnalysisViewResult = <Schema extends z.ZodType>(
|
||||
schema: Schema,
|
||||
parent: AnalysisViewParent,
|
||||
): Result<z.output<Schema>, AnalysisError> => {
|
||||
const parsed = schema.safeParse(parent.normalizedResult);
|
||||
if (parsed.success) return ok(parsed.data);
|
||||
return err(
|
||||
analysisInputErrorFromIssues(
|
||||
"inspect_analysis_view",
|
||||
parsed.error.issues.map((issue) => ({
|
||||
...issue,
|
||||
path: ["source", "normalized_result", ...issue.path],
|
||||
})),
|
||||
{ source: { normalized_result: parent.normalizedResult } },
|
||||
{ cause: parsed.error },
|
||||
),
|
||||
);
|
||||
};
|
||||
@@ -1,8 +1,8 @@
|
||||
import { z } from "zod";
|
||||
import type { AnalysisError } from "../analysisErrorBase.js";
|
||||
import { androidResultSchemas } from "../android/androidAnalysis.js";
|
||||
import { analysisInputErrorFromIssues } from "../inputIssueProjection.js";
|
||||
import { err, ok, type Result } from "../result.js";
|
||||
import { parseAnalysisViewResult } from "./analysisViewParent.js";
|
||||
import {
|
||||
analysisViewInputError,
|
||||
type AnalysisViewParent,
|
||||
@@ -53,19 +53,8 @@ export const projectAndroidPackageView = (
|
||||
},
|
||||
]),
|
||||
);
|
||||
const parsed = packageSchema.safeParse(parent.normalizedResult);
|
||||
if (!parsed.success)
|
||||
return err(
|
||||
analysisInputErrorFromIssues(
|
||||
"inspect_analysis_view",
|
||||
parsed.error.issues.map((issue) => ({
|
||||
...issue,
|
||||
path: ["source", "normalized_result", ...issue.path],
|
||||
})),
|
||||
{ source: { normalized_result: parent.normalizedResult } },
|
||||
{ cause: parsed.error },
|
||||
),
|
||||
);
|
||||
const parsed = parseAnalysisViewResult(packageSchema, parent);
|
||||
if (!parsed.ok) return parsed;
|
||||
// Engine artifact_sha256 identifies the JAR, never the APK. A missing
|
||||
// authenticated APK subject cannot be repaired using the engine identity.
|
||||
if (parent.subject == null)
|
||||
@@ -79,7 +68,7 @@ export const projectAndroidPackageView = (
|
||||
},
|
||||
]),
|
||||
);
|
||||
const report = parsed.data;
|
||||
const report = parsed.value;
|
||||
const shared = {
|
||||
parent_evidence_id: parent.evidenceId,
|
||||
parent_operation: "inspect_android_package" as const,
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
import { z } from "zod";
|
||||
import type { AnalysisError } from "../analysisErrorBase.js";
|
||||
import { flutterResultSchemas } from "../flutter/flutterBuildAnalysis.js";
|
||||
import { analysisInputErrorFromIssues } from "../inputIssueProjection.js";
|
||||
import { err, ok, type Result } from "../result.js";
|
||||
import { parseAnalysisViewResult } from "./analysisViewParent.js";
|
||||
import {
|
||||
analysisViewInputError,
|
||||
type AnalysisViewParent,
|
||||
@@ -59,20 +59,9 @@ export const projectDartAotView = (
|
||||
},
|
||||
]),
|
||||
);
|
||||
const parsed = aotSchema.safeParse(parent.normalizedResult);
|
||||
if (!parsed.success)
|
||||
return err(
|
||||
analysisInputErrorFromIssues(
|
||||
"inspect_analysis_view",
|
||||
parsed.error.issues.map((issue) => ({
|
||||
...issue,
|
||||
path: ["source", "normalized_result", ...issue.path],
|
||||
})),
|
||||
{ source: { normalized_result: parent.normalizedResult } },
|
||||
{ cause: parsed.error },
|
||||
),
|
||||
);
|
||||
const report = parsed.data;
|
||||
const parsed = parseAnalysisViewResult(aotSchema, parent);
|
||||
if (!parsed.ok) return parsed;
|
||||
const report = parsed.value;
|
||||
const { package_uris, dart_uris, class_like_tokens, ...pool } =
|
||||
report.string_pool;
|
||||
const string_pool = {
|
||||
|
||||
@@ -2,8 +2,8 @@ import { z } from "zod";
|
||||
import type { AnalysisError } from "../analysisErrorBase.js";
|
||||
import { uniqueSorted } from "../canonicalOrdering.js";
|
||||
import { evmInterfaceSchema } from "../evm/evmInterface.js";
|
||||
import { analysisInputErrorFromIssues } from "../inputIssueProjection.js";
|
||||
import { err, ok, type Result } from "../result.js";
|
||||
import { parseAnalysisViewResult } from "./analysisViewParent.js";
|
||||
import {
|
||||
analysisViewInputError,
|
||||
type AnalysisViewParent,
|
||||
@@ -60,20 +60,9 @@ export const projectEvmInterfaceView = (
|
||||
},
|
||||
]),
|
||||
);
|
||||
const parsed = evmInterfaceSchema.safeParse(parent.normalizedResult);
|
||||
if (!parsed.success)
|
||||
return err(
|
||||
analysisInputErrorFromIssues(
|
||||
"inspect_analysis_view",
|
||||
parsed.error.issues.map((issue) => ({
|
||||
...issue,
|
||||
path: ["source", "normalized_result", ...issue.path],
|
||||
})),
|
||||
{ source: { normalized_result: parent.normalizedResult } },
|
||||
{ cause: parsed.error },
|
||||
),
|
||||
);
|
||||
const report = parsed.data;
|
||||
const parsed = parseAnalysisViewResult(evmInterfaceSchema, parent);
|
||||
if (!parsed.ok) return parsed;
|
||||
const report = parsed.value;
|
||||
if (
|
||||
parent.subject == null ||
|
||||
parent.subject.format !== "file" ||
|
||||
|
||||
@@ -0,0 +1,168 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
projectAnalysisView,
|
||||
type AnalysisViewParent,
|
||||
} from "./analysisView.js";
|
||||
|
||||
const target = "a".repeat(64);
|
||||
const evidence = (artifact_path: string) => [
|
||||
{
|
||||
kind: "interface_builder_resource" as const,
|
||||
description: "compiled document",
|
||||
location: { address: null, file_offset: null },
|
||||
artifact_path,
|
||||
artifact_sha256: "b".repeat(64),
|
||||
},
|
||||
];
|
||||
const node = (id: string, kind: string, document: string) => ({
|
||||
id,
|
||||
kind,
|
||||
name: id,
|
||||
location: null,
|
||||
attributes: {},
|
||||
evidence: evidence(document),
|
||||
});
|
||||
const main = "Contents/Resources/Main.nib";
|
||||
const second = "Contents/Resources/Second.nib";
|
||||
const document = (relative_path: string) => ({
|
||||
relative_path,
|
||||
archive_sha256: "b".repeat(64),
|
||||
document_kind: "nib" as const,
|
||||
object_count: 2,
|
||||
connection_count: 1,
|
||||
hierarchy_complete: true,
|
||||
});
|
||||
const report = (truncated = false) => ({
|
||||
target_sha256: target,
|
||||
documents: [document(main), document(second)],
|
||||
graph: {
|
||||
target_sha256: target,
|
||||
provider: { id: "rea-ib", version: "1", tool_version: "1" },
|
||||
nodes: [
|
||||
node("main:root", "scene", main),
|
||||
node("main:button", "control", main),
|
||||
node("second:root", "scene", second),
|
||||
],
|
||||
edges: [
|
||||
{
|
||||
id: "main:edge",
|
||||
from: "main:root",
|
||||
to: "main:button",
|
||||
relation: "contains",
|
||||
resolution: "observed",
|
||||
evidence: evidence(main),
|
||||
limitations: [],
|
||||
},
|
||||
],
|
||||
coverage: [
|
||||
{
|
||||
facet: `objects:${main}`,
|
||||
status: "complete",
|
||||
reason: null,
|
||||
examined: 2,
|
||||
omitted: 0,
|
||||
},
|
||||
],
|
||||
truncated,
|
||||
},
|
||||
limitations: ["Recognized archive fields only."],
|
||||
});
|
||||
const parent = (
|
||||
normalizedResult: unknown = report(),
|
||||
sha256 = target,
|
||||
): AnalysisViewParent => ({
|
||||
evidenceId: `ev_${"c".repeat(64)}`,
|
||||
operation: "decode_interface_builder",
|
||||
normalizedResult,
|
||||
subject: {
|
||||
name: "Fixture",
|
||||
digest: { sha256 },
|
||||
format: "mach-o",
|
||||
architecture: "arm64",
|
||||
local_path: "/Applications/Fixture.app/Contents/MacOS/Fixture",
|
||||
},
|
||||
limitations: [],
|
||||
});
|
||||
|
||||
describe("Interface Builder analysis views", () => {
|
||||
it("summarizes documents and graph counts without rows", () => {
|
||||
const projected = projectAnalysisView(parent(), { kind: "summary" });
|
||||
if (!projected.ok) throw projected.error;
|
||||
expect(projected.value).toMatchObject({
|
||||
kind: "summary",
|
||||
parent_operation: "decode_interface_builder",
|
||||
artifact: { sha256: target },
|
||||
summary: {
|
||||
documents: report().documents,
|
||||
available_counts: { nodes: 3, edges: 1, coverage: 1 },
|
||||
node_kinds: { control: 1, scene: 2 },
|
||||
edge_relations: { contains: 1 },
|
||||
coverage_statuses: { complete: 1 },
|
||||
truncated: false,
|
||||
},
|
||||
coverage: { examined: 7, total: 7 },
|
||||
});
|
||||
expect(JSON.stringify(projected.value)).not.toContain("main:button");
|
||||
});
|
||||
|
||||
it("pages one document's nodes and keeps decoder truncation visible", () => {
|
||||
const projected = projectAnalysisView(parent(), {
|
||||
kind: "interface-builder",
|
||||
collection: "nodes",
|
||||
document: main,
|
||||
offset: 1,
|
||||
limit: 5,
|
||||
});
|
||||
if (!projected.ok) throw projected.error;
|
||||
expect(projected.value).toMatchObject({
|
||||
kind: "interface-builder",
|
||||
items: [node("main:button", "control", main)],
|
||||
source_truncated: false,
|
||||
coverage: { status: "page", examined: 1, total: 2, exhausted: true },
|
||||
});
|
||||
const truncated = projectAnalysisView(parent(report(true)), {
|
||||
kind: "interface-builder",
|
||||
collection: "coverage",
|
||||
offset: 0,
|
||||
limit: 64,
|
||||
});
|
||||
if (!truncated.ok) throw truncated.error;
|
||||
expect(truncated.value).toMatchObject({
|
||||
items: report().graph.coverage,
|
||||
source_truncated: true,
|
||||
unknowns: expect.arrayContaining([
|
||||
expect.stringContaining("The decoded graph is incomplete"),
|
||||
]),
|
||||
});
|
||||
});
|
||||
|
||||
it("refuses unknown documents, other views and mismatched targets", () => {
|
||||
const unknown = projectAnalysisView(parent(), {
|
||||
kind: "interface-builder",
|
||||
collection: "edges",
|
||||
document: "Contents/Resources/Missing.nib",
|
||||
offset: 0,
|
||||
limit: 64,
|
||||
});
|
||||
if (unknown.ok) throw new Error("expected an unknown-document refusal");
|
||||
expect(unknown.error).toMatchObject({
|
||||
issues: [{ path: ["view", "document"] }],
|
||||
});
|
||||
const wrongKind = projectAnalysisView(parent(), {
|
||||
kind: "macho",
|
||||
facet: "imports",
|
||||
offset: 0,
|
||||
limit: 64,
|
||||
});
|
||||
if (wrongKind.ok) throw new Error("expected a view-kind refusal");
|
||||
expect(wrongKind.error).toMatchObject({ issues: [{ path: ["view"] }] });
|
||||
const mismatched = projectAnalysisView(parent(report(), "d".repeat(64)), {
|
||||
kind: "summary",
|
||||
});
|
||||
if (mismatched.ok) throw new Error("expected a subject refusal");
|
||||
expect(mismatched.error).toMatchObject({
|
||||
issues: [{ path: ["source", "subject"] }],
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,248 @@
|
||||
import { z } from "zod";
|
||||
import type { AnalysisError } from "../analysisErrorBase.js";
|
||||
import { uniqueSorted } from "../canonicalOrdering.js";
|
||||
import {
|
||||
interfaceBuilderAnalysisSchema,
|
||||
interfaceBuilderDocumentSchema,
|
||||
} from "../apple/interfaceBuilderGraph.js";
|
||||
import {
|
||||
nativeInvestigationCoverageSchema,
|
||||
nativeInvestigationEdgeSchema,
|
||||
nativeInvestigationNodeSchema,
|
||||
} from "../native/nativeInvestigationGraph.js";
|
||||
import { err, ok, type Result } from "../result.js";
|
||||
import { parseAnalysisViewResult } from "./analysisViewParent.js";
|
||||
import {
|
||||
analysisViewInputError,
|
||||
type AnalysisViewParent,
|
||||
type AnalysisViewRequest,
|
||||
type UnsignedAnalysisView,
|
||||
} from "./analysisView.js";
|
||||
import {
|
||||
completeWithinViewCoverage,
|
||||
pageViewCoverage,
|
||||
} from "./analysisViewCoverage.js";
|
||||
|
||||
const page = {
|
||||
kind: z.literal("interface-builder"),
|
||||
offset: z.number().int().nonnegative().default(0),
|
||||
limit: z.number().int().positive().default(64),
|
||||
} as const;
|
||||
|
||||
/** Original decoded rows; graph rows can be narrowed to one decoded document. */
|
||||
export const interfaceBuilderViewRequestSchema = z.discriminatedUnion(
|
||||
"collection",
|
||||
[
|
||||
z.strictObject({
|
||||
...page,
|
||||
collection: z.enum(["nodes", "edges"]),
|
||||
document: z
|
||||
.string()
|
||||
.min(1)
|
||||
.optional()
|
||||
.describe(
|
||||
"Exact documents[].relative_path; selects rows whose evidence names that archive.",
|
||||
),
|
||||
}),
|
||||
z.strictObject({
|
||||
...page,
|
||||
collection: z.enum(["documents", "coverage"]),
|
||||
}),
|
||||
],
|
||||
);
|
||||
|
||||
export const interfaceBuilderViewItemsSchema = z.union([
|
||||
z.array(interfaceBuilderDocumentSchema),
|
||||
z.array(nativeInvestigationNodeSchema),
|
||||
z.array(nativeInvestigationEdgeSchema),
|
||||
z.array(nativeInvestigationCoverageSchema),
|
||||
]);
|
||||
|
||||
/** The decoder's own truncation, independent of page exhaustion. */
|
||||
export const interfaceBuilderViewContextSchema = z.strictObject({
|
||||
source_truncated: z.boolean(),
|
||||
});
|
||||
|
||||
const countsSchema = z.record(z.string(), z.number().int().positive());
|
||||
|
||||
/** Every decoded document with graph counts; rows and evidence stay retained. */
|
||||
export const interfaceBuilderSummarySchema = z.strictObject({
|
||||
target_sha256: interfaceBuilderAnalysisSchema.shape.target_sha256,
|
||||
provider: interfaceBuilderAnalysisSchema.shape.graph.shape.provider,
|
||||
documents: interfaceBuilderAnalysisSchema.shape.documents,
|
||||
available_counts: z.strictObject({
|
||||
nodes: z.number().int().nonnegative(),
|
||||
edges: z.number().int().nonnegative(),
|
||||
coverage: z.number().int().nonnegative(),
|
||||
}),
|
||||
node_kinds: countsSchema,
|
||||
edge_relations: countsSchema,
|
||||
coverage_statuses: countsSchema,
|
||||
truncated: z.boolean(),
|
||||
limitation_count: z.number().int().nonnegative(),
|
||||
});
|
||||
|
||||
const tally = (values: readonly string[]): Record<string, number> => {
|
||||
const counts = new Map<string, number>();
|
||||
for (const value of values) counts.set(value, (counts.get(value) ?? 0) + 1);
|
||||
return Object.fromEntries(
|
||||
[...counts].sort(([left], [right]) => (left < right ? -1 : 1)),
|
||||
);
|
||||
};
|
||||
|
||||
type InterfaceBuilderReport = z.output<typeof interfaceBuilderAnalysisSchema>;
|
||||
|
||||
/** Admit a well-formed report bound to its recorded target, if one exists. */
|
||||
const parseInterfaceBuilderReport = (
|
||||
parent: AnalysisViewParent,
|
||||
): Result<InterfaceBuilderReport, AnalysisError> => {
|
||||
const parsed = parseAnalysisViewResult(
|
||||
interfaceBuilderAnalysisSchema,
|
||||
parent,
|
||||
);
|
||||
if (!parsed.ok) return parsed;
|
||||
const report = parsed.value;
|
||||
if (
|
||||
report.graph.target_sha256 !== report.target_sha256 ||
|
||||
(parent.subject != null &&
|
||||
parent.subject.digest.sha256 !== report.target_sha256)
|
||||
)
|
||||
return err(
|
||||
analysisViewInputError([
|
||||
{
|
||||
path: ["source", "subject"],
|
||||
reason: "invalid_value",
|
||||
message:
|
||||
"Interface Builder parent Evidence subject and graph must identify the report's target SHA-256.",
|
||||
},
|
||||
]),
|
||||
);
|
||||
return ok(report);
|
||||
};
|
||||
|
||||
/** Every decoded document with graph tallies; rows stay in the retained parent. */
|
||||
const summarizeInterfaceBuilder = (
|
||||
report: InterfaceBuilderReport,
|
||||
limitationCount: number,
|
||||
) => {
|
||||
const { graph } = report;
|
||||
const total =
|
||||
report.documents.length +
|
||||
graph.nodes.length +
|
||||
graph.edges.length +
|
||||
graph.coverage.length;
|
||||
return {
|
||||
summary: {
|
||||
target_sha256: report.target_sha256,
|
||||
provider: graph.provider,
|
||||
documents: report.documents,
|
||||
available_counts: {
|
||||
nodes: graph.nodes.length,
|
||||
edges: graph.edges.length,
|
||||
coverage: graph.coverage.length,
|
||||
},
|
||||
node_kinds: tally(graph.nodes.map((node) => node.kind)),
|
||||
edge_relations: tally(graph.edges.map((edge) => edge.relation)),
|
||||
coverage_statuses: tally(graph.coverage.map((row) => row.status)),
|
||||
truncated: graph.truncated,
|
||||
limitation_count: limitationCount,
|
||||
},
|
||||
coverage: completeWithinViewCoverage(total, total),
|
||||
};
|
||||
};
|
||||
|
||||
/** Project retained decode_interface_builder Evidence without reopening archives. */
|
||||
export const projectInterfaceBuilderView = (
|
||||
parent: AnalysisViewParent,
|
||||
view: AnalysisViewRequest,
|
||||
): Result<UnsignedAnalysisView, AnalysisError> => {
|
||||
if (view.kind !== "summary" && view.kind !== "interface-builder")
|
||||
return err(
|
||||
analysisViewInputError([
|
||||
{
|
||||
path: ["view"],
|
||||
reason: "invalid_value",
|
||||
message:
|
||||
"Interface Builder decoding supports summary and interface-builder views.",
|
||||
},
|
||||
]),
|
||||
);
|
||||
const parsed = parseInterfaceBuilderReport(parent);
|
||||
if (!parsed.ok) return parsed;
|
||||
const report = parsed.value;
|
||||
const { graph } = report;
|
||||
const limitations = uniqueSorted([
|
||||
...parent.limitations,
|
||||
...report.limitations,
|
||||
]);
|
||||
const shared = {
|
||||
parent_evidence_id: parent.evidenceId,
|
||||
parent_operation: "decode_interface_builder" as const,
|
||||
parent_digest: parent.evidenceId.slice(3),
|
||||
artifact: {
|
||||
path: parent.subject?.local_path ?? "",
|
||||
sha256: report.target_sha256,
|
||||
},
|
||||
limitations,
|
||||
unknowns: [
|
||||
"Rows describe recognized archive objects and connections only; unrecognized classes, fields and runtime-created UI remain unknown.",
|
||||
...(graph.truncated
|
||||
? [
|
||||
"The decoded graph is incomplete (see coverage and limitations for the cause); page exhaustion does not establish absence of undecoded objects or connections.",
|
||||
]
|
||||
: []),
|
||||
],
|
||||
};
|
||||
if (view.kind === "summary")
|
||||
return ok({
|
||||
kind: "summary",
|
||||
view,
|
||||
...shared,
|
||||
...summarizeInterfaceBuilder(report, limitations.length),
|
||||
});
|
||||
const document =
|
||||
view.collection === "nodes" || view.collection === "edges"
|
||||
? view.document
|
||||
: undefined;
|
||||
if (
|
||||
document !== undefined &&
|
||||
!report.documents.some((row) => row.relative_path === document)
|
||||
)
|
||||
return err(
|
||||
analysisViewInputError([
|
||||
{
|
||||
path: ["view", "document"],
|
||||
reason: "invalid_value",
|
||||
message:
|
||||
"Select an exact documents[].relative_path from this decoded report.",
|
||||
},
|
||||
]),
|
||||
);
|
||||
const inDocument = <
|
||||
Row extends { readonly evidence: readonly { artifact_path: unknown }[] },
|
||||
>(
|
||||
candidates: readonly Row[],
|
||||
): readonly Row[] =>
|
||||
document === undefined
|
||||
? candidates
|
||||
: candidates.filter((row) =>
|
||||
row.evidence.some((evidence) => evidence.artifact_path === document),
|
||||
);
|
||||
const rows =
|
||||
view.collection === "documents"
|
||||
? report.documents
|
||||
: view.collection === "coverage"
|
||||
? graph.coverage
|
||||
: view.collection === "nodes"
|
||||
? inDocument(graph.nodes)
|
||||
: inDocument(graph.edges);
|
||||
const items = rows.slice(view.offset, view.offset + view.limit);
|
||||
return ok({
|
||||
kind: "interface-builder",
|
||||
view,
|
||||
items,
|
||||
...shared,
|
||||
source_truncated: graph.truncated,
|
||||
coverage: pageViewCoverage(view.offset, items.length, rows.length),
|
||||
});
|
||||
};
|
||||
@@ -2,9 +2,9 @@ import { z } from "zod";
|
||||
import type { AnalysisError } from "../analysisErrorBase.js";
|
||||
import { uniqueSorted } from "../canonicalOrdering.js";
|
||||
import { isMachOFormat } from "../binaryTargetTypes.js";
|
||||
import { analysisInputErrorFromIssues } from "../inputIssueProjection.js";
|
||||
import { inspectMachoSchema } from "../native/nativeInspection.js";
|
||||
import { err, ok, type Result } from "../result.js";
|
||||
import { parseAnalysisViewResult } from "./analysisViewParent.js";
|
||||
import {
|
||||
analysisViewInputError,
|
||||
type AnalysisViewParent,
|
||||
@@ -93,19 +93,8 @@ type MachoReport = z.output<typeof inspectMachoSchema>;
|
||||
const parseMachoReport = (
|
||||
parent: AnalysisViewParent,
|
||||
): Result<MachoReport, AnalysisError> => {
|
||||
const parsed = inspectMachoSchema.safeParse(parent.normalizedResult);
|
||||
if (!parsed.success)
|
||||
return err(
|
||||
analysisInputErrorFromIssues(
|
||||
"inspect_analysis_view",
|
||||
parsed.error.issues.map((issue) => ({
|
||||
...issue,
|
||||
path: ["source", "normalized_result", ...issue.path],
|
||||
})),
|
||||
{ source: { normalized_result: parent.normalizedResult } },
|
||||
{ cause: parsed.error },
|
||||
),
|
||||
);
|
||||
const parsed = parseAnalysisViewResult(inspectMachoSchema, parent);
|
||||
if (!parsed.ok) return parsed;
|
||||
if (parent.subject != null && !isMachOFormat(parent.subject.format))
|
||||
return err(
|
||||
analysisViewInputError([
|
||||
@@ -117,7 +106,7 @@ const parseMachoReport = (
|
||||
},
|
||||
]),
|
||||
);
|
||||
return ok(parsed.data);
|
||||
return ok(parsed.value);
|
||||
};
|
||||
|
||||
/** Counts replace rows; every collection caveat stays in the limitations. */
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
import { z } from "zod";
|
||||
import type { AnalysisError } from "../analysisErrorBase.js";
|
||||
import { uniqueSorted } from "../canonicalOrdering.js";
|
||||
import { analysisInputErrorFromIssues } from "../inputIssueProjection.js";
|
||||
import {
|
||||
managedArtifactInspectionSchema,
|
||||
managedMemberInspectionSchema,
|
||||
} from "../managed/managedArtifact.js";
|
||||
import { err, ok, type Result } from "../result.js";
|
||||
import { parseAnalysisViewResult } from "./analysisViewParent.js";
|
||||
import {
|
||||
analysisViewInputError,
|
||||
type AnalysisViewParent,
|
||||
@@ -67,22 +67,12 @@ export const projectManagedArtifactView = (
|
||||
},
|
||||
]),
|
||||
);
|
||||
const parsed = managedArtifactInspectionSchema.safeParse(
|
||||
parent.normalizedResult,
|
||||
const parsed = parseAnalysisViewResult(
|
||||
managedArtifactInspectionSchema,
|
||||
parent,
|
||||
);
|
||||
if (!parsed.success)
|
||||
return err(
|
||||
analysisInputErrorFromIssues(
|
||||
"inspect_analysis_view",
|
||||
parsed.error.issues.map((issue) => ({
|
||||
...issue,
|
||||
path: ["source", "normalized_result", ...issue.path],
|
||||
})),
|
||||
{ source: { normalized_result: parent.normalizedResult } },
|
||||
{ cause: parsed.error },
|
||||
),
|
||||
);
|
||||
const report = parsed.data;
|
||||
if (!parsed.ok) return parsed;
|
||||
const report = parsed.value;
|
||||
const identity_scope = {
|
||||
token_identity: "build-local" as const,
|
||||
requires_artifact_sha256: report.artifact.sha256,
|
||||
|
||||
@@ -2,11 +2,11 @@ import { z } from "zod";
|
||||
import type { AnalysisError } from "../analysisErrorBase.js";
|
||||
import { uniqueSorted } from "../canonicalOrdering.js";
|
||||
import { err, ok, type Result } from "../result.js";
|
||||
import { parseAnalysisViewResult } from "./analysisViewParent.js";
|
||||
import {
|
||||
wasmArtifactSchema,
|
||||
wasmArtifactMatchesSubject,
|
||||
} from "../wasm/wasmArtifact.js";
|
||||
import { analysisInputErrorFromIssues } from "../inputIssueProjection.js";
|
||||
import {
|
||||
analysisViewInputError,
|
||||
type AnalysisViewParent,
|
||||
@@ -81,20 +81,9 @@ export const projectWasmArtifactView = (
|
||||
},
|
||||
]),
|
||||
);
|
||||
const parsed = wasmArtifactSchema.safeParse(parent.normalizedResult);
|
||||
if (!parsed.success)
|
||||
return err(
|
||||
analysisInputErrorFromIssues(
|
||||
"inspect_analysis_view",
|
||||
parsed.error.issues.map((issue) => ({
|
||||
...issue,
|
||||
path: ["source", "normalized_result", ...issue.path],
|
||||
})),
|
||||
{ source: { normalized_result: parent.normalizedResult } },
|
||||
{ cause: parsed.error },
|
||||
),
|
||||
);
|
||||
const report = parsed.data;
|
||||
const parsed = parseAnalysisViewResult(wasmArtifactSchema, parent);
|
||||
if (!parsed.ok) return parsed;
|
||||
const report = parsed.value;
|
||||
if (
|
||||
!wasmArtifactMatchesSubject(
|
||||
report.artifact,
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
import { z } from "zod";
|
||||
import type { AnalysisError } from "../analysisErrorBase.js";
|
||||
import { analysisInputErrorFromIssues } from "../inputIssueProjection.js";
|
||||
import type { JsonValue } from "../jsonValue.js";
|
||||
import { err, ok, type Result } from "../result.js";
|
||||
import { parseAnalysisViewResult } from "./analysisViewParent.js";
|
||||
import {
|
||||
webNetworkCaptureRecordSchema,
|
||||
webNetworkCaptureBinaryDescriptorSchema,
|
||||
@@ -118,20 +118,9 @@ const preview = (record: WebNetworkCaptureRecord) => {
|
||||
const parseCaptureParent = (
|
||||
parent: AnalysisViewParent,
|
||||
): Result<WebNetworkCapture, AnalysisError> => {
|
||||
const parsed = webNetworkCaptureSchema.safeParse(parent.normalizedResult);
|
||||
if (!parsed.success)
|
||||
return err(
|
||||
analysisInputErrorFromIssues(
|
||||
"inspect_analysis_view",
|
||||
parsed.error.issues.map((issue) => ({
|
||||
...issue,
|
||||
path: ["source", "normalized_result", ...issue.path],
|
||||
})),
|
||||
{ source: { normalized_result: parent.normalizedResult } },
|
||||
{ cause: parsed.error },
|
||||
),
|
||||
);
|
||||
const report = parsed.data;
|
||||
const parsed = parseAnalysisViewResult(webNetworkCaptureSchema, parent);
|
||||
if (!parsed.ok) return parsed;
|
||||
const report = parsed.value;
|
||||
if (
|
||||
report.artifact.sha256 !== null &&
|
||||
(parent.subject == null ||
|
||||
|
||||
@@ -337,6 +337,42 @@ describe("call path topology and capacity", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("directional call relationship coverage", () => {
|
||||
it.each([
|
||||
{
|
||||
directions: ["callers"] as ["callers"],
|
||||
status: "not_found",
|
||||
exhaustive: true,
|
||||
},
|
||||
{
|
||||
directions: ["callees"] as ["callees"],
|
||||
status: "unknown",
|
||||
exhaustive: false,
|
||||
},
|
||||
{
|
||||
directions: ["callers", "callees"] as ["callers", "callees"],
|
||||
status: "unknown",
|
||||
exhaustive: false,
|
||||
},
|
||||
])(
|
||||
"uses $directions coverage for the forward search",
|
||||
({ directions, status, exhaustive }) => {
|
||||
const limitation = "The inventory omitted a reference.";
|
||||
const relationshipCoverage: RelationshipCoverage = {
|
||||
status: "partial",
|
||||
limitations: [limitation],
|
||||
limited_directions: directions,
|
||||
};
|
||||
const result = run([
|
||||
observe("0x1000", ["0x2000"], "rea-workflow", { relationshipCoverage }),
|
||||
observe("0x2000", [], "rea-workflow", { relationshipCoverage }),
|
||||
]);
|
||||
expect(result).toMatchObject({ status, search_scope: { exhaustive } });
|
||||
expect(result.limitations).toContain(limitation);
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("explicit call relationship coverage", () => {
|
||||
it("accumulates 150,000 partial reasons without argument-stack exhaustion", () => {
|
||||
const result = run([
|
||||
|
||||
+8
-10
@@ -1,3 +1,4 @@
|
||||
import { relationshipCoverageLimitations } from "./relationshipCoverage.js";
|
||||
import {
|
||||
callPathInputSchema,
|
||||
callPathResultSchema,
|
||||
@@ -10,7 +11,7 @@ import {
|
||||
parseFunctionEvidence,
|
||||
type FunctionSnapshot,
|
||||
} from "./functionDossierEvidence.js";
|
||||
import { isCallerOnlyUnknownCoverage } from "./functionCallCoverage.js";
|
||||
import { isCallerOnlyIncompleteCoverage } from "./functionCallCoverage.js";
|
||||
import { compareUnicodeCodePoints } from "./unicodeCodePointOrder.js";
|
||||
|
||||
/** Minimal directed caller-to-callee adjacency for call-path search. */
|
||||
@@ -240,15 +241,12 @@ const inspectSearch = ({
|
||||
const coverage = snapshot.evidence.relationship_coverage;
|
||||
for (const limitation of snapshot.limitations)
|
||||
limitations.push(limitation);
|
||||
if (coverage?.status === "partial")
|
||||
for (const limitation of coverage.limitations)
|
||||
if (coverage !== null) {
|
||||
for (const limitation of relationshipCoverageLimitations(coverage))
|
||||
limitations.push(limitation);
|
||||
else if (coverage?.status === "unknown")
|
||||
limitations.push(coverage.reason);
|
||||
for (const diagnostic of coverage !== null && "diagnostics" in coverage
|
||||
? coverage.diagnostics
|
||||
: [])
|
||||
limitations.push(diagnostic);
|
||||
for (const diagnostic of coverage.diagnostics ?? [])
|
||||
limitations.push(diagnostic);
|
||||
}
|
||||
// The requested goal is a terminal for this search. Its outgoing edges
|
||||
// cannot add a path to the goal, so its dossier limits are contextual
|
||||
// but do not make the searched start-to-goal closure incomplete.
|
||||
@@ -268,7 +266,7 @@ const inspectSearch = ({
|
||||
coverage === null
|
||||
? snapshot.limitations.length > 0
|
||||
: coverage.status !== "complete" &&
|
||||
!isCallerOnlyUnknownCoverage(coverage)
|
||||
!isCallerOnlyIncompleteCoverage(coverage)
|
||||
)
|
||||
incompleteExpandedCoverage = true;
|
||||
} else if (node === goal) {
|
||||
|
||||
@@ -2,7 +2,7 @@ import { describe, expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
combineFunctionRelationshipCoverage,
|
||||
isCallerOnlyUnknownCoverage,
|
||||
isCallerOnlyIncompleteCoverage,
|
||||
} from "./functionCallCoverage.js";
|
||||
import type { RelationshipCoverage } from "./relationshipCoverage.js";
|
||||
|
||||
@@ -22,14 +22,14 @@ const cappedCallees: RelationshipCoverage = {
|
||||
};
|
||||
const complete: RelationshipCoverage = { status: "complete" };
|
||||
|
||||
describe("isCallerOnlyUnknownCoverage", () => {
|
||||
describe("isCallerOnlyIncompleteCoverage", () => {
|
||||
it("detects unknown callers beside complete callees", () => {
|
||||
const combined = combineFunctionRelationshipCoverage(
|
||||
unknownCallers,
|
||||
complete,
|
||||
);
|
||||
expect(combined.status).toBe("unknown");
|
||||
expect(isCallerOnlyUnknownCoverage(combined)).toBe(true);
|
||||
expect(isCallerOnlyIncompleteCoverage(combined)).toBe(true);
|
||||
});
|
||||
|
||||
it("rejects unknown callees beside complete callers", () => {
|
||||
@@ -37,7 +37,7 @@ describe("isCallerOnlyUnknownCoverage", () => {
|
||||
complete,
|
||||
unknownCallees,
|
||||
);
|
||||
expect(isCallerOnlyUnknownCoverage(combined)).toBe(false);
|
||||
expect(isCallerOnlyIncompleteCoverage(combined)).toBe(false);
|
||||
});
|
||||
|
||||
it("rejects unknown coverage on both sides", () => {
|
||||
@@ -45,7 +45,7 @@ describe("isCallerOnlyUnknownCoverage", () => {
|
||||
unknownCallers,
|
||||
unknownCallees,
|
||||
);
|
||||
expect(isCallerOnlyUnknownCoverage(combined)).toBe(false);
|
||||
expect(isCallerOnlyIncompleteCoverage(combined)).toBe(false);
|
||||
});
|
||||
|
||||
it("rejects unknown callers beside IDA-capped callees", () => {
|
||||
@@ -54,11 +54,37 @@ describe("isCallerOnlyUnknownCoverage", () => {
|
||||
cappedCallees,
|
||||
);
|
||||
expect(combined.status).toBe("unknown");
|
||||
expect(isCallerOnlyUnknownCoverage(combined)).toBe(false);
|
||||
expect(isCallerOnlyIncompleteCoverage(combined)).toBe(false);
|
||||
});
|
||||
|
||||
it("rejects complete coverage", () => {
|
||||
const combined = combineFunctionRelationshipCoverage(complete, complete);
|
||||
expect(isCallerOnlyUnknownCoverage(combined)).toBe(false);
|
||||
expect(isCallerOnlyIncompleteCoverage(combined)).toBe(false);
|
||||
});
|
||||
|
||||
it("retains caller-only partial coverage without relying on limitation wording", () => {
|
||||
const combined = combineFunctionRelationshipCoverage(
|
||||
{ status: "partial", limitations: ["Missing incoming references."] },
|
||||
complete,
|
||||
);
|
||||
expect(combined).toMatchObject({ limited_directions: ["callers"] });
|
||||
expect(isCallerOnlyIncompleteCoverage(combined)).toBe(true);
|
||||
});
|
||||
|
||||
it("uses explicit directions even when explanation text contains direction labels", () => {
|
||||
expect(
|
||||
isCallerOnlyIncompleteCoverage({
|
||||
status: "unknown",
|
||||
reason: "Callers: unavailable. Callees: diagnostic text.",
|
||||
limited_directions: ["callers"],
|
||||
}),
|
||||
).toBe(true);
|
||||
expect(
|
||||
isCallerOnlyIncompleteCoverage({
|
||||
status: "unknown",
|
||||
reason: "Callers: unavailable.",
|
||||
limited_directions: ["callees"],
|
||||
}),
|
||||
).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,4 +1,7 @@
|
||||
import type { RelationshipCoverage } from "./relationshipCoverage.js";
|
||||
import {
|
||||
relationshipCoverageLimitations,
|
||||
type RelationshipCoverage,
|
||||
} from "./relationshipCoverage.js";
|
||||
import type { FunctionDossier } from "./hopperValues.js";
|
||||
import { uniqueSorted } from "./canonicalOrdering.js";
|
||||
|
||||
@@ -15,25 +18,25 @@ export const combineFunctionRelationshipCoverage = (
|
||||
{ direction: "callees" as const, coverage: callees },
|
||||
];
|
||||
const diagnostics = uniqueSorted(
|
||||
reads.flatMap(({ coverage }) =>
|
||||
coverage !== undefined && "diagnostics" in coverage
|
||||
? coverage.diagnostics
|
||||
: [],
|
||||
),
|
||||
reads.flatMap(({ coverage }) => coverage?.diagnostics ?? []),
|
||||
);
|
||||
const details = diagnostics.length > 0 ? { diagnostics } : {};
|
||||
const callersLimited = callers?.status !== "complete";
|
||||
const calleesLimited = callees?.status !== "complete";
|
||||
const limited_directions: Extract<
|
||||
RelationshipCoverage,
|
||||
{ status: "partial" }
|
||||
>["limited_directions"] = callersLimited
|
||||
? calleesLimited
|
||||
? ["callers", "callees"]
|
||||
: ["callers"]
|
||||
: ["callees"];
|
||||
const reasons = reads.flatMap(({ direction, coverage }) => {
|
||||
const label =
|
||||
direction === "callers" ? callerCoveragePrefix : calleeCoveragePrefix;
|
||||
const limitations =
|
||||
coverage === undefined
|
||||
? ["Provider did not report relationship coverage."]
|
||||
: coverage.status === "unknown"
|
||||
? [coverage.reason]
|
||||
: coverage.status === "partial"
|
||||
? coverage.limitations
|
||||
: [];
|
||||
return limitations.map((reason) => `${label}${reason}`);
|
||||
return relationshipCoverageLimitations(coverage).map(
|
||||
(reason) => `${label}${reason}`,
|
||||
);
|
||||
});
|
||||
if (
|
||||
reads.some(
|
||||
@@ -43,54 +46,48 @@ export const combineFunctionRelationshipCoverage = (
|
||||
return {
|
||||
status: "unknown",
|
||||
reason: uniqueSorted(reasons).join(" "),
|
||||
limited_directions,
|
||||
...details,
|
||||
};
|
||||
if (reasons.length > 0)
|
||||
return {
|
||||
status: "partial",
|
||||
limitations: uniqueSorted(reasons),
|
||||
limited_directions,
|
||||
...details,
|
||||
};
|
||||
return { status: "complete", ...details };
|
||||
};
|
||||
|
||||
/**
|
||||
* Whether combined relationship coverage is unknown only on the caller side.
|
||||
* The combine helper above prefixes each direction's reasons, so a reason
|
||||
* that carries the caller prefix and no callee prefix records observed
|
||||
* callees with unobservable callers. Forward (callee-edge) searches stay
|
||||
* exhaustive under that caveat; the caller reason remains a limitation.
|
||||
* Forward searches depend on callee coverage. Preserve the historical
|
||||
* unknown-reason encoding when reading observations without explicit directions.
|
||||
*/
|
||||
export const isCallerOnlyUnknownCoverage = (
|
||||
export const isCallerOnlyIncompleteCoverage = (
|
||||
coverage: RelationshipCoverage,
|
||||
): boolean =>
|
||||
coverage.status === "unknown" &&
|
||||
coverage.reason.startsWith(callerCoveragePrefix) &&
|
||||
!coverage.reason.includes(calleeCoveragePrefix);
|
||||
): boolean => {
|
||||
if (coverage.status === "complete") return false;
|
||||
if (coverage.limited_directions !== undefined)
|
||||
return !coverage.limited_directions.some(
|
||||
(direction) => direction === "callees",
|
||||
);
|
||||
return (
|
||||
coverage.status === "unknown" &&
|
||||
coverage.reason.startsWith(callerCoveragePrefix) &&
|
||||
!coverage.reason.includes(calleeCoveragePrefix)
|
||||
);
|
||||
};
|
||||
|
||||
/** Explain why two resolved call sets cannot establish call-set equality. */
|
||||
export const functionCallCoverageLimitations = (
|
||||
coverage: RelationshipCoverage | null,
|
||||
dossier: Pick<FunctionDossier, "unresolved_calls">,
|
||||
): string[] => {
|
||||
const limitations =
|
||||
coverage === null
|
||||
? ["Provider did not report relationship coverage."]
|
||||
: coverage.status === "unknown"
|
||||
? [
|
||||
coverage.reason,
|
||||
...("diagnostics" in coverage ? coverage.diagnostics : []),
|
||||
]
|
||||
: coverage.status === "partial"
|
||||
? [
|
||||
...coverage.limitations,
|
||||
...("diagnostics" in coverage ? coverage.diagnostics : []),
|
||||
]
|
||||
: [];
|
||||
limitations.push(
|
||||
return uniqueSorted([
|
||||
...relationshipCoverageLimitations(coverage),
|
||||
...(coverage?.status === "complete" ? [] : (coverage?.diagnostics ?? [])),
|
||||
...(dossier.unresolved_calls ?? []).map(
|
||||
({ address, reason }) => `Unresolved call at ${address}: ${reason}`,
|
||||
),
|
||||
);
|
||||
return uniqueSorted(limitations);
|
||||
]);
|
||||
};
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { analyzeJavaScriptStaticSource } from "./javascriptStaticAnalysis.js";
|
||||
|
||||
const endpoints = (source: string) =>
|
||||
analyzeJavaScriptStaticSource(source).endpoints.map(
|
||||
({ kind, value, mechanism }) => ({ kind, value, mechanism }),
|
||||
);
|
||||
|
||||
describe("server-sent event endpoints", () => {
|
||||
it("reports EventSource URLs like WebSocket URLs", () => {
|
||||
expect(
|
||||
endpoints(`
|
||||
new WebSocket("wss://chat.example/ws");
|
||||
new EventSource("/api/chat/stream");
|
||||
new window.EventSource("https://chat.example/events", { withCredentials: true });
|
||||
`),
|
||||
).toEqual([
|
||||
{
|
||||
kind: "network",
|
||||
value: "wss://chat.example/ws",
|
||||
mechanism: "call:WebSocket",
|
||||
},
|
||||
{
|
||||
kind: "network",
|
||||
value: "/api/chat/stream",
|
||||
mechanism: "call:EventSource",
|
||||
},
|
||||
{
|
||||
kind: "network",
|
||||
value: "https://chat.example/events",
|
||||
mechanism: "call:window.EventSource",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("keeps a dynamic EventSource URL unreported", () => {
|
||||
expect(endpoints("new EventSource(url);")).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,34 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { analyzeJavaScriptStaticSource } from "./javascriptStaticAnalysis.js";
|
||||
|
||||
const endpoints = (source: string) =>
|
||||
analyzeJavaScriptStaticSource(source).endpoints.map(
|
||||
({ kind, value, mechanism }) => ({ kind, value, mechanism }),
|
||||
);
|
||||
|
||||
describe("beacon endpoints", () => {
|
||||
it("reports navigator.sendBeacon URLs", () => {
|
||||
expect(
|
||||
endpoints(`
|
||||
navigator.sendBeacon("/api/telemetry", payload);
|
||||
window.navigator.sendBeacon("https://metrics.example/collect");
|
||||
`),
|
||||
).toEqual([
|
||||
{
|
||||
kind: "network",
|
||||
value: "/api/telemetry",
|
||||
mechanism: "call:navigator.sendBeacon",
|
||||
},
|
||||
{
|
||||
kind: "network",
|
||||
value: "https://metrics.example/collect",
|
||||
mechanism: "call:window.navigator.sendBeacon",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("keeps a dynamic beacon URL unreported", () => {
|
||||
expect(endpoints("navigator.sendBeacon(url, payload);")).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -554,7 +554,13 @@ export const endpointArgument = (
|
||||
receiverFact?: JavaScriptOpenReceiverFact,
|
||||
): string | undefined => {
|
||||
if (receiverFact === "window") return undefined;
|
||||
if (name === "fetch" || name.endsWith(".fetch") || name === "WebSocket")
|
||||
if (
|
||||
name === "fetch" ||
|
||||
name.endsWith(".fetch") ||
|
||||
name === "WebSocket" ||
|
||||
name === "EventSource" ||
|
||||
name.endsWith(".EventSource")
|
||||
)
|
||||
return stringValue(args[0]);
|
||||
if (name.endsWith(".open"))
|
||||
return xhrOpenUrl(name, args[0], args[1], receiverFact);
|
||||
@@ -574,6 +580,9 @@ export const endpointArgument = (
|
||||
? undefined
|
||||
: value;
|
||||
}
|
||||
// navigator.sendBeacon(url, data) queues a POST to its first argument.
|
||||
if (name === "sendBeacon" || name.endsWith(".sendBeacon"))
|
||||
return stringValue(args[0]);
|
||||
if (name.endsWith("loadURL")) return stringValue(args[0]);
|
||||
return undefined;
|
||||
};
|
||||
|
||||
@@ -1,24 +1,51 @@
|
||||
import { z } from "zod";
|
||||
|
||||
const diagnostics = z.array(z.string().min(1)).min(1);
|
||||
const completeCoverage = z.strictObject({ status: z.literal("complete") });
|
||||
const diagnostics = z.array(z.string().min(1)).min(1).exactOptional();
|
||||
const completeCoverage = z.strictObject({
|
||||
status: z.literal("complete"),
|
||||
diagnostics,
|
||||
});
|
||||
const limitedDirections = z.union([
|
||||
z.tuple([z.literal("callers")]).check(z.minLength(1), z.maxLength(1)),
|
||||
z.tuple([z.literal("callees")]).check(z.minLength(1), z.maxLength(1)),
|
||||
z
|
||||
.tuple([z.literal("callers"), z.literal("callees")])
|
||||
.check(z.minLength(2), z.maxLength(2)),
|
||||
]);
|
||||
const partialCoverage = z.strictObject({
|
||||
status: z.literal("partial"),
|
||||
limitations: z.array(z.string().min(1)).min(1),
|
||||
limited_directions: limitedDirections.exactOptional(),
|
||||
diagnostics,
|
||||
});
|
||||
const unknownCoverage = z.strictObject({
|
||||
status: z.literal("unknown"),
|
||||
reason: z.string().min(1),
|
||||
limited_directions: limitedDirections.exactOptional(),
|
||||
diagnostics,
|
||||
});
|
||||
|
||||
/** Explicit coverage state for a provider's observed caller/callee set. */
|
||||
export const relationshipCoverageSchema = z.union([
|
||||
export const relationshipCoverageSchema = z.discriminatedUnion("status", [
|
||||
completeCoverage,
|
||||
completeCoverage.extend({ diagnostics }),
|
||||
partialCoverage,
|
||||
partialCoverage.extend({ diagnostics }),
|
||||
unknownCoverage,
|
||||
unknownCoverage.extend({ diagnostics }),
|
||||
]);
|
||||
|
||||
export type RelationshipCoverage = z.infer<typeof relationshipCoverageSchema>;
|
||||
|
||||
/** Reasons the observed relationship inventory cannot establish completeness. */
|
||||
export const relationshipCoverageLimitations = (
|
||||
coverage: RelationshipCoverage | null | undefined,
|
||||
): readonly string[] => {
|
||||
switch (coverage?.status) {
|
||||
case undefined:
|
||||
return ["Provider did not report relationship coverage."];
|
||||
case "complete":
|
||||
return [];
|
||||
case "partial":
|
||||
return coverage.limitations;
|
||||
case "unknown":
|
||||
return [coverage.reason];
|
||||
}
|
||||
};
|
||||
|
||||
@@ -5,5 +5,5 @@ export const PACKAGE_METADATA = {
|
||||
serverSdkVersion: "2.3.1",
|
||||
clientSdkVersion: "2.3.1",
|
||||
coreSdkVersion: "2.3.1",
|
||||
skillVersion: "41",
|
||||
skillVersion: "42",
|
||||
} as const;
|
||||
|
||||
@@ -40,6 +40,8 @@ import {
|
||||
NATIVE_AOT_PROFILE_CONTRACT_REVISION,
|
||||
} from "../domain/native/nativeAotPe.js";
|
||||
|
||||
const callRelationshipCoverage = "function-body-call-coverage-v5";
|
||||
|
||||
/**
|
||||
* The processor language REA pins for this target, or undefined when Ghidra may
|
||||
* select from the header. The committed profile and the session readback both
|
||||
@@ -125,7 +127,7 @@ export const resolveGhidraAnalysisProfile = async (
|
||||
load_image_observations: "source-mappings-entry-context-v2",
|
||||
function_body_evidence: "complete-inclusive-ranges-v1",
|
||||
function_references: "complete-body-and-entry-reference-manager-v2",
|
||||
call_relationship_coverage: "resolved-call-reference-diagnostics-v1",
|
||||
call_relationship_coverage: callRelationshipCoverage,
|
||||
function_dossier_coverage:
|
||||
"observed-facets-and-call-relationships-v1",
|
||||
string_inventory_evidence: "defined-data-coverage-v1",
|
||||
@@ -222,7 +224,7 @@ export const resolveGhidraAnalysisProfile = async (
|
||||
load_image_observations: "source-mappings-entry-context-v2",
|
||||
function_body_evidence: "complete-inclusive-ranges-v1",
|
||||
function_references: "complete-body-and-entry-reference-manager-v2",
|
||||
call_relationship_coverage: "resolved-call-reference-diagnostics-v1",
|
||||
call_relationship_coverage: callRelationshipCoverage,
|
||||
function_dossier_coverage: "observed-facets-and-call-relationships-v1",
|
||||
string_inventory_evidence: "defined-data-coverage-v1",
|
||||
location_resolution: "explicit-address-exact-entry-symbol-first-v3",
|
||||
|
||||
@@ -15,10 +15,13 @@ import { projectGhidraDoctorInspection } from "./GhidraDoctor.js";
|
||||
const INSTALL = "/opt/ghidra_12.1.4_PUBLIC";
|
||||
const installationHost = (
|
||||
overrides: Partial<GhidraInstallationHost> = {},
|
||||
platform: NodeJS.Platform = "linux",
|
||||
): GhidraInstallationHost => ({
|
||||
readText: () => "application.version=12.1.4\napplication.java.min=21\n",
|
||||
executable: () => true,
|
||||
probeJava: () => ({
|
||||
osArch: "amd64",
|
||||
osName: platform === "win32" ? "Windows 11" : "Linux",
|
||||
version: "21.0.11",
|
||||
major: 21,
|
||||
home: "/usr/lib/jvm/jdk-21",
|
||||
@@ -45,7 +48,7 @@ const inspection = (
|
||||
? {}
|
||||
: { installDir: options.installDir }),
|
||||
},
|
||||
installationHost(overrides),
|
||||
installationHost(overrides, options.platform ?? "linux"),
|
||||
);
|
||||
|
||||
const doctorHost = (
|
||||
@@ -129,6 +132,8 @@ describe("Ghidra doctor integration", () => {
|
||||
{ installDir: INSTALL },
|
||||
{
|
||||
probeJava: () => ({
|
||||
osArch: "amd64",
|
||||
osName: "Linux",
|
||||
version: "17.0.12",
|
||||
major: 17,
|
||||
home: "/usr/lib/jvm/jdk-17",
|
||||
|
||||
@@ -43,7 +43,120 @@ const parseFunctionValueFixture = (
|
||||
return parsed.ok ? { ok: true as const, value: parsed.value.value } : parsed;
|
||||
};
|
||||
|
||||
describe("Ghidra unresolved call locations", () => {
|
||||
it("rejects a noncanonical unresolved call site rather than retaining unverifiable location evidence", () => {
|
||||
expect(
|
||||
parseGhidraFunctionResult("analyze_function", {
|
||||
value: {
|
||||
...functionDossierSchema.parse(ghidraFunctionDossier()),
|
||||
unresolved_calls: [
|
||||
{ address: "not-an-address", reason: "No call destination." },
|
||||
],
|
||||
},
|
||||
limitations: [],
|
||||
relationship_coverage: {
|
||||
status: "partial",
|
||||
limitations: ["An unresolved call remains."],
|
||||
},
|
||||
}),
|
||||
).toMatchObject({ ok: false, error: { _tag: "AnalysisOutputError" } });
|
||||
});
|
||||
});
|
||||
|
||||
describe("Ghidra function-analysis result values", () => {
|
||||
it("retains unavailable pseudocode instead of treating its empty representation as observed source", () => {
|
||||
const availability = {
|
||||
pseudocode: {
|
||||
available: false,
|
||||
reason: "Ghidra does not decompile external functions.",
|
||||
},
|
||||
comments: { available: true },
|
||||
};
|
||||
const parsed = parseGhidraFunctionResult("analyze_function", {
|
||||
value: {
|
||||
...functionDossierSchema.parse(ghidraFunctionDossier()),
|
||||
pseudocode: "",
|
||||
facet_availability: availability,
|
||||
},
|
||||
limitations: [],
|
||||
});
|
||||
expect(parsed).toMatchObject({
|
||||
ok: true,
|
||||
value: { value: { pseudocode: "", facet_availability: availability } },
|
||||
});
|
||||
});
|
||||
it.each([
|
||||
{ status: "complete", diagnostics: ["Resolved static references only."] },
|
||||
{
|
||||
status: "partial",
|
||||
limitations: ["Only the first instruction range was examined."],
|
||||
},
|
||||
{ status: "unknown", reason: "Relationship inventory was unavailable." },
|
||||
{
|
||||
status: "partial",
|
||||
limitations: ["Missing callers."],
|
||||
limited_directions: ["callers"],
|
||||
},
|
||||
{
|
||||
status: "unknown",
|
||||
reason: "Missing callees.",
|
||||
limited_directions: ["callees"],
|
||||
},
|
||||
])("preserves observed $status relationship coverage", (coverage) => {
|
||||
const parsed = parseGhidraFunctionResult("analyze_function", {
|
||||
value: ghidraFunctionDossier(),
|
||||
limitations: [],
|
||||
relationship_coverage: coverage,
|
||||
});
|
||||
expect(parsed).toMatchObject({
|
||||
ok: true,
|
||||
value: { relationshipCoverage: coverage },
|
||||
});
|
||||
});
|
||||
|
||||
it("preserves omitted relationship coverage for legacy observations", () => {
|
||||
const parsed = parseGhidraFunctionResult("analyze_function", {
|
||||
value: ghidraFunctionDossier(),
|
||||
limitations: [],
|
||||
});
|
||||
if (!parsed.ok) throw parsed.error;
|
||||
expect(parsed.value).not.toHaveProperty("relationshipCoverage");
|
||||
});
|
||||
});
|
||||
|
||||
describe("Ghidra relationship metadata validation", () => {
|
||||
it.each([
|
||||
{ status: "partial", limitations: [] },
|
||||
{ status: "unknown", reason: "" },
|
||||
{ status: "complete", limitations: ["Truncated."] },
|
||||
{ status: "partial", limitations: ["Truncated."], limited_directions: [] },
|
||||
{
|
||||
status: "partial",
|
||||
limitations: ["Truncated."],
|
||||
limited_directions: ["callers", "callers"],
|
||||
},
|
||||
{ status: "unknown", reason: "Missing.", limited_directions: ["incoming"] },
|
||||
{ status: "complete", limited_directions: ["callers"] },
|
||||
])("rejects malformed relationship coverage %j", (coverage) => {
|
||||
expect(
|
||||
parseGhidraFunctionResult("analyze_function", {
|
||||
value: ghidraFunctionDossier(),
|
||||
limitations: [],
|
||||
relationship_coverage: coverage,
|
||||
}),
|
||||
).toMatchObject({ ok: false, error: { _tag: "AnalysisOutputError" } });
|
||||
});
|
||||
|
||||
it("rejects relationship metadata on unrelated function operations", () => {
|
||||
expect(
|
||||
parseGhidraFunctionResult("procedure_assembly", {
|
||||
value: "ret",
|
||||
limitations: [],
|
||||
relationship_coverage: { status: "complete" },
|
||||
}),
|
||||
).toMatchObject({ ok: false, error: { _tag: "AnalysisOutputError" } });
|
||||
});
|
||||
|
||||
it("preserves file-offset resolution in Evidence locations and rejects inexact offsets", () => {
|
||||
const selection = {
|
||||
file_offset: 0x210,
|
||||
|
||||
@@ -27,6 +27,10 @@ import {
|
||||
import { nativeApiBoundarySchema } from "../domain/native/nativeApiBoundary.js";
|
||||
import { nativeValueFlowSchema } from "../domain/native/nativeValueFlow.js";
|
||||
import type { EvidenceLocation } from "../domain/evidence.js";
|
||||
import {
|
||||
relationshipCoverageSchema,
|
||||
type RelationshipCoverage,
|
||||
} from "../domain/relationshipCoverage.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js";
|
||||
import {
|
||||
@@ -307,6 +311,7 @@ const ghidraFunctionDossier = functionDossierSchema
|
||||
...value.comments.map(({ address }) => address),
|
||||
...value.callers.map(({ address }) => address),
|
||||
...value.callees.map(({ address }) => address),
|
||||
...(value.unresolved_calls ?? []).map(({ address }) => address),
|
||||
...value.incoming_references.flatMap(referenceAddresses),
|
||||
...value.outgoing_references.flatMap(referenceAddresses),
|
||||
...value.referenced_strings.flatMap(({ address, source_address }) => [
|
||||
@@ -419,6 +424,7 @@ const functionBoundaryObservationOperations: ReadonlySet<GhidraFunctionOperation
|
||||
const functionObservationEnvelope = z.strictObject({
|
||||
value: z.unknown(),
|
||||
limitations: z.array(z.string()),
|
||||
relationship_coverage: relationshipCoverageSchema.exactOptional(),
|
||||
selection: z
|
||||
.strictObject({
|
||||
file_offset: z.number().int().min(0).max(Number.MAX_SAFE_INTEGER),
|
||||
@@ -432,6 +438,7 @@ export interface ParsedGhidraFunctionResult {
|
||||
readonly value: JsonValue;
|
||||
readonly limitations: readonly string[];
|
||||
readonly locations?: readonly EvidenceLocation[];
|
||||
readonly relationshipCoverage?: RelationshipCoverage;
|
||||
}
|
||||
|
||||
/** Require exact, bounded Java-bridge function output before creating Evidence. */
|
||||
@@ -442,6 +449,7 @@ export const parseGhidraFunctionResult = (
|
||||
let observation: {
|
||||
readonly value: unknown;
|
||||
readonly limitations: readonly string[];
|
||||
readonly relationship_coverage?: RelationshipCoverage;
|
||||
readonly selection?: z.infer<
|
||||
typeof functionObservationEnvelope
|
||||
>["selection"];
|
||||
@@ -450,13 +458,17 @@ export const parseGhidraFunctionResult = (
|
||||
const envelope = functionObservationEnvelope.safeParse(value);
|
||||
if (
|
||||
!envelope.success ||
|
||||
(operation !== "analyze_function" &&
|
||||
operation !== "procedure_callers" &&
|
||||
operation !== "procedure_callees" &&
|
||||
envelope.data.relationship_coverage !== undefined) ||
|
||||
(operation !== "procedure_pseudo_code" &&
|
||||
envelope.data.selection !== undefined)
|
||||
)
|
||||
return err(
|
||||
new AnalysisOutputError(
|
||||
operation,
|
||||
"Ghidra bridge omitted the function observation envelope",
|
||||
"Ghidra bridge returned an invalid function observation envelope for this operation",
|
||||
envelope.success ? undefined : { cause: envelope.error },
|
||||
),
|
||||
);
|
||||
@@ -469,6 +481,9 @@ export const parseGhidraFunctionResult = (
|
||||
? ok({
|
||||
value: jsonValueSchema.parse(parsed.data),
|
||||
limitations: observation.limitations,
|
||||
...(observation.relationship_coverage === undefined
|
||||
? {}
|
||||
: { relationshipCoverage: observation.relationship_coverage }),
|
||||
...(observation.selection === undefined
|
||||
? {}
|
||||
: {
|
||||
|
||||
@@ -12,5 +12,7 @@ export const ghidraProfileInstallationHost: GhidraInstallationHost = {
|
||||
home: "/jdk-21",
|
||||
bits: 64,
|
||||
runtime: "jdk",
|
||||
osArch: "amd64",
|
||||
osName: "Linux",
|
||||
}),
|
||||
};
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { spawnSync } from "node:child_process";
|
||||
|
||||
import { elf, fat, thinMach } from "../domain/binaryTarget.fixture.js";
|
||||
import { projectGhidraDoctorInspection } from "./GhidraDoctor.js";
|
||||
import {
|
||||
ghidraInstallationDiagnostics,
|
||||
ghidraJavaEnvironment,
|
||||
inspectGhidraInstallation,
|
||||
type GhidraExecutableHeaderProbe,
|
||||
@@ -16,6 +18,14 @@ import {
|
||||
ghidraJavaRequirement,
|
||||
} from "./GhidraInstallationPolicy.js";
|
||||
|
||||
// The real Java probe shells out via spawnSync; stub it so one test can run
|
||||
// the real probeJava against captured `-XshowSettings:properties` output.
|
||||
// Every other test overrides probeJava, so the stub never fires for them.
|
||||
vi.mock("node:child_process", async (importOriginal) => {
|
||||
const original = await importOriginal<typeof import("node:child_process")>();
|
||||
return { ...original, spawnSync: vi.fn() };
|
||||
});
|
||||
|
||||
const INSTALL = "/opt/ghidra";
|
||||
const PROPERTIES = `${INSTALL}/Ghidra/application.properties`;
|
||||
const HEADLESS = `${INSTALL}/support/analyzeHeadless`;
|
||||
@@ -32,6 +42,8 @@ const headerProbe = (bytes: Buffer): GhidraExecutableHeaderProbe => ({
|
||||
});
|
||||
|
||||
const JAVA: GhidraJavaObservation = {
|
||||
osArch: "amd64",
|
||||
osName: "Linux",
|
||||
version: "21.0.11",
|
||||
major: SUPPORTED_GHIDRA_JAVA_MAJOR,
|
||||
home: "/usr/lib/jvm/jdk-21",
|
||||
@@ -174,6 +186,11 @@ describe("Ghidra installation inspection", () => {
|
||||
{ environment: {}, installDir: INSTALL, platform, architecture },
|
||||
host({
|
||||
executable: (path) => path === HEADLESS || path === decompiler,
|
||||
probeJava: () => ({
|
||||
...JAVA,
|
||||
osArch: architecture === "arm64" ? "aarch64" : "amd64",
|
||||
osName: platform === "darwin" ? "Mac OS X" : "Linux",
|
||||
}),
|
||||
}),
|
||||
),
|
||||
).toMatchObject({
|
||||
@@ -202,6 +219,7 @@ describe("Ghidra installation inspection", () => {
|
||||
return {
|
||||
...JAVA,
|
||||
home: "C:\\Java\\jdk-21",
|
||||
osName: "Windows 11",
|
||||
};
|
||||
},
|
||||
};
|
||||
@@ -721,7 +739,10 @@ describe("Ghidra FAT native decompiler admission", () => {
|
||||
platform: "darwin",
|
||||
architecture: "arm64",
|
||||
},
|
||||
host({ executableHeader: () => headerProbe(bytes) }),
|
||||
host({
|
||||
probeJava: () => ({ ...JAVA, osArch: "aarch64", osName: "Mac OS X" }),
|
||||
executableHeader: () => headerProbe(bytes),
|
||||
}),
|
||||
);
|
||||
expect(result.status).toBe(status);
|
||||
expect(
|
||||
@@ -739,7 +760,10 @@ describe("Ghidra FAT native decompiler admission", () => {
|
||||
platform: "darwin",
|
||||
architecture: "arm64",
|
||||
},
|
||||
host({ executableHeader: () => headerProbe(bytes) }),
|
||||
host({
|
||||
probeJava: () => ({ ...JAVA, osArch: "aarch64", osName: "Mac OS X" }),
|
||||
executableHeader: () => headerProbe(bytes),
|
||||
}),
|
||||
);
|
||||
expect(result.status).toBe("available");
|
||||
expect(
|
||||
@@ -788,7 +812,14 @@ describe("Ghidra native decompiler admission", () => {
|
||||
])("distinguishes $name", ({ options, executableHeader }) => {
|
||||
const result = inspectGhidraInstallation(
|
||||
options,
|
||||
host({ executableHeader }),
|
||||
host({
|
||||
probeJava: () => ({
|
||||
...JAVA,
|
||||
osArch: options.architecture === "arm64" ? "aarch64" : "amd64",
|
||||
osName: options.platform === "darwin" ? "Mac OS X" : "Linux",
|
||||
}),
|
||||
executableHeader,
|
||||
}),
|
||||
);
|
||||
expect(result.status).toBe("unavailable");
|
||||
expect(
|
||||
@@ -849,6 +880,7 @@ describe("Ghidra native decompiler admission", () => {
|
||||
architecture: "arm64",
|
||||
},
|
||||
host({
|
||||
probeJava: () => ({ ...JAVA, osArch: "aarch64", osName: "Mac OS X" }),
|
||||
executableHeader: (path: string) =>
|
||||
path === MAC_DECOMPILER ? headerProbe(bytes) : undefined,
|
||||
}),
|
||||
@@ -872,6 +904,7 @@ describe("Ghidra native decompiler admission", () => {
|
||||
architecture: "arm64",
|
||||
},
|
||||
host({
|
||||
probeJava: () => ({ ...JAVA, osArch: "aarch64", osName: "Mac OS X" }),
|
||||
executableHeader: (path: string) =>
|
||||
path === MAC_DECOMPILER ? headerProbe(elf(2, 1, 62)) : undefined,
|
||||
}),
|
||||
@@ -889,3 +922,205 @@ describe("Ghidra native decompiler admission", () => {
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("Ghidra JVM native directory admission", () => {
|
||||
it("rejects a JDK whose os.arch selects a different native decompiler directory", () => {
|
||||
const result = inspectGhidraInstallation(
|
||||
{
|
||||
environment: {},
|
||||
installDir: INSTALL,
|
||||
platform: "darwin",
|
||||
architecture: "arm64",
|
||||
},
|
||||
host({
|
||||
probeJava: () => ({ ...JAVA, osArch: "x86_64", osName: "Mac OS X" }),
|
||||
}),
|
||||
);
|
||||
expect(result.status).toBe("unavailable");
|
||||
if (result.status !== "unavailable") return;
|
||||
expect(
|
||||
result.checks.find(({ name }) => name === "native_decompiler"),
|
||||
).toMatchObject({
|
||||
status: "failed",
|
||||
code: "executable_missing",
|
||||
detail: expect.stringContaining(
|
||||
"JVM reports os.arch=x86_64, so Ghidra selects its native decompiler from os/mac_x86_64",
|
||||
),
|
||||
remediation: expect.stringContaining("JAVA_HOME"),
|
||||
});
|
||||
});
|
||||
|
||||
it("accepts a JDK whose os.arch selects the validated native decompiler directory", () => {
|
||||
const result = inspectGhidraInstallation(
|
||||
{
|
||||
environment: {},
|
||||
installDir: INSTALL,
|
||||
platform: "darwin",
|
||||
architecture: "arm64",
|
||||
},
|
||||
host({
|
||||
probeJava: () => ({ ...JAVA, osArch: "aarch64", osName: "Mac OS X" }),
|
||||
}),
|
||||
);
|
||||
expect(result.status).toBe("available");
|
||||
});
|
||||
|
||||
it("rejects a successful Java probe that omits os.arch", () => {
|
||||
const { osArch: _observedArchitecture, ...withoutArchitecture } = JAVA;
|
||||
const result = inspectGhidraInstallation(
|
||||
{
|
||||
environment: {},
|
||||
installDir: INSTALL,
|
||||
platform: "darwin",
|
||||
architecture: "arm64",
|
||||
},
|
||||
host({ probeJava: () => withoutArchitecture }),
|
||||
);
|
||||
expect(result.status).toBe("unavailable");
|
||||
expect(
|
||||
result.checks.find(({ name }) => name === "native_decompiler"),
|
||||
).toMatchObject({
|
||||
status: "failed",
|
||||
detail: expect.stringContaining("did not report os.arch"),
|
||||
});
|
||||
expect(projectGhidraDoctorInspection(result).available).toBe(false);
|
||||
});
|
||||
|
||||
it.each(["riscv64", "sparcv9", ""])(
|
||||
"refuses an unrecognized JVM os.arch (%s) instead of assuming alignment",
|
||||
(osArch) => {
|
||||
const result = inspectGhidraInstallation(
|
||||
{
|
||||
environment: {},
|
||||
installDir: INSTALL,
|
||||
platform: "darwin",
|
||||
architecture: "arm64",
|
||||
},
|
||||
host({
|
||||
probeJava: () => ({ ...JAVA, osArch }),
|
||||
}),
|
||||
);
|
||||
expect(result.status).toBe("unavailable");
|
||||
if (result.status !== "unavailable") return;
|
||||
expect(
|
||||
result.checks.find(({ name }) => name === "native_decompiler"),
|
||||
).toMatchObject({
|
||||
status: "failed",
|
||||
detail: expect.stringContaining("unrecognized os.arch="),
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("rejects a JDK whose os.name selects a different host OS", () => {
|
||||
const result = inspectGhidraInstallation(
|
||||
{
|
||||
environment: {},
|
||||
installDir: INSTALL,
|
||||
platform: "darwin",
|
||||
architecture: "arm64",
|
||||
},
|
||||
host({
|
||||
probeJava: () => ({
|
||||
...JAVA,
|
||||
osArch: "aarch64",
|
||||
osName: "Windows 11",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
expect(result.status).toBe("unavailable");
|
||||
if (result.status !== "unavailable") return;
|
||||
expect(
|
||||
result.checks.find(({ name }) => name === "native_decompiler"),
|
||||
).toMatchObject({
|
||||
status: "failed",
|
||||
code: "executable_missing",
|
||||
detail: expect.stringContaining("JVM reports os.name=Windows 11"),
|
||||
remediation: expect.stringContaining("JAVA_HOME"),
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("Ghidra JVM OS admission", () => {
|
||||
it("rejects a successful Java probe that omits os.name", () => {
|
||||
const { osName: _observedOsName, ...withoutOsName } = JAVA;
|
||||
const result = inspectGhidraInstallation(
|
||||
{
|
||||
environment: {},
|
||||
installDir: INSTALL,
|
||||
platform: "linux",
|
||||
architecture: "x64",
|
||||
},
|
||||
host({ probeJava: () => withoutOsName }),
|
||||
);
|
||||
expect(result.status).toBe("unavailable");
|
||||
expect(
|
||||
result.checks.find(({ name }) => name === "native_decompiler"),
|
||||
).toMatchObject({
|
||||
status: "failed",
|
||||
detail: expect.stringContaining("did not report os.name"),
|
||||
});
|
||||
expect(projectGhidraDoctorInspection(result).available).toBe(false);
|
||||
});
|
||||
|
||||
it.each(["SunOS", "FreeBSD", ""])(
|
||||
"refuses an unrecognized JVM os.name (%s) instead of assuming alignment",
|
||||
(osName) => {
|
||||
const result = inspectGhidraInstallation(
|
||||
{
|
||||
environment: {},
|
||||
installDir: INSTALL,
|
||||
platform: "linux",
|
||||
architecture: "x64",
|
||||
},
|
||||
host({
|
||||
probeJava: () => ({ ...JAVA, osName }),
|
||||
}),
|
||||
);
|
||||
expect(result.status).toBe("unavailable");
|
||||
if (result.status !== "unavailable") return;
|
||||
expect(
|
||||
result.checks.find(({ name }) => name === "native_decompiler"),
|
||||
).toMatchObject({
|
||||
status: "failed",
|
||||
detail: expect.stringContaining("os.name="),
|
||||
});
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("Ghidra Java probe os.arch extraction", () => {
|
||||
it("extracts os.arch through the real Java probe into the installation diagnostics", () => {
|
||||
vi.mocked(spawnSync).mockReturnValue({
|
||||
status: 0,
|
||||
stdout: "",
|
||||
stderr: [
|
||||
"Property settings:",
|
||||
" file.encoding = UTF-8",
|
||||
" java.home = /usr/lib/jvm/jdk-21",
|
||||
" java.version = 21.0.11",
|
||||
" java.vm.name = OpenJDK 64-Bit Server VM",
|
||||
" os.arch = x86_64",
|
||||
" os.name = Linux",
|
||||
" sun.arch.data.model = 64",
|
||||
"",
|
||||
].join("\n"),
|
||||
} as never);
|
||||
try {
|
||||
// No host override: the real system host runs the real probeJava
|
||||
// against the stubbed spawnSync.
|
||||
const inspection = inspectGhidraInstallation({
|
||||
environment: {},
|
||||
installDir: INSTALL,
|
||||
javaHome: "/usr/lib/jvm/jdk-21",
|
||||
platform: "linux",
|
||||
architecture: "x64",
|
||||
});
|
||||
expect(ghidraInstallationDiagnostics(inspection)).toMatchObject({
|
||||
java_os_arch: "x86_64",
|
||||
java_os_name: "Linux",
|
||||
});
|
||||
} finally {
|
||||
vi.mocked(spawnSync).mockReset();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
@@ -203,6 +203,17 @@ interface GhidraInstallationObservation {
|
||||
readonly javaCommand: string;
|
||||
/** Observed java.vm.name; omitted when the probe cannot establish VM identity. */
|
||||
readonly javaVmName?: string;
|
||||
/**
|
||||
* Observed JVM os.arch; omitted when the probe cannot establish it. Ghidra
|
||||
* selects its native decompiler directory from this value.
|
||||
*/
|
||||
readonly javaOsArch?: string;
|
||||
/**
|
||||
* Observed JVM os.name; omitted when the probe cannot establish it. Ghidra
|
||||
* selects its native decompiler OS directory from the JVM OS/architecture
|
||||
* pair, so a non-host value selects an unvalidated directory.
|
||||
*/
|
||||
readonly javaOsName?: string;
|
||||
readonly checks: readonly GhidraInstallationCheck[];
|
||||
}
|
||||
|
||||
@@ -244,6 +255,17 @@ export interface GhidraJavaObservation {
|
||||
readonly bits: number;
|
||||
readonly runtime: "jdk" | "jre";
|
||||
readonly vmName?: string;
|
||||
/**
|
||||
* Observed JVM os.arch property, which Ghidra uses to select its native
|
||||
* decompiler directory. Omitted when the probe cannot establish it.
|
||||
*/
|
||||
readonly osArch?: string;
|
||||
/**
|
||||
* Observed JVM os.name property, which Ghidra uses with os.arch to select
|
||||
* its native decompiler OS directory. Omitted when the probe cannot
|
||||
* establish it.
|
||||
*/
|
||||
readonly osName?: string;
|
||||
}
|
||||
|
||||
/** A bounded executable-header probe: a small prefix plus the whole file size. */
|
||||
@@ -324,6 +346,12 @@ export const inspectGhidraInstallation = (
|
||||
...(coordinates.java?.vmName === undefined
|
||||
? {}
|
||||
: { javaVmName: coordinates.java.vmName }),
|
||||
...(coordinates.java?.osArch === undefined
|
||||
? {}
|
||||
: { javaOsArch: coordinates.java.osArch }),
|
||||
...(coordinates.java?.osName === undefined
|
||||
? {}
|
||||
: { javaOsName: coordinates.java.osName }),
|
||||
checks,
|
||||
};
|
||||
if (failed !== undefined)
|
||||
@@ -555,6 +583,8 @@ export const ghidraInstallationDiagnostics = (
|
||||
java_home: inspection.javaHome,
|
||||
java_version: inspection.javaVersion,
|
||||
java_vm_name: inspection.javaVmName ?? null,
|
||||
java_os_arch: inspection.javaOsArch ?? null,
|
||||
java_os_name: inspection.javaOsName ?? null,
|
||||
platform: inspection.platform,
|
||||
architecture: inspection.architecture,
|
||||
checks: inspection.checks.map((check) => ({
|
||||
@@ -632,6 +662,61 @@ const nativeDecompilerCheck = ({
|
||||
architecture,
|
||||
host.executableHeader?.(path),
|
||||
);
|
||||
// Ghidra resolves its native decompiler directory from the JVM's os.arch,
|
||||
// not the host process architecture. A JDK running under emulation (for
|
||||
// example x86_64 under Rosetta on Apple silicon) selects a directory that
|
||||
// was never validated, so every analysis fails after a passing doctor.
|
||||
// Ghidra resolves the OS directory from the JVM os.name/os.arch pair, so a
|
||||
// JVM reporting a non-host os.name (for example through -Dos.name=Windows)
|
||||
// selects a different OS directory even when os.arch matches.
|
||||
const osArch = coordinates.java?.osArch;
|
||||
const osName = coordinates.java?.osName;
|
||||
const selectedDirectory = jvmOsArchNativeDirectory(platform, osArch);
|
||||
const validatedDirectory = NATIVE_PLATFORMS[platform]?.[architecture];
|
||||
// Without a JVM observation the java check reports the missing runtime;
|
||||
// the decompiler directory cannot be selected, so this check defers to it.
|
||||
if (coordinates.java !== undefined) {
|
||||
if (selectedDirectory === undefined)
|
||||
return installationCheck({
|
||||
name: "native_decompiler",
|
||||
passed: false,
|
||||
code: "executable_missing",
|
||||
detail:
|
||||
osArch === undefined
|
||||
? "The Java probe did not report os.arch; the JVM-selected native decompiler directory is unknown and cannot be matched to the validated host directory"
|
||||
: `JVM reports unrecognized os.arch=${osArch}; its selected native decompiler directory is unknown and cannot be matched to the validated host directory`,
|
||||
remediation: `Point JAVA_HOME at a JDK whose reported architecture maps to the validated Ghidra decompiler (${validatedDirectory ?? "unsupported host"}). REA does not build or install native tools.`,
|
||||
});
|
||||
if (
|
||||
validatedDirectory !== undefined &&
|
||||
selectedDirectory !== validatedDirectory &&
|
||||
path !== null
|
||||
)
|
||||
return installationCheck({
|
||||
name: "native_decompiler",
|
||||
passed: false,
|
||||
code: "executable_missing",
|
||||
detail: `JVM reports os.arch=${osArch}, so Ghidra selects its native decompiler from os/${selectedDirectory}, but the validated decompiler targets ${validatedDirectory} (host ${platform}/${architecture})`,
|
||||
remediation: `Point JAVA_HOME at a JDK whose architecture matches the built decompiler (${validatedDirectory}). REA does not build or install native tools.`,
|
||||
});
|
||||
if (osName === undefined)
|
||||
return installationCheck({
|
||||
name: "native_decompiler",
|
||||
passed: false,
|
||||
code: "executable_missing",
|
||||
detail:
|
||||
"The Java probe did not report os.name; the JVM-selected native decompiler OS directory is unknown and cannot be matched to the validated host platform",
|
||||
remediation: `Point JAVA_HOME at a JDK whose reported operating system matches the host platform (${platform}). REA does not build or install native tools.`,
|
||||
});
|
||||
if (!jvmOsNameMatchesPlatform(platform, osName) && path !== null)
|
||||
return installationCheck({
|
||||
name: "native_decompiler",
|
||||
passed: false,
|
||||
code: "executable_missing",
|
||||
detail: `JVM reports os.name=${osName}, which does not match the validated host platform ${platform} (host ${platform}/${architecture})`,
|
||||
remediation: `Point JAVA_HOME at a JDK whose reported operating system matches the host platform (${platform}). REA does not build or install native tools.`,
|
||||
});
|
||||
}
|
||||
return installationCheck({
|
||||
name: "native_decompiler",
|
||||
passed: path !== null && compatibility?.status !== "incompatible",
|
||||
@@ -646,6 +731,43 @@ const nativeDecompilerCheck = ({
|
||||
});
|
||||
};
|
||||
|
||||
/**
|
||||
* Whether a JVM os.name value selects the host platform's native OS directory.
|
||||
* Unknown values never align with the validated host directory; admission
|
||||
* must not treat them as matching.
|
||||
*/
|
||||
const jvmOsNameMatchesPlatform = (
|
||||
platform: NodeJS.Platform,
|
||||
osName: string,
|
||||
): boolean => {
|
||||
const normalized = osName.toLowerCase();
|
||||
if (platform === "win32") return normalized.startsWith("windows");
|
||||
if (platform === "darwin")
|
||||
return normalized.startsWith("mac") || normalized.startsWith("darwin");
|
||||
if (platform === "linux") return normalized.startsWith("linux");
|
||||
return false;
|
||||
};
|
||||
|
||||
/**
|
||||
* Map a JVM os.arch value to the Ghidra native directory that JVM selects.
|
||||
* Unknown values stay unknown; admission must not treat them as aligned with
|
||||
* the validated host directory.
|
||||
*/
|
||||
const jvmOsArchNativeDirectory = (
|
||||
platform: NodeJS.Platform,
|
||||
osArch: string | undefined,
|
||||
): string | undefined => {
|
||||
if (osArch === undefined) return undefined;
|
||||
const normalized = osArch.toLowerCase();
|
||||
const node: NodeJS.Architecture | undefined =
|
||||
normalized === "x86_64" || normalized === "amd64" || normalized === "x64"
|
||||
? "x64"
|
||||
: normalized === "aarch64" || normalized === "arm64"
|
||||
? "arm64"
|
||||
: undefined;
|
||||
return node === undefined ? undefined : NATIVE_PLATFORMS[platform]?.[node];
|
||||
};
|
||||
|
||||
const versionCheck = (
|
||||
properties: GhidraApplicationProperties | undefined,
|
||||
): GhidraInstallationCheck => {
|
||||
@@ -781,6 +903,8 @@ const systemGhidraInstallationHost = (): GhidraInstallationHost => ({
|
||||
const version = javaProperty(output, "java.version");
|
||||
const home = javaProperty(output, "java.home");
|
||||
const vmName = javaProperty(output, "java.vm.name");
|
||||
const osArch = javaProperty(output, "os.arch");
|
||||
const osName = javaProperty(output, "os.name");
|
||||
const bits = Number.parseInt(
|
||||
javaProperty(output, "sun.arch.data.model") ?? "0",
|
||||
10,
|
||||
@@ -791,6 +915,8 @@ const systemGhidraInstallationHost = (): GhidraInstallationHost => ({
|
||||
major: javaMajor(version),
|
||||
home,
|
||||
...(vmName === null ? {} : { vmName }),
|
||||
...(osArch === null ? {} : { osArch }),
|
||||
...(osName === null ? {} : { osName }),
|
||||
bits,
|
||||
runtime: executablePath(
|
||||
(process.platform === "win32" ? win32 : posix).join(
|
||||
|
||||
@@ -2,11 +2,43 @@ import { snapshotEnvironment } from "../process/snapshotEnvironment.js";
|
||||
import { dirname, join } from "node:path";
|
||||
import { execFileOutput } from "../process/ExecFileOutput.js";
|
||||
|
||||
/** JVM option environments remove quotes, including quotes inside an option. */
|
||||
const jvmEnvironmentArguments = (value: string): readonly string[] => {
|
||||
const arguments_: string[] = [];
|
||||
let argument = "";
|
||||
let quote: "'" | '"' | undefined;
|
||||
for (const character of value) {
|
||||
if (quote !== undefined) {
|
||||
if (character === quote) quote = undefined;
|
||||
else argument += character;
|
||||
} else if (character === "'" || character === '"') quote = character;
|
||||
else if (/\s/u.test(character)) {
|
||||
if (argument !== "") arguments_.push(argument);
|
||||
argument = "";
|
||||
} else argument += character;
|
||||
}
|
||||
if (quote !== undefined)
|
||||
throw new Error(
|
||||
"Invalid quoted Ghidra LaunchSupport JVM environment option",
|
||||
);
|
||||
if (argument !== "") arguments_.push(argument);
|
||||
return arguments_;
|
||||
};
|
||||
|
||||
const rejectJvmPlatformOverrides = (arguments_: readonly string[]): void => {
|
||||
if (
|
||||
arguments_.some((argument) => /^-Dos\.(?:arch|name)(?:=|$)/u.test(argument))
|
||||
)
|
||||
throw new Error(
|
||||
"Ghidra LaunchSupport may not override os.arch or os.name; the native decompiler was validated for the inspected JVM platform.",
|
||||
);
|
||||
};
|
||||
|
||||
/** Launch coordinates obtained from Ghidra's own release configuration helper. */
|
||||
export const ghidraJavaLaunch = async (options: {
|
||||
readonly platform: NodeJS.Platform;
|
||||
readonly analyzeHeadlessPath: string;
|
||||
readonly javaHome: string;
|
||||
readonly javaHome?: string;
|
||||
readonly javaVmName?: string;
|
||||
readonly homeRoot: string;
|
||||
readonly tempRoot: string;
|
||||
@@ -21,7 +53,17 @@ export const ghidraJavaLaunch = async (options: {
|
||||
}> => {
|
||||
const supportRoot = dirname(options.analyzeHeadlessPath);
|
||||
const installRoot = dirname(supportRoot);
|
||||
const command = join(options.javaHome, "bin", "java");
|
||||
const selectedJavaHome = options.javaHome ?? options.environment.JAVA_HOME;
|
||||
const command =
|
||||
selectedJavaHome === undefined
|
||||
? options.platform === "win32"
|
||||
? "java.exe"
|
||||
: "java"
|
||||
: join(
|
||||
selectedJavaHome,
|
||||
"bin",
|
||||
options.platform === "win32" ? "java.exe" : "java",
|
||||
);
|
||||
const isolatedProperties = [
|
||||
`-Duser.home=${options.homeRoot}`,
|
||||
`-Djava.io.tmpdir=${options.tempRoot}`,
|
||||
@@ -48,6 +90,9 @@ export const ghidraJavaLaunch = async (options: {
|
||||
[...supportArguments, "-envvars"],
|
||||
{
|
||||
env: environment,
|
||||
// Wait for the helper's exit on abort so a cancelled or timed-out
|
||||
// startup never leaves the probe running without an owning process.
|
||||
stopSignal: "SIGTERM",
|
||||
...(options.signal === undefined ? {} : { signal: options.signal }),
|
||||
},
|
||||
);
|
||||
@@ -65,11 +110,25 @@ export const ghidraJavaLaunch = async (options: {
|
||||
if ((environment[key] ?? "") === "")
|
||||
environment[key] = line.slice(separator + 1);
|
||||
}
|
||||
// The merged environment becomes the analysis JVM's environment, and the
|
||||
// JVM applies these options variables automatically. An injected
|
||||
// -Dos.arch/-Dos.name there selects an unvalidated native directory without
|
||||
// appearing in vmArguments, so reject the effective values as well.
|
||||
for (const key of [
|
||||
"JAVA_TOOL_OPTIONS",
|
||||
"_JAVA_OPTIONS",
|
||||
"JDK_JAVA_OPTIONS",
|
||||
] as const) {
|
||||
rejectJvmPlatformOverrides(jvmEnvironmentArguments(environment[key] ?? ""));
|
||||
}
|
||||
const configuredArguments = await execFileOutput(
|
||||
command,
|
||||
[...supportArguments, "-vmargs"],
|
||||
{
|
||||
env: environment,
|
||||
// Wait for the helper's exit on abort so a cancelled or timed-out
|
||||
// startup never leaves the probe running without an owning process.
|
||||
stopSignal: "SIGTERM",
|
||||
...(options.signal === undefined ? {} : { signal: options.signal }),
|
||||
},
|
||||
);
|
||||
@@ -83,6 +142,7 @@ export const ghidraJavaLaunch = async (options: {
|
||||
)
|
||||
)
|
||||
throw new Error("Invalid Ghidra LaunchSupport JVM argument");
|
||||
rejectJvmPlatformOverrides(vmArguments);
|
||||
return {
|
||||
command,
|
||||
environment,
|
||||
|
||||
@@ -172,6 +172,8 @@ export interface GhidraHeadlessLauncherOptions {
|
||||
readonly mipsSoftFloat?: true;
|
||||
/** Spawn seam for provider-boundary lifecycle tests. */
|
||||
readonly spawnProcess?: typeof spawnOwnedProviderProcess;
|
||||
/** LaunchSupport probe seam for Java-free launcher fixtures. */
|
||||
readonly javaLaunch?: typeof ghidraJavaLaunch;
|
||||
}
|
||||
|
||||
/** Launch Ghidra without copying scripts into or modifying its installation. */
|
||||
@@ -303,43 +305,46 @@ export class GhidraHeadlessLauncher implements GhidraLauncher {
|
||||
throw new GhidraLaunchError(
|
||||
"POSIX Ghidra requires its inspected JDK home",
|
||||
);
|
||||
const inspectedJavaLaunch = await (
|
||||
this.options.javaLaunch ?? ghidraJavaLaunch
|
||||
)({
|
||||
platform,
|
||||
analyzeHeadlessPath: this.options.analyzeHeadlessPath,
|
||||
...(this.options.javaHome === undefined
|
||||
? {}
|
||||
: { javaHome: this.options.javaHome }),
|
||||
...(this.options.javaVmName === undefined
|
||||
? {}
|
||||
: { javaVmName: this.options.javaVmName }),
|
||||
homeRoot: paths.homeRoot,
|
||||
tempRoot: paths.tempRoot,
|
||||
arguments: headlessArguments,
|
||||
environment,
|
||||
onStderr: (mode, stderr) => {
|
||||
if (stderr === "") return;
|
||||
const bytes = Buffer.byteLength(stderr);
|
||||
const redacted = redactCapturedTransportCredential(
|
||||
{ text: stderr, bytes, observedBytes: bytes },
|
||||
session.token,
|
||||
"[REDACTED]",
|
||||
);
|
||||
const bounded = boundedUtf8Text(
|
||||
redacted,
|
||||
GHIDRA_LAUNCH_COMMAND_OUTPUT_BYTES,
|
||||
);
|
||||
launchSupport.push({
|
||||
mode,
|
||||
stderr: bounded.text,
|
||||
stderr_bytes: bytes,
|
||||
stderr_truncated: bounded.truncated,
|
||||
});
|
||||
},
|
||||
...(options.signal === undefined ? {} : { signal: options.signal }),
|
||||
});
|
||||
const command =
|
||||
platform !== "win32" && this.options.javaHome !== undefined
|
||||
? await ghidraJavaLaunch({
|
||||
platform,
|
||||
analyzeHeadlessPath: this.options.analyzeHeadlessPath,
|
||||
javaHome: this.options.javaHome,
|
||||
...(this.options.javaVmName === undefined
|
||||
? {}
|
||||
: { javaVmName: this.options.javaVmName }),
|
||||
homeRoot: paths.homeRoot,
|
||||
tempRoot: paths.tempRoot,
|
||||
arguments: headlessArguments,
|
||||
environment,
|
||||
onStderr: (mode, stderr) => {
|
||||
if (stderr === "") return;
|
||||
const bytes = Buffer.byteLength(stderr);
|
||||
const redacted = redactCapturedTransportCredential(
|
||||
{ text: stderr, bytes, observedBytes: bytes },
|
||||
session.token,
|
||||
"[REDACTED]",
|
||||
);
|
||||
const bounded = boundedUtf8Text(
|
||||
redacted,
|
||||
GHIDRA_LAUNCH_COMMAND_OUTPUT_BYTES,
|
||||
);
|
||||
launchSupport.push({
|
||||
mode,
|
||||
stderr: bounded.text,
|
||||
stderr_bytes: bytes,
|
||||
stderr_truncated: bounded.truncated,
|
||||
});
|
||||
},
|
||||
...(options.signal === undefined
|
||||
? {}
|
||||
: { signal: options.signal }),
|
||||
})
|
||||
: { ...scriptCommand, environment };
|
||||
platform === "win32"
|
||||
? { ...scriptCommand, environment }
|
||||
: inspectedJavaLaunch;
|
||||
started = await (this.options.spawnProcess ?? spawnOwnedProviderProcess)({
|
||||
command: command.command,
|
||||
arguments: command.arguments,
|
||||
|
||||
@@ -37,12 +37,16 @@ import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
|
||||
import { ProviderStartupDeadline } from "../process/ProviderDeadline.js";
|
||||
|
||||
const INSTALL = "/opt/ghidra_12.1.4_PUBLIC";
|
||||
const installationHost = (): GhidraInstallationHost => ({
|
||||
platform: "linux",
|
||||
const installationHost = (
|
||||
platform: NodeJS.Platform = "linux",
|
||||
): GhidraInstallationHost => ({
|
||||
platform,
|
||||
architecture: "x64",
|
||||
readText: () => "application.version=12.1.4\napplication.java.min=21\n",
|
||||
executable: () => true,
|
||||
probeJava: () => ({
|
||||
osArch: "amd64",
|
||||
osName: platform === "win32" ? "Windows 11" : "Linux",
|
||||
version: "21.0.11",
|
||||
major: 21,
|
||||
home: "/usr/lib/jvm/jdk-21",
|
||||
@@ -102,7 +106,7 @@ describe("Ghidra provider", () => {
|
||||
},
|
||||
},
|
||||
});
|
||||
const windows = provider({ ...installationHost(), platform: "win32" });
|
||||
const windows = provider(installationHost("win32"));
|
||||
expect(windows.inspectTargetSupport(target).status).toBe("unsupported");
|
||||
});
|
||||
it("discovers the exact installation once without launching Ghidra", () => {
|
||||
@@ -112,6 +116,8 @@ describe("Ghidra provider", () => {
|
||||
probeJava: () => {
|
||||
probeCount += 1;
|
||||
return {
|
||||
osArch: "amd64",
|
||||
osName: "Linux",
|
||||
version: "21.0.11",
|
||||
major: 21,
|
||||
home: "/usr/lib/jvm/jdk-21",
|
||||
@@ -193,7 +199,7 @@ describe("Ghidra provider", () => {
|
||||
|
||||
describe("Ghidra raw-binary profile", () => {
|
||||
it("admits raw 6502 on Windows and binds its 16-bit language without mutation authority", async () => {
|
||||
const ghidra = provider({ ...installationHost(), platform: "win32" });
|
||||
const ghidra = provider(installationHost("win32"));
|
||||
const target: BinaryTarget = {
|
||||
path: "/tmp/6502.bin",
|
||||
sha256: "a".repeat(64),
|
||||
@@ -260,7 +266,7 @@ describe("Ghidra raw-binary profile", () => {
|
||||
expect(shifted.value.profile?.digest).not.toBe(
|
||||
resolved.value.profile.digest,
|
||||
);
|
||||
const windows = provider({ ...installationHost(), platform: "win32" });
|
||||
const windows = provider(installationHost("win32"));
|
||||
expect(windows.inspectTargetSupport(target).status).toBe("unsupported");
|
||||
});
|
||||
});
|
||||
@@ -302,7 +308,7 @@ describe("Ghidra Mach-O slice support", () => {
|
||||
|
||||
describe("Ghidra platform support", () => {
|
||||
it("commits the PE role and managed classification to profile identity", async () => {
|
||||
const ghidra = provider({ ...installationHost(), platform: "win32" });
|
||||
const ghidra = provider(installationHost("win32"));
|
||||
const application = peTarget("x86_64");
|
||||
const library = {
|
||||
...application,
|
||||
@@ -343,7 +349,7 @@ describe("Ghidra platform support", () => {
|
||||
});
|
||||
|
||||
it("keeps Windows annotation mutation unavailable independently of native controls", () => {
|
||||
const ghidra = provider({ ...installationHost(), platform: "win32" });
|
||||
const ghidra = provider(installationHost("win32"));
|
||||
expect(
|
||||
ghidra
|
||||
.capabilities()
|
||||
@@ -363,7 +369,7 @@ describe("Ghidra platform support", () => {
|
||||
reason: WINDOWS_NATIVE_AUTHORITY_UNAVAILABLE_REASON,
|
||||
});
|
||||
onTestFinished(() => native.mockRestore());
|
||||
const ghidra = provider({ ...installationHost(), platform: "win32" });
|
||||
const ghidra = provider(installationHost("win32"));
|
||||
const nativeApplication = peTarget("x86_64");
|
||||
|
||||
for (const architecture of ["x86", "x86_64"] as const)
|
||||
@@ -454,7 +460,7 @@ describe("Ghidra Windows PE fixture admission", () => {
|
||||
await promisify(execFile)(process.execPath, [
|
||||
join(root, "scripts/create-ghidra-windows-fixture.mjs"),
|
||||
]);
|
||||
const ghidra = provider({ ...installationHost(), platform: "win32" });
|
||||
const ghidra = provider(installationHost("win32"));
|
||||
for (const architecture of ["x86", "x86_64"] as const) {
|
||||
for (const dll of [false, true]) {
|
||||
const path = join(
|
||||
@@ -627,7 +633,7 @@ describe("Ghidra client projection", () => {
|
||||
expect(resolved.value.profile.parameters).toMatchObject({
|
||||
language_id: "auto-from-header",
|
||||
compiler_spec_id: "auto-default",
|
||||
call_relationship_coverage: "resolved-call-reference-diagnostics-v1",
|
||||
call_relationship_coverage: "function-body-call-coverage-v5",
|
||||
function_dossier_coverage: "observed-facets-and-call-relationships-v1",
|
||||
});
|
||||
|
||||
@@ -864,7 +870,16 @@ describe("Ghidra relationship coverage", () => {
|
||||
Promise.resolve(
|
||||
ok(
|
||||
operation === "procedure_callees"
|
||||
? { value: ["0x2000"], limitations: [] }
|
||||
? {
|
||||
value: ["0x2000"],
|
||||
limitations: [],
|
||||
relationship_coverage: {
|
||||
status: "complete",
|
||||
diagnostics: [
|
||||
"Only resolved Ghidra call references are returned; unresolved computed or indirect calls remain unknown, while function classifications distinguish thunks and externals.",
|
||||
],
|
||||
},
|
||||
}
|
||||
: { value: [], limitations: [] },
|
||||
),
|
||||
),
|
||||
@@ -1818,6 +1833,8 @@ describe("compatible Ghidra builds", () => {
|
||||
readText: () =>
|
||||
"application.version=12.1.2\napplication.java.min=21\napplication.java.max=\n",
|
||||
probeJava: () => ({
|
||||
osArch: "amd64",
|
||||
osName: "Linux",
|
||||
version: "27",
|
||||
major: 27,
|
||||
home: "/usr/lib/jvm/java-27-openjdk",
|
||||
|
||||
@@ -2,7 +2,6 @@ import type {
|
||||
CapabilityDescriptor,
|
||||
ProviderIdentity,
|
||||
} from "../application/AnalysisProvider.js";
|
||||
import type { RelationshipCoverage } from "../domain/relationshipCoverage.js";
|
||||
import {
|
||||
hasWindowsNativeAuthority,
|
||||
windowsNativeAuthorityUnavailableReason,
|
||||
@@ -44,22 +43,6 @@ const stringInventoryLimitation =
|
||||
const searchLimitation =
|
||||
"Regex mode uses Java Pattern semantics. Matcher or compiler stack exhaustion returns a resource constraint without closing the session; literal mode avoids regex recursion and returns complete matching values.";
|
||||
|
||||
const resolvedCallReferenceLimitation =
|
||||
"Only resolved Ghidra call references are returned; unresolved computed or indirect calls remain unknown, while function classifications distinguish thunks and externals.";
|
||||
|
||||
/** Relationship-only caveats belong to each observed caller/callee result. */
|
||||
export const relationshipCoverageFor = (
|
||||
operation: string,
|
||||
): RelationshipCoverage | undefined =>
|
||||
operation === "procedure_callers" ||
|
||||
operation === "procedure_callees" ||
|
||||
operation === "analyze_function"
|
||||
? {
|
||||
status: "complete",
|
||||
diagnostics: [resolvedCallReferenceLimitation],
|
||||
}
|
||||
: undefined;
|
||||
|
||||
/** Limitation text for one admitted operation, including the common base. */
|
||||
export const limitationsFor = (operation: string): readonly string[] => {
|
||||
const common = [
|
||||
|
||||
@@ -34,6 +34,7 @@ import {
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import type { EvidenceLocation } from "../domain/evidence.js";
|
||||
import type { RelationshipCoverage } from "../domain/relationshipCoverage.js";
|
||||
import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js";
|
||||
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
@@ -86,7 +87,6 @@ import {
|
||||
healthLimitations,
|
||||
windowsP0Limitations,
|
||||
limitationsFor,
|
||||
relationshipCoverageFor,
|
||||
} from "./GhidraProviderCapabilities.js";
|
||||
import type { GhidraSessionError } from "./GhidraSessionError.js";
|
||||
import {
|
||||
@@ -733,8 +733,8 @@ export const createGhidraProviderClient = (input: {
|
||||
rawResult: JsonValue,
|
||||
observationLimitations: readonly string[] = [],
|
||||
locations: readonly EvidenceLocation[] = [],
|
||||
relationshipCoverage?: RelationshipCoverage,
|
||||
) => {
|
||||
const relationshipCoverage = relationshipCoverageFor(operation);
|
||||
return createAnalysisExecution(normalized, committedProfile.provider, {
|
||||
rawResult,
|
||||
locations,
|
||||
@@ -838,12 +838,14 @@ export const createGhidraProviderClient = (input: {
|
||||
let normalized: JsonValue;
|
||||
let observationLimitations: readonly string[] = [];
|
||||
let observationLocations: readonly EvidenceLocation[] = [];
|
||||
let relationshipCoverage: RelationshipCoverage | undefined;
|
||||
if (isGhidraFunctionOperation(operation)) {
|
||||
const result = parseGhidraFunctionResult(operation, called.value);
|
||||
if (!result.ok) return result;
|
||||
normalized = result.value.value;
|
||||
observationLimitations = result.value.limitations;
|
||||
observationLocations = result.value.locations ?? [];
|
||||
relationshipCoverage = result.value.relationshipCoverage;
|
||||
} else {
|
||||
const result = parseGhidraInventoryResult(operation, called.value);
|
||||
if (!result.ok) return result;
|
||||
@@ -939,6 +941,7 @@ export const createGhidraProviderClient = (input: {
|
||||
called.value,
|
||||
observationLimitations,
|
||||
observationLocations,
|
||||
relationshipCoverage,
|
||||
),
|
||||
);
|
||||
},
|
||||
|
||||
@@ -11,6 +11,7 @@ import { silentLogger } from "../logger.js";
|
||||
import { GhidraProvider } from "./GhidraProvider.js";
|
||||
import { GhidraSessionError } from "./GhidraSessionError.js";
|
||||
import { GHIDRA_SESSION_CAPABILITIES } from "./GhidraSessionValues.js";
|
||||
import type { RelationshipCoverage } from "../domain/relationshipCoverage.js";
|
||||
|
||||
const TARGET: BinaryTarget = {
|
||||
path: "/tmp/ghidra-cache-fixture",
|
||||
@@ -23,7 +24,7 @@ const TARGET: BinaryTarget = {
|
||||
|
||||
// Exercise the real provider descriptors, profile, input/output projection and
|
||||
// session cache; only the installed Ghidra process is replaced at its seam.
|
||||
const fixture = (version = "12.1.4") => {
|
||||
const fixture = (version = "12.1.4", coverage?: RelationshipCoverage) => {
|
||||
const config = parseConfig({ GHIDRA_INSTALL_DIR: "/opt/ghidra" });
|
||||
if (!config.ok) throw config.error;
|
||||
const calls: string[] = [];
|
||||
@@ -43,6 +44,8 @@ const fixture = (version = "12.1.4") => {
|
||||
`application.version=${version}\napplication.java.min=21\n`,
|
||||
executable: () => true,
|
||||
probeJava: () => ({
|
||||
osArch: "amd64",
|
||||
osName: "Linux",
|
||||
version: "21.0.11",
|
||||
major: 21,
|
||||
home: "/usr/lib/jvm/jdk-21",
|
||||
@@ -111,7 +114,13 @@ const fixture = (version = "12.1.4") => {
|
||||
return ok(
|
||||
operation === "procedure_pseudo_code" ||
|
||||
operation === "analyze_function"
|
||||
? { value, limitations: [] }
|
||||
? {
|
||||
value,
|
||||
limitations: [],
|
||||
...(operation === "analyze_function" && coverage !== undefined
|
||||
? { relationship_coverage: coverage }
|
||||
: {}),
|
||||
}
|
||||
: value,
|
||||
);
|
||||
},
|
||||
@@ -131,42 +140,119 @@ const open = async (session: ReturnType<typeof fixture>["session"]) => {
|
||||
expect((await session.openResolvedTarget(preview.value)).ok).toBe(true);
|
||||
};
|
||||
|
||||
describe("Ghidra snapshot cache", () => {
|
||||
it("refuses snapshots whose profile predates typed incoming references", async () => {
|
||||
const initial = fixture();
|
||||
await open(initial.session);
|
||||
const snapshot = initial.session.exportAnalysisSnapshot();
|
||||
if (!snapshot.ok) throw snapshot.error;
|
||||
await initial.session.close();
|
||||
const profile = snapshot.value.binding.analysis_profile;
|
||||
expect(profile.parameters.instruction_references).toBe(
|
||||
"exact-address-incoming-v1",
|
||||
);
|
||||
const legacy = {
|
||||
...snapshot.value,
|
||||
binding: {
|
||||
...snapshot.value.binding,
|
||||
analysis_profile: createAnalysisProfile(
|
||||
profile.provider,
|
||||
Object.fromEntries(
|
||||
Object.entries(profile.parameters).filter(
|
||||
([key]) => key !== "instruction_references",
|
||||
),
|
||||
),
|
||||
),
|
||||
},
|
||||
};
|
||||
const replay = fixture();
|
||||
const imported = replay.session.importAnalysisSnapshot(legacy);
|
||||
if (!imported.ok) throw imported.error;
|
||||
expect(await replay.session.previewTarget(TARGET)).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "EvidenceIntegrityError" },
|
||||
});
|
||||
expect(replay.calls).toEqual([]);
|
||||
await replay.session.close();
|
||||
});
|
||||
describe("Ghidra relationship coverage replay", () => {
|
||||
it.each([
|
||||
undefined,
|
||||
{ status: "complete", diagnostics: ["Resolved static references only."] },
|
||||
{ status: "partial", limitations: ["Instruction coverage was truncated."] },
|
||||
{ status: "unknown", reason: "Relationship inventory was unavailable." },
|
||||
{
|
||||
status: "partial",
|
||||
limitations: ["Missing callers."],
|
||||
limited_directions: ["callers"],
|
||||
},
|
||||
] satisfies (RelationshipCoverage | undefined)[])(
|
||||
"preserves relationship coverage %j through live execution, cache and snapshot replay",
|
||||
async (coverage) => {
|
||||
const initial = fixture("12.1.4", coverage);
|
||||
await open(initial.session);
|
||||
const parameters = { procedure: "0x1000" };
|
||||
for (let i = 0; i < 2; i++) {
|
||||
const execution = await initial.session.execute(
|
||||
"analyze_function",
|
||||
parameters,
|
||||
);
|
||||
if (!execution.ok) throw execution.error;
|
||||
expect(execution.value.relationshipCoverage).toEqual(coverage);
|
||||
}
|
||||
expect(initial.calls).toEqual(["analyze_function"]);
|
||||
const snapshot = initial.session.exportAnalysisSnapshot();
|
||||
if (!snapshot.ok) throw snapshot.error;
|
||||
await initial.session.close();
|
||||
const replay = fixture();
|
||||
expect(replay.session.importAnalysisSnapshot(snapshot.value).ok).toBe(
|
||||
true,
|
||||
);
|
||||
await open(replay.session);
|
||||
const execution = await replay.session.execute(
|
||||
"analyze_function",
|
||||
parameters,
|
||||
);
|
||||
if (!execution.ok) throw execution.error;
|
||||
expect(execution.value.relationshipCoverage ?? undefined).toEqual(
|
||||
coverage,
|
||||
);
|
||||
expect(replay.calls).toEqual([]);
|
||||
await replay.session.close();
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("Ghidra snapshot profile admission", () => {
|
||||
it.each([
|
||||
{
|
||||
parameter: "instruction_references",
|
||||
current: "exact-address-incoming-v1",
|
||||
legacy: undefined,
|
||||
},
|
||||
{
|
||||
parameter: "call_relationship_coverage",
|
||||
current: "function-body-call-coverage-v5",
|
||||
legacy: "resolved-call-reference-diagnostics-v1",
|
||||
},
|
||||
{
|
||||
parameter: "call_relationship_coverage",
|
||||
current: "function-body-call-coverage-v5",
|
||||
legacy: "decoded-call-target-coverage-v2",
|
||||
},
|
||||
{
|
||||
parameter: "call_relationship_coverage",
|
||||
current: "function-body-call-coverage-v5",
|
||||
legacy: "decoded-call-target-coverage-v3",
|
||||
},
|
||||
{
|
||||
parameter: "call_relationship_coverage",
|
||||
current: "function-body-call-coverage-v5",
|
||||
legacy: "function-body-call-coverage-v4",
|
||||
},
|
||||
])(
|
||||
"refuses snapshots with outdated $parameter semantics",
|
||||
async ({ parameter, current, legacy: prior }) => {
|
||||
const initial = fixture();
|
||||
await open(initial.session);
|
||||
const snapshot = initial.session.exportAnalysisSnapshot();
|
||||
if (!snapshot.ok) throw snapshot.error;
|
||||
await initial.session.close();
|
||||
const profile = snapshot.value.binding.analysis_profile;
|
||||
expect(profile.parameters[parameter]).toBe(current);
|
||||
const legacy = {
|
||||
...snapshot.value,
|
||||
binding: {
|
||||
...snapshot.value.binding,
|
||||
analysis_profile: createAnalysisProfile(profile.provider, {
|
||||
...Object.fromEntries(
|
||||
Object.entries(profile.parameters).filter(
|
||||
([key]) => key !== parameter,
|
||||
),
|
||||
),
|
||||
...(prior === undefined ? {} : { [parameter]: prior }),
|
||||
}),
|
||||
},
|
||||
};
|
||||
const replay = fixture();
|
||||
const imported = replay.session.importAnalysisSnapshot(legacy);
|
||||
if (!imported.ok) throw imported.error;
|
||||
expect(await replay.session.previewTarget(TARGET)).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "EvidenceIntegrityError" },
|
||||
});
|
||||
expect(replay.calls).toEqual([]);
|
||||
await replay.session.close();
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("Ghidra snapshot cache", () => {
|
||||
it.each([
|
||||
{ operation: "analyze_function", parameters: { procedure: "0x1000" } },
|
||||
{
|
||||
|
||||
@@ -1,39 +1,36 @@
|
||||
import type { EvidenceMcpServer } from "./EvidenceMcpServer.js";
|
||||
import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js";
|
||||
|
||||
import {
|
||||
ARTIFACT_TOOL_CONTRACTS,
|
||||
artifactInspectionInputSchema,
|
||||
} from "../contracts/artifactToolContracts.js";
|
||||
import type { EvidenceLookup } from "../application/EvidenceInputResolver.js";
|
||||
import {
|
||||
requireAnalysisViewRetention,
|
||||
summarizeRetainedAnalysis,
|
||||
} from "../application/analysisView/AnalysisViewService.js";
|
||||
import { createArtifactExtractionDestination } from "../application/artifacts/ArtifactExtractionDestination.js";
|
||||
import { ok } from "../domain/result.js";
|
||||
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
|
||||
import type { Logger } from "pino";
|
||||
import { registerEvidenceTools } from "./registerEvidenceTools.js";
|
||||
import { artifactInspectionResultSchema } from "../domain/artifactInspection.js";
|
||||
import {
|
||||
retainedSummaryDelivery,
|
||||
withoutDeliveryDetail,
|
||||
type EvidenceToolFamilyOptions,
|
||||
} from "./retainedSummaryDelivery.js";
|
||||
|
||||
/** Operations whose MCP delivery may be a retained summary view. */
|
||||
const SUMMARY_DELIVERY: ReadonlySet<string> = new Set([
|
||||
"inspect_artifact",
|
||||
"decode_interface_builder",
|
||||
]);
|
||||
/** inspect_artifact keeps an explicit detail in its Evidence parameters. */
|
||||
const decodeWithoutDelivery = withoutDeliveryDetail(
|
||||
new Set(["decode_interface_builder"]),
|
||||
);
|
||||
|
||||
/** Register deterministic artifact inventory and safe extraction operations. */
|
||||
export const registerArtifactTools = (
|
||||
server: EvidenceMcpServer,
|
||||
options: {
|
||||
readonly logger: Logger;
|
||||
readonly activeTarget: (() => BinaryTarget | undefined) | undefined;
|
||||
readonly recordEvidence: EvidenceWriter["recordEvidence"] | undefined;
|
||||
readonly evidenceById?: EvidenceLookup | undefined;
|
||||
readonly withAdmittedAnalysis: import("./analysisAdmission.js").WithAdmittedAnalysis;
|
||||
},
|
||||
options: EvidenceToolFamilyOptions,
|
||||
): void => {
|
||||
const retention = {
|
||||
recordEvidence: options.recordEvidence,
|
||||
evidenceById: options.evidenceById,
|
||||
};
|
||||
registerEvidenceTools(server, ARTIFACT_TOOL_CONTRACTS, {
|
||||
...options,
|
||||
...retainedSummaryDelivery(SUMMARY_DELIVERY, options),
|
||||
parametersForExecution: (operation, parameters) => {
|
||||
if (operation === "inspect_artifact") {
|
||||
const { detail: _detail, ...execution } =
|
||||
@@ -42,16 +39,9 @@ export const registerArtifactTools = (
|
||||
}
|
||||
return operation === "extract_artifact"
|
||||
? { ...parameters, output_root: createArtifactExtractionDestination() }
|
||||
: parameters;
|
||||
: decodeWithoutDelivery(operation, parameters);
|
||||
},
|
||||
preflight: (operation, parameters) =>
|
||||
operation === "inspect_artifact" && parameters.detail === "summary"
|
||||
? requireAnalysisViewRetention(operation, retention)
|
||||
: ok(null),
|
||||
projectEvidence: (operation, parameters, evidence) =>
|
||||
operation === "inspect_artifact" && parameters.detail === "summary"
|
||||
? summarizeRetainedAnalysis(evidence, retention)
|
||||
: ok(evidence),
|
||||
parametersForEvidence: decodeWithoutDelivery,
|
||||
sourceEvidence: (operation, result) =>
|
||||
operation === "inspect_artifact"
|
||||
? artifactInspectionResultSchema
|
||||
|
||||
@@ -1,6 +1,4 @@
|
||||
import type { EvidenceMcpServer } from "./EvidenceMcpServer.js";
|
||||
import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js";
|
||||
import type { EvidenceLookup } from "../application/EvidenceInputResolver.js";
|
||||
import {
|
||||
requireAnalysisViewRetention,
|
||||
summarizeRetainedAnalysis,
|
||||
@@ -8,24 +6,17 @@ import {
|
||||
|
||||
import type { AnalysisOperationPort } from "../application/AnalysisProvider.js";
|
||||
import { MANAGED_TOOL_CONTRACTS } from "../contracts/managed/managedToolContracts.js";
|
||||
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
|
||||
import { AnalysisInputError } from "../domain/analysisErrorCore.js";
|
||||
import { err, ok } from "../domain/result.js";
|
||||
import type { Logger } from "pino";
|
||||
import { registerEvidenceTools } from "./registerEvidenceTools.js";
|
||||
import type { EvidenceToolFamilyOptions } from "./retainedSummaryDelivery.js";
|
||||
import { runManagedProviderExecution } from "../composition/directAnalysis.js";
|
||||
import { isManagedToolName } from "../contracts/managed/managedToolContracts.js";
|
||||
|
||||
/** Register execution-free managed PE/CLI inspection. */
|
||||
export const registerManagedTools = (
|
||||
server: EvidenceMcpServer,
|
||||
options: {
|
||||
readonly logger: Logger;
|
||||
readonly activeTarget: (() => BinaryTarget | undefined) | undefined;
|
||||
readonly recordEvidence: EvidenceWriter["recordEvidence"] | undefined;
|
||||
readonly evidenceById?: EvidenceLookup | undefined;
|
||||
readonly withAdmittedAnalysis: import("./analysisAdmission.js").WithAdmittedAnalysis;
|
||||
},
|
||||
options: EvidenceToolFamilyOptions,
|
||||
): void => {
|
||||
let selectedManagedPath: string | undefined;
|
||||
const analysisForAdmission = (
|
||||
|
||||
@@ -1,54 +1,25 @@
|
||||
import type { EvidenceMcpServer } from "./EvidenceMcpServer.js";
|
||||
import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js";
|
||||
|
||||
import { NATIVE_TOOL_CONTRACTS } from "../contracts/native/nativeToolContracts.js";
|
||||
import type { EvidenceLookup } from "../application/EvidenceInputResolver.js";
|
||||
import {
|
||||
requireAnalysisViewRetention,
|
||||
summarizeRetainedAnalysis,
|
||||
} from "../application/analysisView/AnalysisViewService.js";
|
||||
import type { JsonValue } from "../domain/jsonValue.js";
|
||||
import { ok } from "../domain/result.js";
|
||||
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
|
||||
import type { Logger } from "pino";
|
||||
import { registerEvidenceTools } from "./registerEvidenceTools.js";
|
||||
import {
|
||||
retainedSummaryDelivery,
|
||||
withoutDeliveryDetail,
|
||||
type EvidenceToolFamilyOptions,
|
||||
} from "./retainedSummaryDelivery.js";
|
||||
|
||||
/** Delivery detail selects a response shape; it never reaches the tools or Evidence identity. */
|
||||
const withoutDelivery = (
|
||||
operation: string,
|
||||
parameters: Readonly<Record<string, JsonValue>>,
|
||||
): Readonly<Record<string, JsonValue>> => {
|
||||
if (operation !== "inspect_macho") return parameters;
|
||||
const { detail: _detail, ...selected } = parameters;
|
||||
return selected;
|
||||
};
|
||||
const SUMMARY_DELIVERY: ReadonlySet<string> = new Set(["inspect_macho"]);
|
||||
|
||||
/** Register macOS native contracts; Mach-O inspection can deliver a retained summary. */
|
||||
export const registerNativeTools = (
|
||||
server: EvidenceMcpServer,
|
||||
options: {
|
||||
readonly logger: Logger;
|
||||
readonly activeTarget: (() => BinaryTarget | undefined) | undefined;
|
||||
readonly recordEvidence: EvidenceWriter["recordEvidence"] | undefined;
|
||||
readonly evidenceById?: EvidenceLookup | undefined;
|
||||
readonly withAdmittedAnalysis: import("./analysisAdmission.js").WithAdmittedAnalysis;
|
||||
},
|
||||
options: EvidenceToolFamilyOptions,
|
||||
): void => {
|
||||
const retention = {
|
||||
recordEvidence: options.recordEvidence,
|
||||
evidenceById: options.evidenceById,
|
||||
};
|
||||
const withoutDelivery = withoutDeliveryDetail(SUMMARY_DELIVERY);
|
||||
registerEvidenceTools(server, NATIVE_TOOL_CONTRACTS, {
|
||||
...options,
|
||||
...retainedSummaryDelivery(SUMMARY_DELIVERY, options),
|
||||
parametersForExecution: withoutDelivery,
|
||||
parametersForEvidence: withoutDelivery,
|
||||
preflight: (operation, parameters) =>
|
||||
operation === "inspect_macho" && parameters.detail === "summary"
|
||||
? requireAnalysisViewRetention(operation, retention)
|
||||
: ok(null),
|
||||
projectEvidence: (operation, parameters, evidence) =>
|
||||
operation === "inspect_macho" && parameters.detail === "summary"
|
||||
? summarizeRetainedAnalysis(evidence, retention)
|
||||
: ok(evidence),
|
||||
});
|
||||
};
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
import { analysisDetailSchema } from "../contracts/analysisDetail.js";
|
||||
|
||||
import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js";
|
||||
import type { EvidenceLookup } from "../application/EvidenceInputResolver.js";
|
||||
import {
|
||||
requireAnalysisViewRetention,
|
||||
summarizeRetainedAnalysis,
|
||||
} from "../application/analysisView/AnalysisViewService.js";
|
||||
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
|
||||
import type { Evidence } from "../domain/evidence.js";
|
||||
import type { JsonValue } from "../domain/jsonValue.js";
|
||||
import { AnalysisInputError } from "../domain/analysisErrorCore.js";
|
||||
import { err, ok } from "../domain/result.js";
|
||||
import type { Logger } from "pino";
|
||||
import type { WithAdmittedAnalysis } from "./analysisAdmission.js";
|
||||
|
||||
/** Registration inputs shared by provider-backed Evidence tool families. */
|
||||
export interface EvidenceToolFamilyOptions {
|
||||
readonly logger: Logger;
|
||||
readonly activeTarget: (() => BinaryTarget | undefined) | undefined;
|
||||
readonly recordEvidence: EvidenceWriter["recordEvidence"] | undefined;
|
||||
readonly evidenceById?: EvidenceLookup | undefined;
|
||||
readonly withAdmittedAnalysis: WithAdmittedAnalysis;
|
||||
}
|
||||
|
||||
type Parameters = Readonly<Record<string, JsonValue>>;
|
||||
|
||||
/**
|
||||
* Summary delivery for the named operations: refuse it before analysis when
|
||||
* this server retains no Evidence, then return the retained summary view.
|
||||
*/
|
||||
export const retainedSummaryDelivery = (
|
||||
operations: ReadonlySet<string>,
|
||||
options: Pick<EvidenceToolFamilyOptions, "recordEvidence" | "evidenceById">,
|
||||
) => {
|
||||
const retention = {
|
||||
recordEvidence: options.recordEvidence,
|
||||
evidenceById: options.evidenceById,
|
||||
};
|
||||
const selected = (operation: string, parameters: Parameters): boolean =>
|
||||
operations.has(operation) && parameters.detail === "summary";
|
||||
return {
|
||||
preflight: (operation: string, parameters: Parameters) => {
|
||||
if (!operations.has(operation)) return ok(null);
|
||||
const detail = analysisDetailSchema
|
||||
.unwrap()
|
||||
.optional()
|
||||
.safeParse(parameters.detail);
|
||||
if (!detail.success)
|
||||
return err(
|
||||
new AnalysisInputError(operation, { cause: detail.error }, [
|
||||
{
|
||||
path: ["detail"],
|
||||
reason: "invalid_value",
|
||||
expected: ["complete", "summary"],
|
||||
},
|
||||
]),
|
||||
);
|
||||
return detail.data === "summary"
|
||||
? requireAnalysisViewRetention(operation, retention)
|
||||
: ok(null);
|
||||
},
|
||||
projectEvidence: (
|
||||
operation: string,
|
||||
parameters: Parameters,
|
||||
evidence: Evidence,
|
||||
) =>
|
||||
selected(operation, parameters)
|
||||
? summarizeRetainedAnalysis(evidence, retention)
|
||||
: ok(evidence),
|
||||
};
|
||||
};
|
||||
|
||||
/** Delivery detail selects a response shape; it never reaches tools or Evidence identity. */
|
||||
export const withoutDeliveryDetail =
|
||||
(operations: ReadonlySet<string>) =>
|
||||
(operation: string, parameters: Parameters): Parameters => {
|
||||
if (!operations.has(operation)) return parameters;
|
||||
const { detail: _detail, ...selected } = parameters;
|
||||
return selected;
|
||||
};
|
||||
@@ -1,4 +1,4 @@
|
||||
import { writeFile } from "node:fs/promises";
|
||||
import { chmod, mkdir, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { expect } from "vitest";
|
||||
@@ -68,6 +68,130 @@ cliTest(
|
||||
},
|
||||
);
|
||||
|
||||
const isWindows = process.platform === "win32";
|
||||
|
||||
const errorBody = (result: {
|
||||
readonly content: readonly { readonly type: string }[];
|
||||
}) => {
|
||||
const content = result.content.find((item) => item.type === "text");
|
||||
if (content?.type !== "text" || !("text" in content))
|
||||
throw new Error("Missing error response");
|
||||
return JSON.parse(String(content.text));
|
||||
};
|
||||
|
||||
const createElectronApplication = async (root: string) => {
|
||||
const application = join(root, "app");
|
||||
await mkdir(application);
|
||||
await writeFile(
|
||||
join(application, "package.json"),
|
||||
JSON.stringify({ name: "fixture", main: "main.js" }),
|
||||
);
|
||||
await writeFile(join(application, "main.js"), "");
|
||||
return application;
|
||||
};
|
||||
|
||||
cliTest.skipIf(isWindows)(
|
||||
"rejects a non-executable Electron executable_path before launch through CLI and MCP",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-electron-not-executable-");
|
||||
const application = await createElectronApplication(root);
|
||||
const executable = join(root, "not-an-executable.txt");
|
||||
await writeFile(executable, "plain text");
|
||||
await chmod(executable, 0o644);
|
||||
const request = {
|
||||
executable_path: executable,
|
||||
application_path: application,
|
||||
actions: [],
|
||||
};
|
||||
const requestPath = join(root, "input.json");
|
||||
await writeFile(requestPath, JSON.stringify(request));
|
||||
|
||||
const response = await cli.run({
|
||||
arguments: ["capture-electron-scenario", requestPath, "--format", "json"],
|
||||
environment: { REA_LOG_LEVEL: "silent" },
|
||||
});
|
||||
expect(response.exitCode).toBe(1);
|
||||
expect(response.json).toMatchObject({
|
||||
code: "invalid_request",
|
||||
details: {
|
||||
issues: [{ path: ["executable_path"], reason: "invalid_value" }],
|
||||
},
|
||||
});
|
||||
|
||||
const { call } = await connectLocalToolsMcp();
|
||||
const result = await call("capture_electron_scenario", request);
|
||||
expect(result.isError).toBe(true);
|
||||
expect(errorBody(result)).toMatchObject({
|
||||
error: {
|
||||
code: "invalid_request",
|
||||
details: {
|
||||
issues: [{ path: ["executable_path"], reason: "invalid_value" }],
|
||||
},
|
||||
},
|
||||
});
|
||||
const session = await call("binary_session", {});
|
||||
expect(session.isError, JSON.stringify(session)).not.toBe(true);
|
||||
},
|
||||
);
|
||||
|
||||
cliTest.skipIf(isWindows)(
|
||||
"returns a typed failure when an executable exits before Electron starts and keeps the MCP server and its Evidence",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-electron-early-exit-");
|
||||
const application = await createElectronApplication(root);
|
||||
const executable = join(root, "exits-immediately");
|
||||
await writeFile(executable, "#!/bin/sh\nexit 0\n");
|
||||
await chmod(executable, 0o755);
|
||||
const request = {
|
||||
executable_path: executable,
|
||||
application_path: application,
|
||||
actions: [],
|
||||
};
|
||||
const requestPath = join(root, "input.json");
|
||||
await writeFile(requestPath, JSON.stringify(request));
|
||||
|
||||
const response = await cli.run({
|
||||
arguments: ["capture-electron-scenario", requestPath, "--format", "json"],
|
||||
environment: { REA_LOG_LEVEL: "silent" },
|
||||
});
|
||||
expect(response.exitCode).toBe(1);
|
||||
expect(response.json).toMatchObject({
|
||||
code: "execution_failure",
|
||||
details: { operation: "capture_electron_scenario" },
|
||||
});
|
||||
expect(response.stdout).toContain("Process failed to launch");
|
||||
|
||||
const { call } = await connectLocalToolsMcp();
|
||||
const analysis = await call("analyze_javascript_application", {
|
||||
input_path: application,
|
||||
});
|
||||
expect(analysis.isError, JSON.stringify(analysis)).not.toBe(true);
|
||||
const retained = async () => {
|
||||
const bundle = await call("get_evidence_bundle", {});
|
||||
expect(bundle.isError, JSON.stringify(bundle)).not.toBe(true);
|
||||
const structured = bundle.structuredContent as {
|
||||
result: { records: { evidence_id: string }[] };
|
||||
};
|
||||
return structured.result.records.map((record) => record.evidence_id);
|
||||
};
|
||||
const before = await retained();
|
||||
expect(before).toHaveLength(1);
|
||||
|
||||
const result = await call("capture_electron_scenario", request);
|
||||
expect(result.isError).toBe(true);
|
||||
expect(errorBody(result)).toMatchObject({
|
||||
error: {
|
||||
code: "execution_failure",
|
||||
details: { operation: "capture_electron_scenario" },
|
||||
},
|
||||
});
|
||||
const session = await call("binary_session", {});
|
||||
expect(session.isError, JSON.stringify(session)).not.toBe(true);
|
||||
expect(await retained()).toEqual(before);
|
||||
},
|
||||
180_000,
|
||||
);
|
||||
|
||||
cliTest(
|
||||
"rejects a file URL path field without resolving an invented local path",
|
||||
async ({ cli }) => {
|
||||
|
||||
@@ -279,26 +279,44 @@ it("preserves bounded instruction coverage separately from using-node coverage",
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects malformed method descriptors at the Android adapter boundary", async () => {
|
||||
const invalidSummary = {
|
||||
...summary,
|
||||
methods: summary.methods.map((method) => ({
|
||||
...method,
|
||||
dex_descriptor: "not-a-dex-descriptor",
|
||||
})),
|
||||
};
|
||||
await expect(
|
||||
analyzeJadxRequest(
|
||||
{
|
||||
json: () => Promise.resolve(invalidSummary),
|
||||
text: () => Promise.reject(new Error("unexpected text call")),
|
||||
it.each([
|
||||
"not-a-dex-descriptor",
|
||||
"(Lbad(name;)V",
|
||||
"(Lbad)name;)V",
|
||||
"()Lbad(name;",
|
||||
"()Lbad)name;",
|
||||
])(
|
||||
"rejects malformed DEX prototype %s at the adapter boundary",
|
||||
async (descriptor) => {
|
||||
const invalidSummary = {
|
||||
...summary,
|
||||
methods: summary.methods.map((method) => ({
|
||||
...method,
|
||||
dex_descriptor: descriptor,
|
||||
})),
|
||||
};
|
||||
await expect(
|
||||
analyzeJadxRequest(
|
||||
{
|
||||
json: () => Promise.resolve(invalidSummary),
|
||||
text: () => Promise.reject(new Error("unexpected text call")),
|
||||
},
|
||||
{
|
||||
operation: "inspect_android_class",
|
||||
input: { path: "/fixture.apk", class_name: "fixture.Target" },
|
||||
},
|
||||
{} as never,
|
||||
engine,
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
_tag: "AnalysisOutputError",
|
||||
cause: {
|
||||
issues: expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
path: ["methods", 0, "dex_descriptor"],
|
||||
}),
|
||||
]),
|
||||
},
|
||||
{
|
||||
operation: "inspect_android_class",
|
||||
input: { path: "/fixture.apk", class_name: "fixture.Target" },
|
||||
},
|
||||
{} as never,
|
||||
engine,
|
||||
),
|
||||
).rejects.toThrow();
|
||||
});
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
@@ -15,6 +15,19 @@ const run = async (arguments_: readonly string[]): Promise<unknown> => {
|
||||
return JSON.parse(stdout);
|
||||
};
|
||||
|
||||
const runFailure = async (arguments_: readonly string[]): Promise<unknown> => {
|
||||
try {
|
||||
await run(arguments_);
|
||||
} catch (cause: unknown) {
|
||||
if (typeof cause !== "object" || cause === null || !("stdout" in cause))
|
||||
throw cause;
|
||||
const stdout = cause.stdout;
|
||||
if (typeof stdout !== "string") throw cause;
|
||||
return JSON.parse(stdout);
|
||||
}
|
||||
throw new Error("Expected the CLI command to fail");
|
||||
};
|
||||
|
||||
it.each(["analyze-javascript-application", "analyze"])(
|
||||
"%s carries the integrity policy into shared JavaScript analysis",
|
||||
async (command) => {
|
||||
@@ -46,3 +59,48 @@ it("rejects the JavaScript integrity policy on binary-only inspection", async ()
|
||||
stdout: expect.stringContaining("Unknown flag: --integrityPolicy"),
|
||||
});
|
||||
});
|
||||
|
||||
it.each(["fail", "record-and-continue"] as const)(
|
||||
"rejects the explicitly selected %s integrity policy on a binary analyze target",
|
||||
async (policy) => {
|
||||
const failure = await runFailure([
|
||||
"analyze",
|
||||
"unused.bin",
|
||||
"--integrity-policy",
|
||||
policy,
|
||||
]);
|
||||
expect(failure).toMatchObject({
|
||||
code: "invalid_request",
|
||||
details: {
|
||||
issues: [
|
||||
{
|
||||
path: ["integrity_policy"],
|
||||
reason: "invalid_value",
|
||||
message:
|
||||
"An integrity policy requires the static JavaScript ASAR/directory analysis lane; binary_overview accepts no input.",
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("still routes a binary analyze target without an integrity policy", async () => {
|
||||
await expect(run(["analyze", "unused.bin"])).rejects.toMatchObject({
|
||||
code: 1,
|
||||
stdout: expect.stringContaining("target does not exist"),
|
||||
});
|
||||
});
|
||||
|
||||
it("advertises the optional integrity policy without an undefined union", async () => {
|
||||
const schema = (await run(["analyze", "--schema"])) as {
|
||||
options?: { properties?: { integrityPolicy?: unknown } };
|
||||
};
|
||||
const policy = schema.options?.properties?.integrityPolicy;
|
||||
expect(policy).toMatchObject({
|
||||
type: "string",
|
||||
enum: ["fail", "record-and-continue"],
|
||||
description: "Behavior when declared application integrity does not match",
|
||||
});
|
||||
expect(JSON.stringify(policy)).not.toContain("undefined");
|
||||
});
|
||||
|
||||
@@ -67,6 +67,7 @@ describe("client configuration filesystem lifecycle", () => {
|
||||
"kimi",
|
||||
"vscode",
|
||||
"trae",
|
||||
"workbuddy",
|
||||
"grok_build",
|
||||
"omp",
|
||||
"pi",
|
||||
|
||||
@@ -55,6 +55,8 @@ const host: GhidraInstallationHost = {
|
||||
readText: () => "application.version=12.1.3\napplication.java.min=21\n",
|
||||
executable: () => true,
|
||||
probeJava: () => ({
|
||||
osArch: "amd64",
|
||||
osName: "Linux",
|
||||
version: "21.0.12",
|
||||
major: 21,
|
||||
home: "/jdk-21",
|
||||
|
||||
@@ -0,0 +1,159 @@
|
||||
import { mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
import { readClientRegistrationStatuses } from "../../../src/application/ClientRegistrationStatus.js";
|
||||
import { npxRegistrationCommand } from "../../../src/application/ClientRegistrationIdentity.js";
|
||||
import { configureClientConfiguration } from "../../../src/application/SetupClientConfiguration.js";
|
||||
import { skillDestinations } from "../../../src/application/SetupSkill.js";
|
||||
import { supportedClients } from "../../../src/application/SupportedClients.js";
|
||||
import { systemUninstallHost } from "../../../src/application/Uninstall.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
import { clearClientLocationEnvironment } from "../../fixtures/clientEnvironment.js";
|
||||
|
||||
/**
|
||||
* The registration command setup writes on the host running the tests. Windows
|
||||
* wraps it in `cmd.exe /d /c`, so the expected entry follows the host too.
|
||||
*/
|
||||
const command = npxRegistrationCommand(process.platform);
|
||||
|
||||
beforeEach(clearClientLocationEnvironment);
|
||||
afterEach(() => vi.unstubAllEnvs());
|
||||
|
||||
const workbuddyPath = (
|
||||
home: string,
|
||||
platform: NodeJS.Platform = process.platform,
|
||||
env: Parameters<typeof supportedClients>[2] = {},
|
||||
) =>
|
||||
supportedClients(home, platform, env).find(
|
||||
({ name }) => name === "workbuddy",
|
||||
);
|
||||
|
||||
describe("WorkBuddy configuration paths", () => {
|
||||
it("resolves the global mcp.json under the home root", () => {
|
||||
expect(workbuddyPath("/home/a")?.configPath).toBe(
|
||||
join("/home/a", ".workbuddy", "mcp.json"),
|
||||
);
|
||||
expect(workbuddyPath("/Users/a")?.markerPath).toBe(
|
||||
join("/Users/a", ".workbuddy"),
|
||||
);
|
||||
expect(workbuddyPath("C:\\Users\\a", "win32")?.configPath).toBe(
|
||||
join("C:\\Users\\a", ".workbuddy", "mcp.json"),
|
||||
);
|
||||
});
|
||||
|
||||
it("resolves the registration format as the plain mcpServers JSON dialect", () => {
|
||||
expect(workbuddyPath("/home/a")?.format).toBe("json");
|
||||
});
|
||||
|
||||
it("resolves the personal skills directory under the home root", () => {
|
||||
expect(
|
||||
skillDestinations("/home/a", ["workbuddy"]).map(({ path }) => path),
|
||||
).toEqual([
|
||||
join("/home/a", ".workbuddy", "skills", "reverse-engineer-anything"),
|
||||
]);
|
||||
expect(
|
||||
skillDestinations("C:\\Users\\a", ["workbuddy"], {}, "win32").map(
|
||||
({ path }) => path,
|
||||
),
|
||||
).toEqual([
|
||||
join("C:\\Users\\a", ".workbuddy", "skills", "reverse-engineer-anything"),
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("WorkBuddy JSON registration", () => {
|
||||
const workbuddyClient = async () => {
|
||||
const home = await createTestTempDirectory("rea-workbuddy-");
|
||||
const client = workbuddyPath(home, process.platform);
|
||||
if (client?.markerPath === undefined)
|
||||
throw new Error("missing WorkBuddy client");
|
||||
await mkdir(client.markerPath, { recursive: true });
|
||||
return { home, client };
|
||||
};
|
||||
|
||||
it("upserts rea into mcpServers while preserving unrelated settings", async () => {
|
||||
const { home, client } = await workbuddyClient();
|
||||
await writeFile(
|
||||
client.configPath,
|
||||
`{
|
||||
"mcpServers": {
|
||||
"other": { "command": "other-server" }
|
||||
}
|
||||
}`,
|
||||
);
|
||||
|
||||
expect(
|
||||
(await configureClientConfiguration(client, {}, command)).status,
|
||||
).toBe("configured");
|
||||
const parsed = JSON.parse(await readFile(client.configPath, "utf8"));
|
||||
expect(parsed.mcpServers.other).toEqual({ command: "other-server" });
|
||||
expect(parsed.mcpServers.rea).toEqual({
|
||||
command: command[0],
|
||||
args: command.slice(1),
|
||||
});
|
||||
|
||||
expect(await configureClientConfiguration(client, {}, command)).toEqual({
|
||||
status: "unchanged",
|
||||
});
|
||||
expect(
|
||||
(
|
||||
await readClientRegistrationStatuses(home, undefined, {
|
||||
platform: process.platform,
|
||||
environment: {},
|
||||
})
|
||||
).find(({ client }) => client === "workbuddy"),
|
||||
).toMatchObject({ state: "aligned", command });
|
||||
});
|
||||
|
||||
it("creates the mcpServers table in an existing mcp.json without servers", async () => {
|
||||
const { client } = await workbuddyClient();
|
||||
await writeFile(
|
||||
client.configPath,
|
||||
JSON.stringify({ unrelated: { setting: true } }, null, 2),
|
||||
);
|
||||
expect(
|
||||
(await configureClientConfiguration(client, {}, command)).status,
|
||||
).toBe("configured");
|
||||
const parsed = JSON.parse(await readFile(client.configPath, "utf8"));
|
||||
expect(parsed.unrelated).toEqual({ setting: true });
|
||||
expect(parsed.mcpServers.rea).toEqual({
|
||||
command: command[0],
|
||||
args: command.slice(1),
|
||||
});
|
||||
});
|
||||
|
||||
it("registers into an empty mcp.json", async () => {
|
||||
const { client } = await workbuddyClient();
|
||||
await writeFile(client.configPath, "");
|
||||
expect(
|
||||
(await configureClientConfiguration(client, {}, command)).status,
|
||||
).toBe("configured");
|
||||
const parsed = JSON.parse(await readFile(client.configPath, "utf8"));
|
||||
expect(parsed.mcpServers.rea).toEqual({
|
||||
command: command[0],
|
||||
args: command.slice(1),
|
||||
});
|
||||
});
|
||||
|
||||
it("removes only the rea entry on uninstall", async () => {
|
||||
const { home, client } = await workbuddyClient();
|
||||
await writeFile(
|
||||
client.configPath,
|
||||
JSON.stringify(
|
||||
{ mcpServers: { other: { command: "other-server" } } },
|
||||
null,
|
||||
2,
|
||||
),
|
||||
);
|
||||
await configureClientConfiguration(client, {}, command);
|
||||
|
||||
expect((await systemUninstallHost(home).removeClient(client)).status).toBe(
|
||||
"removed",
|
||||
);
|
||||
const parsed = JSON.parse(await readFile(client.configPath, "utf8"));
|
||||
expect(parsed.mcpServers.other).toEqual({ command: "other-server" });
|
||||
expect(parsed.mcpServers.rea).toBeUndefined();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,138 @@
|
||||
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
|
||||
import { Ajv2020 } from "ajv/dist/2020.js";
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import { FUNCTION_COMPARISON_EXAMPLE } from "../../../src/contracts/functionComparisonExample.js";
|
||||
import { createEvidence } from "../../../src/domain/evidence.js";
|
||||
import { functionDossierSchema } from "../../../src/domain/hopperValues.js";
|
||||
import type { RelationshipCoverage } from "../../../src/domain/relationshipCoverage.js";
|
||||
import { createServer } from "../../../src/server/createServer.js";
|
||||
import { createTestBinarySession } from "../../fixtures/binarySession.js";
|
||||
|
||||
const observe = (relationshipCoverage: RelationshipCoverage) => {
|
||||
const template = FUNCTION_COMPARISON_EXAMPLE.left;
|
||||
if (template.subject === null)
|
||||
throw new Error("Expected artifact-bound fixture");
|
||||
const dossier = functionDossierSchema.parse(template.normalized_result);
|
||||
return createEvidence(
|
||||
{
|
||||
path: template.subject.local_path,
|
||||
sha256: template.subject.digest.sha256,
|
||||
format: template.subject.format,
|
||||
},
|
||||
template.provider,
|
||||
{
|
||||
operation: "analyze_function",
|
||||
parameters: template.parameters,
|
||||
result: { ...dossier, callees: [], unresolved_calls: [] },
|
||||
confidence: template.confidence,
|
||||
authority: template.authority,
|
||||
relationshipCoverage,
|
||||
},
|
||||
);
|
||||
};
|
||||
|
||||
it("preserves directional gaps through advertised schemas and public path reconstruction", async () => {
|
||||
const session = createTestBinarySession(() => {
|
||||
throw new Error("Offline call paths must not launch a provider");
|
||||
});
|
||||
const server = createServer({ kind: "session", session });
|
||||
const client = new Client({ name: "function-call-directions", version: "1" });
|
||||
const [clientTransport, serverTransport] =
|
||||
InMemoryTransport.createLinkedPair();
|
||||
try {
|
||||
await server.connect(serverTransport);
|
||||
await client.connect(clientTransport);
|
||||
const advertised = (await client.listTools()).tools.find(
|
||||
({ name }) => name === "build_call_path",
|
||||
);
|
||||
if (advertised === undefined)
|
||||
throw new Error("Missing build_call_path contract");
|
||||
const ajv = new Ajv2020({ strict: false, validateFormats: false });
|
||||
expect(ajv.validateSchema(advertised.inputSchema)).toBe(true);
|
||||
const validate = ajv.compile(advertised.inputSchema);
|
||||
for (const { directions, status, exhaustive } of [
|
||||
{
|
||||
directions: ["callers"] as ["callers"],
|
||||
status: "not_found",
|
||||
exhaustive: true,
|
||||
},
|
||||
{
|
||||
directions: ["callees"] as ["callees"],
|
||||
status: "unknown",
|
||||
exhaustive: false,
|
||||
},
|
||||
{
|
||||
directions: ["callers", "callees"] as ["callers", "callees"],
|
||||
status: "unknown",
|
||||
exhaustive: false,
|
||||
},
|
||||
]) {
|
||||
const limitation =
|
||||
"An observed call reference is absent from the inventory.";
|
||||
const coverage: RelationshipCoverage = {
|
||||
status: "partial",
|
||||
limitations: [limitation],
|
||||
limited_directions: directions,
|
||||
};
|
||||
const observation = observe(coverage);
|
||||
expect(session.recordEvidence(observation).ok).toBe(true);
|
||||
const input = {
|
||||
functions: [observation],
|
||||
start: { address: "0x1000" },
|
||||
goal: { address: "0x9999" },
|
||||
};
|
||||
expect(validate(input), JSON.stringify(validate.errors)).toBe(true);
|
||||
for (const limited_directions of [
|
||||
[],
|
||||
["callers", "callers"],
|
||||
["callees", "callees"],
|
||||
["callers", "callees", "callers"],
|
||||
["incoming"],
|
||||
])
|
||||
expect(
|
||||
validate({
|
||||
...input,
|
||||
functions: [
|
||||
{
|
||||
...observation,
|
||||
relationship_coverage: {
|
||||
...coverage,
|
||||
limited_directions,
|
||||
},
|
||||
},
|
||||
],
|
||||
}),
|
||||
).toBe(false);
|
||||
const result = await client.callTool({
|
||||
name: "build_call_path",
|
||||
arguments: input,
|
||||
});
|
||||
if (result.isError) throw new Error(JSON.stringify(result));
|
||||
expect(result.structuredContent).toMatchObject({
|
||||
normalized_result: {
|
||||
status,
|
||||
search_scope: { exhaustive },
|
||||
limitations: expect.arrayContaining([limitation]),
|
||||
},
|
||||
});
|
||||
const comparison = await client.callTool({
|
||||
name: "compare_functions",
|
||||
arguments: { left: observation, right: observation },
|
||||
});
|
||||
if (comparison.isError) throw new Error(JSON.stringify(comparison));
|
||||
expect(comparison.structuredContent).toMatchObject({
|
||||
normalized_result: {
|
||||
dimensions: expect.arrayContaining([
|
||||
expect.objectContaining({ dimension: "calls", status: "unknown" }),
|
||||
]),
|
||||
},
|
||||
});
|
||||
// Each scenario owns its Evidence and residual-unknown ledger.
|
||||
const closed = await session.close();
|
||||
if (!closed.ok) throw closed.error;
|
||||
}
|
||||
} finally {
|
||||
await Promise.allSettled([session.close(), client.close(), server.close()]);
|
||||
}
|
||||
});
|
||||
@@ -15,9 +15,17 @@ import {
|
||||
type GhidraFunctionOperation,
|
||||
} from "../../../src/ghidra/GhidraFunctionValues.js";
|
||||
|
||||
it.each([true, false])(
|
||||
"compares observed empty Ghidra facets while preserving pseudocode availability (%s)",
|
||||
async (pseudocodeAvailable) => {
|
||||
it.each([
|
||||
{ pseudocodeAvailable: true, reportsCoverage: true },
|
||||
{ pseudocodeAvailable: false, reportsCoverage: true },
|
||||
{ pseudocodeAvailable: true, reportsCoverage: false },
|
||||
{ pseudocodeAvailable: false, reportsCoverage: false },
|
||||
])(
|
||||
"compares observed empty Ghidra facets with pseudocode=$pseudocodeAvailable and coverage=$reportsCoverage",
|
||||
async ({ pseudocodeAvailable, reportsCoverage }) => {
|
||||
const coverage = reportsCoverage
|
||||
? { status: "complete" as const }
|
||||
: undefined;
|
||||
const dossier = functionDossierSchema.parse(ghidraFunctionDossier());
|
||||
Object.assign(dossier, {
|
||||
comments: [],
|
||||
@@ -47,7 +55,13 @@ it.each([true, false])(
|
||||
if (operation !== "analyze_function")
|
||||
throw new Error(`Unexpected operation: ${operation}`);
|
||||
return Promise.resolve(
|
||||
ok({ value: jsonValueSchema.parse(dossier), limitations: [] }),
|
||||
ok({
|
||||
value: jsonValueSchema.parse(dossier),
|
||||
limitations: [],
|
||||
...(coverage === undefined
|
||||
? {}
|
||||
: { relationship_coverage: coverage }),
|
||||
}),
|
||||
);
|
||||
},
|
||||
);
|
||||
@@ -61,9 +75,7 @@ it.each([true, false])(
|
||||
harness.session,
|
||||
analyzed.structuredContent,
|
||||
);
|
||||
expect(evidence.relationship_coverage).toMatchObject({
|
||||
status: "complete",
|
||||
});
|
||||
expect(evidence.relationship_coverage).toEqual(coverage ?? null);
|
||||
expect(evidence.normalized_result).toEqual(dossier);
|
||||
const reply = await harness.mcp.callTool({
|
||||
name: "compare_functions",
|
||||
@@ -86,7 +98,7 @@ it.each([true, false])(
|
||||
pseudocode: pseudocodeAvailable ? "unchanged" : "unknown",
|
||||
assembly: "unchanged",
|
||||
comments: "unchanged",
|
||||
calls: "unchanged",
|
||||
calls: reportsCoverage ? "unchanged" : "unknown",
|
||||
references: "unchanged",
|
||||
strings_names: "unchanged",
|
||||
cfg: "unchanged",
|
||||
@@ -96,9 +108,7 @@ it.each([true, false])(
|
||||
evidence.parameters,
|
||||
);
|
||||
if (!replay.ok) throw replay.error;
|
||||
expect(replay.value.relationshipCoverage).toEqual(
|
||||
evidence.relationship_coverage,
|
||||
);
|
||||
expect(replay.value.relationshipCoverage).toEqual(coverage);
|
||||
expect(harness.calls).toEqual(["analyze_function"]);
|
||||
} finally {
|
||||
await harness.close();
|
||||
|
||||
@@ -106,6 +106,8 @@ export const ghidraInstallationHost = (): GhidraInstallationHost => ({
|
||||
readText: () => "application.version=12.1.4\napplication.java.min=21\n",
|
||||
executable: () => true,
|
||||
probeJava: () => ({
|
||||
osArch: "amd64",
|
||||
osName: "Linux",
|
||||
version: "21.0.11",
|
||||
major: 21,
|
||||
home: "/usr/lib/jvm/jdk-21",
|
||||
|
||||
@@ -102,6 +102,8 @@ const installationHost: GhidraInstallationHost = {
|
||||
readText: () => "application.version=12.1.4\napplication.java.min=21\n",
|
||||
executable: () => true,
|
||||
probeJava: () => ({
|
||||
osArch: "amd64",
|
||||
osName: "Linux",
|
||||
version: "21.0.11",
|
||||
major: 21,
|
||||
home: "/fixture/jdk-21",
|
||||
|
||||
@@ -0,0 +1,164 @@
|
||||
import { mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { expect } from "vitest";
|
||||
import { z } from "zod";
|
||||
|
||||
import { interfaceBuilderAnalysisSchema } from "../../../src/domain/apple/interfaceBuilderGraph.js";
|
||||
import { thinMach } from "../../../src/domain/binaryTarget.fixture.js";
|
||||
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
import { mcpTest } from "../../support/mcp/mcpFixture.js";
|
||||
import { openProductionTarget } from "../../support/mcp/productionSession.js";
|
||||
|
||||
const evidenceOf = <Schema extends z.ZodType>(schema: Schema) =>
|
||||
z.object({
|
||||
evidence_id: z.string(),
|
||||
parameters: z.record(z.string(), z.unknown()),
|
||||
normalized_result: schema,
|
||||
});
|
||||
const summarySchema = z.object({
|
||||
kind: z.literal("summary"),
|
||||
parent_evidence_id: z.string(),
|
||||
summary: z.object({
|
||||
documents: z.array(z.unknown()),
|
||||
available_counts: z.object({
|
||||
nodes: z.number(),
|
||||
edges: z.number(),
|
||||
coverage: z.number(),
|
||||
}),
|
||||
}),
|
||||
});
|
||||
const pageSchema = z.object({
|
||||
kind: z.literal("interface-builder"),
|
||||
items: z.array(z.unknown()),
|
||||
coverage: z.object({ next_offset: z.number().nullable() }),
|
||||
});
|
||||
|
||||
/** A two-document app; each nib is a real rooted UIKit outlet archive. */
|
||||
const fixtureApp = async (): Promise<string> => {
|
||||
const app = join(
|
||||
await createTestTempDirectory("rea-ib-summary-mcp-"),
|
||||
"Fixture.app",
|
||||
);
|
||||
const contents = join(app, "Contents");
|
||||
const resources = join(contents, "Resources");
|
||||
await mkdir(resources, { recursive: true });
|
||||
await mkdir(join(contents, "MacOS"));
|
||||
await writeFile(
|
||||
join(contents, "MacOS", "Fixture"),
|
||||
thinMach(0xfeedfacf, 0x0100000c),
|
||||
);
|
||||
await writeFile(
|
||||
join(contents, "Info.plist"),
|
||||
"<plist><dict><key>CFBundleExecutable</key><string>Fixture</string></dict></plist>",
|
||||
);
|
||||
const nib = await readFile(
|
||||
join(process.cwd(), "tests/fixtures/interface-builder/UIKitOutlet.xml"),
|
||||
);
|
||||
await writeFile(join(resources, "Main.nib"), nib);
|
||||
await writeFile(join(resources, "Second.nib"), nib);
|
||||
return app;
|
||||
};
|
||||
|
||||
mcpTest(
|
||||
"retains complete Interface Builder Evidence and pages one document's rows",
|
||||
async ({ mcp, onTestFinished }) => {
|
||||
const app = await fixtureApp();
|
||||
const client = await openProductionTarget(mcp, onTestFinished, app);
|
||||
|
||||
const invalidDetail = await client.callTool({
|
||||
name: "decode_interface_builder",
|
||||
arguments: { detail: "brief" },
|
||||
});
|
||||
expect(invalidDetail.isError).toBe(true);
|
||||
expect(invalidDetail.content).toEqual([
|
||||
expect.objectContaining({
|
||||
type: "text",
|
||||
text: expect.stringMatching(/Input validation error:.*detail/su),
|
||||
}),
|
||||
]);
|
||||
|
||||
// Summary first: its own retained parent must serve every later page.
|
||||
const summarized = await client.callTool({
|
||||
name: "decode_interface_builder",
|
||||
arguments: { detail: "summary" },
|
||||
});
|
||||
expect(summarized.isError, JSON.stringify(summarized.content)).not.toBe(
|
||||
true,
|
||||
);
|
||||
const summary = evidenceOf(summarySchema).parse(
|
||||
summarized.structuredContent,
|
||||
).normalized_result;
|
||||
const source = {
|
||||
kind: "retained-evidence",
|
||||
evidence_id: summary.parent_evidence_id,
|
||||
};
|
||||
const readAll = async (view: Record<string, unknown>) => {
|
||||
const rows: unknown[] = [];
|
||||
let offset: number | null = 0;
|
||||
while (offset !== null) {
|
||||
const paged = await client.callTool({
|
||||
name: "inspect_analysis_view",
|
||||
arguments: {
|
||||
source,
|
||||
view: { kind: "interface-builder", ...view, offset, limit: 2 },
|
||||
},
|
||||
});
|
||||
expect(paged.isError, JSON.stringify(paged.content)).not.toBe(true);
|
||||
const page = evidenceOf(pageSchema).parse(
|
||||
paged.structuredContent,
|
||||
).normalized_result;
|
||||
rows.push(...page.items);
|
||||
offset = page.coverage.next_offset;
|
||||
}
|
||||
return rows;
|
||||
};
|
||||
const main = "Contents/Resources/Main.nib";
|
||||
const edges = await readAll({ collection: "edges" });
|
||||
const mainNodes = await readAll({ collection: "nodes", document: main });
|
||||
|
||||
// Only then compare with a complete call made for reference.
|
||||
const complete = await client.callTool({
|
||||
name: "decode_interface_builder",
|
||||
arguments: {},
|
||||
});
|
||||
expect(complete.isError, JSON.stringify(complete.content)).not.toBe(true);
|
||||
const full = evidenceOf(interfaceBuilderAnalysisSchema).parse(
|
||||
complete.structuredContent,
|
||||
);
|
||||
expect(full.parameters).not.toHaveProperty("detail");
|
||||
const report = full.normalized_result;
|
||||
expect(report.documents).toHaveLength(2);
|
||||
// Delivery detail never reaches the decoder or the source Evidence ID.
|
||||
expect(summary.parent_evidence_id).toBe(full.evidence_id);
|
||||
expect(summary.summary.documents).toEqual(report.documents);
|
||||
expect(summary.summary.available_counts).toEqual({
|
||||
nodes: report.graph.nodes.length,
|
||||
edges: report.graph.edges.length,
|
||||
coverage: report.graph.coverage.length,
|
||||
});
|
||||
expect(edges).toEqual(report.graph.edges);
|
||||
const expectedMainNodes = report.graph.nodes.filter(({ evidence }) =>
|
||||
evidence.some(({ artifact_path }) => artifact_path === main),
|
||||
);
|
||||
expect(expectedMainNodes.length).toBeGreaterThan(0);
|
||||
expect(expectedMainNodes.length).toBeLessThan(report.graph.nodes.length);
|
||||
expect(mainNodes).toEqual(expectedMainNodes);
|
||||
|
||||
const unknownDocument = await client.callTool({
|
||||
name: "inspect_analysis_view",
|
||||
arguments: {
|
||||
source,
|
||||
view: {
|
||||
kind: "interface-builder",
|
||||
collection: "nodes",
|
||||
document: "Contents/Resources/Missing.nib",
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(parseMcpToolError(unknownDocument)).toMatchObject({
|
||||
error: { code: "invalid_request" },
|
||||
});
|
||||
},
|
||||
);
|
||||
@@ -1,40 +1,24 @@
|
||||
import { silentLogger } from "../../../src/logger.js";
|
||||
import { expect } from "vitest";
|
||||
import { z } from "zod";
|
||||
|
||||
import { createBinarySession } from "../../../src/composition/binary.js";
|
||||
import { parseConfig } from "../../../src/config/parseConfig.js";
|
||||
import { keyedArchiveResultSchema } from "../../../src/domain/apple/keyedArchive.js";
|
||||
import { createServer } from "../../../src/server/createServer.js";
|
||||
import {
|
||||
archiveNumberMetadataCases,
|
||||
expectKeyedArchiveNumberMetadata,
|
||||
keyedArchiveNumberMetadataFixture,
|
||||
} from "../../fixtures/keyedArchiveNumberMetadata.js";
|
||||
import { mcpTest } from "../../support/mcp/mcpFixture.js";
|
||||
import { openProductionTarget } from "../../support/mcp/productionSession.js";
|
||||
|
||||
mcpTest.for(archiveNumberMetadataCases)(
|
||||
"MCP preserves numeric meaning for $name",
|
||||
async (item, { mcp, onTestFinished }) => {
|
||||
const fixture = await keyedArchiveNumberMetadataFixture(item);
|
||||
const configured = parseConfig({ REA_ANALYSIS_PROVIDER: "auto" });
|
||||
if (!configured.ok) throw configured.error;
|
||||
const session = createBinarySession(
|
||||
configured.value,
|
||||
silentLogger,
|
||||
process.env,
|
||||
const client = await openProductionTarget(
|
||||
mcp,
|
||||
onTestFinished,
|
||||
fixture.path,
|
||||
);
|
||||
onTestFinished(async () => {
|
||||
await session.close();
|
||||
});
|
||||
const client = await mcp.connect(
|
||||
createServer({ kind: "session", session }),
|
||||
);
|
||||
const opened = await client.callTool({
|
||||
name: "open_binary",
|
||||
arguments: { path: fixture.path },
|
||||
});
|
||||
expect(opened.isError, JSON.stringify(opened.content)).not.toBe(true);
|
||||
const result = await client.callTool({
|
||||
name: "inspect_keyed_archive",
|
||||
arguments: {},
|
||||
|
||||
@@ -1,39 +1,23 @@
|
||||
import { expect } from "vitest";
|
||||
import { z } from "zod";
|
||||
|
||||
import { createBinarySession } from "../../../src/composition/binary.js";
|
||||
import { parseConfig } from "../../../src/config/parseConfig.js";
|
||||
import { keyedArchiveResultSchema } from "../../../src/domain/apple/keyedArchive.js";
|
||||
import { silentLogger } from "../../../src/logger.js";
|
||||
import { createServer } from "../../../src/server/createServer.js";
|
||||
import {
|
||||
expectKeyedArchivePrototypeKeys,
|
||||
keyedArchivePrototypeKeyFixture,
|
||||
} from "../../fixtures/keyedArchivePrototypeKeys.js";
|
||||
import { mcpTest } from "../../support/mcp/mcpFixture.js";
|
||||
import { openProductionTarget } from "../../support/mcp/productionSession.js";
|
||||
|
||||
mcpTest(
|
||||
"MCP retains ordinary evidence with formatted XML prototype keys",
|
||||
async ({ mcp, onTestFinished }) => {
|
||||
const fixture = await keyedArchivePrototypeKeyFixture();
|
||||
const configured = parseConfig({ REA_ANALYSIS_PROVIDER: "auto" });
|
||||
if (!configured.ok) throw configured.error;
|
||||
const session = createBinarySession(
|
||||
configured.value,
|
||||
silentLogger,
|
||||
process.env,
|
||||
const client = await openProductionTarget(
|
||||
mcp,
|
||||
onTestFinished,
|
||||
fixture.path,
|
||||
);
|
||||
onTestFinished(async () => {
|
||||
await session.close();
|
||||
});
|
||||
const client = await mcp.connect(
|
||||
createServer({ kind: "session", session }),
|
||||
);
|
||||
const opened = await client.callTool({
|
||||
name: "open_binary",
|
||||
arguments: { path: fixture.path },
|
||||
});
|
||||
expect(opened.isError, JSON.stringify(opened.content)).not.toBe(true);
|
||||
const result = await client.callTool({
|
||||
name: "inspect_keyed_archive",
|
||||
arguments: {},
|
||||
|
||||
@@ -1,15 +1,12 @@
|
||||
import { silentLogger } from "../../../src/logger.js";
|
||||
import { copyFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { describe, expect } from "vitest";
|
||||
import { z } from "zod";
|
||||
|
||||
import { createBinarySession } from "../../../src/composition/binary.js";
|
||||
import { parseConfig } from "../../../src/config/parseConfig.js";
|
||||
import { createServer } from "../../../src/server/createServer.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
import { mcpTest } from "../../support/mcp/mcpFixture.js";
|
||||
import { openProductionTarget } from "../../support/mcp/productionSession.js";
|
||||
|
||||
const demangledSchema = z.object({
|
||||
normalized_result: z.object({
|
||||
@@ -30,24 +27,7 @@ describe.skipIf(process.platform !== "darwin")(
|
||||
"tool",
|
||||
);
|
||||
await copyFile("/usr/bin/true", target);
|
||||
const configured = parseConfig({ REA_ANALYSIS_PROVIDER: "auto" });
|
||||
if (!configured.ok) throw configured.error;
|
||||
const session = createBinarySession(
|
||||
configured.value,
|
||||
silentLogger,
|
||||
process.env,
|
||||
);
|
||||
onTestFinished(async () => {
|
||||
await session.close();
|
||||
});
|
||||
const client = await mcp.connect(
|
||||
createServer({ kind: "session", session }),
|
||||
);
|
||||
const opened = await client.callTool({
|
||||
name: "open_binary",
|
||||
arguments: { path: target },
|
||||
});
|
||||
expect(opened.isError, JSON.stringify(opened.content)).not.toBe(true);
|
||||
const client = await openProductionTarget(mcp, onTestFinished, target);
|
||||
|
||||
const called = await client.callTool({
|
||||
name: "demangle_swift",
|
||||
|
||||
@@ -1,12 +1,9 @@
|
||||
import { silentLogger } from "../../../src/logger.js";
|
||||
import { copyFile, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { describe, expect } from "vitest";
|
||||
import { z } from "zod";
|
||||
|
||||
import { createBinarySession } from "../../../src/composition/binary.js";
|
||||
import { parseConfig } from "../../../src/config/parseConfig.js";
|
||||
import { inspectMachoSchema } from "../../../src/domain/native/nativeInspection.js";
|
||||
import { createServer } from "../../../src/server/createServer.js";
|
||||
import { thinMach } from "../../../src/domain/binaryTarget.fixture.js";
|
||||
@@ -16,6 +13,7 @@ import { ok } from "../../../src/domain/result.js";
|
||||
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
import { mcpTest } from "../../support/mcp/mcpFixture.js";
|
||||
import { openProductionTarget } from "../../support/mcp/productionSession.js";
|
||||
|
||||
const evidenceOf = <Schema extends z.ZodType>(schema: Schema) =>
|
||||
z.object({
|
||||
@@ -54,24 +52,7 @@ describe.skipIf(process.platform !== "darwin")(
|
||||
"ls",
|
||||
);
|
||||
await copyFile("/bin/ls", target);
|
||||
const configured = parseConfig({ REA_ANALYSIS_PROVIDER: "auto" });
|
||||
if (!configured.ok) throw configured.error;
|
||||
const session = createBinarySession(
|
||||
configured.value,
|
||||
silentLogger,
|
||||
process.env,
|
||||
);
|
||||
onTestFinished(async () => {
|
||||
await session.close();
|
||||
});
|
||||
const client = await mcp.connect(
|
||||
createServer({ kind: "session", session }),
|
||||
);
|
||||
const opened = await client.callTool({
|
||||
name: "open_binary",
|
||||
arguments: { path: target },
|
||||
});
|
||||
expect(opened.isError, JSON.stringify(opened.content)).not.toBe(true);
|
||||
const client = await openProductionTarget(mcp, onTestFinished, target);
|
||||
|
||||
const complete = await client.callTool({
|
||||
name: "inspect_macho",
|
||||
|
||||
@@ -0,0 +1,323 @@
|
||||
import { chmod, mkdir, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { expect, vi } from "vitest";
|
||||
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
import {
|
||||
processTest,
|
||||
type TestProcesses,
|
||||
} from "../../support/process/processFixture.js";
|
||||
|
||||
const script = `
|
||||
import { PlaywrightElectronActiveProvider } from "./dist/browser/PlaywrightElectronActiveProvider.js";
|
||||
import { electronActiveObservationInputSchema } from "./dist/domain/javascript/electronActiveObservation.js";
|
||||
|
||||
const [scenario, executable, root] = JSON.parse(process.argv[1]);
|
||||
let observedRejections = 0;
|
||||
if (scenario === "oneFailureWithManySuccesses" || scenario === "externalListenerSeesUnrelatedOnce")
|
||||
process.on("unhandledRejection", () => { observedRejections += 1; });
|
||||
const orphanStack = (message, file) =>
|
||||
message + "\\n at waitForLine (/app/node_modules/playwright-core/lib/server/" + file + ":247:21)";
|
||||
const playwrightError = (message, file = "electron/electron.js") => {
|
||||
const error = new Error(message);
|
||||
error.stack = "Error: " + orphanStack(message, file);
|
||||
return error;
|
||||
};
|
||||
const rejectLater = (error, ms) => setTimeout(() => void Promise.reject(error), ms);
|
||||
const fakeApplication = { process: () => ({ pid: undefined }), close: async () => undefined };
|
||||
const launchFailure = (message) =>
|
||||
new Error("electron.launch: " + message + "\\nCall log:\\n - <launching>");
|
||||
const after = (ms, action) => new Promise((resolve, reject) => setTimeout(() => {
|
||||
try { resolve(action()); } catch (error) { reject(error); }
|
||||
}, ms));
|
||||
const orphans = (message) => {
|
||||
for (let index = 0; index < 3; index += 1) void Promise.reject(playwrightError(message));
|
||||
};
|
||||
|
||||
let during = -1;
|
||||
const launches = {
|
||||
realLaunch: undefined,
|
||||
realWithUnrelatedPlaywrightRejection: async (options) => {
|
||||
void Promise.reject(playwrightError("unrelated playwright failure", "page.js"));
|
||||
return (await import("playwright-core"))._electron.launch(options);
|
||||
},
|
||||
failureWithOrphans: async () => {
|
||||
during = process.listenerCount("unhandledRejection");
|
||||
orphans("Process failed to launch!");
|
||||
await after(10, () => { throw launchFailure("Process failed to launch!"); });
|
||||
},
|
||||
failureInOneTick: async () => {
|
||||
orphans("Process failed to launch!");
|
||||
throw launchFailure("Process failed to launch!");
|
||||
},
|
||||
failureWithUnrelatedPlaywrightRejection: async () => {
|
||||
orphans("Process failed to launch!");
|
||||
void Promise.reject(playwrightError("Target page, context or browser has been closed", "page.js"));
|
||||
await after(10, () => { throw launchFailure("Process failed to launch!"); });
|
||||
},
|
||||
externalListenerSeesUnrelatedOnce: async () => {
|
||||
void Promise.reject(playwrightError("unrelated playwright failure", "page.js"));
|
||||
await after(10, () => { throw launchFailure("Process failed to launch!"); });
|
||||
},
|
||||
pendingWithUnrelatedRejection: () => {
|
||||
setInterval(() => {}, 1000);
|
||||
rejectLater(new Error("unrelated application failure"), 20);
|
||||
return new Promise(() => {});
|
||||
},
|
||||
success: async () => {
|
||||
during = process.listenerCount("unhandledRejection");
|
||||
return fakeApplication;
|
||||
},
|
||||
successWithPlaywrightRejection: async () => {
|
||||
void Promise.reject(playwrightError("stray playwright failure", "page.js"));
|
||||
await after(10, () => fakeApplication);
|
||||
return fakeApplication;
|
||||
},
|
||||
};
|
||||
|
||||
const input = (executablePath) => ({
|
||||
...electronActiveObservationInputSchema.parse({
|
||||
executable_path: executablePath,
|
||||
application_path: root,
|
||||
}),
|
||||
application_root: root,
|
||||
});
|
||||
const capture = (launch, executablePath = executable) =>
|
||||
new PlaywrightElectronActiveProvider({}, launch).capture(input(executablePath));
|
||||
|
||||
const before = process.listenerCount("unhandledRejection");
|
||||
let results;
|
||||
if (scenario === "concurrentFailures" || scenario === "failureThenSuccess" || scenario === "successThenFailure" || scenario === "oneFailureWithManySuccesses") {
|
||||
const failing = (message, orphanAt, settleAt) => async () => {
|
||||
during = Math.max(during, process.listenerCount("unhandledRejection"));
|
||||
await after(orphanAt, () => orphans(message));
|
||||
await after(settleAt - orphanAt, () => { throw launchFailure(message); });
|
||||
};
|
||||
const succeeding = (settleAt) => () => {
|
||||
during = Math.max(during, process.listenerCount("unhandledRejection"));
|
||||
return after(settleAt, () => fakeApplication);
|
||||
};
|
||||
const pair = {
|
||||
concurrentFailures: [failing("alpha failed", 10, 30), failing("beta failed", 20, 50)],
|
||||
failureThenSuccess: [failing("alpha failed", 10, 20), succeeding(40)],
|
||||
successThenFailure: [failing("alpha failed", 20, 40), succeeding(30)],
|
||||
oneFailureWithManySuccesses: [failing("alpha failed", 10, 80), ...[20, 30, 40, 50].map(succeeding)],
|
||||
}[scenario];
|
||||
results = await Promise.all(pair.map((launch) => capture(launch)));
|
||||
} else {
|
||||
results = [await capture(launches[scenario] ?? undefined)];
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
console.log(JSON.stringify({
|
||||
before,
|
||||
during,
|
||||
after: process.listenerCount("unhandledRejection"),
|
||||
observedRejections,
|
||||
reasons: results.map((result) => result.ok ? null : result.error.reason ?? null),
|
||||
ok: results.map((result) => result.ok),
|
||||
causeReasons: results.map((result) => result.ok ? null : result.error.cause?.reason ?? null),
|
||||
cleanupResources: results.map((result) => result.ok ? null : result.error.cleanupResources ?? null),
|
||||
causes: results.map((result) => result.ok ? null : String(result.error.cause?.message).split("\\n")[0]),
|
||||
}));
|
||||
`;
|
||||
|
||||
// A cold macOS Swift toolchain may spend up to 120 seconds building the
|
||||
// ownership reader before any child launch reaches its assertion.
|
||||
vi.setConfig({ testTimeout: 180_000 });
|
||||
|
||||
type Outcome = {
|
||||
readonly before: number;
|
||||
readonly during: number;
|
||||
readonly after: number;
|
||||
readonly ok: readonly boolean[];
|
||||
readonly causes: readonly (string | null)[];
|
||||
readonly observedRejections: number;
|
||||
readonly reasons: readonly (string | null)[];
|
||||
readonly causeReasons: readonly (string | null)[];
|
||||
readonly cleanupResources: readonly (readonly string[] | null)[];
|
||||
};
|
||||
|
||||
const run = async (
|
||||
processes: TestProcesses,
|
||||
scenario: string,
|
||||
executable = "/not/launched",
|
||||
root = "/",
|
||||
) =>
|
||||
processes.run(
|
||||
process.execPath,
|
||||
[
|
||||
"--input-type=module",
|
||||
"--eval",
|
||||
script,
|
||||
JSON.stringify([scenario, executable, root]),
|
||||
],
|
||||
{ timeoutMs: 150_000 },
|
||||
);
|
||||
|
||||
const outcome = (result: { readonly stdout: string }): Outcome =>
|
||||
JSON.parse(result.stdout.trim().split("\n").at(-1) ?? "") as Outcome;
|
||||
|
||||
const earlyExit = async () => {
|
||||
const root = await createTestTempDirectory("rea-launch-rejections-");
|
||||
const executable = join(root, "exits-immediately");
|
||||
await mkdir(join(root, "app"));
|
||||
await writeFile(executable, "#!/bin/sh\nexit 0\n");
|
||||
await chmod(executable, 0o755);
|
||||
return { executable, root: join(root, "app") };
|
||||
};
|
||||
|
||||
processTest(
|
||||
"absorbs the orphaned rejections of a failed launch and removes its listener",
|
||||
async ({ processes }) => {
|
||||
const result = await run(processes, "failureWithOrphans");
|
||||
expect(result.exitCode, result.stderr).toBe(0);
|
||||
expect(outcome(result)).toMatchObject({
|
||||
during: outcome(result).before + 1,
|
||||
after: outcome(result).before,
|
||||
ok: [false],
|
||||
causes: ["electron.launch: Process failed to launch!"],
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
processTest(
|
||||
"absorbs orphans raised in the same tick as the launch failure",
|
||||
async ({ processes }) => {
|
||||
const result = await run(processes, "failureInOneTick");
|
||||
expect(result.exitCode, result.stderr).toBe(0);
|
||||
expect(outcome(result)).toMatchObject({ ok: [false] });
|
||||
},
|
||||
);
|
||||
|
||||
processTest.skipIf(process.platform === "win32")(
|
||||
"absorbs the orphans of a real Playwright launch that exits at once",
|
||||
async ({ processes }) => {
|
||||
const { executable, root } = await earlyExit();
|
||||
const result = await run(processes, "realLaunch", executable, root);
|
||||
expect(result.exitCode, result.stderr).toBe(0);
|
||||
expect(outcome(result)).toMatchObject({
|
||||
ok: [false],
|
||||
causes: ["electron.launch: Process failed to launch!"],
|
||||
});
|
||||
expect(outcome(result).after).toBe(outcome(result).before);
|
||||
},
|
||||
);
|
||||
|
||||
processTest.skipIf(process.platform === "win32")(
|
||||
"still fails on an unrelated Playwright rejection during a real failing launch",
|
||||
async ({ processes }) => {
|
||||
const { executable, root } = await earlyExit();
|
||||
const result = await run(
|
||||
processes,
|
||||
"realWithUnrelatedPlaywrightRejection",
|
||||
executable,
|
||||
root,
|
||||
);
|
||||
expect(result.exitCode, result.stderr).toBe(1);
|
||||
expect(result.stderr).toContain("unrelated playwright failure");
|
||||
expect(result.stdout).toBe("");
|
||||
},
|
||||
);
|
||||
|
||||
processTest(
|
||||
"does not replay an unrelated orphan to a listener that already received it",
|
||||
async ({ processes }) => {
|
||||
const result = await run(processes, "externalListenerSeesUnrelatedOnce");
|
||||
expect(result.exitCode, result.stderr).toBe(0);
|
||||
expect(outcome(result)).toMatchObject({
|
||||
observedRejections: 1,
|
||||
ok: [false],
|
||||
causes: ["electron.launch: Process failed to launch!"],
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
processTest(
|
||||
"still fails on an unrelated Playwright rejection beside a failing launch's own orphans",
|
||||
async ({ processes }) => {
|
||||
const result = await run(
|
||||
processes,
|
||||
"failureWithUnrelatedPlaywrightRejection",
|
||||
);
|
||||
expect(result.exitCode, result.stderr).toBe(1);
|
||||
expect(result.stderr).toContain(
|
||||
"Target page, context or browser has been closed",
|
||||
);
|
||||
expect(result.stdout).toBe("");
|
||||
},
|
||||
);
|
||||
|
||||
processTest(
|
||||
"still fails at once on an unrelated rejection while a launch is pending",
|
||||
async ({ processes }) => {
|
||||
const result = await run(processes, "pendingWithUnrelatedRejection");
|
||||
expect(result.exitCode, result.stderr).toBe(1);
|
||||
expect(result.stderr).toContain("unrelated application failure");
|
||||
expect(result.stdout).toBe("");
|
||||
},
|
||||
);
|
||||
|
||||
processTest(
|
||||
"removes its listener after a successful launch",
|
||||
async ({ processes }) => {
|
||||
const result = await run(processes, "success");
|
||||
expect(result.exitCode, result.stderr).toBe(0);
|
||||
expect(outcome(result)).toMatchObject({
|
||||
during: outcome(result).before + 1,
|
||||
after: outcome(result).before,
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
processTest(
|
||||
"re-raises a Playwright rejection when the launch succeeds",
|
||||
async ({ processes }) => {
|
||||
const result = await run(processes, "successWithPlaywrightRejection");
|
||||
expect(result.exitCode, result.stderr).toBe(1);
|
||||
expect(result.stderr).toContain("stray playwright failure");
|
||||
expect(result.stdout).toBe("");
|
||||
},
|
||||
);
|
||||
|
||||
processTest.for([
|
||||
"concurrentFailures",
|
||||
"failureThenSuccess",
|
||||
"successThenFailure",
|
||||
])(
|
||||
"keeps overlapping launches from re-raising each other's orphans (%s)",
|
||||
async (scenario, { processes }) => {
|
||||
const result = await run(processes, scenario);
|
||||
expect(result.exitCode, result.stderr).toBe(0);
|
||||
expect(outcome(result).after).toBe(outcome(result).before);
|
||||
},
|
||||
);
|
||||
|
||||
processTest(
|
||||
"holds each orphan once across many overlapping successful launches",
|
||||
async ({ processes }) => {
|
||||
const result = await run(processes, "oneFailureWithManySuccesses");
|
||||
expect(result.exitCode, result.stderr).toBe(0);
|
||||
const observed = outcome(result);
|
||||
expect(observed.during).toBe(observed.before + 1);
|
||||
expect(observed.after).toBe(observed.before);
|
||||
expect(observed.observedRejections).toBe(3);
|
||||
expect(observed.ok).toEqual([false, false, false, false, false]);
|
||||
expect(observed.causes[0]).toBe("electron.launch: alpha failed");
|
||||
expect(observed.reasons).toEqual([
|
||||
null,
|
||||
"cleanup_failed",
|
||||
"cleanup_failed",
|
||||
"cleanup_failed",
|
||||
"cleanup_failed",
|
||||
]);
|
||||
expect(observed.causeReasons).toEqual([
|
||||
null,
|
||||
...Array.from({ length: 4 }, () => "process_ownership_unavailable"),
|
||||
]);
|
||||
expect(observed.cleanupResources).toEqual([
|
||||
[],
|
||||
...Array.from({ length: 4 }, () => ["owned_process_group"]),
|
||||
]);
|
||||
},
|
||||
);
|
||||
@@ -50,9 +50,16 @@ if (mode === "env-failure" || (probe === "-vmargs" && mode === "vm-failure")) {
|
||||
process.stderr.write("failed " + probe + "\\n");
|
||||
process.exitCode = 3;
|
||||
} else if (probe === "-envvars") {
|
||||
process.stdout.write(mode === "env-malformed" ? "invalid environment\\n" : "GHIDRA_MAXMEM=1G\\n");
|
||||
process.stdout.write(mode === "env-malformed" ? "invalid environment\\n" : mode.startsWith("env-injection:") ? mode.slice("env-injection:".length) + "\\n" : "GHIDRA_MAXMEM=1G\\n");
|
||||
} else {
|
||||
process.stdout.write(mode === "vm-malformed" ? "invalid JVM argument\\n" : "-Dfile.encoding=UTF8\\n");
|
||||
const vmArguments = mode === "vm-malformed"
|
||||
? "invalid JVM argument\\n"
|
||||
: mode === "os-arch-override"
|
||||
? "-Dos.arch=x86_64\\n"
|
||||
: mode === "os-name-override"
|
||||
? "-Dos.name=Windows\\n"
|
||||
: "-Dfile.encoding=UTF8\\n";
|
||||
process.stdout.write(vmArguments);
|
||||
}
|
||||
`,
|
||||
{ mode: 0o755 },
|
||||
@@ -215,6 +222,57 @@ describe.skipIf(process.platform === "win32")(
|
||||
},
|
||||
);
|
||||
|
||||
it.each(["os-arch-override", "os-name-override"])(
|
||||
"rejects LaunchSupport %s before starting analysis",
|
||||
async (mode) => {
|
||||
const { session, launcher, analysisArguments } = await fixture(mode);
|
||||
const launched = await launcher.launch(session);
|
||||
expect(launched.ok).toBe(false);
|
||||
expect(analysisArguments).toEqual([]);
|
||||
if (launched.ok)
|
||||
throw new Error("Expected a JVM platform override failure");
|
||||
expect(launched.error.cause).toMatchObject({
|
||||
message: expect.stringContaining(
|
||||
"may not override os.arch or os.name",
|
||||
),
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it.each([
|
||||
["JAVA_TOOL_OPTIONS", "-Dos.arch=x86_64"],
|
||||
["JAVA_TOOL_OPTIONS", '"-Dos.arch=x86_64"'],
|
||||
["_JAVA_OPTIONS", "'-Dos.name=Windows'"],
|
||||
["JDK_JAVA_OPTIONS", '-D"os.name"=Windows'],
|
||||
["JDK_JAVA_OPTIONS", "-D'os.arch'=x86_64"],
|
||||
])(
|
||||
"rejects helper-environment platform injection from %s=%s before analysis",
|
||||
async (key, value) => {
|
||||
const { session, launcher, analysisArguments } = await fixture(
|
||||
`env-injection:${key}=${value}`,
|
||||
);
|
||||
const launched = await launcher.launch(session);
|
||||
expect(launched.ok).toBe(false);
|
||||
expect(analysisArguments).toEqual([]);
|
||||
if (launched.ok)
|
||||
throw new Error("Expected a JVM platform override failure");
|
||||
expect(launched.error.cause).toMatchObject({
|
||||
message: expect.stringContaining(
|
||||
"may not override os.arch or os.name",
|
||||
),
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("admits a quoted non-platform option without mistaking its value for a platform argument", async () => {
|
||||
const { session, launcher, analysisArguments } = await fixture(
|
||||
'env-injection:JAVA_TOOL_OPTIONS=-Ddescription="text -Dos.arch=x86_64"',
|
||||
);
|
||||
const launched = await launcher.launch(session);
|
||||
if (!launched.ok) throw launched.error;
|
||||
expect(analysisArguments).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("bounds retained UTF-8 warnings and reports original byte counts", async () => {
|
||||
const { session, launcher } = await fixture("large");
|
||||
const launched = await launcher.launch(session);
|
||||
@@ -290,6 +348,13 @@ describe.skipIf(process.platform === "win32")(
|
||||
"application.version=12.1.4\napplication.java.min=21\n",
|
||||
executable: () => true,
|
||||
probeJava: () => ({
|
||||
osArch: process.arch === "arm64" ? "aarch64" : "amd64",
|
||||
osName:
|
||||
process.platform === "darwin"
|
||||
? "Mac OS X"
|
||||
: process.platform === "win32"
|
||||
? "Windows 11"
|
||||
: "Linux",
|
||||
version: "21.0.12",
|
||||
major: 21,
|
||||
home: "/fixture/jdk",
|
||||
|
||||
@@ -524,6 +524,13 @@ describe("Ghidra Windows launcher runtime", () => {
|
||||
),
|
||||
javaHome: "C:\\Java\\jdk-21",
|
||||
bridgeScriptPath: join(parent, "ReaGhidraBridge.java"),
|
||||
// The preflight validates the inspected JVM's LaunchSupport output;
|
||||
// these fixtures assert batch-spawn behavior without a working JDK.
|
||||
javaLaunch: async () => ({
|
||||
command: "C:\\Java\\jdk-21\\bin\\java.exe",
|
||||
arguments: [],
|
||||
environment: {},
|
||||
}),
|
||||
spawnProcess,
|
||||
});
|
||||
const launched = await launcher.launch({
|
||||
@@ -569,6 +576,13 @@ describe("Ghidra Windows launcher runtime", () => {
|
||||
platform: "win32",
|
||||
javaHome,
|
||||
bridgeScriptPath: "/package/bridge/ReaGhidraBridge.java",
|
||||
// The preflight validates the inspected JVM's LaunchSupport output;
|
||||
// this fixture captures the batch launcher without a working JDK.
|
||||
javaLaunch: async () => ({
|
||||
command: `${javaHome}\\bin\\java.exe`,
|
||||
arguments: [],
|
||||
environment: {},
|
||||
}),
|
||||
});
|
||||
const launched = await launcher.launch({
|
||||
runtimeRoot,
|
||||
|
||||
@@ -12,6 +12,8 @@ import ghidra.program.model.listing.FlowOverride;
|
||||
import ghidra.program.model.listing.Function;
|
||||
import ghidra.program.model.listing.Instruction;
|
||||
import ghidra.program.model.symbol.SourceType;
|
||||
import ghidra.program.model.symbol.Reference;
|
||||
import ghidra.program.model.symbol.RefType;
|
||||
import java.io.File;
|
||||
import java.lang.reflect.Method;
|
||||
import java.nio.file.Files;
|
||||
@@ -20,9 +22,12 @@ import java.util.HexFormat;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.ArrayList;
|
||||
|
||||
/** Injects reported analysis flags into imported ELF/Mach-O functions; never executes target code. */
|
||||
public class ReaNoReturnProbe extends GhidraScript {
|
||||
private record MemoryCallReference(Address source, Address target, RefType type,
|
||||
SourceType sourceType, int operandIndex, boolean primary) {}
|
||||
private final JsonArray checks = new JsonArray();
|
||||
private final Map<Function, Address> originalEnds = new LinkedHashMap<>();
|
||||
private final Map<Function, String> originalSignatures = new LinkedHashMap<>();
|
||||
@@ -85,6 +90,114 @@ public class ReaNoReturnProbe extends GhidraScript {
|
||||
return instance;
|
||||
}
|
||||
|
||||
private void knownInteriorCall(GhidraScript bridge) throws Exception {
|
||||
Function caller = function("rea_tls_chain");
|
||||
Function target = function("rea_tls");
|
||||
Address interior = target.getEntryPoint().add(1);
|
||||
require("interior_target_is_owned", currentProgram.getFunctionManager().getFunctionAt(interior) == null &&
|
||||
target.equals(currentProgram.getFunctionManager().getFunctionContaining(interior)));
|
||||
Instruction call = null;
|
||||
for (Instruction instruction : currentProgram.getListing().getInstructions(caller.getBody(), true)) {
|
||||
if (instruction.getFlowType().isCall()) { call = instruction; break; }
|
||||
}
|
||||
if (call == null) throw new IllegalStateException("Missing real call instruction");
|
||||
var references = currentProgram.getReferenceManager();
|
||||
List<Reference> originals = new ArrayList<>();
|
||||
List<MemoryCallReference> saved = new ArrayList<>();
|
||||
for (Reference reference : call.getReferencesFrom()) {
|
||||
if (!reference.getReferenceType().isCall()) continue;
|
||||
originals.add(reference);
|
||||
saved.add(new MemoryCallReference(reference.getFromAddress(), reference.getToAddress(),
|
||||
reference.getReferenceType(), reference.getSource(), reference.getOperandIndex(), reference.isPrimary()));
|
||||
}
|
||||
require("original_call_reference_present", !originals.isEmpty());
|
||||
Reference changed = null;
|
||||
try {
|
||||
for (Reference reference : originals) references.delete(reference);
|
||||
changed = references.addMemoryReference(call.getAddress(), interior,
|
||||
RefType.UNCONDITIONAL_CALL, SourceType.USER_DEFINED, 0);
|
||||
require("interior_target_omitted_by_callee_getter", !caller.getCalledFunctions(monitor).contains(target));
|
||||
JsonObject query = new JsonObject();
|
||||
query.addProperty("document", currentProgram.getName());
|
||||
query.addProperty("procedure", caller.getName());
|
||||
query.addProperty("direction", "outgoing");
|
||||
Method read = bridge.getClass().getDeclaredMethod("procedureReferences", JsonObject.class);
|
||||
read.setAccessible(true);
|
||||
JsonObject output = (JsonObject) read.invoke(bridge, query);
|
||||
require("interior_reference_keeps_known_procedure", output.getAsJsonArray("references").asList().stream().anyMatch(
|
||||
edge -> !edge.getAsJsonObject().get("target_procedure").isJsonNull() &&
|
||||
edge.getAsJsonObject().getAsJsonObject("target_procedure").get("name").getAsString().equals(target.getName())));
|
||||
require("known_reference_is_not_also_unresolved", output.getAsJsonArray("unresolved_calls").isEmpty());
|
||||
query.remove("direction");
|
||||
Method callees = bridge.getClass().getDeclaredMethod("procedureCalls", JsonObject.class, boolean.class);
|
||||
callees.setAccessible(true);
|
||||
JsonObject inventory = (JsonObject) callees.invoke(bridge, query, false);
|
||||
require("omitted_callee_inventory_stays_partial", inventory.getAsJsonObject("relationship_coverage").get("status").getAsString().equals("partial"));
|
||||
require("omitted_callee_direction_is_explicit",
|
||||
inventory.getAsJsonObject("relationship_coverage").getAsJsonArray("limited_directions").toString().equals("[\"callees\"]"));
|
||||
require("interior_caller_omitted_by_entry_getter", !target.getCallingFunctions(monitor).contains(caller));
|
||||
query.addProperty("procedure", target.getName());
|
||||
JsonObject incoming = (JsonObject) callees.invoke(bridge, query, true);
|
||||
require("off_entry_caller_inventory_stays_partial",
|
||||
incoming.getAsJsonObject("relationship_coverage").get("status").getAsString().equals("partial"));
|
||||
require("off_entry_caller_direction_is_explicit",
|
||||
incoming.getAsJsonObject("relationship_coverage").getAsJsonArray("limited_directions").toString().equals("[\"callers\"]"));
|
||||
JsonObject dossierQuery = new JsonObject();
|
||||
dossierQuery.addProperty("procedure", target.getName());
|
||||
Method analyze = bridge.getClass().getDeclaredMethod("analyzeFunctionObservation", JsonObject.class);
|
||||
analyze.setAccessible(true);
|
||||
Object observation = analyze.invoke(bridge, dossierQuery);
|
||||
Method coverage = observation.getClass().getDeclaredMethod("relationshipCoverage");
|
||||
coverage.setAccessible(true);
|
||||
require("dossier_preserves_caller_only_gap",
|
||||
((JsonObject) coverage.invoke(observation)).getAsJsonArray("limited_directions").toString().equals("[\"callers\"]"));
|
||||
} finally {
|
||||
if (changed != null) references.delete(changed);
|
||||
for (MemoryCallReference reference : saved) {
|
||||
Reference restored = references.addMemoryReference(reference.source(), reference.target(),
|
||||
reference.type(), reference.sourceType(), reference.operandIndex());
|
||||
references.setPrimary(restored, reference.primary());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void nonCallInstructionReference(GhidraScript bridge) throws Exception {
|
||||
Function caller = function("rea_tls_chain");
|
||||
Function target = function("rea_memcpy");
|
||||
Instruction source = null;
|
||||
for (Instruction instruction : currentProgram.getListing().getInstructions(caller.getBody(), true)) {
|
||||
if (!instruction.getFlowType().isCall()) { source = instruction; break; }
|
||||
}
|
||||
if (source == null) throw new IllegalStateException("Missing non-call instruction");
|
||||
var references = currentProgram.getReferenceManager();
|
||||
Reference added = references.addMemoryReference(source.getAddress(), target.getEntryPoint().add(1),
|
||||
RefType.UNCONDITIONAL_CALL, SourceType.USER_DEFINED, -1);
|
||||
try {
|
||||
require("non_call_reference_target_omitted", !caller.getCalledFunctions(monitor).contains(target));
|
||||
JsonObject query = new JsonObject();
|
||||
query.addProperty("document", currentProgram.getName());
|
||||
query.addProperty("procedure", caller.getName());
|
||||
Method read = bridge.getClass().getDeclaredMethod("procedureCalls", JsonObject.class, boolean.class);
|
||||
read.setAccessible(true);
|
||||
JsonObject output = (JsonObject) read.invoke(bridge, query, false);
|
||||
require("non_call_instruction_reference_stays_partial",
|
||||
output.getAsJsonObject("relationship_coverage").get("status").getAsString().equals("partial"));
|
||||
query.addProperty("direction", "outgoing");
|
||||
Method edges = bridge.getClass().getDeclaredMethod("procedureReferences", JsonObject.class);
|
||||
edges.setAccessible(true);
|
||||
JsonObject known = (JsonObject) edges.invoke(bridge, query);
|
||||
require("non_call_known_reference_is_not_unresolved", known.getAsJsonArray("unresolved_calls").isEmpty());
|
||||
references.delete(added);
|
||||
Address unknown = currentProgram.getMemory().getMaxAddress().add(0x1000);
|
||||
added = references.addMemoryReference(source.getAddress(), unknown,
|
||||
RefType.UNCONDITIONAL_CALL, SourceType.USER_DEFINED, -1);
|
||||
JsonObject unresolved = (JsonObject) edges.invoke(bridge, query);
|
||||
require("non_call_unknown_reference_keeps_site", unresolved.getAsJsonArray("unresolved_calls").size() == 1);
|
||||
} finally {
|
||||
references.delete(added);
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
String[] args = getScriptArgs();
|
||||
@@ -164,6 +277,8 @@ public class ReaNoReturnProbe extends GhidraScript {
|
||||
Object output = pseudocode.invoke(bridge, query);
|
||||
require("recovered_pseudocode", output instanceof JsonObject value &&
|
||||
value.get("value").getAsString().contains("rea_after_call"));
|
||||
nonCallInstructionReference(bridge);
|
||||
knownInteriorCall(bridge);
|
||||
} finally {
|
||||
((ghidra.app.decompiler.DecompInterface) field.get(bridge)).dispose();
|
||||
}
|
||||
|
||||
@@ -1,199 +0,0 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { test } from "vitest";
|
||||
import { z } from "zod";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
type Delivery = "complete" | "summary";
|
||||
type Observation = {
|
||||
evidence_id: string;
|
||||
parent_evidence_id?: string;
|
||||
limitations: string[];
|
||||
kind?: string;
|
||||
view?: { kind: string; facet?: string; offset?: number; limit?: number };
|
||||
coverage?:
|
||||
| string
|
||||
| {
|
||||
examined: number;
|
||||
total: number;
|
||||
next_offset: number | null;
|
||||
exhausted: boolean;
|
||||
};
|
||||
unknowns?: string[];
|
||||
};
|
||||
type Fixture = {
|
||||
summary: {
|
||||
artifact: string;
|
||||
delivery: Delivery;
|
||||
provider_search_path_sha256: string;
|
||||
calls: unknown[];
|
||||
};
|
||||
answer: {
|
||||
capture: { coverage: string; capture_start: null };
|
||||
http_request_uris_first2: string[];
|
||||
delivery_observations: Observation[];
|
||||
};
|
||||
};
|
||||
const reportSchema = z.object({
|
||||
facts_match: z.boolean(),
|
||||
settings_match: z.boolean(),
|
||||
fact_differences: z.array(z.string()),
|
||||
preserved_run_specific_values: z.array(z.object({ value: z.unknown() })),
|
||||
});
|
||||
|
||||
// Synthetic measurement directories exercise the comparator CLI, not providers.
|
||||
const observation = (kind: string): Observation => ({
|
||||
evidence_id: `ev_${kind}`,
|
||||
parent_evidence_id: "ev_parent",
|
||||
limitations: ["saved bytes only"],
|
||||
kind,
|
||||
view:
|
||||
kind === "summary"
|
||||
? { kind }
|
||||
: { kind, facet: "http-uris", offset: 0, limit: 2 },
|
||||
coverage: { examined: 2, total: 3, next_offset: 2, exhausted: false },
|
||||
unknowns: ["unrecorded traffic"],
|
||||
});
|
||||
const fixture = (delivery: Delivery, artifact: string): Fixture => ({
|
||||
summary: {
|
||||
artifact,
|
||||
delivery,
|
||||
provider_search_path_sha256: "a".repeat(64),
|
||||
calls: [],
|
||||
},
|
||||
answer: {
|
||||
capture: { coverage: "complete", capture_start: null },
|
||||
http_request_uris_first2: [
|
||||
"http://example.test/one",
|
||||
"http://example.test/two",
|
||||
],
|
||||
delivery_observations:
|
||||
delivery === "summary"
|
||||
? [observation("summary"), observation("pcap")]
|
||||
: [
|
||||
{
|
||||
evidence_id: "ev_complete",
|
||||
limitations: ["saved bytes only"],
|
||||
coverage: "complete",
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
function compare(left: Fixture, right: Fixture) {
|
||||
const root = mkdtempSync(join(tmpdir(), "rea-compare-test-"));
|
||||
try {
|
||||
const directories = [join(root, "left"), join(root, "right")] as const;
|
||||
for (const [index, value] of [left, right].entries()) {
|
||||
const directory = directories[index];
|
||||
assert.ok(directory);
|
||||
mkdirSync(directory);
|
||||
for (const name of ["summary", "answer"] as const)
|
||||
writeFileSync(
|
||||
join(directory, `${name}.json`),
|
||||
JSON.stringify(value[name]),
|
||||
);
|
||||
}
|
||||
const result = spawnSync(
|
||||
process.execPath,
|
||||
[
|
||||
fileURLToPath(
|
||||
new URL(
|
||||
"../../experiments/analysis-efficiency/compare.mjs",
|
||||
import.meta.url,
|
||||
),
|
||||
),
|
||||
"--left",
|
||||
directories[0],
|
||||
"--right",
|
||||
directories[1],
|
||||
],
|
||||
{ encoding: "utf8" },
|
||||
);
|
||||
assert.ifError(result.error);
|
||||
assert.equal(result.stderr, "");
|
||||
return {
|
||||
status: result.status,
|
||||
report: reportSchema.parse(JSON.parse(result.stdout)),
|
||||
};
|
||||
} finally {
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
for (const artifact of [
|
||||
"wireshark-http.cap",
|
||||
"wireshark-http_redirects.pcapng",
|
||||
]) {
|
||||
test(`${artifact}: repeated modes ignore only receipt IDs`, () => {
|
||||
for (const delivery of ["complete", "summary"] as const) {
|
||||
const left = fixture(delivery, artifact);
|
||||
const right = structuredClone(left);
|
||||
for (const value of right.answer.delivery_observations) {
|
||||
value.evidence_id += "_second";
|
||||
if (typeof value.parent_evidence_id === "string")
|
||||
value.parent_evidence_id += "_second";
|
||||
}
|
||||
const { status, report } = compare(left, right);
|
||||
assert.equal(status, 0);
|
||||
assert.equal(report.facts_match, true);
|
||||
assert.deepEqual(
|
||||
report.preserved_run_specific_values.map((value) => value.value),
|
||||
[left.answer.delivery_observations, right.answer.delivery_observations],
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test(`${artifact}: repeated modes reject stable observation changes`, () => {
|
||||
for (const delivery of ["complete", "summary"] as const) {
|
||||
const left = fixture(delivery, artifact);
|
||||
const mutations: ((values: Observation[]) => void)[] = [
|
||||
(values) => values[0]!.limitations.push("new limitation"),
|
||||
(values) => {
|
||||
values[0]!.coverage = "partial";
|
||||
},
|
||||
(values) => {
|
||||
delete values[0]!.coverage;
|
||||
},
|
||||
];
|
||||
if (delivery === "summary")
|
||||
mutations.push(
|
||||
(values) => {
|
||||
values[1]!.view!.offset = 1;
|
||||
},
|
||||
(values) => {
|
||||
const coverage = values[1]!.coverage;
|
||||
assert.ok(typeof coverage === "object" && coverage !== null);
|
||||
coverage.next_offset = null;
|
||||
},
|
||||
(values) => values[1]!.unknowns!.push("extra unknown"),
|
||||
(values) => values.reverse(),
|
||||
);
|
||||
for (const mutate of mutations) {
|
||||
const right = structuredClone(left);
|
||||
mutate(right.answer.delivery_observations);
|
||||
const { status, report } = compare(left, right);
|
||||
assert.equal(status, 1);
|
||||
assert.equal(report.settings_match, true);
|
||||
assert.equal(report.facts_match, false);
|
||||
assert.deepEqual(report.fact_differences, ["delivery_observations"]);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test(`${artifact}: different modes preserve their different receipt scopes`, () => {
|
||||
const left = fixture("complete", artifact);
|
||||
const right = fixture("summary", artifact);
|
||||
const { status, report } = compare(left, right);
|
||||
assert.equal(status, 0);
|
||||
assert.equal(report.facts_match, true);
|
||||
assert.deepEqual(
|
||||
report.preserved_run_specific_values.map((value) => value.value),
|
||||
[left.answer.delivery_observations, right.answer.delivery_observations],
|
||||
);
|
||||
right.answer.capture.coverage = "partial";
|
||||
assert.equal(compare(left, right).report.facts_match, false);
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
import type { Client } from "@modelcontextprotocol/client";
|
||||
import { expect, type OnTestFinishedHandler } from "vitest";
|
||||
|
||||
import { createBinarySession } from "../../../src/composition/binary.js";
|
||||
import { parseConfig } from "../../../src/config/parseConfig.js";
|
||||
import { silentLogger } from "../../../src/logger.js";
|
||||
import { createServer } from "../../../src/server/createServer.js";
|
||||
import type { TestMcp } from "./mcpFixture.js";
|
||||
|
||||
/** Open one real target in a production session served over MCP. */
|
||||
export const openProductionTarget = async (
|
||||
mcp: TestMcp,
|
||||
onTestFinished: (handler: OnTestFinishedHandler) => void,
|
||||
path: string,
|
||||
): Promise<Client> => {
|
||||
const configured = parseConfig({ REA_ANALYSIS_PROVIDER: "auto" });
|
||||
if (!configured.ok) throw configured.error;
|
||||
const session = createBinarySession(
|
||||
configured.value,
|
||||
silentLogger,
|
||||
process.env,
|
||||
);
|
||||
onTestFinished(async () => {
|
||||
await session.close();
|
||||
});
|
||||
const client = await mcp.connect(createServer({ kind: "session", session }));
|
||||
const opened = await client.callTool({
|
||||
name: "open_binary",
|
||||
arguments: { path },
|
||||
});
|
||||
expect(opened.isError, JSON.stringify(opened.content)).not.toBe(true);
|
||||
return client;
|
||||
};
|
||||
Reference in New Issue
Block a user