Commit Graph
28 Commits
Author SHA1 Message Date
morluto 34a4bb6508 fix(elf): centralize fixed header validation across consumers
Validate ELF identification/header versions and declared header bounds at the shared owner. Reuse parsed facts in target selection, Go, Dart and MIPS readers while preserving their stricter format policies and x32 support.

Repair synthetic ELF fixtures and remove a superseded Go error-text assertion. Validation: 74 source tests, 202 filesystem/Go/MIPS boundary tests, typecheck and lint. No new real-provider claims.
2026-10-13 08:07:58 +08:00
morluto 921bb95755 fix(artifacts): distinguish Java classes from universal Mach-O
Classify FAT containers using shared architecture, range, and observed slice-header facts rather than the CAFEBABE signature alone. Preserve typed unsupported and malformed reasons, reject malformed sibling slices consistently, and remove repeated range parsing in consumers.

Validation: valid small and large JVM classes in public JAR/ZIP inspection, four FAT width/byte-order controls, and focused executable/Mach-O boundary tests. The class fixtures were checked with JDK javap; no new JVM analysis capability is advertised.

The Apple selection and Hopper readers retain their existing parsing because they also own oversized-count diagnostics, reserved fields, streaming final-size validation, or source freshness that the shared header result does not represent.

Representative public mixed-ZIP inventory probe, four fresh processes with an already warm filesystem cache: first process elapsed 626.8 -> 548.3 ms, CPU 740 -> 740 ms, sampled family peak RSS 375.5 -> 345.4 MiB. Subsequent three-process medians: elapsed 767.2 -> 454.0 ms, CPU 890 -> 640 ms, peak RSS 368.7 -> 346.2 MiB. Estimated output tokens 2310 -> 2292. Concurrent load was uncontrolled, so these observations do not establish a speed improvement. Whole-agent task tokens and schema/call overhead were not measured.

Fixes #2317
2026-10-13 04:12:04 +08:00
morluto c9fed50691 fix(targets): recognize Windows headers after short DOS stubs
Observe MZ header declarations and signature uncertainty at the domain boundary, then reuse those facts in bounded target resolution, inventory, and Go PE parsing. Preserve DOS relocation ownership and fail closed when the relevant bytes remain unexamined.

Validation: 59 domain tests, 130 target/Go boundary tests, and public CLI inspection of a clang/lld-link PE DLL with a short DOS header. Fixes #2172.
2026-10-13 03:47:45 +08:00
0baad58e60 feat(ghidra): admit RISC-V ELF analysis through a pinned generic lane (#2151)
* feat(ghidra): admit RISC-V ELF analysis through a pinned generic lane

Refs #2063.

Recognize `EM_RISCV` (243) as a provider-neutral family that carries its own ELF
facts, and bound the first Ghidra lane to little-endian `ET_EXEC` images that
declare the double-float ABI, pinned to `RISCV:LE:32:default` or
`RISCV:LE:64:default` with the `gcc` compiler specification.

Ghidra 12.1.4 is not reliable without the pin: its `riscv.opinion` pins
`variant="default"` for the 64-bit constraint and leaves the 32-bit constraint
with the comment that the loader "must choose between 'default' and 'AndeStar'
variants". Measured on the installed release tree, a source-owned generic RV32
ELF with no Andes declaration imports as `RISCV:LE:32:AndeStar_v5:gcc`, while
RV64 resolves to `RISCV:LE:64:default:gcc`. The lane therefore states the
language per width and refuses any caller-selected processor outside it instead
of inheriting the header guess.

Refusals name the declaration rather than falling back to a guessed profile:
big-endian images (Ghidra 12.1.4 ships no big-endian RISC-V language),
non-`ET_EXEC` images, `EF_RISCV_RVE`, `EF_RISCV_TSO`, and every float ABI other
than double, which the pinned generic compiler specifications model. The raw
e_flags bits stay in Evidence as declarations, not as instruction semantics.

Family recognition now has one owner for the ELF-metadata families:
`ELF_METADATA_ARCHITECTURES`, `isElfMetadataArchitecture`, and
`HeaderArchitecture` replace ten `Exclude<BinaryArchitecture, "mips">` sites that
would each have needed a second hand-edited member. `ghidraPinnedLanguage` is
that owner for the import commitment: the session readback previously re-derived
only the MIPS pin, so a profile committing any second pinned language failed with
a language-mismatch error before the bridge started.

Hopper and IDA refuse RISC-V by name, matching the MIPS wording. Caller-supplied
raw-binary entry keeps its existing four-family list; this lane is ELF-header
admission only.

Verification on macOS 15.7.7 arm64, Node 24.18.0, Ghidra 12.1.4 PUBLIC and JDK
21.0.12.1, with the host decompiler built from the release sources for
`os/mac_arm_64`: `npm run check` passes; `npm run test:fast` passes 396 test
files; `npm run verify:test-discovery` discovers all 917 test files exactly once.
Source-owned RV32 and RV64 double-float `ET_EXEC` fixtures complete
`rea analyze --provider ghidra` and `rea decompile rea_riscv_entry`, committing
`RISCV:LE:32:default` and `RISCV:LE:64:default` with `language_selection:
provider-pinned-v1`; the soft-float RV64 shape is refused with the named
float-ABI reason, and the full refusal matrix including the RV32 and
width-mismatch shapes is covered by unit tests. No `verify:ghidra:riscv` lane
script is included in this change; the runs above were performed manually against
the installed engine. Fork pull requests receive no CI check runs, so the
real-provider scope above is the evidence available for review.

* fix(ghidra): bound the RISC-V lane by declared flags and document it

Address review feedback on #2151.

`e_flags` bits outside `EF_RISCV_RVC | EF_RISCV_FLOAT_ABI` are now refused with
the offending value named, so a reserved or newer producer declaration cannot
enter the verified generic lane by falling through the RVE, TSO and float-ABI
checks.

Admission reads `e_type` only, so the refusal text no longer calls `ET_EXEC`
static: a fixed-address declaration does not establish static linkage, and
`PT_INTERP` and dynamic dependencies are not inspected. The session records that
boundary as a limitation rather than inferring a load model.

Add `docs/ghidra-riscv.md` in the shape of the MIPS lane guide (admission
boundary, refusal table, why the language is pinned per class, Evidence fields,
and what the fixtures do not establish), and correct
`docs/firmware-analysis.md`, which still told users that MIPS and RISC-V
decompilation had not been added.

Verification after the change: `npm run check` and `npm run docs:check` pass; the
RISC-V module tests pass (17 lane cases plus 9 parser cases); on Ghidra 12.1.4
with JDK 21.0.12.1 the RV32 and RV64 double-float fixtures still analyze and
decompile with `RISCV:LE:32:default` / `RISCV:LE:64:default` and
`language_selection: provider-pinned-v1`, the soft-float fixture is still refused
by name, and the new linkage limitation appears in the emitted Evidence.

* fix(ghidra): align RISC-V pinning with admission

Reuse the complete lane admission boundary when selecting a language, retain unsigned unsupported flag bits in diagnostics, and derive float ABI metadata from the source declaration.

* test(ghidra): verify RISC-V CLI and MCP workflows

Build source-owned RV32/RV64 fixtures with Clang and LLD, establish independent ELF and symbol facts with LLVM readelf, and verify real provider evidence, instruction bytes, calls, decompilation, schema projections, snapshot replay, refusal diagnostics, source immutability and cleanup.

Verified on macOS ARM64 with Ghidra 12.1.4, JDK 21 and LLVM/LLD 23.1.2. Dynamic linkage, big-endian, Andes and RVE/TSO workflows remain unverified; ISA attributes are not inspected.

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-13 00:11:35 +08:00
akushonkamenandGitHub 10400fdb57 fix(ida): thin the selected FAT slice into the private headless input (#2201)
Universal Mach-O admission reported the selected slice architecture while the IDA provider's private input copy handed the whole FAT container to upstream idalib, which then picked its own slice (x86_64 for /bin/cat) while Evidence labeled the session arm64. FAT parsing now exports the selected slice's byte range as universalSlice, and the private input copy writes exactly those bytes: the whole source digest still anchors the copy against admission-time changes, and the thinned result anchors the post-operation identity check.

Thin targets keep the whole-file copy, and mach-o-universal remains the container format across Evidence, snapshots, and provider routing.

Validation: typecheck, oxlint (no new warnings), oxfmt, and targeted vitest runs — ida adapters 12 files/116 tests, domain and services projects 60 files/736 tests, ghidra/hopper behavior 2 files/50 tests, filesystem and universal-extraction boundary 2 files/49 tests, native signature MCP 1 file/2 tests, plus a new thinning suite (5 tests, red before the fix: 3 failed). The thinned product was verified byte-identical to the `clang -arch arm64` input of a `lipo -create` universal fixture. Controlled fixtures only; no live IDA analysis was performed for this change.

Closes #2139
2026-10-12 22:07:49 +08:00
morluto b38429cb6a fix(native): preserve universal Mach-O container identity
Keep FAT container format distinct from selected slice architecture through target admission, Evidence, snapshots, and CLI/MCP lifecycle results. Preserve native, Hopper, Ghidra, extraction, and dylib routing for the Mach-O family.

Reject selected FAT ranges overlapping their table or too short for a Mach-O header. Replace empty-slice fixtures with in-bounds executable headers and cover thin, universal, single-slice, FAT64, and swapped containers.

Validation: static and architecture checks, generated docs, targeted producer and public MCP tests. Provider routing uses controlled fixtures; no live Hopper, IDA, JADX, or Ghidra analysis was performed for this change.
2026-10-12 09:29:07 +08:00
Jeremy LevartovskyandGitHub 0d11f71e0d fix(managed): admit ReadyToRun images built for non-Windows hosts (#1985)
ReadyToRun images compiled for Linux, macOS or the BSDs store the COFF machine XORed with IMAGE_FILE_MACHINE_NATIVE_OS_OVERRIDE (dotnet/runtime src/coreclr/inc/pedecoder.h). Every framework assembly in a Linux .NET install has machine 0xFD1D, so target admission and the managed PE reader rejected them as an unsupported architecture.

Decode the PE machine once in src/domain/peMachine.ts, replacing the two duplicated switches, and accept an OS override only on images with a CLI directory.
2026-10-12 08:41:23 +08:00
Jay-Alexander ElliotGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
bddddccdee feat(native): admit bounded MIPS ELF analysis through Ghidra (#1362)
* feat(native): admit bounded MIPS ELF analysis through Ghidra

Retain MIPS ELF identity and interpretation through Evidence and snapshots.
Keep provider admission bounded to ELF32 ET_EXEC o32 MIPS32r2, without
implicitly widening Hopper, IDA or Windows P0 coverage. Add source-owned
fixtures and an optional real-provider verifier; real Ghidra verification
remains pending in this contribution environment.

Refs #718

* fix(ghidra): bound MIPS floating-point mode admission

Reject explicit EF_MIPS_FP64 and EF_MIPS_NAN2008 declarations until their
semantics have a real-provider verification lane. Add PSP PRX/Allegrex,
o64/EABI and floating-point-mode refusal regressions. Document that the
header gate is not a full .MIPS.abiflags check and advance the profile lane
identity to invalidate older interpretation bindings.

Validation: 24 standalone checks against the transpiled production profile
module passed; the two new flag cases failed before this change. Clang 17
and LLD compiled the source-owned fixture for both endian targets. The
complete Vitest/typecheck suite and real Ghidra verification remain pending.

Refs morluto/rea#718; PSP support remains follow-up morluto/rea#1330.

* fix(native): validate MIPS ABI records before Ghidra admission

Read bounded ELF32 PT_MIPS_ABIFLAGS and SHT_MIPS_ABIFLAGS records through
the already-open target handle. Preserve declared ABI fields, reject
conflicting/duplicate/truncated records, and keep missing records unknown.
Wire inspection into target resolution before provider selection. Require
compatible declarations for the deliberately narrow Ghidra lane and bind
all ABI fields into profile identity (v3) for snapshot invalidation.

Clang 17 produced eight source-owned fixtures spanning both byte orders
and FPXX/FP32/FP64/soft-float modes with identical ELF header flags. The
new reader and policy distinguish them. 24 focused regression test bodies
passed after TypeScript transpilation using node:test in place of Vitest;
strict isolated TypeScript checking of the reader and target/result types
passed. These are limited checks, not full Vitest/project typechecking or
real-Ghidra verification, which remain pending.

Advances morluto/rea#718. PSP/Allegrex remains morluto/rea#1330 follow-up.

* fix(native): repair MIPS integration boundaries and test resolver behavior

Keep ELF targets out of Hopper Mach-O preparation and scope existing PE fixture types correctly. Move ABI tests into the filesystem boundary lane, use shared cleanup, add resolver regressions, apply the pinned formatter, and register the real MIPS verifier through the existing package-command pattern. Validated with repository-pinned Node/npm, check:ci, actual focused Vitest, discovery and docs:check. Real Ghidra conformance remains pending.

* fix(test): honor caller-selected JDK in real MIPS verification

* test(native): strengthen MIPS Ghidra conformance

Cross-check the source-owned fixture with bounded GNU readelf inspection, compare production ELF/ABI interpretation and independent symbol addresses, and verify effective Ghidra language/compiler identity through existing public operations. Add a source-owned fixed-encoding probe for immediate and conditional branch checks plus global/string byte assertions.

Add 16 assertion/producer-boundary regression cases without widening production admission, changing provider contracts, or introducing proprietary dependencies. Local focused suite: 99 passed, one existing root-permission test skipped. Typecheck, lint/module boundaries, formatting and 700-file discovery pass; full hosted checks and real Ghidra verification remain pending for this checkpoint.

* fix(ghidra): preserve native-tool architecture checks after MIPS integration

The conflict-free upstream merge failed TypeScript compilation because includes on the union of MIPS-only and other architecture arrays requires never. Use equality-based membership without casts or changing accepted native hosts. Extend the existing native-decompiler rejection table with a MIPS executable at a Linux x64 tool path; retain upstream FAT and unknown-header behavior. Hosted verification follows this commit; no success is claimed yet.

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-10 23:04:27 +08:00
morluto ad59531a96 refactor: remove forwarding exports and redundant aliases 2026-10-09 22:46:09 +08:00
Kaoru0822andClaude Opus 5.5 0b04d384cc fix(native): refuse truncated executable headers at target admission
Target admission accepted a thin Mach-O after reading only its magic and
CPU type, so a 12-byte prefix opened as a valid arm64 executable and the
first provider query failed as a generic execution failure. ELF headers
were likewise accepted after 20 bytes, and a FAT file whose selected slice
lies beyond its end was admitted.

Require the complete fixed header (Mach-O 28 or 32 bytes, ELF 52 or 64),
Mach-O load commands that fit in the file, and a selected FAT slice within
the file, checking commitments against the file size rather than the
probed prefix. Each refusal names the failed constraint.

Fixes #1189

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 03:51:37 +00:00
N0zoM1z0andGitHub 5bcb6645b5 feat(ghidra): admit explicit DOS COM analysis (#702)
* feat(ghidra): admit explicit DOS COM analysis

Carry caller-selected COM interpretation through CLI, MCP and target
identity, seed real-mode entry/context before Ghidra analysis, and verify
complete loaded bytes, source mappings and measured register context.
Keep original binaries unchanged and PSP/runtime coverage explicit.

Assisted-by: Nitori (gensokyo-skills:nitori-reverse-engineering)

* fix(verification): validate COM package selection before allocating fixtures
2026-10-06 16:26:40 +08:00
Rudy Mizrahi CelekliandGitHub 0ce529af4e fix(managed): recognize PE Thumb and ARMNT machines (#605)
Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com>
2026-10-06 02:09:00 +08:00
N0zoM1z0andGitHub 4f099a10f8 feat(ghidra): add DOS MZ analysis and complete function extents (#556)
Validate DOS MZ load modules and import them with Ghidra's 16-bit
real-mode profile. Preserve complete observed inclusive function body
ranges through inventory, function analysis, Evidence and snapshots.

Add a public source-generated DOS CLI/MCP verification lane and
regressions for malformed headers, body membership and cache profiles.

Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing)
Assisted-by: Suika (gensokyo-skills:suika-scatter-gather-planning)
2026-10-05 20:09:45 +08:00
morlutoandGitHub 2160d4fae5 fix: consolidate boundary ownership and cleanup failure paths (#477) 2026-09-08 10:59:13 +08:00
morlutoandGitHub b8a9da0c0f refactor(mcp): make analysis states unrepresentable (#468)
* refactor(config): parse capability settings into exact states

* refactor(runtime): encode replay and process outcomes exactly

* refactor: make analysis states unrepresentable

* refactor(mcp): register canonical schemas with SDK

* fix: preserve capability and catalog invariants

* fix(build): remove stale compiled modules

* fix(replay): update real verifier policy dependency

* fix(package): verify the complete MCP catalog

* fix(test): supply complete replay reload policy

* refactor(contracts): centralize output schema lookup

* refactor(mcp): delegate protocol boundaries to SDK

* fix(browser): approve real source capture

* fix(mcp): preserve catalog guidance
2026-08-10 02:28:11 +08:00
morluto 6e6cce1bf3 feat(artifacts): support MSIX and AppX packages 2026-07-23 07:09:04 +00:00
N0zoM1z0andN0zoM1z0 24274fe6b6 feat(target): classify Windows PE admission metadata
Parse PE optional-header commitments before provider selection and expose provider-neutral executable role and managed-code observations. Hash targets through the same open file handle used for classification so later provider snapshots can reject path replacement by digest. Cover native applications, DLLs, managed images, malformed directory declarations, and delayed PE headers while preserving managed-provider behavior.
2026-07-17 03:05:19 +09:00
N0zoM1z0andN0zoM1z0 21da71ec6d refactor(domain): remove provider-specific target and snapshot state
Move Hopper loader derivation and compatibility projection into the Hopper adapter so BinaryTarget carries only provider-neutral identity and format metadata.

Introduce bounded canonical analysis-profile commitments, exact provider/profile snapshot v2 bindings, legacy snapshot rejection guidance, and backward-compatible profiled Evidence v2 records. Partition cache and replay behavior by target, concrete provider build, and profile while preserving custom Hopper loader overrides.

Propagate profiles consistently through CLI, MCP, composed workflows, and session lifecycle; preserve the 78-tool contract and legacy loaderArgs response projection. Add migration documentation, generated API updates, property regressions, package verification, and real Hopper coverage.
2026-07-15 20:31:56 +09:00
morluto 4ff4c11a00 Add evidence-backed investigation workflows 2026-07-12 21:12:18 +00:00
morluto 4f18e26ee5 feat: evolve REA analysis platform 2026-07-12 19:32:13 +08:00
morluto fc21c9a2d7 Simplify REA onboarding and app targeting 2026-07-12 12:26:16 +08:00
morluto 4dec415fc8 Remove dead exports found by knip
- Un-export HOMEBREW_COMMANDS, systemSetupHost, ToolKind, BinaryArchitecture,
  HopperDiagnostic -- only consumed within their own files
- Remove unused HopperUnavailableError class (never instantiated)
- Reformat knip.json entry/project arrays to multi-line
2026-07-12 10:20:45 +08:00
morluto 66b8d6ea90 fix(targets): cancel startup and probe PE offsets 2026-07-12 09:05:16 +08:00
morluto 15e8cf3234 fix(boundaries): reject unsafe local inputs 2026-07-12 08:59:18 +08:00
morluto a67054f815 fix(setup): preserve consent and startup target kind 2026-07-12 08:51:35 +08:00
morluto 62ede515b7 docs: record runtime and release constraints 2026-07-12 08:45:35 +08:00
morluto 2b7ef8610d fix(hopper): make loader selection non-interactive 2026-07-12 08:33:33 +08:00
morluto f32e718cc7 feat(session): add dynamic binary lifecycle 2026-07-12 06:34:18 +08:00