chore: sync main for native IDA verification

The native IDA follow-up is being verified on Windows IDA Pro 9.3. IDA Free is not used in this follow-up.
This commit is contained in:
N0zoM1z0
2026-10-13 00:47:56 +08:00
85 changed files with 4560 additions and 478 deletions
+5
View File
@@ -148,6 +148,11 @@ existing analysis families directly, for example
- Cancellation during a package pull preserves caller-selected output. The
error includes partial observations identifying completed artifacts and the
interrupted local path; remaining APKs were not attempted.
- Bugreports collect into isolated staging and publish with exclusive creation;
filename collisions select the next numeric suffix. If publication fails, the
completed archive remains in staging. The error's partial observation gives
its recoverable `local_path`, the original `adb_reported_path`, and an unknown
digest status. Later operations and shutdown preserve this retained evidence.
- Failed package acquisition rollback retains its directory identity and retries
cleanup before later work and during shutdown. Cleanup refuses a replacement
directory. Terminal shutdown stops admission and drains active operations.
+4 -2
View File
@@ -123,8 +123,10 @@ Select a verified native child for the existing tools:
```
Then use `analyze_function`, `inspect_native_api` or the other native primitives.
Native architecture and executable validation still apply. This milestone does
not add MIPS/PPC/RISC-V decompilation, raw flash load profiles, bootloader/device
Native architecture and executable validation still apply. Generic
[MIPS ELF](ghidra-mips.md) and [RISC-V ELF](ghidra-riscv.md) lanes are separate
documents with their own admission boundaries. This milestone does
not add PPC decompilation, raw flash load profiles, bootloader/device
tree semantics, vulnerability scanning, emulation or hardware runtime capture.
## Resource and lifecycle policy
+105
View File
@@ -0,0 +1,105 @@
# RISC-V ELF analysis through Ghidra
This lane extends native ELF analysis for `EM_RISCV` targets. It does not
execute RISC-V code, emulate a boot sequence, or rehost firmware. It addresses
[#2063](https://github.com/morluto/rea/issues/2063) and follows the same
lane shape as [generic MIPS support](ghidra-mips.md).
## Admission boundary
ELF `EM_RISCV` (243) is retained as the provider-neutral `riscv` family. The
target also retains the raw ELF declarations that decide a lane: class, byte
order, `e_type`, and `e_flags`. Recognition does not mean every provider or
RISC-V variant is supported. Existing PE, managed and Mach-O architecture sets
are unchanged, and caller-supplied headerless raw-binary entry still accepts only
its existing four families.
The Ghidra lane admits little-endian ELF32 and ELF64 `ET_EXEC` images that
declare `EF_RISCV_FLOAT_ABI_DOUBLE` in `e_flags`, with no `EF_RISCV_RVE`,
`EF_RISCV_TSO`, or any bit outside `EF_RISCV_RVC | EF_RISCV_FLOAT_ABI`. These are
header declarations, not proof that every instruction conforms to that ISA.
Provider selection returns the specific unsupported constraint rather than
guessing a compatible profile:
| Declaration | Outcome |
| ----------------------------------------------- | --------------------------------------------------------------------------- |
| Little-endian ELF64 `ET_EXEC`, double-float ABI | admitted, `RISCV:LE:64:default` + `gcc` |
| Little-endian ELF32 `ET_EXEC`, double-float ABI | admitted, `RISCV:LE:32:default` + `gcc` |
| Big-endian | refused; Ghidra 12.1.4 ships no big-endian RISC-V language |
| `ET_REL`, `ET_DYN`, other `e_type` | refused with the declared value |
| `EF_RISCV_RVE` | refused; the embedded 15-register ABI needs its own processor specification |
| `EF_RISCV_TSO` | refused; the total-store-order memory model is unverified here |
| Soft, single, or quad float ABI | refused; the pinned generic compiler specification models double-float |
| Reserved or newer `e_flags` bits | refused with the offending bits named |
`e_type` is the only load-shape declaration this lane reads. A fixed-address
`ET_EXEC` declaration does **not** establish static linkage: `PT_INTERP` and
dynamic dependencies are not inspected, and the session records that boundary as
a limitation rather than inferring a load model.
## Why the language is pinned
Ghidra's header-based selection is not reliable for RV32. The 12.1.4
`riscv.opinion` pins `variant="default"` for the 64-bit constraint and leaves the
32-bit constraint unpinned with the comment that the loader "must choose between
'default' and 'AndeStar' variants". Measured against the installed release tree:
| Fixture | Ghidra auto-selection |
| ----------------------------------------------- | ----------------------------- |
| RV64 ELF, `e_flags 0x5` (RVC, double-float ABI) | `RISCV:LE:64:default:gcc` |
| RV32 ELF, `e_flags 0x5`, no Andes declaration | `RISCV:LE:32:AndeStar_v5:gcc` |
So an ordinary RV32 image lands on an Andes processor model unless the language
is stated. REA therefore pins `RISCV:LE:32:default` or `RISCV:LE:64:default`
with the `gcc` compiler specification per class, records
`language_selection: provider-pinned-v1`, and refuses a caller-selected
`REA_GHIDRA_LANGUAGE_ID` or `REA_GHIDRA_COMPILER_SPEC_ID` that disagrees with the
pin without changing the requested override. An override that matches the pin is
accepted and recorded as `configured-v1`.
## Evidence
The lane identity is `elf32-exec-le-double-float-generic-v1` or
`elf64-exec-le-double-float-generic-v1`, and `riscv_elf` retains `elf_class`,
`byte_order`, `type`, and the raw `flags` alongside the read-only booleans
`float_abi_double`, `rvc_declared`, `rve_declared`, and `tso_declared`. Those
fields report declarations the header makes; none of them is evidence of the
original compiler, of instruction conformance, or of a floating-point calling
convention beyond the ABI the pinned language models.
## Verification and limits
Ghidra 12.1.4 PUBLIC with a 64-bit JDK 21+ is required, and on macOS the release
tree ships no host decompiler binary, so `Decompiler/os/<platform>/decompile` and
`sleigh` must come from your own build; see
[installation](installation.md#ghidra).
Run `npm run verify:ghidra:riscv` with an existing Ghidra 12.1.4 installation,
JDK 21+, and a RISC-V-capable Clang, LLD and LLVM readelf. The verifier acquires
no tools and never executes target code. Select installed tools explicitly when
they are not on `PATH`:
```sh
GHIDRA_INSTALL_DIR=/path/to/ghidra_12.1.4_PUBLIC \
JAVA_HOME=/path/to/jdk-21 \
REA_RISCV_CLANG=/path/to/llvm/bin/clang \
REA_RISCV_LLD=/path/to/lld/bin/ld.lld \
REA_RISCV_READELF=/path/to/llvm/bin/llvm-readelf \
npm run verify:ghidra:riscv
```
The lane builds the committed freestanding C and assembly sources into RV32 and
RV64 `ET_EXEC` fixtures. Independent LLVM readelf observations establish ELF
declarations and symbol addresses. Public CLI and stdio MCP workflows verify
function enumeration, decompilation, pinned language/compiler identity, decoded
probe bytes, global data, and direct calls. It validates advertised MCP output
schemas and both JSON representations, saves and reopens snapshots, rejects
header-mutated soft-float and unsupported high-bit declarations, and checks
unchanged source bytes and owned cleanup. Failures retain the diagnostic
workspace; successful runs remove it and report the observed process lineage.
Module tests cover admission and malformed headers. Call-graph completeness,
string inventories, dynamic linkage, Andes or RVE/TSO images, and big-endian
targets are **not** established by this lane. `.riscv.attributes` ISA extensions
are not inspected; the generic language pin remains an analysis convention.
+2
View File
@@ -1,3 +1,5 @@
<div id="native-ui-and-dispatch-investigation"></div>
# Native binary investigation
Use a native provider to discover functions, inspect instructions and pseudocode,
+1 -1
View File
@@ -36,7 +36,7 @@ check project-owned conformance fixtures and fail-closed report behavior.
| ------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| `discover-classify` | target-free startup, artifact inventory, compatible workflow selection, and exact visible limits |
| `diagnose-environment` | broken runtime separated from analysis failure, version skew detected, and revision changed after recovery |
| `declare-request` | exact target, operation, and lifecycle are carried by the request before launch |
| `acquire-authority` | missing scope reported, narrow grant resumed, expansion reprompted, and denial launched nothing |
| `static-analysis` | native and JavaScript routing plus preserved ambiguity, truncation, unsupported syntax, and residual unknowns |
| `reactive-scenarios` | predicate-driven interaction, descendant settlement, Electron correlation, and cancellation cleanup Evidence |
| `compare-authority-candidate` | explicit partial-order/finite-trace semantics, primary divergence localization, and unknown for incomplete comparison |
+27 -9
View File
@@ -19,11 +19,12 @@ behavior, and complete CI evidence before merging.
| `npm run test:boundary` | Boundary, CLI boundary, MCP boundary, process boundary, and process-global projects |
| `npm run test:mcp` | MCP boundary project |
| `npm run test:acceptance` | Complete compiled CLI and MCP acceptance workflows |
| `npm run test:python` | LLDB producer retention and launch contracts with a fake LLDB API; no native debugger required |
| `npm run test:watch` | Dirty source tests in watch mode, without build |
| `npm run test:watch:all` | Changed tests from every project; builds at startup, so rebuild after production edits before relying on compiled tests |
| `npm run check:changed` | Cached typecheck/lint and branch-related source feedback |
| `npm run check:pr` | Opt-in complete local deterministic gate and generated-file checks |
| `npm run docs:check` | Generated-document validation from current source and build outputs |
| `npm run docs:check` | Generated-document validation and fresh onboarding example consistency checks |
For example:
@@ -97,6 +98,12 @@ runner. Preserve simultaneous launches where concurrency is itself under test.
Build/documentation writers hold checkout-local output locks. Tests do not
hold a broad command lock; `check:pr` finishes tests before document validation.
Turbo caches deterministic preparation and static checks, but `test:run` always
executes. Repository files do not fully describe installed tool contents or host
state. A previous test pass therefore cannot replace current validation, even
when the source tree is unchanged. `check:pr` also runs the fast Python producer
suite; CI runs it in the static job for code changes. These producer tests do not
establish real LLDB engine coverage.
Vitest's persistent compile cache remains disabled. The `cliTest` fixture shares
a temporary Node compile cache among its fresh CLI subprocesses. The worker
@@ -247,17 +254,25 @@ Coverage does not replace named boundary or real-provider scenarios.
CI runs six native Vitest shards without retries. Each shard emits a blob
report; the merge job produces aggregate coverage plus JUnit and JSON timing
reports, uploads them together, and writes the slowest files to the workflow
summary. Static checks, documentation, build, package verification, and
reports even when a shard fails, uploads available diagnostics, and writes the
slowest files to the workflow summary. Partial reports are marked diagnostic
only. Failed, cancelled, or missing selected shards still fail the required gate,
and aggregate coverage thresholds remain enforced by the merge command.
Only successful complete merges publish the small `test-timings` artifact, with
checkout-relative paths; the build job downloads that artifact for the sequencer
instead of the coverage/JUnit bundle. Missing history falls back to native
sharding. Static checks, documentation, build, package verification, and
real-system lanes remain separate jobs so one kind of evidence cannot stand in
for another.
The build job saves portable Turbo outputs across runs. Eight Linux real-system
lanes restore this cache without waiting for the build job. The shared action
derives its key from `test:prepare --dry=json` task hashes, runner OS and
architecture, and `.npmrc`; test-only changes can reuse compiled outputs. Every
architecture, `.npmrc`, `.nvmrc`, and the lockfile; test-only changes can reuse compiled outputs. Every
build and verification command still runs, so Turbo checks task inputs and
rebuilds misses. There is no prefix fallback to accumulate older build outputs.
rebuilds misses. A restore prefix with the same platform and toolchain permits
partial task reuse when one generator's inputs change; Turbo still invalidates
that generator's entry. Native outputs are not stored in this cache.
Native package jobs, the Windows curated lane, and the Apple artifact lane reuse
the same workflow's Turbo cache for portable compiled JavaScript, skills, and
@@ -266,9 +281,10 @@ commands, which restore matching outputs or rebuild on a cache miss. Native
artifacts, dependency installs, package archives, and runtime verification remain
host-local. Keep platform-dependent outputs out of this shared cache.
The PR acceptance target is a median `npm run check:pr` wall time below three
minutes across three warm-build runs on the benchmark host. Keep Vitest caches
cold unless separately identified. A PR that touches packaging or real-system
Record cold and warm preparation separately from fresh test execution when
benchmarking the optional `npm run check:pr` gate. Compare multiple runs on the
same host, including subprocess and queue costs; a cached pass is not a test
runtime measurement. A PR that touches packaging or real-system
behavior requires the applicable `verify:*` lanes; packaging and installation
changes also require `npm run verify:package`. The full-gate benchmark measures
that explicit lane, not the routine iteration requirement.
@@ -292,6 +308,7 @@ prove behavior on the verified Ghidra 12.1.4 and JDK 21 build.
| `npm run verify:ghidra` | Linux x64/arm64 ELF or macOS x64/arm64 Mach-O | Host C compiler, Ghidra 12.1.4, and full JDK 21 |
| `npm run verify:ghidra:seeds` | POSIX; raw x86 fixture; seed ordering, BSS labels, analyzer readback and unknown-name rejection | Ghidra 12.1.4 and full JDK 21 |
| `npm run verify:ghidra:swift` | macOS x64/arm64 Mach-O | Host Swift compiler, Ghidra 12.1.4, and full JDK 21 |
| `npm run verify:ghidra:riscv` | Linux/macOS x64/arm64 host; source-owned RV32/RV64 ELF through CLI and MCP | RISC-V Clang, LLD, LLVM readelf, Ghidra 12.1.4, and full JDK 21 |
| `npm run verify:ghidra:switch` | Linux x64 ELF; GCC/Clang optimized and stripped switch fixtures | GCC, Clang, GNU nm/objdump/strip, Ghidra 12.1.4, and full JDK 21 |
| `npm run verify:ghidra:aarch64-jump-table` | Any supported Ghidra host; AArch64 ELF; signed/unsigned byte/halfword tables; host ARM64 Mach-O | Clang with AArch64 target support, Ghidra 12.1.4, and full JDK 21 |
| `npm run verify:ghidra:cross-format` | Any supported Ghidra host; also analyzes AArch64 ELF, x86-64 PE, and x86-64 Mach-O | `clang`, LLD, and `lld-link` in addition to host-lane prerequisites |
@@ -684,7 +701,8 @@ not make native host acceptance unavailable.
Pull requests that change only root `README*.md` files, `docs/`, `AGENTS.md`,
`CONTRIBUTING.md`, `CHANGELOG.md`, `SECURITY.md`, `native/windows/README.md`,
or the GitHub issue and pull-request templates run formatting and
generated-document validation without the source-test shards or native package
generated-document validation, including the small onboarding example consistency
suite, without the source-test shards or native package
lanes. Classification compares the PR head with its merge base, so later
base-branch changes do not expand that scope. Static and coverage aggregate jobs
remain present and fail if classification fails. Every code change retains the
+6 -4
View File
@@ -48,10 +48,12 @@ satisfy. A plan without execution does not establish runtime behavior.
- Use strict inputs with explicit target identity and required fields. Preserve
meaningful target, action, capture, and output choices; remove ignored options
and repeated permission declarations.
- Return complete evidence inline, with artifact/provider identity, source
locations, Evidence references, and relevant confidence and limitations.
Missing coverage is unknown, not absence. Avoid extra lookups solely to
understand the answer or its provenance.
- Return the requested facts inline with artifact/provider identity, source
locations, Evidence references, coverage, and relevant confidence and
limitations. Use supported selection, summaries or views when they answer
the question; preserve access to complete source Evidence and honor existing
contracts and requested detail. Missing coverage is unknown, not absence.
Avoid extra lookups solely to understand the answer or its provenance.
- Keep observed, derived, and inferred relationships distinct. Preserve
unresolved edges and partial provider facts instead of requiring every
provider to supply the same metadata.