docs: organize canonical documentation layout (URGENTE)

Group all existing authored guides by domain and ownership, expose complete navigation, and preserve their public routes. Update physical path consumers and release configuration while retaining historical records. Document domain-based specs and ADR slugs without global numbering.

Co-Authored-By: Codex <noreply@openai.com>
This commit is contained in:
Matheus Breguêz
2026-10-12 18:44:55 -03:00
co-authored by Codex
parent 3ee6ff43c1
commit ea504fac97
119 changed files with 1646 additions and 902 deletions
+488
View File
@@ -0,0 +1,488 @@
# Client and agent verification
Use these optional lanes to verify an installed client or evaluate an agent's
REA workflow. [Testing REA](testing.md) provides the normal development commands
and provider lanes. Each record below states its host, client version, model
fixture, and remaining coverage; a native-client run with a loopback model does
not establish live model-service acceptance.
## End-to-end, integration and golden evidence
The optional `verify:qwen-client`, `verify:pi-client`, and `verify:hermes-client`
lanes require an installed native client. Select its executable with
`REA_VERIFY_QWEN_COMMAND`, `REA_VERIFY_PI_COMMAND`, or `REA_VERIFY_HERMES_COMMAND`.
They configure disposable profiles through the public REA CLI, preserve caller
settings and backups, and exercise real stdio MCP calls and skill loading. The
`call` mode checks complete catalog discovery and actual JavaScript Evidence;
pass `-- chat` to check plain chat with REA enabled. Qwen also reads the full
result that its client offloads, and Pi exercises default codemode execution.
Use `REA_VERIFY_RUNTIME_ROOT` to select a production-only installed REA package.
Receipts retain client versions, result digests, host coverage and owned-process
cleanup. These POSIX lanes use a deterministic loopback model, so they do not
prove live model-provider or native Windows compatibility. Qwen, Pi and Hermes
bind both `HOME` and `USERPROFILE` to the disposable account. Qwen and Pi discover
the shared personal skill through their default search paths without an explicit
skill setting; Hermes discovers it through the selected `HERMES_HOME`.
Set `REA_VERIFY_HERMES_STICKY_PROFILE=1` to exercise native Hermes selection of
a named sticky profile with `HERMES_HOME` still pointing to its root.
`verify:gemini-client` requires an installed Gemini CLI (verified with
`@google/gemini-cli@0.63.0` on POSIX and native Windows); select it with
`REA_VERIFY_GEMINI_COMMAND`. A selected `.js`, `.mjs` or `.cjs` entry point runs
under the verifier's Node. On Windows, select the npm install's
`node_modules/@google/gemini-cli/bundle/gemini.js`, because only a `gemini.cmd`
shim reaches `PATH`.
The optional lane uses native `GEMINI_CLI_HOME` discovery in an isolated
Git project, checks setup plans, backups and idempotence, activates the installed
personal skill, validates all forwarded input JSON Schemas against their declared
dialect, forwards the complete REA catalog and checks full JavaScript
Evidence for a Unicode path. Use `-- chat` for ordinary chat with REA enabled,
or `REA_VERIFY_RUNTIME_ROOT` for a production-only installed REA package.
The local fixture exercises the real Gemini API adapter, including its
`untrusted_context` tool-result envelope. Token counts are synthetic; live
Google API compatibility remains unverified. The lane disables
the client's memory-based relaunch to preserve the caller's Node heap budget.
`verify:opencode-client` requires an installed OpenCode (verified with
`opencode-ai@1.18.35`); select it with `REA_VERIFY_OPENCODE_COMMAND`.
The optional POSIX lane configures isolated XDG roots and `OPENCODE_CONFIG_DIR`,
checks setup plans, backups, preserved JSONC comments and idempotence, activates
the installed skill, validates all forwarded input schemas and verifies complete
JavaScript Evidence for a Unicode path. JSONC is the default fixture; set
`REA_VERIFY_OPENCODE_CONFIG_FORMAT=json` for JSON. Use `-- chat` for ordinary
chat or `REA_VERIFY_RUNTIME_ROOT` for a production-only installed package.
The native core runs with external plugins disabled through `OPENCODE_PURE`;
the local OpenAI-compatible fixture has a caller-declared one-million-token
context and synthetic usage. Skills use an explicit isolated `skills.paths`
directory, so default OS-home discovery remains unverified. Live model APIs and
native Windows are also unverified.
`verify:claude-client` requires installed Claude Code (verified with
`@anthropic-ai/claude-code@2.1.296` on POSIX and native Windows); select it
with `REA_VERIFY_CLAUDE_COMMAND`. On Windows, select
`claude.exe`: npm installs put only a `claude.cmd` shim on `PATH`, and the real
executable is under `node_modules/@anthropic-ai/claude-code/bin`.
The optional lane uses native `CLAUDE_CONFIG_DIR` discovery, an independent
Git workspace, user settings and default built-in tools. It preserves caller
preferences and backups, checks idempotent setup, waits through the native
`WaitForMcpServers` tool when discovery is pending, and validates the complete
catalog and forwarded input schemas. Call mode loads the personal skill through
native `Skill`, checks its full body and verifies named-schema JavaScript Evidence
for a Unicode path. Use `-- chat` for ordinary chat with REA enabled, or
`REA_VERIFY_RUNTIME_ROOT` for a production-only installed package. The local
Anthropic Messages/SSE fixture preserves native resource-hint envelopes in its
request artifacts. Usage is synthetic; live Anthropic API and bare-mode
personal-skill activation remain unverified by this lane.
`verify:copilot-client` requires installed GitHub Copilot CLI (verified with
`@github/copilot@1.0.95`); select it with `REA_VERIFY_COPILOT_COMMAND`.
The optional POSIX lane uses native `COPILOT_HOME` discovery, an independent
Git workspace, guarded setup plans, preserved registrations, backups and
idempotence. Native `skill add` registers the isolated installed skill; call
mode loads its full body, validates all forwarded input schemas and checks
named-schema JavaScript Evidence for a Unicode path. When Copilot spills a
large MCP result, the model requests native `view` with `forceReadLargeFiles`
and validates the complete returned Evidence rather than its preview.
Use `-- chat` for ordinary
chat or `REA_VERIFY_RUNTIME_ROOT` for a production-only installed package.
The native offline BYOK adapter uses a loopback OpenAI completions/SSE fixture
and `gpt-5.4` model metadata; model inference and token usage are synthetic.
Set `REA_VERIFY_COPILOT_MODEL` to examine another client model ID and
`REA_VERIFY_COPILOT_WIRE_MODEL` for its backend model name (defaults to the client
ID). Every actual HTTP request's model is checked and recorded separately.
Both `HOME` and `USERPROFILE` point to the disposable account; native `skill add`
still selects the installed skill directory explicitly.
The same native client passed full-profile chat and analysis with the actual
wire model `gpt-4.1` under a distinct BYOK model ID:
```bash
REA_VERIFY_COPILOT_MODEL=rea-gpt-4.1 REA_VERIFY_COPILOT_WIRE_MODEL=gpt-4.1 npm run verify:copilot-client -- chat
REA_VERIFY_COPILOT_MODEL=rea-gpt-4.1 REA_VERIFY_COPILOT_WIRE_MODEL=gpt-4.1 npm run verify:copilot-client -- call
```
These runs retain full canonical input/output schemas, all tools, installed-skill
loading and complete Evidence readback. They use the fixture's requested
one-million-token BYOK prompt setting; effective capacity and live API acceptance
remain unknown. With the built-in `gpt-4.1` model ID, the verified client still
blocks the full input-schema profile before HTTP with
`compaction_static_context_blocked`, even when requesting a larger BYOK prompt
capacity. The effective capacity is unknown. The default verifier preserves
the complete catalog and full schemas. To verify the existing compact profile:
```bash
REA_VERIFY_COPILOT_MODEL=gpt-4.1 REA_VERIFY_COPILOT_SCHEMA_PROFILE=compact npm run verify:copilot-client -- chat
REA_VERIFY_COPILOT_MODEL=gpt-4.1 REA_VERIFY_COPILOT_SCHEMA_PROFILE=compact npm run verify:copilot-client -- call
```
Both native workflows passed with the complete tool inventory. The verifier selects the
profile only in its disposable Copilot registration and records it in the
receipt. Compact advertisements can reduce nested validation structure; the
server still validates complete canonical inputs. This is a tested alternate
workflow, not a fix for the full-profile client/model limit tracked in #1554.
Live model APIs, native Windows and default OS-home skill discovery remain
unverified.
`verify:grok-client` requires installed Grok Build (verified with the official
Linux x64 1.0.50 binary); select it with `REA_VERIFY_GROK_COMMAND`. This optional
POSIX lane isolates `GROK_HOME`, an independent Git workspace and additional
skill roots, disables foreign configuration discovery, guards setup writes,
preserves unrelated registrations/backups and checks idempotence. Call mode
loads the full skill through native `read_file`, queries all REA names and input
schemas through native `search_tool`, and calls analysis through `use_tool`.
If discovery reports `partial`, the native agent diagnoses REA registration and
retries discovery. No startup-timeout override or fixed readiness delay is used.
A larger Unicode-path fixture exercises native result offloading: the full
retained Evidence is validated against its named schema, then native terminal
queries recover the selected export facts, subject and full artifact digest in
the next model request. This verifies useful artifact recovery without claiming
that every offloaded graph fact enters model context. Native line-number and
truncation envelopes remain in the request artifacts. Use `-- chat` for ordinary
chat, or `REA_VERIFY_RUNTIME_ROOT` for a production-only installed package.
The loopback OpenAI completions/SSE custom model declares a one-million-token
context and synthetic usage. Live xAI APIs, native Windows and default OS-home
skill discovery remain unverified.
`verify:commandcode-client` requires installed Command Code (verified with npm
1.79.2); select it with `REA_VERIFY_COMMANDCODE_COMMAND`. Auth, MCP tokens and
sessions use the actual OS home, so provision a disposable POSIX account rather
than overriding `HOME`. Set `REA_VERIFY_COMMANDCODE_ACCOUNT_HOME` to that account's
actual home and create `.rea-client-verification` there with exactly
`Disposable REA client verification account` followed by a newline. The lane
owns that account's `.commandcode` configuration and shared REA skill. Build the
runtime as the checkout owner, then run `node scripts/verify-commandcode-client.mjs`
as the disposable account; the npm shortcut also needs a writable checkout.
The verifier does not create accounts or install clients. It guards setup
targets, preserves an unrelated registration and backups, checks idempotence,
and verifies native default shared-skill discovery and complete activation.
Default deferred-schema delivery stays enabled: the native prompt advertises
every REA tool, exact-name `search_tools` returns every input schema, and a native
shell query recovers the saved catalog's count and digest when the client spills
it. Actual analysis of a Unicode-path fixture delivers full named-schema Evidence
to the next model request. Use `-- chat` for ordinary chat or
`REA_VERIFY_RUNTIME_ROOT` for a production-only installed package. The loopback
OpenAI completions/SSE model is a keyless BYOK endpoint declaring a million-token
context with synthetic usage. A synthetic account-key value satisfies the
client's print-mode gate; native local-only mode refuses hosted API calls.
Updates, telemetry and cron are disabled. Hosted authentication, live models
and Windows remain unverified.
`verify:omp-client` requires installed OMP (verified with the official Linux x64
18.8.7 binary); select it with `REA_VERIFY_OMP_COMMAND`. This optional POSIX lane
isolates the default agent, global config and XDG roots, guards setup targets,
preserves unrelated registrations and backups, and checks idempotence. It
compares native rejection of an invalid profile with REA's refusal to plan
fallback writes. OMP's default `xd://` interface mounts the complete REA catalog
as devices: call mode reads the complete installed skill and tool documentation,
then dispatches analysis through native `write`. It checks full named-schema
JavaScript Evidence with a Unicode path in the next model request. Device
metadata and documentation are distinct from forwarding all JSON schemas as
model functions. Native print-mode MCP readiness uses its defaults.
Use `-- chat` for ordinary chat or `REA_VERIFY_RUNTIME_ROOT` for a production-only
installed package. The loopback OpenAI completions/SSE model fixture declares a
one-million-token context and synthetic usage. Live model APIs, native Windows,
named-profile native execution and default OS-home skill discovery are
unverified; skill discovery uses an explicit isolated custom directory.
`verify:deepseek-client` requires an installed DeepSeek Harness (`dsh`;
verified with `@deepseek-ai/dsh@0.2.0-rc.2`) and Git. Set
`REA_VERIFY_DEEPSEEK_COMMAND` to select its executable. This optional POSIX lane
seeds caller model settings and a stale Cordis registration in an isolated
`DSH_HOME`, then runs public `setup --client dsh` dry-run, setup, repeated setup,
and scoped doctor. It checks the backup and unrelated patches before booting
Harness with the generated registration. After native execution it uninstalls
through the public CLI and checks that only caller patches remain. `HOME` and
`USERPROFILE` point to the disposable account. The native default
`~/.agents/skills` search path discovers
the installed skill without a `DSH_AGENTS_HOME` override. A separate Git root prevents inherited project skills from
masking a missing installation. The default `call` mode checks skill loading,
complete catalog discovery, the actual forwarded input schemas and regexes,
and full JavaScript Evidence for a Unicode path. Use `-- chat` for ordinary
chat with REA enabled, or `REA_VERIFY_RUNTIME_ROOT` for an installed package.
The fixture uses Harness's custom OpenAI adapter and the one-million-token
context capacity declared by its default DeepSeek Flash model; it does not
establish live DeepSeek API or native Windows coverage. Receipts include
request artifacts, result digests and owned-process lineage.
The DSH acceptance journey also exercises public setup, approval, doctor,
generated-launcher MCP startup and uninstall without an installed Harness.
That journey proves REA workflow behavior, not native Cordis expression
evaluation. Filesystem regressions retain tagged values and unrelated rows
across alias dependencies and check refusal of unsafe YAML without mutation.
Full E2E tests invoke the production command dispatcher and real providers,
without fake launchers, runners or responses. `verify:keyed-archive` writes
Foundation binary, XML, and integer archives using one run-owned Swift module
cache. It checks CLI and separate stdio MCP parity, pagination, escaping-path
rejection, an XML graph golden, and numeric meaning for safe controls, unsafe
integers, boxed numbers, and integral-valued reals. `verify:asset-catalog`
compiles source-owned colors with
`actool`, invokes real `assetutil`, then checks CLI/MCP results, exact catalog
digest, every raw metadata field, pagination and malformed input rejection.
Neither artifact workflow requires Hopper or launches it. Both run in macOS CI.
`verify:macos-bundle` needs only macOS with Command Line Tools. It compiles a
source-owned app with `clang`: a versioned framework, XPC services, an app
extension, a login item, a privileged helper, launchd plists, and a helper tool,
signed ad hoc. It packs the app as a directory, a `ditto` ZIP, and an APFS DMG,
then checks that `inspect-artifact` plus `project-apple-application-graph`
report the same bundle anatomy for all three through the CLI, with stdio MCP
parity. It also checks that the DMG is detached afterwards. The same app
covers `trace-dylib-resolution`:
- each resolution status and finding, with CLI/MCP parity;
- for every traced image, dependencies, rpaths, and install names equal to
`otool -l`;
- for the main executable and an XPC service, a predicted load order equal to
the images dyld actually loads under `DYLD_PRINT_LIBRARIES`.
The lane also compiles executable/library pairs with empty embedded directory,
versioned-path, and suffix settings. It checks present and removed dependencies
through CLI and MCP, and compares actual `DYLD_PRINT_SEARCHING` diagnostics for
root-level candidates. These cases distinguish an empty search directory from
an empty versioned scan or a suffix that only repeats the original path.
It runs in macOS CI.
Prepare the native inspector before cases that measure producer output or exit
behavior; keep startup deadlines and cancellation in distinct cases. Run real
process-capture verification separately from package or build checks. On macOS,
new npm processes can become token-unreadable after changing their display
title and prevent verified cleanup during a capture's ownership sweep.
MCP SDK transport tests with recording providers remain integration tests.
They are useful for schema drift and failure projection but do not prove that
Hopper, Ghidra or another substituted engine works. `verify:package` proves
packaging/install behavior and fake-provider integration; use the corresponding
real-provider lanes for engine claims. Packed-bridge checks verify shipped files and Python syntax.
Real Apple dispatch and Interface
Builder verifiers currently prove format integration through production readers.
`verify:hopper` exercises an installed Hopper through the production stdio MCP
server and CLI. It checks source-owned call chains, CFG edges, references and
complete large inventories, then probes unnamed bookmarks, annotation clearing,
batch validation before mutation, malformed addresses and regexes, segment-end
partial reads, and synthetic file-offset rejection. Advertised schemas are
validated in their JSON Schema dialect and successful replies are checked against
their advertised output schemas. Navigation checks cover interior-object cursor
readback and mapped-memory boundaries. Annotation probes verify invalid native
text and duplicate destinations/names before mutation, preserve unselected label
owners, and exercise explicit batch label swaps. Function locals retain observed
names and stack displacements. Graph probes check symbol/interior-address parity
and a source-owned recursive cycle; literal tracing preserves complete queries
and whitespace. Disposable binary copies prove that switching and closing
actually removes the native document, and that CLI byte, function-dossier,
literal-trace results and invalid-address diagnostics agree with MCP.
No provider is mocked in this lane.
Real search probes cover Unicode names, literal metacharacters, case and regex
modes, annotation cache invalidation, complete native fragments of long literals
checked against byte reads, escaped UTF-8/control text, byte-preserving Latin-1
decoding, and Hopper's UTF-16 symbol-name truncation boundary. It verifies native
CallReference classifications across reference and dossier results, retains long
string fragment metadata in dossiers, and exercises pathological regex deadline
and cancellation followed by successful requests in the same native session. When the macOS
Objective-C fixture is present, the lane also verifies native UTF-16 string objects
and their inferred encodings against their actual bytes. Native terminal calls
are checked across reference, instruction, assembly, block-range and procedure-length
projections; block endpoints are normalized using actual native membership.
Exact Objective-C names also exercise named CLI selectors for function, instruction,
decompilation, reference and search operations. A literal `--help` trace query proves
that selector data is preserved independently of global CLI flags.
Unmapped annotation destinations and
oversized later batch names fail before any earlier edit is applied.
`verify:hopper:deadlines` checks the native client's optional request deadlines
on macOS with an owned source-built fixture. Zero and elapsed deadlines must
leave native comments unchanged, including when a synchronous progress observer
delays timer dispatch. It also observes a short analysis deadline and verifies
subsequent wire recovery and clean shutdown. Caller timeout does not interrupt
Hopper's synchronous native operation. Socket boundary tests deterministically
cover active and queued expiry, late replies and timer cleanup.
The Linux demo lane remains a separate `verify:hopper:linux` command. Its
lifecycle checks attempt competing CLI launches for the active target and a
different target, requiring rejection before either can change the owning
MCP session's documents or procedures. Lease boundary tests also preserve
unresponsive or malformed live endpoints while allowing confirmed stale
sockets to recover.
`verify:hopper:fat` is a separate macOS lane requiring installed Hopper and the
existing Xcode clang/lipo toolchain. It compiles arm64/x86-64 thin executables and
one- and two-slice FAT32 containers, verifies exact/interior address mappings
against bytes in the original files, checks CLI/MCP parity, and checks owned
runtime cleanup. Source byte changes, removal, permission denial (for non-root
callers), and nonregular replacement must retain native partial mapping facts,
reject unverified original-file coordinates,
and recover after restoration. Single-slice FAT cases also relocate the slice
without changing its loaded bytes. `verify:hopper:fat64` additionally checks
FAT64 preparation through Hopper's native Mach-O loader, source-container
mappings, profile identity,
malformed and ambiguous slice rejection, and temporary-image lifetime. Both
lanes have been verified on Hopper 6.1.0-demo; this establishes REA's prepared
FAT64 workflow, not native FAT64-loader support.
Cross-architecture fixture compilation is not required by `verify:hopper`.
Golden tests use immutable captured text inputs with producer/source provenance
under `tests/fixtures/golden/`. Expected results are reviewed for the semantic
claim; capture commands do not automatically approve new expected outputs.
Do not call handcrafted utility output or synthetic binary builders real-data
goldens. Keep unsupported binary layouts and malformed boundaries as targeted
regressions until a real fixture establishes equivalent coverage.
`verify:browser` exercises source-map failure isolation and expanded-output limits through the compiled CLI and stdio MCP with real Chrome. It also submits five 2 MiB source-map annotations, checks retained script identities and explicit map omissions, and closes its owned fixture target. Replacement and document-reset behavior has separate producer-boundary coverage.
The raw Chrome fixture launchers use `--password-store=basic` so Linux startup
does not wait for an unlock prompt from a reachable but locked desktop keyring.
Such a stall leaves the initial page unloaded with empty or placeholder CDP URLs;
it does not establish permission to capture that page. See
[the keyring diagnosis](https://github.com/morluto/rea/issues/1384#issuecomment-6093781686).
`verify:browser` also captures a source-owned noise canvas as a real PNG above
8 MiB through the CLI and stdio MCP, with complete byte/digest parity and real PNG
decoding. Its SDK client explicitly permits the larger inline JSON response;
this lane does not establish large image-comparison request transport coverage.
The scenario checks read locale, timezone, device scale and viewport dimensions
from actual page JavaScript through CLI and stdio MCP DOM captures. They also
verify that attached-page emulation is released after success, initialization
failure, action failure and cancellation while the external target stays open.
`verify:browser:network` is a focused real-browser lane for transaction identity,
selected request/response bytes, binary and compressed responses, duplicate
headers, credential and declared-secret redaction, redirects, streaming cutoff,
CLI/MCP parity, and owned-profile cleanup. Set `REA_BROWSER_EXECUTABLE` to an
installed Chrome-family browser. An optional script argument selects an already
installed package's `scripts/rea.mjs` entry point for packaged-artifact checks.
The complete `verify:browser` lane includes these same checks.
After building, `verify:browser:dom` checks empty and HTML-whitespace form
destinations against a native Chrome DOM-property oracle through CLI and stdio
MCP. It requires `REA_BROWSER_EXECUTABLE` and accepts an optional installed REA
entrypoint. The full browser lane includes the same public-adapter assertions
before other fixtures navigate the selected page.
`verify:browser:scripts` checks active script capture → exact-byte export →
existing static JavaScript analysis through CLI and stdio MCP, including
manifest readback, competing query variants, and resolved relative imports.
It uses an installed browser and accepts an optional installed REA entrypoint.
The complete `verify:browser` lane also exercises passive script export through
both public adapters. See [website script export](../guides/web/website-script-export.md).
`verify:browser:modules` compares CLI and stdio MCP traces against an independent
real Chromium module-loading fixture: import-map scopes, package prefixes,
null/backtracking rejection, query/fragment identities, repeated module instances,
lazy and computed unknowns, exact source/map identities and no implicit refetch.
Set `REA_BROWSER_EXECUTABLE`; the optional script argument selects an installed
package entrypoint. The complete `verify:browser` lane and existing conditional
Chrome CI include this verifier. See [module relationships](../guides/web/website-module-trace.md).
## Agent evaluation and conformance records
Evaluate native, JavaScript, managed and browser investigation tasks through a
real local Codex CLI with:
```bash
npm run verify:agent
```
The lane runs scenarios sequentially in standalone Codex (`--no-daemon`) with a
disposable HOME and CODEX_HOME under `~/.cache/rea-agent-evaluations/`. Set
`REA_AGENT_EVAL_ROOT` to choose another parent directory outside the OS temporary
directory: current Codex refuses helper aliases under `/tmp`, preventing shell
commands and skill reads even if MCP works. Each owned run directory is removed
after verification unless fixture retention is requested. The lane
installs the packaged skill and MCP registration through public `rea setup`,
rather than copying a skill into a project that also inherits the caller's
skills and configuration. Only authentication is copied, when present, and it
is removed even with `REA_AGENT_EVAL_KEEP_FIXTURES=true`. No user configuration,
rules, plugins or history is copied; shell snapshots and additional agents are
disabled. Existing API-key environment authentication remains available.
The native missing-provider scenario preapproves only `open_binary` and
`close_binary` in the disposable client so it reaches REA's actual provider
boundary instead of stopping at Codex approval. This does not validate a real
native engine or authorize arbitrary runtime execution.
Select a smaller run with `REA_AGENT_EVAL_SCENARIOS`, a comma-separated list of
scenario IDs. `asar-zh` exercises the packaged desktop rubric from a Chinese
request. `javascript-module-view` exercises summary analysis, module paging and
item inspection in a tree with unrelated modules; its routing and text checks
remain heuristic, so review the retained transcript for selector choice and
coverage claims. Set `REA_CODEX_CLI` to an installed Codex executable and
`REA_AGENT_EVAL_MODEL` to an account-supported model. Use an external resource
limit for the whole process tree when testing on a constrained host; a Node
heap limit alone does not bound client and child-process memory.
`missing-target` leaves desktop fixtures visible without selecting one in the
request. It requires zero REA calls and a structured request for an app name or
artifact path. Its `expectedFirstTool` is `null`, and
`targetClarificationPassed` grades that limited routing outcome independently
of analysis prose heuristics. Analyzing a nearby example cannot pass this case.
The packaged desktop and parser-comparison scenarios now assess a closed set of
known fixture claims. Each final answer must be a strict JSON manifest containing
exactly the requested claim IDs, values, Evidence IDs, and Evidence authority and
confidence. Golden values are not included in the prompt. The evaluator checks
exact values against both the fixture oracle and authenticated successful REA
results; an Evidence ID by itself is insufficient. Missing, duplicate, extra,
contradictory, incorrectly sourced, or unsupported claims fail the scenario.
The desktop rubric covers the exposed bridge API and members, renderer and main
IPC operations, and the resolved preload. It binds its Evidence to the exact
packaged artifact path and SHA-256. The parser rubric checks the precise heading
depth addition, discriminant, complete comparison counts, and the explicit limit
on runtime semantics. Its comparison must link to the two delivered source
analyses and use the requested module and export selectors. Boundary tests package
the same source-owned desktop fixture and compare the same source-owned parser
files through actual MCP tool results, then verify that fabricated answers fail.
Per-scenario `factualCorrectness` is `passed`, `failed`, or `not_assessed` within
the `configured_fixture_claims` scope. A pass establishes only the configured
claims, not unrestricted factual correctness. Native, managed, browser,
navigation-context, and address-context scenarios have no factual rubric and
remain `not_assessed`; the managed workflow requires both artifact and member
inspection to answer its type and entry-point question.
The closed factual scenarios request complete producer Evidence on the initial
analysis so their source selectors can authenticate all configured facts.
Summary/page/item workflows are exercised separately by `javascript-module-view`;
its heuristic gate does not establish full-producer factual correctness.
All scenarios retain routing, workflow, validation, repetition, process-exit,
and token-use gates. Configured factual scenarios additionally require a factual
pass. Their answer-text heuristics are diagnostic and do not affect the gate.
Validation counts include complete text-only REA `invalid_request` envelopes
and failed SDK calls whose arguments violate the named current REA contract.
Client approval rejections and provider unavailability remain distinct failures;
truncated JSON previews are not parsed into invented error codes.
Repeated `binary_session` observations separated by a successful `open_binary`
or `close_binary` are treated as lifecycle verification. An unchanged retry,
including one after a failed lifecycle call, still counts as repetition.
Scenarios without a rubric retain the legacy text-heuristic gate.
`answerTermCoverageMet` checks case-insensitive substrings,
`epistemicCuePresent` checks keywords, and `finalCitesEvidence` checks only an ID's
presence. These metrics can still accept fabricated prose and must not be read
as factual assessments. Set `REA_AGENT_EVAL_TRANSCRIPT_DIR` to retain complete
tool results and final answers for review.
Report schema version 4 adds negative routing scenarios with nullable
`expectedFirstTool` and a `targetClarificationPassed` outcome. Consumers must
allow an expected absence of REA calls; this does not make an analysis scenario
pass without its required tools. Version 3 changed the top-level
`factualCorrectness` from a constant
string to an assessment summary with status, scope, and assessed/passed/failed/
not-assessed scenario counts. Scenario records include the factual assessment and
configured claim IDs. The evaluation scope is
`routing_workflow_and_configured_fixture_claims`. Update report consumers for
these changes. The narrower heuristic field names introduced in version 2 remain:
`answerHeuristicsMet`, `epistemicCuePresent`, `answerTermCoverageMet`, and
`requiredAnswerTermGroups`.
Regenerate the managed conformance manifest and Evidence completion ledger from
live verification results, or check them for drift:
```bash
npm run evidence:generate
npm run evidence:check
```
The records preserve unsupported and unverified coverage as explicit unknowns.
Run the matching real-tool prerequisites described in this guide.
+81
View File
@@ -0,0 +1,81 @@
# Documentation ownership and authoring
Use the [documentation directory](../README.md) to find existing material before
adding another guide. Preserve one canonical owner for each contract or workflow;
link to that owner instead of copying its claims into several guides.
## Choose a destination
| Document | Destination |
| -------------------------------------------------------- | -------------------------------------- |
| Installation, client setup and first CLI/MCP steps | `docs/getting-started/` |
| Analyst workflows and evidence interpretation | `docs/guides/<domain>/` |
| Analysis-engine configuration and format prerequisites | `docs/providers/` |
| Shared runtime contracts and evidence semantics | `docs/reference/` |
| Development, testing, release and authoring instructions | `docs/development/` |
| Composition diagrams | `docs/architecture/` |
| Compatibility changes for a released version | `docs/migrations/` |
| One feature specification, plan and tasks | `docs/specs/<domain>/<feature-slug>/` |
| One architectural decision | `docs/adr/<domain>/<decision-slug>.md` |
Use meaningful, lowercase domain and feature slugs. Global numeric prefixes are
unnecessary: different domains can add features independently, and branches do
not compete for the next sequence number. Check the existing domain directory
and linked issues before creating a feature with the same slug.
Guides describe existing behavior and limitations. Specs define one proposed
feature or domain change; an existing guide does not become a specification by
moving it into `specs/`.
## Feature lifecycle
Create the specification before implementation. Each feature directory contains:
- `spec.md`: problem, desired behavior, requirements, acceptance and non-goals.
- `plan.md`: the single implementation plan, contracts and architectural choices.
- `tasks.md`: incremental, testable steps linked to requirements and verification.
Split independent domains or separately reviewable features before writing their
specs. A broad `ROADMAP.md`, when useful, only lists child specifications, order
and dependencies; requirements and implementation tasks belong to each child.
Cross-repository contracts have one producer-owned specification. Consumers link
to that contract instead of maintaining competing copies.
Record an isolated architectural decision separately in
`docs/adr/<domain>/<decision-slug>.md`, with **Status**, **Context**,
**Decision** and **Consequences**. Link the ADR from affected specifications.
Do not create decision records for routine file moves or invent past decisions.
Specs do not replace the contribution-scope decision required for a new provider
or expanded execution authority. Follow
[contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope).
## Links and public routes
Use relative links to physical Markdown files so GitHub readers can follow them.
Link repository files outside `docs/` with their full GitHub URLs. Preserve
existing heading anchors when moving a guide.
[`documentation.ts`](https://github.com/morluto/rea/blob/main/docs/.vitepress/documentation.ts)
owns categorized site navigation and the existing guides' stable public routes.
Update it when adding or moving a guide. VitePress rewrites categorized sources
to the existing public paths; its Markdown hook resolves repository links from
the physical source and renders them against those public routes. Keep basenames
and route assignments unique. Edit links use the physical source file.
For a move, search for both the basename and its old full path. Update README
translations, authored skills, verification instructions, documentation-fact
readers and release configuration when they consume that path. Preserve
historical release records and references bound to earlier commits or tags.
## Generated ownership and checks
The [documentation directory](../README.md#machine-owned-documentation) identifies
the machine-owned outputs and their producers. Their existing paths are retained
because scripts, package consumers and release history depend on them. Change
canonical sources and regenerate; do not fix a generated projection by hand.
Run `npm run docs:check` for documentation facts and generated consistency, and
`npm run docs:build` for the complete production site and dead-link validation.
When a physical path moves, also run the tests that read it. Follow
[development checks](testing.md#developer-commands) before requesting review.
@@ -0,0 +1,93 @@
# Reconstruction readiness conformance
Run the source-owned native, JavaScript CLI, Electron, Evidence, comparison,
obligation, replay, and cleanup conformance verifier from a clean checkout:
```sh
npm ci
npm run verify:readiness
```
The command builds the redistributable native conformance fixtures, inventories
the native executable through the public CLI, analyzes the generated JavaScript
CLI and Electron fixtures through both CLI and MCP, evaluates an internal
nine-stage conformance report, and proves stale/tampered replay rejection. The
report evaluator is part of the repository verifier, not a public REA tool. It
prints one compact JSON summary.
Set `REA_READINESS_REPORT_PATH` to retain the complete machine-readable report:
```sh
REA_READINESS_REPORT_PATH=readiness-report.json npm run verify:readiness
```
CI runs this command on Linux, publishes the compact summary, and retains the
full report as the `reconstruction-readiness` artifact.
## Project conformance contract
The verifier runs as `npm run verify:readiness`; it is not exposed as a public
CLI command or MCP tool. The synthetic report input and evaluator exist to
check project-owned conformance fixtures and fail-closed report behavior.
## Required stages
| Stage | Required proof |
| ------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| `discover-classify` | target-free startup, artifact inventory, compatible workflow selection, and exact visible limits |
| `diagnose-environment` | broken runtime separated from analysis failure, version skew detected, and revision changed after recovery |
| `acquire-authority` | missing scope reported, narrow grant resumed, expansion reprompted, and denial launched nothing |
| `static-analysis` | native and JavaScript routing plus preserved ambiguity, truncation, unsupported syntax, and residual unknowns |
| `reactive-scenarios` | predicate-driven interaction, descendant settlement, Electron correlation, and cancellation cleanup Evidence |
| `compare-authority-candidate` | explicit partial-order/finite-trace semantics, primary divergence localization, and unknown for incomplete comparison |
| `preserve-contradictions` | record-and-continue, affected equivalence blocked, and unaffected comparison retained |
| `verify-reconstruction-closure` | incomplete reconstruction detected, incremental verifier closure, and authority-delegating candidates rejected |
| `export-replay` | CLI/MCP parity, process/structured failure parity, deterministic replay, tamper rejection, and stale-input rejection |
Every stage and required check must occur exactly once and link to Evidence in
the embedded Evidence bundle. `fail`, `unknown`, `unsupported`, `truncated`,
and `skipped` never aggregate to `pass`.
## Fail-closed rules
The evaluator rejects or withholds readiness when:
- a bounded capability omits its effective numeric limits;
- a fixture lacks exactly one compatible selected workflow;
- an incompatible or unavailable provider was invoked;
- a typed CLI failure exits zero, or an MCP failure reports success;
- concurrent behavior uses total-order comparison;
- truncation, instability, unavailable authority, or contradiction is labeled
equivalent;
- a deliberate primary divergence is not localized;
- contradiction Evidence or its affected comparison is missing;
- the reconstruction obligation ledger is not `ready`, has no required
obligation, or retains an unowned/unverified required obligation;
- a process exceeds its declared target/lifecycle, cancellation loses diagnostics, or owned
resources remain;
- replay is nondeterministic, tampering/staleness is not detected, or the
expected source digest differs.
Each finding names its stage, stable reason code, Evidence IDs, and detail.
Stage `capability_issue` and `next_action` fields map unsupported or failing
work to the responsible capability and remediation without claiming that the
requested analysis ran.
## Fixtures
The verifier uses only source-owned, redistributable fixtures:
- `tests/conformance` native executables with stable symbols and deliberate
version divergence;
- `tests/conformance/readiness/javascript-cli`, which exposes prompts,
alternate-screen behavior, configuration precedence, child process,
persistence, loopback HTTP, and signal teardown to static reconstruction;
- `tests/conformance/readiness/electron`, which exposes main, preload,
renderer, utility, IPC, renderer restart, blocked shell effect, and deep-link
surfaces;
- the shared incomplete reconstruction and broken-runtime observations in
`RECONSTRUCTION_READINESS_EXAMPLE`.
The synthetic example is a public typed input, not a global equivalence claim.
It demonstrates that every deliberate fault remains represented in the final
report and that only comparable Evidence can close the associated obligation.
+259
View File
@@ -0,0 +1,259 @@
# Releasing REA
Pushes to main automatically open or refresh a Release Please PR with the next
version and changelog. Review its migration notes and wait for its current CI
checks to pass. Merging that PR into main starts publication automatically.
The Release workflow validates the merged release metadata and ancestry before
creating a tag. npm and MCP Registry publication use the exact reviewed merge
SHA returned by Release Please. Later main commits cannot change that source.
Ordinary main pushes and closed, unmerged PRs cannot publish packages. Check
both publication jobs and the published-package canary; a merged PR or GitHub
tag alone does not establish that publication finished. Retry failed jobs in
the original run after a partial publication.
For a release that needs an independently frozen application checkpoint, use
the optional manual path below. It retains explicit prepare and publish phases.
## Contributor-aware release notes
Use [the rea-changelog-update skill](https://github.com/morluto/rea/blob/main/.agents/skills/rea-changelog-update/SKILL.md)
to turn the bot draft into grouped Highlights, Changes, and Fixes with inline
`Thanks @...` credit. `npm run release:notes -- inventory` collects the full
selected Git range and paginated GitHub PR/author/closing-issue metadata;
`record --summary` produces a compact provenance link for public notes, while
`record` produces the full record for review. Credit a contribution once even
when it appears in highlights, details and migrations; omit maintainer
`@morluto` self-thanks. Keep complete authorship metadata in the inventory
instead of duplicating a credit ledger in the release body. `check` validates credits
and optional PR-body equality, and `render` writes the verified release section.
Run a subcommand without its required options to see its usage. GitHub reads
require an authenticated `gh`; record/check/render work offline from the saved
inventory and complete local Git history.
Save a finalized inventory on the release branch at
`docs/releases/VERSION.contributions.json`. Review unknown identities and
reverted/backported work explicitly; GitHub metadata does not establish the
quality of a user-facing claim. Preserve the existing breaking-change section
and checkpoint checks. The contribution inventory is a reviewed snapshot, not
a signed attestation; the offline checker verifies local range coverage and
credit consistency, not GitHub's authorship independently.
Before editing the live PR, add `rea:release-notes-finalized` to prevent ordinary
main-push proposals from overwriting the editorial notes. Wait for any already
running proposal to finish. The guard applies to main-push automation; do not
run manual checkpoint preparation over finalized notes. Remove the label to
resume bot regeneration. The label does not freeze the base branch: new base
commits require re-inventorying and re-reviewing the notes.
Update both the release branch's `CHANGELOG.md` and the PR body, preserving
Release Please's surrounding metadata. Before creating a release, the workflow
checks an inventory when present, requires its target to equal the reviewed
merge's first parent, and compares the freshly fetched PR body to the changelog.
It rejects changes outside the release metadata paths and stale credit records
before any tag or publication. Review the diffs within allowed metadata paths
as well, especially package dependencies and scripts. Existing candidates without inventories retain their current
checks. Merging still starts publication; a notes edit does not authorize a
merge or release by itself.
The checker supports inline Release Please PR bodies up to GitHub's 65,536
character limit, including the surrounding header/footer. Overflow-link PR
bodies need a separate integration and fail explicitly; credits are never
silently truncated.
## 1. Select a source for a manual checkpoint
Use the next minor version proposed by Release Please, including releases with
breaking changes. Record the full source SHA and create `release/VERSION` at
that commit. The selected version and Release Please's proposed version must
agree before publication.
When choosing the manual path, leave the automatic main release PR unmerged.
Select the application commit from main, create the frozen branch, and prepare
its own release PR using the steps below. Merging the main release PR starts
automatic publication instead. Close the superseded main PR after the manual
release has published and its metadata has been synchronized back to main.
`release/VERSION` also sets the expected version for the workflow. Use an exact
SemVer such as `release/6.1.0` or `release/6.1.0-rc.1`, without build metadata.
The checkpoint validator rejects a candidate whose package, lockfile, manifest,
registry metadata or changelog disagrees with that version before creating a tag.
### Compatibility and version selection
For the 6.2 candidate, review the [6.1 to 6.2 migration guide](../migrations/migration-6.2.md)
alongside the changelog before publication.
REA uses Release Please's `always-bump-minor` strategy: every release increments
minor and resets patch, including releases with breaking changes. For example,
the next release after 6.0.0 is 6.1.0. Version numbers use the SemVer format,
but a minor increment does not promise backward compatibility. Review the
changelog's breaking-change section and migration notes before upgrading.
The supported public surface includes documented CLI commands/options, MCP tool
names and input/result contracts, saved evidence formats and supported runtime
requirements. Before adding `!` or a `BREAKING CHANGE` footer, a PR must identify
a previously valid call or configuration that will fail or change meaning,
explain why compatibility cannot be preserved, and give its migration. Prefer
optional additions, compatibility adapters and a documented deprecation period.
Keep breaking markers so the bot includes migration notes without incrementing
major. Maintainers review this impact before merging; the bot parses markers
and cannot infer compatibility.
For example, adding an optional inspection tool is a minor change. Rejecting a
relative MCP path previously accepted by the public contract is breaking;
rejecting an input that the existing contract already prohibited is a fix.
Changing the version number alone does not restore compatibility.
For example, after selecting a reviewed commit for 5.0.0:
```bash
git fetch origin main
git branch release/5.0.0 SOURCE_SHA
git push origin release/5.0.0
```
Replace `SOURCE_SHA` with the selected full commit SHA. The checkpoint must
include this release workflow and CI support for release branches. For an
older checkpoint, backport only the release infrastructure first and record
that additional commit. Do not merge later implementation changes into the
release branch or rebase the candidate onto a moving main.
## 2. Prepare the bot PR
Run the workflow definition from main and select the frozen source separately:
```bash
gh workflow run release.yml --ref main \
-f release_branch=release/5.0.0 -f phase=prepare
```
Release Please targets that branch and prepares its version, changelog, and
registry metadata. The workflow uses an optional `RELEASE_PLEASE_TOKEN`
repository secret and otherwise uses the built-in `GITHUB_TOKEN`. A dedicated
GitHub App or personal access token can start PR CI automatically. With the
built-in token, `opened`, `synchronize`, and `reopened` PR events create runs
that require a maintainer with write access to approve them. See
[GitHub's workflow trigger documentation](https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/trigger-a-workflow).
The generation step normalizes the product catalog. Preparation cannot create
a GitHub release or publish a package. Record the final bot PR head after
normalization, then approve that head's blocked runs from the PR page. Updates
to the bot branch can create new approval-required runs; approval of an older
head does not verify the normalized candidate. Wait for the final head's CI.
The validator reads the actual ancestry range from the published baseline tag
to the selected checkpoint, including visible first-parent Conventional Commits
and the PR titles in GitHub's default merge messages. The report preserves the
original merge subject alongside the extracted Conventional Commit title.
This matters after merging a side-branch release back into a newer main:
Release Please's chronological history cutoff can omit unreleased mainline
commits. Under `always-bump-minor`, unreleased breaking markers allow a minor
increment and still require references in the new release's breaking-change
migration section. Checkpoints using the default strategy retain the major
increment requirement. Other omitted entries are reported for review.
Historical notes from an older release cannot satisfy
the new release's migration check. This audits declared markers and references;
it does not prove API compatibility or the quality of a migration explanation.
If preparation reports a mismatch or omitted breaking changes, inspect the
recorded ancestry report, correct the release PR's version artifacts and notes,
regenerate documentation, and review/test its final head. Do not repeatedly
prepare over manual corrections. Publication independently checks the merged
candidate again before Release Please can create a tag.
Review the candidate's version, notes, generated metadata, and package
contents. Wait for the candidate's CI and relevant real-provider checks.
Routine local iterations need focused checks; CI owns full deterministic
coverage and platform lanes. If an artifact or real-provider check is
unavailable, report that limit before deciding to publish.
Merge the reviewed PR into `release/5.0.0` with its head SHA matched. Further
main commits do not change this candidate. A necessary release fix belongs
on the release branch, must be reviewed and tested, and establishes a new
recorded checkpoint.
## 3. Publish the reviewed merge
After the release PR has merged:
```bash
gh workflow run release.yml --ref release/5.0.0 \
-f release_branch=release/5.0.0 -f phase=publish
```
Publication creates the release from the merged bot PR without preparing or
updating another PR. Both npm and MCP Registry jobs check out Release Please's
exact release SHA. They do not build the current main tip or a mutable branch.
Stable versions publish to npm's `latest` tag; prerelease versions publish to
the `next` tag so they cannot replace the stable install by default.
The publish dispatch runs from the frozen release branch so npm's provenance
records the actual release commit. Before Release Please creates a tag, the
workflow accepts only `prepare` or `publish` and requires the selected branch
tip to equal the dispatch SHA. Registry jobs run only during `publish`. After
a tag exists, a mismatch between that tag's SHA and the dispatch SHA stops
both registry publishes.
See [npm's provenance implementation](https://github.com/npm/cli/blob/v11.16.0/workspaces/libnpmpublish/lib/provenance.js)
for the use of GitHub's workflow ref and commit SHA.
The workflow builds the bundled Windows controls and verifies the packaged
artifact before npm publication. It then verifies the published CLI, the
capability-scoped MCP catalog, and the isolated package update path before
publishing MCP Registry metadata.
Record these outcomes separately:
- GitHub release and tag, including the resolved commit SHA.
- npm's exact version and integrity, with the published-package canary passed.
- MCP Registry's exact server version and matching npm package version.
A GitHub tag alone does not establish npm or MCP Registry publication.
## 4. Sync metadata back to main
Manual checkpoint synchronization is part of completing that release. Finish it
before cutting the next checkpoint; otherwise main retains the previous
release baseline and can propose an already-published version again.
After publication, open a PR from the release branch back to main. Preserve
main's later implementation changes and regenerate ignored build outputs from
the combined contracts with the released package version.
Review and test this synchronization PR, then use a merge commit so the release
tag remains in main's ancestry. Keep the released tag unchanged.
The synchronization must include `.release-please-manifest.json`,
`package.json`, both root versions in `package-lock.json`, `CHANGELOG.md`,
`server.json`, and the versioned documentation examples. Run
`npm run docs:generate`, `npm run docs:check`, and the release configuration
tests against the combined tree. After the merge, verify that the released
tag is an ancestor of main:
```bash
git fetch origin main --tags
git merge-base --is-ancestor rea-agents-5.0.0 origin/main
```
Close the superseded main proposal before merging the synchronization PR.
That merge's main push opens a proposal for the following release using the
updated baseline. Future releases can use the automatic main PR or repeat the
manual checkpoint procedure; automatic proposals do not move frozen branches.
Preparation generates and validates the catalog, portable conformance
projections, and packaged skill from the candidate checkout without committing
them. The checkpoint validator binds tracked version authority to the source
SHA; generated-catalog and package checks validate the derived representation.
No generated-metadata workflow pushes follow-up commits onto feature or release
branches.
## Partial publication and retries
Inspect the failed job and public registry state before retrying. When npm is
already published, the npm job verifies that exact version instead of
publishing it again. Re-run failed jobs in the original publication run so its
release SHA and outputs stay fixed. If only MCP publication failed, retry that
job after checking that the npm canary succeeded.
A branch tip that moved after dispatch fails before Release Please creates a
tag. Do not dispatch a fresh publish phase to repair an already-created
release: Release Please will not create the same release again. Do not move
the tag, delete the release, or unpublish npm as a retry. A defective public
package requires a reviewed correction and a new version.
+35
View File
@@ -0,0 +1,35 @@
# REA roadmap
The [platform tracker](https://github.com/morluto/rea/issues/32),
[process/Hopper tracker](https://github.com/morluto/rea/issues/48), and
[browser tracker](https://github.com/morluto/rea/issues/39) track capability and
verification work. Available tools and setup are documented in the
[investigation guides](https://rea.tools/guides/),
[installation guide](../getting-started/installation.md), and
[generated catalog](../reference/mcp-contracts.md#generated-catalog).
## Priorities
- Expand native architecture, type, and indirect-call verification.
- Connect static findings and runtime observations across application layers.
- Improve obfuscated managed-code comparisons and verified native links.
- Extend process, protocol, filesystem, reconnect, and build-comparison coverage.
- Evaluate additional providers and targets when an analyst workflow demonstrates
a capability gap.
## Admission criteria
New capabilities must return useful evidence through shared CLI/MCP workflows,
preserve observations and unknowns, and declare actual authority and lifecycle
constraints. Agents compose experiments using ordinary commands, scripts, and
fixture servers; custom orchestration needs a demonstrated requirement.
Provider and platform claims require their corresponding
[real verification lanes](testing.md#real-toolchain-verification-lanes).
See [tool design](tool-design.md) for choosing a tool boundary and
[Ghidra semantics](../providers/ghidra-provider.md) for the existing deep-provider contract.
Setup remains additive, reuses existing tools, and requires approval for disclosed
configuration and Hopper installation changes. Additional installer choices need
an implemented toolchain and explicit scope; setup does not install or upgrade
unrelated runtimes or dependencies.
File diff suppressed because it is too large Load Diff
+94
View File
@@ -0,0 +1,94 @@
# MCP tool design
## Decide whether to add a tool
Design around the analyst question. Inspect the current canonical contracts,
provider capabilities, and nearest existing tool before changing the surface.
Extend a tool when intent and result contract match; add one for a distinct
outcome or materially different authority. Public names and semantics remain
provider-neutral, with exact coverage reported per provider.
Keep the catalog complete; report capability/session availability instead of
truncating schemas. Serialized bytes alone do not establish agent usability.
## Choose the tool shape
| Shape | Use it for | Result |
| --------------------- | ------------------------------------------- | ------------------------------------------------------------------------- |
| `inspect` | Facts about one target, object, or resource | Relevant fields, locations, and facet availability |
| `search` / `list` | Candidate discovery | Stable ordering and context; pagination when the query or format needs it |
| `trace` | Relationships across evidence | Typed edges, supporting facts, and unresolved paths |
| `compare` | Explicitly paired artifacts or observations | Identity, comparable coverage, and supported deltas |
| `workflow` | A recurring outcome needing composition | Inline answer, contributing evidence, and partial facets |
| `observe` / `capture` | Runtime behavior | Authority, launch/attach effects, lifecycle, and cleanup status |
These are task shapes, not mandatory prefixes. Name tools for the action and
object callers reason about. Avoid unrelated discovery/execution/mutation modes
inside one tool.
## Prefer primitives; compose workflows
A primitive reports a reusable fact about an identified object or relationship.
A format-specific decoder qualifies; one application's business interpretation
belongs outside the general contract. Expose provider APIs when they add a
useful capability that existing tools cannot express.
Compose a workflow when repeated use demonstrates a multi-source outcome REA
can join without hiding caller choices or uncertainty. Batch homogeneous reads
when that removes repeated work while preserving per-item outcomes. Keep prompts
optional and avoid prescribing call sequences that a direct tool can replace.
Agents can compose experiments with ordinary commands, scripts, and local
fixture servers. Custom orchestration languages, replay engines, or separate
prepare/execute plans need an observed requirement those primitives cannot
satisfy. A plan without execution does not establish runtime behavior.
## Define the contract
- Use strict inputs with explicit target identity and required fields. Preserve
meaningful target, action, capture, and output choices; remove ignored options
and repeated permission declarations.
- Return the requested facts inline with artifact/provider identity, source
locations, Evidence references, coverage, and relevant confidence and
limitations. Use supported selection, summaries or views when they answer
the question; preserve access to complete source Evidence and honor existing
contracts and requested detail. Missing coverage is unknown, not absence.
Avoid extra lookups solely to understand the answer or its provenance.
- Keep observed, derived, and inferred relationships distinct. Preserve
unresolved edges and partial provider facts instead of requiring every
provider to supply the same metadata.
- For caller/callee reads, `relationship_coverage` describes only the
provider-observed direct relationship set. `complete` means that set was
returned without a known read limit; it does not claim every possible runtime
edge. Provider and execution caveats stay in `limitations` and do not imply
incomplete relationship coverage. Missing coverage remains `unknown`.
- Declare process, filesystem, network, UI, and mutation effects truthfully;
enforce the requested target and lifecycle.
- Add limits only for real format, protocol, authority, or measured resource
constraints. Report pagination, truncation, cancellation, and partial failure.
An assumed agent budget is insufficient justification for a limit.
## Implement and evaluate
Put provider protocols/parsing in adapters and shared CLI/MCP workflows in the
application layer. Share Evidence provenance, unknown projection, and session
retention semantics; matching payloads alone do not establish parity.
Measure the resource before choosing a budget and enforce it before the costly
step. Account for value alternatives, trace-frame products, retained records,
serialization expansion, and temporary-file growth separately. Derive facet
completeness from examined records and exhaustiveness, rather than diagnostic
prose or an empty failure list. Preserve observations independently of cleanup
success.
Update canonical contracts, output schemas, examples, generated artifacts, and
relevant docs together. Verify converted advertised schemas and success,
malformed, unavailable, cancellation, and partial outcomes through affected
consumers. Real-engine claims require the matching provider lane; see
[testing.md](testing.md).
When discoverability changes, evaluate representative broad, direct, and negative
tasks on the intended host. Record host/model/catalog, selected tools, arguments,
errors, retries, and task outcomes. Schema validity and keyword heuristics do not
establish successful tool selection. See [mcp-contracts.md](../reference/mcp-contracts.md) for
shipped runtime behavior.