diff --git a/AGENTS.md b/AGENTS.md index b0e98f54d..aec33ebf1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -90,6 +90,8 @@ See [docs/tool-design.md](docs/tool-design.md) for the design checklist. When us ## Testing Guidelines +Prioritize real end-to-end workflows, boundary integration, and golden producer data. Keep a focused module test when it covers a distinct failure or semantic case absent from those workflows; test paths do not establish depth. See [docs/testing.md](docs/testing.md) for pruning and classification rules. + Name tests `*.test.ts`. Use Vitest and production seams (`tests/fixtures/`) rather than module mocks. Domain tests assert pure behavior; adapter tests use fake launcher/socket seams; MCP tests connect with the client SDK version pinned in `package.json`. Preserve the canonical tool inventory defined by `TOOL_CONTRACTS` and verified through `CATALOG_IDENTITY` and generated product metadata. Cover malformed input, cancellation, lifecycle cleanup, and actual format, protocol, host-permission, and target-identity boundaries. Do not add tests that merely freeze arbitrary caps or prescribed call sequences. Real Hopper, Ghidra, browser, managed conformance, and any real managed-tool claims cannot be replaced by mocks; use the corresponding `verify:*` command. Keep tool catalogs complete and self-describing; prefer capability- and session-scoped availability over schema truncation. Serialized bytes alone do not measure agent usability or model context cost. diff --git a/docs/testing.md b/docs/testing.md index 8221305c3..c6a571010 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -7,8 +7,10 @@ semantic cases that these workflows cannot reliably reproduce. A test's path or suite name does not establish its behavioral depth. Avoid tests that mirror getters, enum mappings, fixture helpers, or prescribed -call sequences. Prefer one representative workflow over a Cartesian matrix -when every row crosses the same boundary. Keep distinct command handlers, +call sequences, or assert cache object identity without a caller-visible contract. +Prefer representative workflows over a Cartesian matrix when rows cross the +same boundary. Retain combinations that exercise a distinct interaction, such as +filtering within a full-output envelope. Keep distinct command handlers, producer representations, failure reasons, and lifecycle states covered. Consolidate their shared setup without hiding the inputs or expected evidence. @@ -21,6 +23,19 @@ tests consume a production module, trace CLI/MCP and verifier imports, including imports of compiled files, and remove abandoned scaffolding with its tests when it has no runtime consumer. +When a test expects a `Result` to succeed, throw its error before asserting the +returned value. An `.ok` assertion immediately before the same throwing guard +adds no coverage and hides the typed failure. A removed parser matrix can leave +one golden malformed-input case and a real filesystem lifecycle that proves +configure, update, readback, and removal; an add-only journey does not cover the +whole lifecycle. + +Keep in-memory MCP sessions in the MCP boundary lane and direct filesystem +workflows in the filesystem boundary lane. Reserve acceptance paths for journeys +through the compiled public CLI or MCP runtime. Calling a compiled private +workflow or injecting a provider still establishes its narrower integration +claim, even if a directory name calls it acceptance. + Measure slow files before removing capacity regressions. Optimize repeated process startup, fixture construction, and production algorithms when they dominate runtime. Keep at least one input beyond each formerly failing size or