Merge origin/main for artifact admission and evidence fixes

This commit is contained in:
64579525
2026-10-13 14:32:52 +08:00
188 changed files with 8882 additions and 937 deletions
@@ -83,6 +83,12 @@ explicit path or endpoint and do not need it.
frames; use `match_offset` / `match_limit` for later pages, or explicit null
limit for complete matching output. Preserve the source digest across pages.
This does not perform live capture.
For address/DNS/SNI/protocol summaries use `inspect_pcap_capture` with MCP
detail summary, then retained kind pcap pages. Requires Wireshark 4.x tshark
and capinfos, with Linux prlimit or Windows Job Objects; see
[packet summaries](https://github.com/morluto/rea/blob/main/docs/pcap-analysis.md).
Select an exact DNS question for address pages; page exhaustion does not prove
network completeness. This reads recordings and does not acquire traffic.
- Recorded logic/signal traces: `decode_signal_recording` with explicit decoder,
channel and protocol settings. On Linux with util-linux prlimit, supply sigrok-cli 0.7.2 through
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/signal-recordings.md).
+1
View File
@@ -515,6 +515,7 @@ jobs:
tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts
tests/boundary/filesystem/setupUnavailableProvider.test.ts
tests/boundary/filesystem/windowsNpxRegistration.test.ts
tests/acceptance/setup/homeDirectoryFallback.test.ts
tests/boundary/filesystem/webScriptManifestPaths.test.ts
tests/boundary/process/ProcessEvidence.host.test.ts
tests/acceptance/webModuleCli.test.ts
@@ -7,6 +7,13 @@ on:
- "tests/boundary/cli/captureCleanup.test.ts"
- "tests/boundary/mcp/packetCaptureMcp.test.ts"
- "src/captures/**"
- "src/pcap/**"
- "src/domain/pcap/**"
- "src/domain/analysisView/pcapCaptureView.ts"
- "src/application/pcap/**"
- "src/contracts/pcap/**"
- "src/server/registerPcapTools.ts"
- "tests/boundary/pcap/**"
- "src/domain/captures/**"
- "src/application/captures/**"
- "src/contracts/captures/**"
@@ -48,3 +55,4 @@ jobs:
- run: node scripts/verify-packet-capture.mjs
env:
REA_TSHARK_COMMAND: /usr/bin/tshark
REA_CAPINFOS_COMMAND: /usr/bin/capinfos
+2 -1
View File
@@ -170,6 +170,7 @@ Additional tools and host support depend on the target:
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/flutter-analysis.md) |
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/firmware-analysis.md) |
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/packet-captures.md) |
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/pcap-analysis.md) |
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/signal-recordings.md) |
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/javascript-artifact-reconstruction.md), [Apple applications](docs/apple-application-analysis.md) |
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/process-capture.md) |
@@ -321,7 +322,7 @@ Report vulnerabilities through [SECURITY.md](SECURITY.md).
## Star history
🎉 **110,000 GitHub stars — thank you!**
🎉 **170,000 GitHub stars — thank you!**
Thanks to everyone using REA, reporting bugs, requesting features, testing builds, and contributing fixes.
+2 -1
View File
@@ -139,6 +139,7 @@ npx rea-agents@latest setup
| JavaScript / Electron | الوحدات، وعمليات الاستيراد، وخرائط المصدر، والمسارات، وIPC، وعلاقات إضافات الشيفرة الأصلية | Node.js وnpm؛ [تحليل التطبيقات](https://rea.tools/ar/guides/javascript/) |
| المواقع | بنية الصفحة، والسكربتات، وملاحظات الشبكة، ولقطات الشاشة المطلوبة | متصفح من عائلة Chrome؛ [تحليل المتصفح](https://rea.tools/ar/guides/browser/) |
| تسجيلات الشبكة المحفوظة | الطلبات، والاستجابات، ومحتويات الحمولة المتاحة، ومواضعها في المصدر | HAR؛ وmitmdump على Linux لتسجيلات mitmproxy بصيغتها الأصلية؛ [دليل تسجيلات الشبكة](docs/web-network-captures.md) |
| التقاط الحزم | نقاط النهاية وDNS وTLS SNI وتوزيع البروتوكولات وطلبات HTTP | tshark وcapinfos يوفّرهما المُستدعي؛ [دليل pcap](docs/pcap-analysis.md) |
| تجميعات .NET | البيانات الوصفية، وتعليمات CIL، واعتماديات الشيفرة الأصلية المعلنة، ومقارنات البناء | فحص ثابت؛ [دليل الشيفرة المُدارة](docs/managed-code-analysis.md) |
| حزم Android APK | تصريحات ملف manifest، والأصناف، والدوال الناتجة عن فك الترجمة، والمراجع | JADX دون واجهة رسومية وJDK كامل على Linux/macOS/Windows x64؛ [دليل Android](docs/android-analysis.md) |
| أجهزة Android | الأجهزة والحزم والعمليات والسجلات والدامبات ولقطات الشاشة ونقل APK/الملفات | adb يوفّره المُستدعي (محاكي أو جهاز)؛ [دليل ADB](docs/adb-device-analysis.md) |
@@ -263,7 +264,7 @@ npx rea-agents@latest setup
## سجل النجوم
🎉 **110,000 نجمة على GitHub — شكرًا لكم!**
🎉 **170,000 نجمة على GitHub — شكرًا لكم!**
شكرًا لكل من يستخدم REA، ويبلّغ عن الأخطاء، ويقترح ميزات، ويختبر إصدارات البناء، ويساهم بالإصلاحات.
+2 -1
View File
@@ -139,6 +139,7 @@ REA benötigt Node.js 22.x (>=22.19), 24.x (>=24.11) oder 26+ sowie npm. Zusätz
| JavaScript / Electron | Module, Imports, Source Maps, Routen, IPC und Beziehungen zu nativen Add-ons | Node.js und npm; [Anwendungsanalyse](https://rea.tools/de/guides/javascript/) |
| Websites | Seitenstruktur, Skripte, Netzwerkbeobachtungen und angeforderte Screenshots | Ein Browser aus der Chrome-Familie; [Browseranalyse](https://rea.tools/de/guides/browser/) |
| Gespeicherte Netzwerkaufzeichnungen | Anfragen, Antworten, zugängliche Nutzdaten und Quellpositionen | HAR; mitmdump unter Linux für Aufzeichnungen im nativen mitmproxy-Format; [Leitfaden zu Netzwerkaufzeichnungen](docs/web-network-captures.md) |
| Paketerfassungen | Endpunkte, DNS, TLS SNI, Protokollverteilung und HTTP-Anfragen | Vom Aufrufer bereitgestellte tshark- und capinfos-Binaries; [pcap-Leitfaden](docs/pcap-analysis.md) |
| .NET-Assemblies | Metadaten, CIL-Anweisungen, deklarierte native Abhängigkeiten und Build-Vergleiche | Statische Untersuchung; [Leitfaden für verwalteten Code](docs/managed-code-analysis.md) |
| Android-APKs | Manifest-Deklarationen, Klassen, dekompilierte Methoden und Verweise | JADX ohne grafische Oberfläche und ein vollständiges JDK unter Linux/macOS/Windows x64; [Android-Leitfaden](docs/android-analysis.md) |
| Android-Geräte | Geräte, Pakete, Prozesse, Logs, Dumps, Bildschirmaufnahmen sowie APK- und Dateiübertragungen | Vom Aufrufer bereitgestelltes adb (Emulator oder Gerät); [ADB-Leitfaden](docs/adb-device-analysis.md) |
@@ -261,7 +262,7 @@ Melde Sicherheitslücken gemäß [SECURITY.md](SECURITY.md).
## Sternverlauf
🎉 **110.000 GitHub-Sterne – vielen Dank!**
🎉 **170.000 GitHub-Sterne – vielen Dank!**
Danke an alle, die REA nutzen, Fehler melden, Funktionswünsche äußern, Builds testen und Fehlerbehebungen beitragen.
+2 -1
View File
@@ -139,6 +139,7 @@ REA requiere Node.js 22.x (>=22.19), 24.x (>=24.11) o 26+, además de npm. Las h
| JavaScript / Electron | Módulos, importaciones, mapas de código fuente, rutas, IPC y relaciones con complementos nativos | Node.js y npm; [análisis de aplicaciones](https://rea.tools/es/guides/javascript/) |
| Sitios web | Estructura de la página, scripts, observaciones de red y capturas de pantalla solicitadas | Un navegador de la familia Chrome; [análisis del navegador](https://rea.tools/es/guides/browser/) |
| Capturas de red guardadas | Solicitudes, respuestas, contenido accesible de las cargas útiles y ubicaciones de origen | HAR; mitmdump en Linux para capturas en formato nativo de mitmproxy; [guía de capturas](docs/web-network-captures.md) |
| Capturas de paquetes | Endpoints, DNS, TLS SNI, distribución de protocolos y solicitudes HTTP | tshark y capinfos suministrados por el autor de la llamada; [guía pcap](docs/pcap-analysis.md) |
| Ensamblados .NET | Metadatos, instrucciones CIL, dependencias nativas declaradas y comparaciones de compilaciones | Inspección estática; [guía de código administrado](docs/managed-code-analysis.md) |
| APK de Android | Declaraciones del manifiesto, clases, métodos descompilados y referencias | JADX sin interfaz gráfica y un JDK completo en Linux/macOS/Windows x64; [guía de Android](docs/android-analysis.md) |
| Dispositivos Android | Dispositivos, paquetes, procesos, registros, volcados, capturas de pantalla y transferencias de APK/archivos | adb suministrado por el autor de la llamada (emulador o dispositivo); [guía de ADB](docs/adb-device-analysis.md) |
@@ -261,7 +262,7 @@ Informa de vulnerabilidades siguiendo [SECURITY.md](SECURITY.md).
## Historial de estrellas
🎉 **110.000 estrellas en GitHub: ¡gracias!**
🎉 **170.000 estrellas en GitHub: ¡gracias!**
Gracias a quienes usan REA, informan de errores, solicitan funciones, prueban compilaciones y contribuyen con correcciones.
+2 -1
View File
@@ -139,6 +139,7 @@ REA به Node.js 22.x (>=22.19)، 24.x (>=24.11) یا 26+، به‌همراه np
| برنامه‌های فلاتر | هش اسنپ‌شات دارت، هش کتابخانه‌ها به تفکیک ABI و اصالت بیلد موتور | هیچ (تحلیل ایستا APK)؛ [راهنمای فلاتر](docs/flutter-analysis.md) |
| Firmware | ناحیه‌ها، نتایج استخراج و تحویل به ابزارهای تحلیل بومی | Binwalk / Unblob در لینوکس؛ [راهنمای Firmware](docs/firmware-analysis.md) |
| بسته‌های ضبط‌شده | فیلدهای منتخب، مقادیر تکراری، فیلترها و قواعد رمزگشایی | Linux: TShark 4.x + prlimit; [راهنمای ضبط بسته](docs/packet-captures.md) |
| ضبط بسته‌ها | نقاط پایانی، DNS، TLS SNI، توزیع پروتکل و درخواست‌های HTTP | tshark و capinfos تأمین‌شده توسط فراخوان‌کننده؛ [راهنمای pcap](docs/pcap-analysis.md) |
| سیگنال‌های منطقی ضبط‌شده | توضیحات رمزگشا، بازه‌های نمونه و اطلاعات عیب‌یابی | Linux: sigrok-cli 0.7.2 + prlimit; [راهنمای سیگنال‌ها](docs/signal-recordings.md) |
| بسته‌ها و منابع | فهرست فایل‌ها، digestها، فایل‌های plist، ساختار bundle اپل و منابع استخراج‌شده | [راهنمای آرتیفکت و JavaScript](docs/javascript-artifact-reconstruction.md)، [برنامه‌های اپل](docs/apple-application-analysis.md) |
| رفتار فرایند | خروجی ترمینال، تعاملات، وضعیت خروج، مشاهدات سیستم فایل و مقایسهٔ اجراها | Linux/macOS با PTY بومی؛ [ضبط فرایند](docs/process-capture.md) |
@@ -246,7 +247,7 @@ npx rea-agents@latest setup
## تاریخچهٔ ستاره‌ها
🎉 **۱۱۰٬۰۰۰ ستارهٔ GitHub — سپاسگزاریم!**
🎉 **۱۷۰٬۰۰۰ ستارهٔ GitHub — سپاسگزاریم!**
از همهٔ کسانی که از REA استفاده می‌کنند، خطاها را گزارش می‌دهند، قابلیت‌های جدید پیشنهاد می‌کنند، buildها را آزمایش می‌کنند و در بهبود پروژه مشارکت دارند سپاسگزاریم.
+2 -1
View File
@@ -143,6 +143,7 @@ REA nécessite Node.js 22.x (>=22.19), 24.x (>=24.11) ou 26+, ainsi que npm. Les
| JavaScript / Electron | Modules, imports, source maps, routes, IPC et relations avec les modules natifs | Node.js et npm ; [analyse d'applications](https://rea.tools/fr/guides/javascript/) |
| Sites web | Structure des pages, scripts, observations réseau et captures d'écran demandées | Un navigateur de la famille Chrome ; [analyse du navigateur](https://rea.tools/fr/guides/browser/) |
| Captures réseau enregistrées | Requêtes, réponses, payloads exposés et emplacements dans le code source | HAR ; mitmdump sous Linux pour les captures mitmproxy natives ; [guide des captures](docs/web-network-captures.md) |
| Captures de paquets | Points de terminaison, DNS, TLS SNI, distribution des protocoles et requêtes HTTP | tshark et capinfos fournis par l'appelant ; [guide pcap](docs/pcap-analysis.md) |
| Assemblys .NET | Métadonnées, instructions CIL, dépendances natives déclarées et comparaisons de builds | Inspection statique ; [guide du code managé](docs/managed-code-analysis.md) |
| APK Android | Déclarations du manifeste, classes, méthodes décompilées et références | JADX headless et un JDK complet sous Linux/macOS/Windows x64 ; [guide Android](docs/android-analysis.md) |
| Appareils Android | Appareils, paquets, processus, journaux, dumps, captures d'écran et transferts d'APK/fichiers | adb fourni par l'appelant (émulateur ou appareil) ; [guide ADB](docs/adb-device-analysis.md) |
@@ -265,7 +266,7 @@ Signalez les vulnérabilités via [SECURITY.md](SECURITY.md).
## Historique des étoiles
🎉 **110 000 étoiles GitHub — merci !**
🎉 **170 000 étoiles GitHub — merci !**
Merci à toutes les personnes qui utilisent REA, signalent des bugs, proposent des fonctionnalités, testent les builds et contribuent des correctifs.
+2 -1
View File
@@ -146,6 +146,7 @@ Alat tambahan dan dukungan sistem host bergantung pada target yang dianalisis:
| JavaScript / Electron | Modul, impor, source map, rute, IPC, dan hubungan add-on native | Node.js dan npm; [analisis aplikasi](https://rea.tools/id/guides/javascript/) |
| Situs web | Struktur halaman, skrip, pengamatan jaringan, dan tangkapan layar yang diminta | Peramban berbasis Chrome; [analisis peramban](https://rea.tools/id/guides/browser/) |
| Rekaman jaringan tersimpan | Permintaan, respons, payload yang terekspos, dan lokasi sumber | HAR; mitmdump di Linux untuk rekaman native mitmproxy; [panduan rekaman](docs/web-network-captures.md) |
| Capturan paket | Endpoint, DNS, TLS SNI, distribusi protokol, dan permintaan HTTP | tshark dan capinfos buatan pemanggil; [panduan pcap](docs/pcap-analysis.md) |
| Assembly .NET | Metadata, instruksi CIL, dependensi native yang dideklarasikan, dan perbandingan build | Tidak ada (hanya analisis statis); [panduan kode terkelola](docs/managed-code-analysis.md) |
| APK Android | Deklarasi manifest, kelas, metode hasil dekompilasi, dan referensi | JADX mode headless dan JDK lengkap di Linux/macOS/Windows x64; [panduan Android](docs/android-analysis.md) |
| Perangkat Android | Perangkat, paket, proses, log, dump, tangkapan layar, serta transfer APK/berkas | adb buatan pemanggil (emulator atau perangkat); [panduan ADB](docs/adb-device-analysis.md) |
@@ -286,7 +287,7 @@ Laporkan kerentanan melalui [SECURITY.md](SECURITY.md).
## Riwayat Bintang
🎉 **110.000 bintang GitHub, terima kasih!**
🎉 **170.000 bintang GitHub, terima kasih!**
Terima kasih kepada semua orang yang menggunakan REA, melaporkan bug, meminta fitur, menguji build, dan berkontribusi dalam perbaikan.
+2 -1
View File
@@ -143,6 +143,7 @@ REA richiede Node.js 22.x (>=22.19), 24.x (>=24.11) o 26+, oltre a npm. Strument
| JavaScript / Electron | Moduli, import, source map, route, IPC e relazioni con add-on nativi | Node.js e npm; [analisi delle applicazioni](https://rea.tools/it/guides/javascript/) |
| Siti web | Struttura della pagina, script, osservazioni di rete e screenshot richiesti | Un browser della famiglia Chrome; [analisi browser](https://rea.tools/it/guides/browser/) |
| Acquisizioni di rete salvate | Richieste, risposte, payload esposti e posizioni sorgente | HAR; mitmdump su Linux per acquisizioni mitmproxy native; [guida alle acquisizioni](docs/web-network-captures.md) |
| Captured di pacchetti | Endpoint, DNS, TLS SNI, distribuzione dei protocolli e richieste HTTP | tshark e capinfos forniti dal chiamante; [guida pcap](docs/pcap-analysis.md) |
| Assembly .NET | Metadati, istruzioni CIL, dipendenze native dichiarate e confronti di build | Nessuno (solo statico); [guida managed-code](docs/managed-code-analysis.md) |
| APK Android | Dichiarazioni manifest, classi, metodi decompilati e riferimenti | JADX headless e un JDK completo su Linux/macOS/Windows x64; [guida Android](docs/android-analysis.md) |
| Dispositivi Android | Dispositivi, pacchetti, processi, log, dump, screenshot e trasferimenti APK/file | adb fornito dal chiamante (emulatore o hardware); [guida ADB](docs/adb-device-analysis.md) |
@@ -265,7 +266,7 @@ Segnala vulnerabilità tramite [SECURITY.md](SECURITY.md).
## Cronologia delle stelle
🎉 **110.000 stelle su GitHub — grazie!**
🎉 **170.000 stelle su GitHub — grazie!**
Grazie a tutti coloro che usano REA, segnalano bug, propongono funzionalità, testano le build e contribuiscono con correzioni.
+2 -1
View File
@@ -139,6 +139,7 @@ REA には Node.js 22.x(>=22.19)、24.x(>=24.11)、または 26+ と npm
| JavaScript / Electron | モジュール、インポート、ソースマップ、ルート、IPC、ネイティブアドオンの関係 | Node.js と npm;[アプリケーション分析](https://rea.tools/ja/guides/javascript/) |
| ウェブサイト | ページ構造、スクリプト、ネットワークの観測結果、依頼されたスクリーンショット | Chrome 系ブラウザー;[ブラウザー分析](https://rea.tools/ja/guides/browser/) |
| 保存済みのネットワークキャプチャ | リクエスト、レスポンス、取得可能なペイロード、ソース内の位置 | HAR;mitmproxy ネイティブ形式のキャプチャには Linux 上の mitmdump;[キャプチャガイド](docs/web-network-captures.md) |
| パケットキャプチャ | エンドポイント、DNS、TLS SNI、プロトコル分布、HTTPリクエスト | 呼び出し側が用意した tshark と capinfos;[pcapガイド](docs/pcap-analysis.md) |
| .NET アセンブリ | メタデータ、CIL 命令、宣言されたネイティブ依存関係、ビルドの比較 | 静的な検査;[マネージドコードのガイド](docs/managed-code-analysis.md) |
| Android APK | マニフェストの宣言、クラス、逆コンパイルされたメソッド、参照 | Linux/macOS/Windows x64 上のヘッドレス JADX と完全な JDK;[Android ガイド](docs/android-analysis.md) |
| Android デバイス | デバイス、パッケージ、プロセス、ログ、ダンプ、スクリーンキャプチャ、APK/ファイル転送 | 呼び出し側が用意した adb(エミュレーターまたは実機);[ADB ガイド](docs/adb-device-analysis.md) |
@@ -263,7 +264,7 @@ npx rea-agents@latest setup
## スター履歴
🎉 **GitHub スター 110,000 件、ありがとうございます!**
🎉 **GitHub スター 170,000 件、ありがとうございます!**
REA の利用、バグ報告、機能の要望、ビルドのテスト、修正への貢献に感謝します。
+2 -1
View File
@@ -139,6 +139,7 @@ REA에는 Node.js 22.x(>=22.19), 24.x(>=24.11) 또는 26+와 npm이 필요합니
| JavaScript / Electron | 모듈, 임포트, 소스 맵, 라우트, IPC, 네이티브 애드온 관계 | Node.js와 npm; [애플리케이션 분석](https://rea.tools/ko/guides/javascript/) |
| 웹사이트 | 페이지 구조, 스크립트, 네트워크 관찰 결과, 요청한 스크린샷 | Chrome 계열 브라우저; [브라우저 분석](https://rea.tools/ko/guides/browser/) |
| 저장된 네트워크 캡처 | 요청, 응답, 접근 가능한 페이로드, 소스 위치 | HAR; mitmproxy 네이티브 캡처에는 Linux의 mitmdump; [캡처 가이드](docs/web-network-captures.md) |
| 패킷 캡처 | 엔드포인트, DNS, TLS SNI, 프로토콜 분포, HTTP 요청 | 호출자 제공 tshark 및 capinfos; [pcap 가이드](docs/pcap-analysis.md) |
| .NET 어셈블리 | 메타데이터, CIL 명령, 선언된 네이티브 의존성, 빌드 비교 | 정적 검사; [관리 코드 가이드](docs/managed-code-analysis.md) |
| Android APK | 매니페스트 선언, 클래스, 디컴파일된 메서드, 참조 | Linux/macOS/Windows x64의 헤드리스 JADX와 전체 JDK; [Android 가이드](docs/android-analysis.md) |
| Android 기기 | 기기, 패키지, 프로세스, 로그, 덤프, 화면 캡처, APK/파일 전송 | 호출자 제공 adb(에뮬레이터 또는 실제 기기); [ADB 가이드](docs/adb-device-analysis.md) |
@@ -263,7 +264,7 @@ npx rea-agents@latest setup
## 스타 기록
🎉 **GitHub 스타 110,000개, 감사합니다!**
🎉 **GitHub 스타 170,000개, 감사합니다!**
REA를 사용하고, 버그를 보고하고, 기능을 요청하고, 빌드를 테스트하고, 수정에 기여해 주신 모든 분께 감사드립니다.
+2 -1
View File
@@ -139,6 +139,7 @@ REA wymaga Node.js 22.x (>=22.19), 24.x (>=24.11) lub 26+ oraz npm. Dodatkowe na
| JavaScript / Electron | Moduły, importy, mapy źródeł, trasy, IPC i powiązania z natywnymi dodatkami | Node.js i npm; [analiza aplikacji](https://rea.tools/pl/guides/javascript/) |
| Strony internetowe | Struktura strony, skrypty, obserwacje sieciowe i zamówione zrzuty ekranu | Przeglądarka z rodziny Chrome; [analiza w przeglądarce](https://rea.tools/pl/guides/browser/) |
| Zapisane przechwycenia ruchu sieciowego | Żądania, odpowiedzi, dostępne dane przesyłane i lokalizacje źródłowe | HAR; mitmdump w systemie Linux dla natywnych przechwyceń mitmproxy; [przewodnik przechwytywania](docs/web-network-captures.md) |
| Przechwytywanie pakietów | Punkty końcowe, DNS, TLS SNI, dystrybucja protokołów i żądania HTTP | Dostarczone przez wywołującego tshark i capinfos; [przewodnik pcap](docs/pcap-analysis.md) |
| Zestawy .NET | Metadane, instrukcje CIL, zadeklarowane zależności natywne i porównania kompilacji | Analiza statyczna; [przewodnik kodu zarządzanego](docs/managed-code-analysis.md) |
| Pakiety APK dla Androida | Deklaracje manifestu, klasy, zdekompilowane metody i odwołania | JADX bez interfejsu graficznego i pełny JDK w systemach Linux/macOS/Windows x64; [przewodnik Androida](docs/android-analysis.md) |
| Urządzenia z Androidem | Urządzenia, pakiety, procesy, logi, zrzuty, zrzuty ekranu oraz transfery APK/plików | adb dostarczony przez wywołującego (emulator lub urządzenie); [przewodnik ADB](docs/adb-device-analysis.md) |
@@ -261,7 +262,7 @@ Zgłaszaj podatności zgodnie z [SECURITY.md](SECURITY.md).
## Historia gwiazdek
🎉 **110 000 gwiazdek na GitHubie — dziękujemy!**
🎉 **170 000 gwiazdek na GitHubie — dziękujemy!**
Dziękujemy wszystkim, którzy używają REA, zgłaszają błędy, proponują nowe funkcje, testują kompilacje i przesyłają poprawki.
+2 -1
View File
@@ -139,6 +139,7 @@ O REA requer Node.js 22.x (>=22.19), 24.x (>=24.11) ou 26+, além do npm. As fer
| JavaScript / Electron | Módulos, importações, source maps, rotas, IPC e relações com complementos nativos | Node.js e npm; [análise de aplicativos](https://rea.tools/pt-BR/guides/javascript/) |
| Sites | Estrutura da página, scripts, observações de rede e capturas de tela solicitadas | Um navegador da família Chrome; [análise do navegador](https://rea.tools/pt-BR/guides/browser/) |
| Capturas de rede salvas | Requisições, respostas, payloads acessíveis e localizações de origem | HAR; mitmdump no Linux para capturas no formato nativo do mitmproxy; [guia de capturas](docs/web-network-captures.md) |
| Capturas de pacotes | Endpoints, DNS, TLS SNI, distribuição de protocolos e requisições HTTP | tshark e capinfos fornecidos pelo chamador; [guia pcap](docs/pcap-analysis.md) |
| Assemblies .NET | Metadados, instruções CIL, dependências nativas declaradas e comparações de builds | Inspeção estática; [guia de código gerenciado](docs/managed-code-analysis.md) |
| APKs Android | Declarações do manifesto, classes, métodos descompilados e referências | JADX sem interface gráfica e um JDK completo no Linux/macOS/Windows x64; [guia de Android](docs/android-analysis.md) |
| Dispositivos Android | Dispositivos, pacotes, processos, logs, despejos, capturas de tela e transferências de APK/arquivos | adb fornecido pelo chamador (emulador ou aparelho); [guia de ADB](docs/adb-device-analysis.md) |
@@ -261,7 +262,7 @@ Relate vulnerabilidades conforme [SECURITY.md](SECURITY.md).
## Histórico de estrelas
🎉 **110.000 estrelas no GitHub — muito obrigado!**
🎉 **170.000 estrelas no GitHub — muito obrigado!**
Obrigado a todos que usam o REA, relatam bugs, sugerem funcionalidades, testam builds e contribuem com correções.
+2 -1
View File
@@ -147,6 +147,7 @@ REA требует Node.js 22.x (>=22.19), 24.x (>=24.11) или 26+, а так
| Приложения Flutter | Хеш снапшота Dart, дайджесты библиотек по ABI и происхождение сборки движка | Не требуется (статический разбор APK); [руководство по Flutter](docs/flutter-analysis.md) |
| Прошивки | Области, результаты извлечения и передача в нативный анализ | Binwalk / Unblob в Linux; [руководство по прошивкам](docs/firmware-analysis.md) |
| Записанные пакеты | Выбранные поля, повторяющиеся значения, фильтры и правила декодирования | Linux: TShark 4.x + prlimit; [руководство по захватам](docs/packet-captures.md) |
| Захват пакетов | Конечные точки, DNS, TLS SNI, распределение протоколов и HTTP-запросы | Предоставленные вызывающей стороной tshark и capinfos; [руководство по pcap](docs/pcap-analysis.md) |
| Записанные логические сигналы | Аннотации декодера, диапазоны отсчётов и диагностика | Linux: sigrok-cli 0.7.2 + prlimit; [руководство по сигналам](docs/signal-recordings.md) |
| Пакеты и ресурсы | Списки файлов, хеши, plist, структура пакетов Apple и извлечённые ресурсы | [Руководство по артефактам и JavaScript](docs/javascript-artifact-reconstruction.md), [приложения Apple](docs/apple-application-analysis.md) |
| Поведение процессов | Вывод терминала, взаимодействия, наблюдения за завершением и файловой системой, сравнения запусков | Linux/macOS с нативным PTY; [захват процессов](docs/process-capture.md) |
@@ -261,7 +262,7 @@ npx rea-agents@latest setup
## История звёзд
🎉 **110 000 звёзд на GitHub — спасибо!**
🎉 **170 000 звёзд на GitHub — спасибо!**
Спасибо всем, кто использует REA, сообщает об ошибках, предлагает новые функции, тестирует сборки и помогает с исправлениями.
+2 -1
View File
@@ -139,6 +139,7 @@ REA ต้องใช้ Node.js 22.x (>=22.19), 24.x (>=24.11) หรือ 26
| JavaScript / Electron | โมดูล การนำเข้า ซอร์สแมป เส้นทาง IPC และความสัมพันธ์กับส่วนเสริมเนทีฟ | Node.js และ npm; [การวิเคราะห์แอปพลิเคชัน](https://rea.tools/th/guides/javascript/) |
| เว็บไซต์ | โครงสร้างหน้า สคริปต์ ข้อมูลเครือข่ายที่สังเกตได้ และภาพหน้าจอที่ร้องขอ | เบราว์เซอร์ในตระกูล Chrome; [การวิเคราะห์ผ่านเบราว์เซอร์](https://rea.tools/th/guides/browser/) |
| ข้อมูลเครือข่ายที่บันทึกไว้ | คำขอ คำตอบ เพย์โหลดที่ปรากฏ และตำแหน่งต้นทาง | HAR; mitmdump บน Linux สำหรับข้อมูลที่บันทึกในรูปแบบเนทีฟของ mitmproxy; [คู่มือการบันทึกข้อมูล](docs/web-network-captures.md) |
| การจับแพ็กเก็ต | จุดเชื่อมต่อ, DNS, TLS SNI, การกระจายโปรโตคอล และคำขอ HTTP | tshark และ capinfos ที่ผู้เรียกจัดเตรียม; [คู่มือ pcap](docs/pcap-analysis.md) |
| แอสเซมบลี .NET | เมทาดาทา คำสั่ง CIL การพึ่งพาเนทีฟที่ประกาศไว้ และการเปรียบเทียบผลการบิลด์ | การตรวจสอบแบบสถิต; [คู่มือโค้ดแบบจัดการ](docs/managed-code-analysis.md) |
| APK ของ Android | การประกาศใน manifest คลาส เมธอดที่ดีคอมไพล์ และการอ้างอิง | JADX แบบไม่มี GUI และ JDK แบบเต็มบน Linux/macOS/Windows x64; [คู่มือ Android](docs/android-analysis.md) |
| อุปกรณ์ Android | อุปกรณ์ แพ็กเกจ โพรเซส บันทึก ดัมพ์ ภาพหน้าจอ และการถ่ายโอน APK/ไฟล์ | adb ที่ผู้เรียกจัดเตรียม (โปรแกรมจำลองหรืออุปกรณ์จริง); [คู่มือ ADB](docs/adb-device-analysis.md) |
@@ -261,7 +262,7 @@ npx rea-agents@latest setup
## ประวัติดาว
🎉 **110,000 ดาวบน GitHub — ขอบคุณทุกคน!**
🎉 **170,000 ดาวบน GitHub — ขอบคุณทุกคน!**
ขอบคุณทุกคนที่ใช้ REA รายงานบั๊ก เสนอฟีเจอร์ ทดสอบบิลด์ และช่วยแก้ไข
+2 -1
View File
@@ -139,6 +139,7 @@ REA; Node.js 22.x (>=22.19), 24.x (>=24.11) veya 26+ ve npm gerektirir. Ek araç
| JavaScript / Electron | Modüller, içe aktarmalar, kaynak haritaları, rotalar, IPC ve yerel eklenti ilişkileri | Node.js ve npm; [uygulama analizi](https://rea.tools/tr/guides/javascript/) |
| Web siteleri | Sayfa yapısı, betikler, ağ gözlemleri ve istenen ekran görüntüleri | Chrome ailesinden bir tarayıcı; [tarayıcı analizi](https://rea.tools/tr/guides/browser/) |
| Kaydedilmiş ağ yakalamaları | İstekler, yanıtlar, erişilebilir yük içerikleri ve kaynak konumları | HAR; mitmproxy'nin kendi biçimindeki yakalamalar için Linux üzerinde mitmdump; [ağ yakalama rehberi](docs/web-network-captures.md) |
| Paket yakalamaları | Uç noktalar, DNS, TLS SNI, protokol dağılımı ve HTTP istekleri | Çağıranın sağladığı tshark ve capinfos; [pcap rehberi](docs/pcap-analysis.md) |
| .NET derlemeleri | Meta veriler, CIL talimatları, bildirilmiş yerel bağımlılıklar ve derleme karşılaştırmaları | Statik inceleme; [yönetilen kod rehberi](docs/managed-code-analysis.md) |
| Android APK'leri | Manifest bildirimleri, sınıflar, tersine derlenmiş metotlar ve referanslar | Linux/macOS/Windows x64 üzerinde arayüzsüz JADX ve tam bir JDK; [Android rehberi](docs/android-analysis.md) |
| Android cihazları | Cihazlar, paketler, süreçler, günlükler, dökümler, ekran görüntüleri ve APK/dosya aktarımları | Çağıranın sağladığı adb (emülatör veya cihaz); [ADB rehberi](docs/adb-device-analysis.md) |
@@ -261,7 +262,7 @@ Güvenlik açıklarını [SECURITY.md](SECURITY.md) üzerinden bildirin.
## Yıldız geçmişi
🎉 **GitHub'da 110.000 yıldız — teşekkürler!**
🎉 **GitHub'da 170.000 yıldız — teşekkürler!**
REA'yı kullanan, hata bildiren, özellik talep eden, derlemeleri test eden ve düzeltmelere katkıda bulunan herkese teşekkürler.
+2 -1
View File
@@ -139,6 +139,7 @@ REA потребує Node.js 22.x (>=22.19), 24.x (>=24.11) або 26+, а та
| JavaScript / Electron | Модулі, імпорти, карти вихідного коду, маршрути, IPC і зв'язки з нативними доповненнями | Node.js і npm; [аналіз застосунків](https://rea.tools/uk/guides/javascript/) |
| Сайти | Структура сторінки, скрипти, мережеві спостереження й запитані знімки екрана | Браузер сімейства Chrome; [аналіз браузера](https://rea.tools/uk/guides/browser/) |
| Збережені мережеві захоплення | Запити, відповіді, доступні корисні дані та місця в джерелі | HAR; mitmdump у Linux для захоплень у власному форматі mitmproxy; [посібник мережевих захоплень](docs/web-network-captures.md) |
| Захоплення пакетів | Кінцеві точки, DNS, TLS SNI, розподіл протоколів та HTTP-запити | Надані викликаючою стороною tshark та capinfos; [підручник pcap](docs/pcap-analysis.md) |
| Збірки .NET | Метадані, інструкції CIL, оголошені нативні залежності й порівняння збірок | Статична перевірка; [посібник керованого коду](docs/managed-code-analysis.md) |
| Android APK | Оголошення маніфесту, класи, декомпільовані методи й посилання | JADX без графічного інтерфейсу й повний JDK у Linux/macOS/Windows x64; [посібник Android](docs/android-analysis.md) |
| Пристрої Android | Пристрої, пакети, процеси, журнали, дампи, знімки екрана та передавання APK/файлів | adb, наданий викликаючою стороною (емулятор або пристрій); [підручник ADB](docs/adb-device-analysis.md) |
@@ -261,7 +262,7 @@ npx rea-agents@latest setup
## Історія зірок
🎉 **110 000 зірок на GitHub — дякуємо!**
🎉 **170 000 зірок на GitHub — дякуємо!**
Дякуємо всім, хто використовує REA, повідомляє про помилки, пропонує нові функції, тестує збірки й допомагає з виправленнями.
+2 -1
View File
@@ -139,6 +139,7 @@ REA yêu cầu Node.js 22.x (>=22.19), 24.x (>=24.11) hoặc 26+, cùng với np
| JavaScript / Electron | Mô-đun, các khai báo import, source map, tuyến, IPC và quan hệ với tiện ích mở rộng mã máy | Node.js và npm; [phân tích ứng dụng](https://rea.tools/vi/guides/javascript/) |
| Trang web | Cấu trúc trang, script, quan sát mạng và ảnh chụp màn hình được yêu cầu | Trình duyệt thuộc họ Chrome; [phân tích trình duyệt](https://rea.tools/vi/guides/browser/) |
| Bản ghi lưu lượng mạng đã lưu | Yêu cầu, phản hồi, payload có thể truy cập và vị trí nguồn | HAR; mitmdump trên Linux cho bản ghi định dạng gốc của mitmproxy; [hướng dẫn bản ghi mạng](docs/web-network-captures.md) |
| Bắt gói tin | Điểm cuối, DNS, TLS SNI, phân bố giao thức và yêu cầu HTTP | tshark và capinfos do nơi gọi cung cấp; [hướng dẫn pcap](docs/pcap-analysis.md) |
| Assembly .NET | Siêu dữ liệu, lệnh CIL, các phụ thuộc mã máy được khai báo và so sánh bản dựng | Kiểm tra tĩnh; [hướng dẫn mã được quản lý](docs/managed-code-analysis.md) |
| APK Android | Khai báo manifest, lớp, phương thức được dịch ngược và tham chiếu | JADX không giao diện đồ họa và JDK đầy đủ trên Linux/macOS/Windows x64; [hướng dẫn Android](docs/android-analysis.md) |
| Thiết bị Android | Thiết bị, gói, tiến trình, nhật ký, dump, ảnh chụp màn hình và truyền APK/tệp | adb do nơi gọi cung cấp (trình giả lập hoặc thiết bị); [hướng dẫn ADB](docs/adb-device-analysis.md) |
@@ -261,7 +262,7 @@ Báo cáo lỗ hổng theo [SECURITY.md](SECURITY.md).
## Lịch sử sao
🎉 **110.000 sao trên GitHub — xin cảm ơn!**
🎉 **170.000 sao trên GitHub — xin cảm ơn!**
Cảm ơn mọi người đã sử dụng REA, báo lỗi, đề xuất tính năng, kiểm thử bản dựng và đóng góp bản sửa lỗi.
+2 -1
View File
@@ -139,6 +139,7 @@ REA 需要 Node.js 22.x(>=22.19)、24.x(>=24.11)或 26+,以及 npm。
| JavaScript / Electron | 模組、匯入、source map、路由、IPC 和原生附加元件關係 | Node.js 和 npm;[應用分析](https://rea.tools/zh-TW/guides/javascript/) |
| 網站 | 頁面結構、腳本、網路觀察結果和按請求獲取的螢幕擷取畫面 | Chrome 系瀏覽器;[瀏覽器分析](https://rea.tools/zh-TW/guides/browser/) |
| 已保存的網路擷取 | 請求、回應、可訪問的酬載和來源位置 | HAR;原生 mitmproxy 擷取需要 Linux 上的 mitmdump;[擷取指南](docs/web-network-captures.md) |
| 封包擷取 | 端點、DNS、TLS SNI、通訊協定分佈和 HTTP 請求 | 呼叫端提供的 tshark 和 capinfos;[pcap 指南](docs/pcap-analysis.md) |
| .NET 程式集 | 中繼資料、CIL 指令、聲明的原生依賴和建置對比 | 靜態檢查;[托管程式碼指南](docs/managed-code-analysis.md) |
| Android APK | 清單聲明、類別、反編譯的方法和引用 | Linux/macOS/Windows x64 上的無界面 JADX 和完整 JDK;[Android 指南](docs/android-analysis.md) |
| Android 裝置 | 裝置、套件、程序、日誌、傾印、螢幕擷取以及 APK/檔案傳輸 | 呼叫端提供的 adb(模擬器或實體裝置);[ADB 指南](docs/adb-device-analysis.md) |
@@ -263,7 +264,7 @@ npx rea-agents@latest setup
## Star 歷史
🎉 **GitHub Star 達到 110,000 個,感謝大家!**
🎉 **GitHub Star 達到 170,000 個,感謝大家!**
感謝每一位使用 REA、報告錯誤、提出功能需求、測試建置和貢獻修復的朋友。
+2 -1
View File
@@ -139,6 +139,7 @@ REA 需要 Node.js 22.x(>=22.19)、24.x(>=24.11)或 26+,以及 npm。
| JavaScript / Electron | 模块、导入、source map、路由、IPC 和原生扩展关系 | Node.js 和 npm;[应用分析](https://rea.tools/zh-CN/guides/javascript/) |
| 网站 | 页面结构、脚本、网络观察结果和按请求获取的截图 | Chrome 系浏览器;[浏览器分析](https://rea.tools/zh-CN/guides/browser/) |
| 已保存的网络捕获 | 请求、响应、可访问的载荷和来源位置 | HAR;原生 mitmproxy 捕获需要 Linux 上的 mitmdump;[捕获指南](docs/web-network-captures.md) |
| 数据包捕获 | 端点、DNS、TLS SNI、协议分布和 HTTP 请求 | 调用方提供的 tshark 和 capinfos;[pcap 指南](docs/pcap-analysis.md) |
| .NET 程序集 | 元数据、CIL 指令、声明的原生依赖和构建对比 | 静态检查;[托管代码指南](docs/managed-code-analysis.md) |
| Android APK | 清单声明、类、反编译的方法和引用 | Linux/macOS/Windows x64 上的无界面 JADX 和完整 JDK;[Android 指南](docs/android-analysis.md) |
| Android 设备 | 设备、软件包、进程、日志、转储、屏幕截图以及 APK/文件传输 | 调用方提供的 adb(模拟器或实体设备);[ADB 指南](docs/adb-device-analysis.md) |
@@ -263,7 +264,7 @@ npx rea-agents@latest setup
## Star 历史
🎉 **GitHub Star 达到 110,000 个,感谢大家!**
🎉 **GitHub Star 达到 170,000 个,感谢大家!**
感谢每一位使用 REA、报告错误、提出功能需求、测试构建和贡献修复的朋友。
+13 -6
View File
@@ -317,8 +317,8 @@ The Hopper launcher action used by REA creates a document from an executable;
its supported command-line interface does not attach to an already-open
document. While another live REA session owns the same target and loader
profile, a second session reports the owning run ID instead of opening a
duplicate document. Closing the owning REA session releases this guard; Hopper
keeps its document open. A later REA session can therefore open another
duplicate document. On macOS, closing the owning REA session releases this guard;
Hopper keeps its document open. A later REA session can therefore open another
document for the same target. REA does not currently identify, focus, or reuse
that existing GUI document, and the supported launcher exposes no attach or
reuse action for it.
@@ -341,8 +341,14 @@ Hopper application per user across cooperating REA processes. A competing
CLI or MCP session receives the owning session ID before Hopper is launched,
for both the same target and a different target. Switch targets through that
MCP session, or close it before starting another session. Closing the owner
releases the reservation. This does not attach to or claim ownership of
manually opened Hopper applications.
releases the reservation and stops REA's owned demo application and private
display; the headless Linux demo does not retain documents after MCP shutdown.
Its helper watches a private lifetime pipe, so an owner crash or SIGKILL also
stops those children, without relying on a reusable PID or a process-name scan.
A later session recovers the dead owner's stale lease after child cleanup;
private runtime files can remain when REA itself was killed. This does not attach to
or claim ownership of manually opened Hopper applications. An application left
running by an older REA version must still be closed before starting a new session.
Stale lease recovery is serialized. If an interrupted recovery leaves its
reservation directory, the diagnostic identifies that exact path. Stop REA
@@ -435,8 +441,9 @@ For cursor navigation, string decoding, symbol-name limits, and regex matching,
see [Hopper analysis behavior](native-investigation.md#hopper-analysis-behavior).
Closing or switching a target closes its bound Hopper document, shuts down REA's
bridge and removes its temporary socket directory while preserving the Hopper
application and unrelated documents. A `cleanup_incomplete`
bridge and removes its temporary socket directory. The macOS Hopper application
and unrelated documents remain open; the owned Linux demo application stops.
A `cleanup_incomplete`
result identifies resources whose cleanup could not be verified.
REA retains unresolved cleanup ownership and any confirmed shutdown phases.
After addressing the reported failure, retry `close_binary` on the same
+6 -3
View File
@@ -156,9 +156,12 @@ to the comparison Evidence in a live session.
`compare_source_to_bundle` compares one cryptographically committed
`HistoricalSourceGraph` with authenticated application-graph Evidence. The
stable scoring model reports every admitted signal and weight: exact source
digest, source-map original path, exact current path, path suffix, basename, and
language extension. Exact digest wins over location inference; weak basename
or extension evidence never forces a mapping.
digest, source-map original path, exact current path, path suffix, Unicode-
equivalent path, basename, and language extension. Exact digest wins over
location inference; weak basename, extension, and Unicode-equivalence evidence
never forces a mapping. Unicode-equivalent spellings are candidate evidence
only because distinct filesystem paths can preserve those spellings as separate
files; multiple canonical-equivalent candidates remain unknown.
Each historical source file is classified as `unchanged`, `modified`,
`removed`, `split`, `merged`, `duplicated`, or `unknown`. Multiple equally
+6
View File
@@ -21,6 +21,12 @@ directly to `compare_functions`, and `inspect_artifact` Evidence to
`compare_artifacts`. Use `get_evidence_bundle` when the task needs broader
retained session history or an explicit bundle for transfer.
Evidence and bundle imports reject unsupported fields in their envelopes,
identities, manifests and other fixed metadata before authentication. Imports
leave retained session state unchanged when validation fails. Keep application
metadata in the supported JSON payloads (`parameters`, `raw_result`,
`normalized_result`, or analysis-profile parameters), where keys are preserved.
Tool execution failures return `isError: true` and the complete canonical
`{ "error": ... }` diagnostic as JSON in text `content`. Read that diagnostic
for the error code, target details, partial observations, and remediation.
+118
View File
@@ -0,0 +1,118 @@
# Packet capture summaries
REA summarizes already saved pcap/pcapng files through caller-selected Wireshark
4.x `tshark` and `capinfos`. Use `inspect_packet_capture` for selected fields,
filters and decode-as rules; use this family for address, DNS, SNI, protocol-stack
and URI inventories. It does not acquire traffic or fetch recorded targets.
## Supply tools
```sh
export REA_TSHARK_COMMAND=/absolute/path/tshark
export REA_CAPINFOS_COMMAND=/absolute/path/capinfos
# Linux only; default /usr/bin/prlimit
export REA_TSHARK_PRLIMIT_COMMAND=/absolute/path/prlimit
rea inspect-pcap-client
rea inspect-pcap-capture /captures/session.pcapng --json
```
Without explicit tool commands, this family resolves them through PATH and
records that selection source. Client inspection reads no capture file. Capture
inspection requires Linux with util-linux prlimit or Windows with owned Job
Objects. macOS capture inspection returns a capability gap because the new
native memory boundary is unverified; its version probes remain available.
Tools and installed libraries/plugins are caller-trusted. An empty private
personal Wireshark configuration is used; this is not an OS sandbox.
## Inspect only the needed facts
MCP `inspect_pcap_capture` accepts `path` and optional `detail: "summary"`.
Complete delivery remains the default. Summary commits and retains the same
complete Evidence, returning identity, capture metadata, observed counts,
retained counts, source coverage and diagnostic lengths without large lists.
Then use its `normalized_result.parent_evidence_id` with
`inspect_analysis_view` source kind `retained-evidence`:
```json
{"kind":"pcap","facet":"endpoints","offset":0,"limit":20}
{"kind":"pcap","facet":"dns-queries","offset":0,"limit":20}
{"kind":"pcap","facet":"dns-addresses","query":"example.com","offset":0,"limit":20}
{"kind":"pcap","facet":"tls-sni","offset":0,"limit":20}
{"kind":"pcap","facet":"diagnostics","stream":"tshark","offset":0,"limit":4096}
```
Other row facets are `protocols`, `http-uris` and `unattributed-addresses`.
Defaults are offset 0 and limit 64. DNS question pages give address counts;
select one exact retained question to page its addresses. Diagnostic windows
count UTF-16 code units; other windows count rows. Missing/ambiguous selectors
fail. Unknown legacy diagnostics remain unknown, not an observed empty stream.
Portable CLI views use the existing `inspect-analysis-view` workflow with inline
complete Evidence. Views never reread the source or launch Wireshark and work
after the original file is deleted. Page exhaustion covers the retained facet;
it does not establish network completeness or recover missing traffic.
## Interpret provenance and coverage
`target.sha256` binds observations to an admitted stable capture snapshot. Both
tools read that copy, which is hashed again after analysis. Capture metadata and
literal field values are producer observations; address/protocol counts and DNS
associations are derivations. Capture Evidence has historical-reference authority.
Addresses count once per packet, including loopback and nested IP occurrences.
DNS addresses are associated only with one question in the same response packet;
additional records may be included, so this does not prove final resolution.
Ambiguous addresses remain in a separate inventory. Protocol distribution counts
exact `frame.protocols` stack strings. TLS SNI is recorded handshake metadata,
not process identity or proof of the destination's authenticity.
All examined inventories are retained within the resource budget; no fixed
200/100/50-item truncation is applied. HTTP transport credentials are excluded
without rewriting other URI bytes; ambiguous credential spans cause omission.
Counts preserve distinct observed source URIs before omission or coalescing.
`coverage: "partial"` records unresolved DNS attribution or URI item loss;
component credential exclusion is also counted explicitly. Diagnostics retain
both tool stderr streams. Recorded fields do not establish decrypted payloads,
causality, process identity, absence of unrecorded traffic or current deployments.
## Resources and lifecycle
One inspection is admitted at a time per provider. Concurrent calls return a
capacity error; close drains active work before releasing resources. Captures
are streamed into a read-only owned snapshot with a 2 GiB disk budget. Native
capture commands enforce Linux address-space ceilings of at most 1 GiB,
300 CPU seconds, 64 MiB per file and no core dumps, preserving stricter inherited
limits. Windows uses a 1 GiB committed-memory limit for each owned tool tree.
These are not aggregate machine or filesystem quotas.
TShark output is bounded at 8 MiB; its known five-level selected-field shape and
128 MiB graph-expansion estimate are checked before allocation. Retained facts
have a separate 16 MiB estimate. Capinfos output is limited to 64 KiB. Command
wall deadlines are 300 s for dissection, 60 s for metadata and 20 s for probes.
Native ownership preparation has its own bounded lifecycle.
Malformed/oversized output returns no partial success. Unvalidated partial JSON
is not returned as facts. Cancellation reaches probes, acquisition, commands and
projection. Cleanup failure returns a typed error, retains completed Evidence
when available, and retains owned resources for retry on close.
## Verification scope
Current follow-up verification uses Linux/Node 24.18.0 and Wireshark 4.0.17.
Private real recorded corpora are checked against independent raw-frame/IP
parsing and Python aggregation, public CLI/stdio MCP parity, SDK schemas, source
hashes and retained views. Synthetic native tools separately test malformed
output, cancellation, source/snapshot mutation, capacity and cleanup; they do
not establish real Wireshark or other-platform behavior.
The original proposal reported Linux 4.6.6, macOS/Windows 4.6.9 verification on
its earlier implementation. Those reports do not verify the new final memory
boundary. Actual Windows/Bun Wireshark execution is unverified here; macOS
capture support needs a separate enforceable-bound decision.
```sh
npm run verify:pcap -- --pcap /absolute/saved-capture.pcapng
```
This verifier exercises built CLI/stdio MCP, schemas, artifact identity and
cleanup. Without `--pcap`, capture inspection is reported as unverified. Private
captures, keys and provider runtime files are not committed.
+8
View File
@@ -849,6 +849,14 @@ establish passing Android coverage; inspect its execution receipt.
The `termux-browser-diagnostics` artifact retains the Termux command log, exit
status, and Android logcat on success or failure. The emulator is disposable.
## Packet capture analysis
`npm run verify:pcap -- --pcap PATH` exercises caller-supplied tshark and
capinfos against a saved capture. Linux verification uses Wireshark 4.6.6
and captures containing DNS, TLS and HTTP traffic. Native Windows and macOS
engine behavior remains unverified; process fixtures do not establish it.
See [Packet capture analysis](pcap-analysis.md) for contracts and budgets.
## Apple native metadata and UI
`npm run verify:apple-dispatch` compiles the Objective-C fixture (classes,
@@ -15,6 +15,10 @@ Follow-up: [Dart AOT summary and retained string pools](flutter-aot-summary.md)
records real LocalSend delivery pairs and old Evidence continuation, preserving
ABI selection, snapshot identity, original lower bounds and partial coverage.
Follow-up: [SQLite existing-control measurements](sqlite-existing-controls.md)
tests QGIS's shipped GeoPackage through the existing column selection, preserving
full selected-table schema and typed cells before considering another API.
Follow-up: [native PE/DOS entry views and real model verification](native-entry-views.md)
records the post-merge native continuation, including a small-function case where
four selected replies cost more bytes than the complete dossier.
@@ -178,6 +178,7 @@ def main():
'inspect_android_package', 'search_android_classes',
'inspect_android_class', 'inspect_android_method',
'inspect_firmware_regions', 'inspect_managed_artifact',
'inspect_sqlite_database',
'inspect_managed_members', 'inspect_managed_native_boundaries',
'inspect_artifact', 'inspect_wasm_artifact',
'inspect_evm_interface', 'get_evidence_bundle', 'export_evidence_bundle',
@@ -250,6 +250,19 @@
"transport_bytes": 46558706,
"derivation": "Official arm64 release APK, compared with GitHub release asset digest; byte-only analysis without execution"
}
},
{
"id": "qgis-world-map.gpkg",
"file": "qgis-world-map.gpkg",
"bytes": 12914688,
"sha256": "01d293159d1b77bd9212b5252587853f61aa9e25b2387135f34afaba8d057190",
"provenance": {
"project": "QGIS bundled world map",
"source_commit": "e1223b8e9e71fc50ee995ed556f711c54ccc101b",
"git_blob_sha1": "c9ff80228dff0de93c9acdef16a97545ba360244",
"url": "https://raw.githubusercontent.com/qgis/QGIS/e1223b8e9e71fc50ee995ed556f711c54ccc101b/resources/data/world_map.gpkg",
"derivation": "unmodified official resource file; streaming Git blob and SHA-256 verification"
}
}
]
}
@@ -28,6 +28,7 @@ const { values } = parseArgs({
"javascript-question": { type: "string", default: "inventory" },
"managed-question": { type: "string", default: "inventory" },
schemas: { type: "string", default: "default" },
"sqlite-selection": { type: "string", default: "all" },
},
});
if (
@@ -43,6 +44,11 @@ if (
) ||
!["inventory", "selected"].includes(values["javascript-question"]) ||
!["default", "selected"].includes(values.schemas) ||
!["all", "columns"].includes(values["sqlite-selection"]) ||
(values.artifact !== "qgis-world-map.gpkg" &&
values["sqlite-selection"] !== "all") ||
(values.artifact === "qgis-world-map.gpkg" &&
values.delivery !== "complete") ||
!["inventory", "identity"].includes(values["managed-question"]) ||
(!["sharex.dll", "sharex.helpers.dll"].includes(values.artifact) &&
values["managed-question"] !== "inventory") ||
@@ -518,6 +524,54 @@ const inspectEvmTarget = async () => {
};
const workflows = {
"qgis-world-map.gpkg": async () => {
const columns = ["fid", "NAME", "ISO_A3"];
const input = {
path: values.path,
table: "countries",
row_limit: 1,
...(values["sqlite-selection"] === "columns" ? { columns } : {}),
};
const definition = catalogDefinitions.get("inspect_sqlite_database");
if (
definition === undefined ||
!new Ajv2020({ strict: false, validateFormats: false }).validate(
definition.inputSchema,
input,
)
)
throw new Error(
"Selected SQLite request is not advertised by this runtime",
);
const inspected = await call("inspect_sqlite_database", input);
const report = normalized(inspected);
if (report.rows === null || report.rows.table !== "countries")
throw new Error("Runtime returned no selected country sample");
const indexes = columns.map((column) => {
const index = report.rows.columns.indexOf(column);
if (index < 0)
throw new Error(`Missing requested SQLite column ${column}`);
return index;
});
return {
artifact_sha256: targetSha(inspected),
artifact: report.artifact,
wal: report.wal,
engine: report.engine,
database_encoding: report.database_encoding,
schema: report.schema,
rows: {
...report.rows,
columns,
values: report.rows.values.map((row) =>
indexes.map((index) => row[index]),
),
},
producer_limitations: report.limitations,
limitations: inspected.limitations,
evidence_ids: [inspected.evidence_id],
};
},
"obsidian.asar": async () => {
if (values["javascript-question"] === "selected") {
const inspected = await call("analyze_javascript_application", {
@@ -902,6 +956,10 @@ const report = {
)
? values["managed-question"]
: null,
sqlite_selection:
values.artifact === "qgis-world-map.gpkg"
? values["sqlite-selection"]
: null,
provider_settings_sha256: sha(JSON.stringify(selectedProviders)),
provider_search_path_sha256: sha(
selectedProviders.PATH ?? process.env.PATH ?? "",
@@ -0,0 +1,113 @@
# Process trace validation
A 6,000-event directed cycle previously overflowed the JavaScript stack inside
the schema refinement. Cycle detection now uses iterative topological traversal.
Cyclic inputs retain independent declaration diagnostics without expanding all
event pairs. Unordered declarations retain group membership, with one diagnostic
per overlapping or conflicting group, instead of allocating a pair set and
repeating the same error for every pair.
A unique topological order proves that every event pair is ordered, allowing
rank comparisons without retaining transitive closure. Other acyclic graphs
retain only one origin's forward and backward reachability sets at a time.
Complete pair validation preserves the missing count and first declaration-order
missing pair. General partial orders still require quadratic pair examination
and repeated graph traversal; they trade CPU time for bounded retained state.
## Initial cycle fix comparison
Flameox ran three randomized paired blocks (seed 917), each limited to 20 seconds
and 512 MiB. Node 24.18.0 used a 384-MiB old-space limit on macOS arm64. Each
process validated the same 2,500-event acyclic chain four times, then checked the
known 6,000-event cycle failure with per-event not-before constraints and one
unordered group containing every event. All six executions passed the embedded answer
digest and expected rejection checks.
The baseline source is from `bb540826d`; the candidate is the initial cycle fix
in `10935b7b1`, which still memoized reachability sets for every queried origin.
Both modules had TypeScript annotations removed with Node's identical built-in
transform and used the same copied runtime dependencies. This isolates schema
validation rather than comparing complete packaged CLI/MCP processes.
| Median measurement | Baseline | Candidate |
| -------------------------------------------- | -------: | --------: |
| First validation elapsed milliseconds | 335.80 | 177.25 |
| Warm validation elapsed milliseconds | 313.36 | 165.23 |
| First validation CPU milliseconds | 366.85 | 218.39 |
| Warm validation CPU milliseconds | 335.32 | 194.08 |
| Whole-process elapsed milliseconds | 1,344.13 | 787.00 |
| Process CPU milliseconds before final report | 1,457.67 | 941.05 |
| Peak process-family MiB | 328.36 | 395.52 |
The input is identical: 233,095 JSON bytes, estimated at 77,010 tokens by tokenx
1.3.0. The valid parsed result digest is
`28bd99ce83f0da8e45600ef20f114cb52bb26b5f5e6bfc1601ca3e712eaf3147`.
Actual agent tokens, catalog injection, transport, model reasoning and provider
costs were not measured. This is a component measurement, not an end-to-end
investigation speed claim.
First validation excludes module import and fixture preparation; warm runs reuse
the process but create fresh per-validation graph state. Whole-process elapsed
includes startup, imports, all four validations, the cyclic-input check, token
estimation and reporting. CPU includes startup through metrics publication;
peak RSS covers the complete single-process workload, with no children. Host
caches were not flushed. Peak memory increased in this cohort. Its retained
valid-order closure still required quadratic memory, prompting the follow-up.
Three pairs describe this fixture only.
Preserved Flameox evidence:
`d6211639446e8b6ec2e82636913eaf471def96673e28f03e9c7f98bec07f8e13`;
analysis `e663b6bd8f5693975542d5b5e2b18437b3c167a0be939bb7386eb68549936231`.
An earlier cohort lost modules during another checkout build; its three failed
executions and timing results are excluded. The successful cohort uses an
isolated runtime copy. Source-module SHA-256 values are
`a7729faa2183b0e493b9ac4fc55f6feaee5b2e5294be13d1b29b57a44b899788`
(baseline) and `725409ce84902125941251037a8a214ccc3c1eec577e484c164c361ca580d693`
(candidate).
## Bounded retention follow-up
Flameox compared `10935b7b1` with the rank/streaming implementation in three
randomized paired blocks (seed 624), under the same host, Node, heap, memory and
20-second execution bounds. Each execution validated the chain above four times,
then a 1,200-event partial order with 600 unordered two-event layers and all four
edges between consecutive layers four times, then the same cyclic failure.
All six executions passed both parsed-answer digests and the cycle diagnostic.
Runtime dependencies were copied and isolated from concurrent checkout builds.
| Median measurement | Memoized closure | Rank/streaming |
| ---------------------------------------- | ---------------: | -------------: |
| Chain first elapsed milliseconds | 166.11 | 11.37 |
| Chain warm elapsed milliseconds | 154.25 | 5.32 |
| Chain first CPU milliseconds | 210.13 | 24.38 |
| Chain warm CPU milliseconds | 184.76 | 16.84 |
| Partial order first elapsed milliseconds | 49.44 | 116.54 |
| Partial order warm elapsed milliseconds | 50.67 | 110.29 |
| Partial order first CPU milliseconds | 72.47 | 137.69 |
| Partial order warm CPU milliseconds | 70.65 | 121.77 |
| Capture process elapsed milliseconds | 901.24 | 557.30 |
| Process CPU milliseconds before report | 1,101.12 | 665.03 |
| Peak process-family MiB | 391.13 | 127.72 |
First samples exclude module import and fixture preparation. Partial-order first
samples follow four chain validations, so they are first use of that fixture
within an already running process. Warm medians pool the three subsequent samples
from each of three processes. CPU and peak RSS cover the combined workload;
capture elapsed additionally includes Flameox capture startup and overhead.
Host caches were not flushed. The partial-order fixture is slower after this
change; these results establish a retention tradeoff, not universal acceleration.
The partial-order input is 165,590 JSON bytes, estimated at 52,783 tokens by tokenx
1.3.0; its parsed-answer SHA-256 is
`41d06e593813e497221c36793bf00d5fece7055081895b5f6c123ba6ad9a8227`.
Chain bytes, token estimate and answer digest remain unchanged. Actual agent
tokens, transport, catalog injection, model reasoning and provider costs remain
unmeasured. Both cohorts are component measurements, not complete investigations.
Preserved evidence:
`e29086af99afd8e67cbbd92e43d960ea40f73b4d318ea0cbca60f493575462b8`;
analysis `086133516f2acf031032d357a3ddb1ac611ce4f12556a24065a2becd567c5f7b`.
The baseline module SHA-256 is
`725409ce84902125941251037a8a214ccc3c1eec577e484c164c361ca580d693`;
the candidate module SHA-256 is
`2109f1096406e9ecac8bbf4f7ae8abb5113404790a9382487285d2831fcddf26`.
@@ -0,0 +1 @@
Statically inspect the supplied QGIS world-map GeoPackage at {{artifact_path}} using only the registered REA inspect_sqlite_database tool. Select the ordinary countries table with row_limit 1 and its default selected_table schema scope. Report the complete returned schema exactly as reported, including SQL, every declared column, indexes, views, triggers, completeness and explicit nulls. For the one returned sample, report only fid, NAME and ISO_A3 in that order, preserving the original tagged SQLite cells exactly: decimal integer strings, text values with bytes_hex and encoding, and SQL null. Preserve row_limit, returned_rows, truncated and order; row order is unspecified and this is not a complete country inventory. Preserve original artifact identity, wal including null, engine, database_encoding, both normalized-result producer limitations and outer Evidence limitations exactly including order. Do not return geometry blobs or unrequested sample cells, interpret boundaries/political labels as current facts, or infer missing values. Final JSON fields: artifact_sha256, artifact, wal, engine, database_encoding, schema, rows, producer_limitations, limitations, evidence_ids. Include real response Evidence IDs. Do not use shell or host file tools, network, other agents or execute QGIS. Stop once these facts are supported.
@@ -69,7 +69,7 @@ def scope_with_verified_counts(value, expected_scope, returned_counts):
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--domain', choices=['wasm', 'evm', 'android', 'flutter-aot', 'managed', 'managed-nonempty', 'firmware', 'inventory', 'javascript', 'native', 'browser'], required=True)
parser.add_argument('--domain', choices=['wasm', 'evm', 'android', 'flutter-aot', 'sqlite', 'managed', 'managed-nonempty', 'firmware', 'inventory', 'javascript', 'native', 'browser'], required=True)
parser.add_argument('--trial', type=Path, required=True)
parser.add_argument('--reference', type=Path)
parser.add_argument('--resource-log', type=Path, required=True)
@@ -133,6 +133,11 @@ def main():
for key, expected in {'artifact_bytes': n['artifact']['bytes'], 'bytecode_sha256': n['bytecode']['sha256'], 'bytecode_bytes': n['bytecode']['bytes'], 'functions': n['functions'], 'evidence_kind': n['evidence_kind'], 'discovery_completeness': n['discovery_completeness'], 'runtime_execution': n['runtime_execution'], 'bytecode_kind': n['bytecode']['kind'], 'hardfork': n['bytecode']['hardfork'], 'deployment_authenticity': n['bytecode']['deployment_authenticity']}.items():
check(key, answer.get(key), expected)
check('evm_original_limitations', answer.get('limitations'), n['limitations'])
elif domain == 'sqlite':
for key in ['artifact', 'wal', 'engine', 'database_encoding', 'schema', 'rows', 'producer_limitations', 'limitations']:
# Missing is not SQL/JSON null; booleans are not numeric counters.
check(key, (key in answer, json.dumps(answer.get(key), sort_keys=True)),
(True, json.dumps(reference[key], sort_keys=True)))
elif domain == 'flutter-aot':
for key in ['target', 'abi', 'libapp', 'string_pool', 'coverage', 'limitations']:
check(key, answer.get(key), reference[key])
@@ -0,0 +1,181 @@
# SQLite existing-control measurements
Continues [#2016](https://github.com/morluto/rea/issues/2016) after the Flutter
lane. The measured main runtime is pinned to
`172407787f1842dedbcd27b5385177595f786ce4`. Its only post-Flutter
commit repairs Windows transient runtime admission (#2467); this Linux round
does not establish Windows performance. No production/API change is proposed
merely because another summary or cache could be added.
Before PR integration, main advanced to
`bb540826d65316c164d2de8ba4ef10f81a8ad073`. Its five intervening commits cover
MCP archive contracts/documentation, an advertised-schema inventory test,
pwntools cleanup identity, concurrent Mach-O slice inspection, and transient
DMG verification retries. None changes the SQLite selector. Rebase and current
main verification do not relabel the frozen measurements below.
The committed post-format driver passes a real SDK smoke against the frozen
runtime. After rebase, a fresh build, real all/selected-column SDK pair and docs
check pass on the updated main. Selected facts match each other and the frozen
runtime; artifact bytes remain unchanged. These are integration checks, not
additional model performance pairs.
## Real application data and question
Use QGIS's bundled `world_map.gpkg`, unmodified from official immutable source
commit `e1223b8e9e71fc50ee995ed556f711c54ccc101b`. `corpus.json` pins its URL,
12,914,688-byte size, Git blob `c9ff80228dff0de93c9acdef16a97545ba360244`, and
SHA-256 `01d293159d1b77bd9212b5252587853f61aa9e25b2387135f34afaba8d057190`.
Streaming acquisition matches the upstream blob. The upstream resource install
list includes `data/*`; [its README](https://github.com/qgis/QGIS/blob/e1223b8e9e71fc50ee995ed556f711c54ccc101b/resources/data/README_world_map.md)
records data provenance. This is shipped application data, not a generated REA
fixture. QGIS is neither installed nor executed.
The source question asks for the complete returned `countries` table schema
and one sample's `fid`, `NAME`, `ISO_A3` tagged cells, in that order. It preserves
database/WAL identity, engine/encoding, exact original SQL and columns, integer
strings, text bytes, nulls, row limit/count, unspecified order, truncation and
both producer/outer limitations. It does not require geometry blobs or a full
country inventory. Neither political labels nor the sampled ordering are
inferred to be current or globally exhaustive.
SQLite already defaults to compact schema inventory without a table, detailed
selected-table schema with a table, and exposes exact `columns`. Selecting
columns avoids constructing unrelated cells without changing the requested
table schema. Thus compare existing all-visible columns against exact columns
on the same main runtime, rather than changing an old user's default or adding
a new delivery mechanism. Inventory scope cannot substitute for the detailed
schema requested by this question.
## Public SDK observations
Frozen runtime: main `172407787f1842dedbcd27b5385177595f786ce4`, compiled
dist/bridge file-table SHA-256
`72a4ac6c6db491c455f3e4a798f8f52ed5850c08831e7cdc9f25c1525a5a1d11`.
Node 24.18.0, SDK client/server 2.3.1, observed SQLite 3.53.1, WAL null. Driver,
provider settings/search-path fingerprints, artifact and locked dependencies
remain equal. Each workflow starts a fresh server and validates the selected
actual advertised schema. Host page caches are not flushed.
The initial default inventory returns 26 table entries, including explicit
virtual/shadow classifications, no rows and `not_requested` column details:
27,185 compact client-result JSON bytes. Selecting `countries` reveals eight
columns, including a geometry blob. Its one-row full reply is 485,277 bytes.
The selected request retains every requested declared-schema field and exact
tagged sample values while omitting unrelated sample cells.
| Trial | Reply bytes | SDK workflow seconds | Payload CPU seconds | Kernel peak bytes |
| ------------- | ----------: | -------------------: | ------------------: | ----------------: |
| sdk-all-1 | 485,277 | 11.820 | 12.755 | 562,884,608 |
| sdk-columns-1 | 18,509 | 14.017 | 13.715 | 557,510,656 |
| sdk-columns-2 | 18,509 | 15.826 | 14.162 | 548,343,808 |
| sdk-all-2 | 485,277 | 5.679 | 7.310 | 561,704,960 |
Both pairs make one successful call, pass settings/selected-fact comparison,
retain complete original schema and all requested typed cells, and preserve
source bytes. Reply bytes fall about 96.2%. These observations **do not show SDK
speed or CPU improvement**: selected workflows are slower in both pairs and
timing varies substantially. No cause is inferred from these data. Payload CPU
includes runner/catalog work outside the SDK workflow timer. Memory changes
are small and do not establish a general reduction. Raw replies and the exact
pre-format driver source are archived privately.
Reproduce with `measure.mjs --artifact qgis-world-map.gpkg` and
`--sqlite-selection all|columns`, using the same `--runtime-root`, `--revision`,
artifact path and `--schemas selected`. `--delivery` stays `complete`; this is
an existing request selection, not a new summary API. Original caller arguments
and schemas remain in the private receipts. The source question is
`questions/sqlite-qgis.txt` and the selected factual grader is
`review-model.py --domain sqlite`.
## Decision and scope
The initial four actual Codex attempts never executed SQLite: the client rejected
the tool because its private snapshot writes require approval and the isolated
driver lacked this exact tool's approval override. Every factual grade fails;
these are failed client attempts, not completed-task performance comparisons.
All profiles are removed. The existing experiment-only per-tool approval list
now includes `inspect_sqlite_database`; each corrected trial exposes only that
exact enabled tool. REA annotations, target authority and global Codex configuration
are unchanged. The [official configuration reference](https://learn.chatgpt.com/docs/config-file/config-reference)
documents the per-tool override. Corrected cohorts use fresh output directories
and retain the original failures separately.
Actual repeated Codex comparisons determine the task token/time benefit before
crediting smaller replies. An existing selector that answers the question is
evidence against adding another production mechanism. Keep negative outcomes
and unavailable coverage, including failure/retry costs and profile cleanup.
Four corrected serial trials use Codex CLI 0.161.0, requested `gpt-5.6-terra`
high, one enabled tool, identical owned context/discovery capture, and the same
frozen main runtime. The requested alias is not a verified server-resolved model.
Equal treatment instructions select the existing all-visible or exact-column
request; the source question and requested schema/sample facts stay fixed.
| Corrected trial | Input | Cached | Uncached | Output | Seconds | MCP calls | Facts |
| ------------------------ | -----: | -----: | -------: | -----: | ------: | --------: | ----- |
| approved-model-all-1 | 88,131 | 52,480 | 35,651 | 3,369 | 84.007 | 2 | pass |
| approved-model-columns-1 | 37,340 | 20,992 | 16,348 | 3,006 | 87.514 | 1 | pass |
| approved-model-columns-2 | 56,148 | 36,096 | 20,052 | 3,298 | 97.466 | 2 | pass |
| approved-model-all-2 | 88,340 | 52,480 | 35,860 | 3,084 | 113.847 | 2 | pass |
Both pairs pass every selected producer fact, exact SQL/schema/null/tagged
cells, observed response Evidence IDs, target-byte preservation and profile
cleanup. The grader checks producer observations, not independent geometry
semantics. Actual events contain only the chosen MCP tool and agent messages;
no shell or other agent execution appears. Usage is the final aggregate actually
reported by the CLI, one usage record per trial; cached input remains included
in total input. Output includes 853 / 357 / 523 / 664 reasoning tokens.
Total input falls **57.63% / 36.44%**, uncached input **54.14% / 44.08%**.
Time is mixed: columns are **4.17% slower** in pair 1 and **14.39% faster** in
pair 2. Output is 10.77% lower / 6.94% higher. Repeated calls remain counted;
pair 1 has two all-column calls versus one selected call, while pair 2 has two
calls on both sides. This supports scoped token savings from using the existing
selector, not a dependable speedup or a new API requirement. Client context and
usage are observed separately from raw SDK reply bytes.
Model payload CPU seconds / kernel peaks are 7.596 / 763,838,464;
7.772 / 761,143,296; 8.409 / 762,597,376; and 11.759 / 774,025,216. CPU is mixed
and memory differences are small. Payload cleanup/limit observations pass;
missing terminal counters keep every strict whole-resource grade false.
The original blocked attempts remain separate and never provide comparison
denominators. They each have one failed call, no SQLite result and failed facts:
| Blocked trial | Input | Cached | Uncached | Output | Seconds |
| --------------- | -----: | -----: | -------: | -----: | ------: |
| model-all-1 | 35,290 | 29,952 | 5,338 | 798 | 59.881 |
| model-columns-1 | 35,356 | 22,016 | 13,340 | 670 | 44.764 |
| model-columns-2 | 35,034 | 20,992 | 14,042 | 639 | 30.007 |
| model-all-2 | 35,104 | 20,992 | 14,112 | 673 | 25.132 |
**Decision: no production/API change.** Existing column selection materially
reduces unnecessary delivery and actual model input while retaining the complete
requested schema and exact cells. Existing documentation already recommends
table/column selection. No additional summary, schema-scope default change,
automatic field omission, plugin layer or cross-session cache is justified by
these results. Preserve the negative/mixed time, output and CPU findings.
Real one-shot CLI inventory, all-column sample and exact-column sample match
the corresponding full MCP normalized results, subjects, parameters and IDs;
the source digest remains unchanged. An independent Python binding using SQLite
3.40.1 opens this immutable artifact read-only and agrees with the exact stored
table SQL, eight declared columns and selected tagged sample cells. This is a
second engine/binding observation of this file, not a general row-order or
geometry-semantics guarantee. The six existing context/driver regression tests
pass; Python compilation and JavaScript syntax checks pass. No new product test
or production implementation is added for this measurement-only finding.
Manual review tightens the SQLite grader to distinguish missing fields from
explicit null and preserve JSON scalar types. Regrading all four stored model
answers still passes every selected fact; deliberate omission of `wal` and a
boolean substituted for numeric `row_limit` both fail. Original transcripts,
usage and previous review receipts stay intact; no model trial is rerun.
Heavy tasks remain serial under one CPU, unchanged 3 GiB early stop / 4 GiB hard
memory, no swap, owned task scopes and host guards. SDK payload observations
verify cleanup and zero phase limit events; final whole-scope counters remain
unknown. This does not establish live/WAL atomicity, all SQLite applications,
other runtimes/platforms or geospatial geometry semantics. Other packet/signal
and capture lanes remain incremental follow-ups using their existing controls.
@@ -0,0 +1,69 @@
# Windows empty-profile startup verification
Follow-up to [#2345](https://github.com/morluto/rea/pull/2345), including
@gefrezim's diagnosis of the import-time Windows home lookup.
## Source and boundary
Baseline: main `172407787f1842dedbcd27b5385177595f786ce4`. Verification used
the real Windows 11 10.0.22631.0 host, official Windows x64 Node 24.18.0,
locked production dependencies and REA's compiled public CLI / SDK 2.3.1 MCP
server. No IDA or Ghidra engine was used. Runtime copies and compact receipts
live outside the checkout; no account profile path or ambient environment is
recorded here.
The tested dispatcher SHA-256 is
`526105418604cc6e5a202a6275b700814e27450bfaffd45e7510530ba669de8d`.
Only the dispatcher changed in the deployed runtime. It treats an empty Windows
`USERPROFILE` as unset before dependency imports. Nonempty selections continue
to reach the existing home resolver; no additional lookup or fallback owner is
introduced.
## Real Windows observations
With `HOME` empty in each case, the baseline produced:
| USERPROFILE selection | `--version` | `--help` | MCP discovery |
| --------------------------- | ----------------------- | ----------------------- | ------------- |
| Unset | Exit 0 | Exit 0 | Healthy |
| Empty | Exit 1, `uv_os_homedir` | Exit 1, `uv_os_homedir` | Healthy |
| Nonempty test-owned profile | Exit 0 | Exit 0 | Healthy |
The corrected CLI passed all six invocations with no stderr. Version output
remained 6 bytes and help output 15,497 bytes. Real setup dry runs with empty
and nonempty profile selections both exited 0, planned their configuration
under the expected home, preserved existing configuration bytes and created no
configuration under the working directory. No installation was performed.
MCP was already healthy in the baseline; this change does not claim to repair
an MCP startup failure. Complete baseline discovery was 187 tools and 2,545,790
serialized bytes, SHA-256
`1e6bbca7f62852134eb703ea098f5a302d1fe333b8433475429f6fc1b1aba18f`.
All three corrected MCP controls also passed with no stderr and the same
catalogue bytes and digest.
The existing `homeDirectoryFallback` acceptance test exercises the import-time
failure through the public setup dry run. Adding it to the Windows curated lane
makes that regression execute on the affected platform. No duplicate startup
test was retained. Nine related tests passed, including the existing MCP doctor
and curated-lane path checks; Linux passing alone does not prove Windows behavior.
Review caught a missing package-scope registration for the newly listed test;
the original CI shard failed that ownership invariant. The path is now also
registered in `scripts/ci/scopes.mjs`, and all 84 scope checks plus the two
curated-path checks pass. Runtime source and its tested digest are unchanged.
## Efficiency and resource limits
Heavy host probes ran serially at BelowNormal priority, with one CPU and a
512 MiB CLI/server heap setting (256 MiB for the probe parent). Heap settings do
not bound total process memory. An exit-time `process.resourceUsage()` observer
recorded individual corrected CLI invocations: CPU time ranged from 7.968 to
9.874 seconds and peak working set from 299,956 to 301,852 KiB. These figures
exclude the probe parent and any observer overhead outside the CLI process.
The six observed wall times ranged from 8.656 to 10.961 seconds.
The baseline did not use that resource observer, and cold/warm caches were not
normalized. These observations establish functional behavior and bounded probe
scope; no CPU, memory or latency improvement is claimed. The fix adds one
constant-time environment check, no filesystem lookup, engine call, retry or
cache. Output bytes are not model tokens; actual model usage was not measured.
+1 -1
View File
@@ -167,7 +167,7 @@ main() {
[ "$installed_version" = "$version" ] || fail "installed version $installed_version does not match $version. Reinstall the requested version, then retry."
printf 'REA %s is installed.\n' "$version"
if [ "$start_setup" = true ] && [ -r /dev/tty ] && [ -w /dev/tty ] && [ -t 1 ]; then
if [ "$start_setup" = true ] && [ -r /dev/tty ] && [ -w /dev/tty ] && [ -t 1 ] && [ -t 2 ]; then
printf 'Starting REA setup...\n'
"$install_bin" setup </dev/tty >/dev/tty
else
+1
View File
@@ -167,6 +167,7 @@
"verify:legacy-simulator": "npm run build:cached && node scripts/verify/apple/legacy-simulator.mjs",
"verify:architecture-guards": "node scripts/verify/architecture-guards.mjs",
"verify:apktool": "npm run build:cached && node scripts/verify-apktool.mjs",
"verify:pcap": "npm run build:cached && node scripts/verify-pcap.mjs",
"verify:flutter": "npm run build:cached && node scripts/verify-flutter.mjs",
"verify:adb": "npm run build:cached && node scripts/verify-adb.mjs",
"verify:ghidra": "npm run build:cached && node scripts/verify-real-ghidra.mjs",
+1
View File
@@ -61,6 +61,7 @@ export const SCOPES = {
"tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts",
"tests/boundary/filesystem/setupUnavailableProvider.test.ts",
"tests/boundary/filesystem/windowsNpxRegistration.test.ts",
"tests/acceptance/setup/homeDirectoryFallback.test.ts",
"tests/boundary/filesystem/webScriptManifestPaths.test.ts",
"tests/boundary/process/ProcessEvidence.host.test.ts",
"tests/acceptance/webModuleCli.test.ts",
+53 -3
View File
@@ -10,6 +10,7 @@ import json
import os
from pathlib import Path
import secrets
import select
import stat
import subprocess
import tempfile
@@ -118,6 +119,7 @@ class XWindowAttributes(ctypes.Structure):
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser()
parser.add_argument("--probe", action="store_true")
parser.add_argument("--owner-stdin", action="store_true")
parser.add_argument("--hopper")
parser.add_argument("--socket")
parser.add_argument(
@@ -133,6 +135,43 @@ def parse_args() -> argparse.Namespace:
return parser.parse_args()
class OwnerLifetime:
"""The launcher's private pipe is a lifetime capability, independent of PIDs."""
def __init__(self, enabled: bool) -> None:
self._poll = select.poll() if enabled else None
if self._poll is not None:
# Node/libuv can implement piped stdio with a Unix socket pair.
mode = os.fstat(0).st_mode
if not (stat.S_ISFIFO(mode) or stat.S_ISSOCK(mode)):
raise AdapterFailure(
INVALID_LAUNCH_COMMAND,
"invalid_launch_command",
"owner lifetime requires a private stdin pipe",
)
self._poll.register(0, select.POLLIN | select.POLLHUP | select.POLLERR)
def alive(self) -> bool:
if self._poll is None:
return True
for _, events in self._poll.poll(0):
if events & (select.POLLHUP | select.POLLERR | select.POLLNVAL):
return False
if events & select.POLLIN and not os.read(0, 4096):
return False
return True
def wait_for_hopper(child: subprocess.Popen[bytes], owner: OwnerLifetime) -> int:
while owner.alive():
try:
return child.wait(timeout=0.1)
except subprocess.TimeoutExpired:
continue
# run_launch's finally owns cleanup of both direct, unreaped children.
return 0
def digest(path: Path) -> str:
value = hashlib.sha256()
with path.open("rb") as file:
@@ -572,6 +611,7 @@ def start_xvfb(
"-auth",
str(authority),
],
stdin=subprocess.DEVNULL,
stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE,
)
@@ -627,6 +667,9 @@ def run_launch(args: argparse.Namespace) -> int:
"invalid_launch_command",
"missing Hopper launch coordinates",
)
owner = OwnerLifetime(args.owner_stdin)
if not owner.alive():
return 0
hopper = Path(args.hopper)
socket = Path(args.socket)
try:
@@ -654,6 +697,8 @@ def run_launch(args: argparse.Namespace) -> int:
child: subprocess.Popen[bytes] | None = None
display: int | None = None
try:
if not owner.alive():
return 0
hopper_environment = dict(os.environ)
for name in ("WAYLAND_DISPLAY", "WAYLAND_SOCKET"):
hopper_environment.pop(name, None)
@@ -666,7 +711,10 @@ def run_launch(args: argparse.Namespace) -> int:
"XDG_SESSION_TYPE": "x11",
}
)
child = subprocess.Popen(command, env=hopper_environment)
# Children must not consume or retain REA's lifetime pipe.
child = subprocess.Popen(
command, env=hopper_environment, stdin=subprocess.DEVNULL
)
leader = os.getpid()
os.environ["XAUTHORITY"] = str(authority)
@@ -702,10 +750,12 @@ def run_launch(args: argparse.Namespace) -> int:
observed: list[tuple[int, int, int, int, int]] = []
dialog: tuple[int, int, int, int, int] | None = None
while time.monotonic() < deadline:
if not owner.alive():
return 0
if socket.exists():
x11.XCloseDisplay(display)
display = None
return child.wait()
return wait_for_hopper(child, owner)
if not hopper_is_owned(leader, hopper):
if child.poll() is None:
time.sleep(0.1)
@@ -749,7 +799,7 @@ def run_launch(args: argparse.Namespace) -> int:
x11.XFlush(display)
x11.XCloseDisplay(display)
display = None
return child.wait()
return wait_for_hopper(child, owner)
except AdapterFailure as error:
raise error.with_stderr(capture)
finally:
+9
View File
@@ -52,6 +52,11 @@ export const toolFamilyCatalog = (sources) => {
surface: "android-provider",
contracts: sources.androidContracts.ANDROID_TOOL_CONTRACTS,
},
{
id: "pcap",
surface: "pcap-provider",
contracts: sources.pcapContracts.PCAP_TOOL_CONTRACTS,
},
{
id: "flutter",
surface: "flutter-provider",
@@ -251,6 +256,10 @@ export const providerCatalog = (sources) => {
identity: sources.androidProvider.JADX_PROVIDER_IDENTITY,
contracts: sources.androidContracts.ANDROID_TOOL_CONTRACTS,
},
{
identity: sources.pcapProvider.PCAP_PROVIDER_IDENTITY,
contracts: sources.pcapContracts.PCAP_TOOL_CONTRACTS,
},
{
identity: sources.flutterProvider.FLUTTER_PROVIDER_IDENTITY,
contracts: sources.flutterContracts.FLUTTER_TOOL_CONTRACTS,
+2 -2
View File
@@ -68,8 +68,8 @@ const SOURCE_PATHS = {
flutterProvider: "dist/flutter/FlutterBuildProvider.js",
androidProvider: "dist/android/JadxRelease.js",
apktoolContracts: "dist/contracts/apktool/apktoolToolContracts.js",
flutterContracts: "dist/contracts/flutter/flutterToolContracts.js",
flutterProvider: "dist/flutter/FlutterBuildProvider.js",
pcapContracts: "dist/contracts/pcap/pcapToolContracts.js",
pcapProvider: "dist/pcap/PcapProvider.js",
apktoolProvider: "dist/apktool/ApktoolProvider.js",
jebContracts: "dist/contracts/jeb/jebToolContracts.js",
jebProvider: "dist/jeb/JebProvider.js",
+130
View File
@@ -0,0 +1,130 @@
import assert from "node:assert/strict";
import { execFile } from "node:child_process";
import { setTimeout as delay } from "node:timers/promises";
import { promisify } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { verifyNoTokenOwnedProcesses } from "../../dist/process/ProcessOwnership.js";
import {
observeProcessStartIdentity,
signalProcessWithStartIdentity,
} from "../../dist/process/ProcessOwnershipObservation.js";
import { startUnrelatedHopperSentinel } from "./unrelated-hopper-sentinel.mjs";
import {
requireMcpEvidenceResult,
requireMcpLifecycleResult,
} from "./mcp-verifier-results.mjs";
const execFileAsync = promisify(execFile);
/** Prove actual MCP owner exit releases the demo singleton for subsequent CLI/MCP sessions. */
export async function verifyLinuxHopperOwnerExit({
primary,
secondary,
serverCommand,
serverArgs,
environment,
}) {
const sentinel = await startUnrelatedHopperSentinel();
const outcomes = [];
try {
for (const mode of ["graceful", "SIGKILL"]) {
const transport = new StdioClientTransport({
command: serverCommand,
args: serverArgs,
env: environment,
stderr: "pipe",
});
// Drain launcher diagnostics even on the deliberately interrupted owner.
transport.stderr?.resume();
const client = new Client({ name: "hopper-owner-exit", version: "1" });
const options = { timeout: 180_000 };
try {
await client.connect(transport);
const ownerPid = transport.pid;
assert.ok(ownerPid !== null);
const owner = await observeProcessStartIdentity(ownerPid);
assert.equal(owner?.state, "readable");
requireMcpLifecycleResult(
await client.callTool(
{
name: "open_binary",
arguments: { path: primary },
},
options,
),
"open_binary",
);
const procedures = requireMcpEvidenceResult(
await client.callTool(
{ name: "list_procedures", arguments: {} },
options,
),
"list_procedures",
);
assert.ok(procedures.length > 0);
const status = requireMcpLifecycleResult(
await client.callTool(
{ name: "binary_session", arguments: {} },
options,
),
"binary_session",
);
const runId = status.analysis_run?.run_id;
assert.equal(typeof runId, "string");
assert.equal(status.analysis_provider_binding?.provider.id, "hopper");
assert.equal((await verifyNoTokenOwnedProcesses(runId)).cleaned, false);
if (mode === "graceful") await client.close();
else {
assert.equal(
await signalProcessWithStartIdentity(
ownerPid,
owner.identity,
"SIGKILL",
),
"signaled",
);
}
const deadline = Date.now() + 10_000;
let cleanup;
do {
cleanup = await verifyNoTokenOwnedProcesses(runId);
if (cleanup.cleaned) break;
await delay(100);
} while (Date.now() < deadline);
assert.equal(cleanup.cleaned, true, cleanup.reason);
// The SIGKILL case must recover the stale lease for both target identities.
for (const target of [primary, secondary]) {
const result = await execFileAsync(
process.execPath,
[
"scripts/rea.mjs",
"inspect",
target,
"--provider",
"hopper",
"--format",
"json",
],
{ env: environment, timeout: 180_000, maxBuffer: 16 * 1024 * 1024 },
);
assert.ok(JSON.parse(result.stdout).normalized_result);
}
outcomes.push({
mode,
ownedProcessesStopped: true,
sameTargetReopened: true,
differentTargetReopened: true,
});
} finally {
await client.close();
await transport.close();
}
}
} finally {
assert.equal(await sentinel.verifyAndClose(), true);
}
return { outcomes, unrelatedHopperSurvived: true };
}
+6
View File
@@ -16,6 +16,12 @@ const isMcpMode =
const isMcpDoctorMode = args[0] === "mcp" && args[1] === "doctor";
const packageRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
// Windows home lookup throws for an empty USERPROFILE. Dependencies resolve
// home during import, before REA's own resolver can apply its account fallback.
if (process.platform === "win32" && process.env.USERPROFILE === "") {
delete process.env.USERPROFILE;
}
// Playwright's registry assumes one of its three desktop host platforms when
// it chooses the browser cache directory. Termux reports `android`, although
// its Node runtime and filesystem are POSIX-compatible. Keep the cache path
+46
View File
@@ -236,6 +236,52 @@ try {
const hci = await inspect({ path: bluetooth, fields: ["bthci_cmd.opcode"] });
assert.deepEqual(hci.packets[0]["bthci_cmd.opcode"], ["0x0c03"]);
assert.deepEqual(await readFile(path), source);
// Exercise the incremental summary workflow against the same owned bytes
// using real installed Wireshark tools; these inputs are format fixtures.
const pcapTool = (await client.listTools()).tools.find(
(t) => t.name === "inspect_pcap_capture",
);
assert.ok(pcapTool?.outputSchema);
const pcapAjv = new Ajv2020({ strict: false, validateFormats: false });
assert.ok(pcapAjv.validateSchema(pcapTool.inputSchema));
assert.ok(pcapAjv.validateSchema(pcapTool.outputSchema));
for (const capturePath of [path, bluetooth]) {
const response = await client.callTool({
name: pcapTool.name,
arguments: { path: capturePath, detail: "summary" },
});
assert.notEqual(response.isError, true, JSON.stringify(response));
assert.ok(
pcapAjv.validate(pcapTool.outputSchema, response.structuredContent),
pcapAjv.errorsText(),
);
const summary = parseEvidence(response.structuredContent).normalized_result;
assert.equal(
summary.summary.target.sha256,
createHash("sha256")
.update(await readFile(capturePath))
.digest("hex"),
);
const source = {
kind: "retained-evidence",
evidence_id: summary.parent_evidence_id,
};
const page = await client.callTool({
name: "inspect_analysis_view",
arguments: {
source,
view: { kind: "pcap", facet: "protocols", limit: 10 },
},
});
assert.notEqual(page.isError, true, JSON.stringify(page));
assert.equal(
parseEvidence(page.structuredContent).normalized_result
.parent_evidence_id,
source.evidence_id,
);
cases++;
}
assert.deepEqual(await readFile(bluetooth), bluetoothSource);
for (const input of [
{ path, fields: ["frame.number"], display_filter: "this is not a filter" },
+131
View File
@@ -0,0 +1,131 @@
import assert from "node:assert/strict";
import { execFile } from "node:child_process";
import { mkdtemp, readdir, rmdir } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { promisify } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { toolContract } from "../dist/contracts/toolContracts.js";
import { hashStableFile } from "../dist/filesystem/StableFileHash.js";
import { mcpTextValue } from "./lib/mcp-verifier-results.mjs";
const args = process.argv.slice(2);
if (args.includes("--help") || args.includes("-h")) {
console.log(
"Usage: npm run verify:pcap -- [--pcap PATH]\nRequires caller-supplied tshark/capinfos. Without --pcap, only client readiness is verified.",
);
process.exit(0);
}
if (!(args.length === 0 || (args.length === 2 && args[0] === "--pcap")))
throw new Error("Expected --pcap PATH");
const path = args[1] === undefined ? null : resolve(args[1]);
// This parent is shared by independent CLI/MCP owners. A leased Windows
// runtime root would prevent their native admission of its directory handle.
const root = await mkdtemp(join(tmpdir(), "rea-pcap-verifier-"));
const environment = Object.fromEntries(
Object.entries(process.env).filter(([, value]) => typeof value === "string"),
);
Object.assign(environment, {
TMPDIR: root,
TMP: root,
TEMP: root,
});
const entrypoint = resolve("scripts/rea.mjs");
const execute = promisify(execFile);
const client = new Client({ name: "pcap-real-verifier", version: "1" });
const transport = new StdioClientTransport({
command: process.execPath,
args: [entrypoint, "mcp"],
env: environment,
stderr: "pipe",
});
transport.stderr?.resume();
try {
await client.connect(transport);
const readiness = await run("inspect_pcap_client", {});
assert(readiness.client.tshark_version && readiness.client.capinfos_version);
if (path === null)
console.log("[unverified] Capture inspection: no --pcap was supplied");
else {
const identity = await hashStableFile(path);
const result = await run("inspect_pcap_capture", { path });
assert.deepEqual(result.target, { path, ...identity });
assert(result.packet_count >= 0);
assert(
result.endpoints.every(
(entry) => entry.packet_count <= result.packet_count,
),
);
assert(
result.protocol_distribution.every(
(entry) => entry.packet_count <= result.packet_count,
),
);
const partial =
result.endpoint_count > result.endpoints.length ||
result.dns_query_count > result.dns_queries.length ||
result.tls_sni_count > result.tls_sni_values.length ||
result.protocol_count > result.protocol_distribution.length ||
result.http_request_uri_count > result.http_request_uris.length ||
result.unattributed_dns_packet_count > 0 ||
result.dns_queries.some(
(entry) =>
entry.response_address_count > entry.response_addresses.length,
);
assert.equal(result.coverage, partial ? "partial" : "complete");
assert.deepEqual(
await hashStableFile(path),
identity,
"Input capture must remain unchanged",
);
console.log(
`[ok] ${result.packet_count} packets; ${result.endpoint_count} addresses; coverage=${result.coverage}`,
);
}
} finally {
try {
await client.close();
} finally {
await transport.close();
}
assert.deepEqual(
await readdir(root),
[],
`Owned capture resources remain under ${root}`,
);
await rmdir(root);
}
console.log("All asserted pcap CLI/MCP checks passed");
async function run(name, input) {
const contract = toolContract(name);
const { stdout } = await execute(
process.execPath,
[
entrypoint,
name.replaceAll("_", "-"),
...(input.path === undefined ? [] : [input.path]),
"--json",
],
{ env: environment, timeout: 360_000, maxBuffer: 8 * 1024 * 1024 },
);
const cli = contract.outputSchema.parse(JSON.parse(stdout));
const response = await client.callTool(
{ name, arguments: input },
{ timeout: 360_000 },
);
assert.notEqual(response.isError, true, mcpTextValue(response));
const mcp = contract.outputSchema.parse(JSON.parse(mcpTextValue(response)));
assert.deepEqual(mcp.normalized_result, cli.normalized_result);
assert.deepEqual(mcp.subject, cli.subject);
if (input.path !== undefined) {
assert.equal(cli.subject.local_path, input.path);
assert.equal(
cli.subject.digest.sha256,
cli.normalized_result.target.sha256,
);
}
console.log(`[ok] ${name}: named output schema and CLI/MCP Evidence parity`);
return cli.normalized_result;
}
+9
View File
@@ -47,6 +47,7 @@ import {
} from "./lib/real-hopper-boundaries.mjs";
import { startUnrelatedHopperSentinel } from "./lib/unrelated-hopper-sentinel.mjs";
import { snapshotHopperRuntime } from "./lib/real-hopper-cleanup.mjs";
import { verifyLinuxHopperOwnerExit } from "./lib/real-hopper-owner-exit.mjs";
const execFileAsync = promisify(execFile);
const verifierRun = createVerifierRun();
const timeout = 180_000;
@@ -577,6 +578,14 @@ if (sessionsAfter.length > 0) {
}
if (summary === undefined)
throw new Error("Real-Hopper verification did not produce a summary");
if (process.platform === "linux")
summary.ownerExit = await verifyLinuxHopperOwnerExit({
primary: targetA,
secondary: targetB,
serverCommand,
serverArgs,
environment: serverEnvironment,
});
const completedVerifierRun = await completeVerifierRun(verifierRun);
if (
completedVerifierRun.process_lineage.status !== "verified" ||
+47 -1
View File
@@ -96,6 +96,7 @@ const decodeStub = (
readonly withStrings?: boolean;
readonly withGerman?: boolean;
readonly failDecode?: boolean;
readonly ymlFile?: string;
readonly manifestFile?: string;
readonly stringsFile?: string;
readonly stringsXml?: string;
@@ -104,6 +105,10 @@ const decodeStub = (
const withStrings = options.withStrings ?? true;
const withGerman = options.withGerman ?? true;
const stringsXml = options.stringsXml ?? REAL_STRINGS;
const ymlWrite =
options.ymlFile === undefined
? `printf '%s\\n' '${REAL_YML.replace(/'/gu, "'\\''")}' > "$5/apktool.yml"`
: `cp '${options.ymlFile}' "$5/apktool.yml"`;
const manifestWrite =
options.manifestFile === undefined
? `printf '%s\\n' '${REAL_MANIFEST.replace(/'/gu, "'\\''")}' > "$5/AndroidManifest.xml"`
@@ -117,7 +122,7 @@ if [ "$1" = "d" ]; then
${
options.failDecode === true
? `printf 'Exception in thread "main" brut.androlib.AndrolibException: Could not decode the APK\\n' >&2; exit 1;`
: `printf '%s\\n' '${REAL_YML.replace(/'/gu, "'\\''")}' > "$5/apktool.yml"
: `${ymlWrite}
${manifestWrite}
mkdir -p "$5/res/values"${withGerman ? ' "$5/res/values-de"' : ""}
${withStrings ? stringsWrite : ""}
@@ -192,6 +197,47 @@ it("decodes resources with target identity, metadata, and strings", async () =>
expect(result.coverage).toBe("complete");
});
it.each([
{
name: "malformed UTF-8",
bytes: Buffer.concat([
Buffer.from("versionInfo:\n versionName: com."),
Buffer.from([0xff]),
Buffer.from(".app\n"),
]),
valid: false,
},
{
name: "a literal replacement character",
bytes: Buffer.from("versionInfo:\n versionName: com.�.app\n", "utf8"),
valid: true,
},
])("handles $name in producer metadata", async ({ bytes, valid }) => {
const { command, root } = await writeApktoolStub((root) =>
decodeStub({ ymlFile: join(root, "malformed-apktool.yml") })(root),
);
await writeFile(join(root, "malformed-apktool.yml"), bytes);
const apk = await apkFixture(root);
if (valid) {
const result = await evidenceResult(providerOver(command), {
operation: "decode_android_resources",
input: { path: apk.path, include_strings: false },
});
expect(result.metadata).toMatchObject({ version_name: "com.�.app" });
expect(result.coverage).toBe("complete");
return;
}
const result = await providerOver(command).execute({
operation: "decode_android_resources",
input: { path: apk.path, include_strings: false },
});
if (result.ok) throw new Error("expected invalid producer output");
const projected = projectAnalysisError(result.error);
expect(projected).toMatchObject({ code: "unreadable_output" });
expect(result.error.message).toContain("invalid UTF-8");
expect(result.error.message).toContain("apktool.yml");
});
it("preserves empty resource strings and reports markup omissions as partial", async () => {
const stringsXml =
'<resources><string name="empty"/><string name="raw"> A &amp; B </string><string name="styled">Hello <b>world</b></string></resources>';
+14 -7
View File
@@ -117,13 +117,20 @@ const readBoundedText = async (
throwIfCancelled(operation, signal);
const truncated = length > budget;
// A prefix may end within a UTF-8 character; omit those trailing bytes.
const decoder = new TextDecoder("utf-8");
return {
text: decoder.decode(bytes.subarray(0, Math.min(length, budget)), {
stream: truncated,
}),
truncated,
};
try {
const decoder = new TextDecoder("utf-8", { fatal: true });
return {
text: decoder.decode(bytes.subarray(0, Math.min(length, budget)), {
stream: truncated,
}),
truncated,
};
} catch (cause: unknown) {
throw new AnalysisProtocolError(
`apktool produced invalid UTF-8 in ${path}`,
{ cause },
);
}
} finally {
await file.close();
}
+44 -30
View File
@@ -54,6 +54,15 @@ export interface AnalysisExecution {
readonly subject: EvidenceSubjectTarget | null;
}
interface CreateAnalysisExecutionOptions {
readonly rawResult?: unknown;
readonly analysisProfile?: AnalysisProfileCommitment;
readonly limitations?: readonly string[];
readonly relationshipCoverage?: RelationshipCoverage;
readonly locations?: readonly EvidenceLocation[];
readonly subject?: EvidenceSubjectTarget;
}
/**
* Build a validated atomic provider observation at an adapter boundary.
*
@@ -61,38 +70,43 @@ export interface AnalysisExecution {
* `result`. Omitting it records `null` rather than a second complete copy of
* the normalized result in every Evidence record and tool response.
*/
export const createAnalysisExecution = (
export function createAnalysisExecution<Subject extends EvidenceSubjectTarget>(
result: unknown,
provider: ProviderIdentity,
options: {
readonly rawResult?: unknown;
readonly analysisProfile?: AnalysisProfileCommitment;
readonly limitations?: readonly string[];
readonly relationshipCoverage?: RelationshipCoverage;
readonly locations?: readonly EvidenceLocation[];
readonly subject?: EvidenceSubjectTarget;
} = {},
): AnalysisExecution => ({
result: jsonValueSchema.parse(result),
rawResult:
options.rawResult === undefined
? null
: jsonValueSchema.parse(options.rawResult),
provider,
...(options.analysisProfile === undefined
? {}
: { analysisProfile: structuredClone(options.analysisProfile) }),
limitations: [...(options.limitations ?? [])],
...(options.relationshipCoverage === undefined
? {}
: {
relationshipCoverage: relationshipCoverageSchema.parse(
options.relationshipCoverage,
),
}),
locations: [...(options.locations ?? [])],
subject: options.subject ?? null,
});
options: CreateAnalysisExecutionOptions & { readonly subject: Subject },
): Omit<AnalysisExecution, "subject"> & { readonly subject: Subject };
export function createAnalysisExecution(
result: unknown,
provider: ProviderIdentity,
options?: CreateAnalysisExecutionOptions,
): AnalysisExecution;
export function createAnalysisExecution(
result: unknown,
provider: ProviderIdentity,
options: CreateAnalysisExecutionOptions = {},
): AnalysisExecution {
return {
result: jsonValueSchema.parse(result),
rawResult:
options.rawResult === undefined
? null
: jsonValueSchema.parse(options.rawResult),
provider,
...(options.analysisProfile === undefined
? {}
: { analysisProfile: structuredClone(options.analysisProfile) }),
limitations: [...(options.limitations ?? [])],
...(options.relationshipCoverage === undefined
? {}
: {
relationshipCoverage: relationshipCoverageSchema.parse(
options.relationshipCoverage,
),
}),
locations: [...(options.locations ?? [])],
subject: options.subject ?? null,
};
}
/** Provider-neutral application capability for one analysis operation. */
export interface AnalysisOperationPort {
+1 -1
View File
@@ -481,7 +481,7 @@ const workflowAvailabilityFor = ({
remediation: "Run process capture on a supported Linux or macOS host.",
};
if (kind === "application") return { reason: "available", remediation: null };
if (kind === "flutter-provider")
if (kind === "pcap-provider" || kind === "flutter-provider")
return { reason: "available", remediation: null };
const electron = electronProviderAvailability(name, kind, policy);
if (electron !== null) return electron;
+65 -41
View File
@@ -7,6 +7,7 @@ import {
clientServerForcedEnabled,
clientServerListedDisabled,
effectiveClientServer,
type ClientConfigurationDocument,
} from "./ClientConfigurationDocument.js";
import { access } from "node:fs/promises";
import { resolve } from "node:path";
@@ -74,24 +75,48 @@ const registrationSchema = z
})
.passthrough();
/** Inspect supported client registrations without reading their environment. */
interface ClientRegistrationReadOptions {
readonly platform?: NodeJS.Platform;
readonly environment?: NodeJS.ProcessEnv;
readonly resources?: ArtifactResourceScope;
}
/** Registration facts shared by diagnostics and update maintenance. */
export interface ClientRegistrationInspection {
readonly status: ClientRegistrationStatus;
readonly activation: "enabled" | "disabled" | "unknown";
}
/** Project diagnostics from the same inspection used to select maintenance. */
export const readClientRegistrationStatuses = async (
home: string,
currentCommandPath: string = resolve(process.argv[1] ?? "unknown"),
options: {
readonly platform?: NodeJS.Platform;
readonly environment?: NodeJS.ProcessEnv;
readonly resources?: ArtifactResourceScope;
} = {},
): Promise<readonly ClientRegistrationStatus[]> => {
const statuses: ClientRegistrationStatus[] = [];
options: ClientRegistrationReadOptions = {},
): Promise<readonly ClientRegistrationStatus[]> =>
(
await readClientRegistrationInspections(home, currentCommandPath, options)
).map(({ status }) => status);
/** Parse effective registrations and their client-specific activation once. */
export const readClientRegistrationInspections = async (
home: string,
currentCommandPath: string = resolve(process.argv[1] ?? "unknown"),
options: ClientRegistrationReadOptions = {},
): Promise<readonly ClientRegistrationInspection[]> => {
const inspections: ClientRegistrationInspection[] = [];
const record = (
status: ClientRegistrationStatus,
activation: ClientRegistrationInspection["activation"] = "unknown",
): void => {
inspections.push({ status, activation });
};
for (const client of supportedClients(
home,
options.platform,
options.environment,
)) {
if (client.configPathError !== undefined) {
statuses.push({
record({
client: client.name,
config_path: client.configPath,
command: [],
@@ -104,7 +129,7 @@ export const readClientRegistrationStatuses = async (
const manualRemediation = manualRegistrationRemediation(client.name);
if (client.format === "unsupported") {
if (manualRemediation !== undefined)
statuses.push({
record({
client: client.name,
config_path: client.markerPath ?? client.configPath,
command: [],
@@ -119,7 +144,7 @@ export const readClientRegistrationStatuses = async (
options.resources,
);
if (!files.ok) {
statuses.push({
record({
client: client.name,
config_path: client.configPath,
command: [],
@@ -130,9 +155,7 @@ export const readClientRegistrationStatuses = async (
}
const parsed = effectiveClientConfiguration(files.value);
if (parsed === undefined) {
statuses.push(
unavailableStatus(client.name, client.configPath, "missing"),
);
record(unavailableStatus(client.name, client.configPath, "missing"));
continue;
}
const policy = await readClientPolicyBlock(
@@ -142,7 +165,7 @@ export const readClientRegistrationStatuses = async (
options.resources,
);
if (!policy.ok) {
statuses.push({
record({
client: client.name,
config_path: client.configPath,
command: [],
@@ -159,7 +182,7 @@ export const readClientRegistrationStatuses = async (
? dshDocument(parsed.document).editabilityError
: undefined;
if (editabilityError !== undefined) {
statuses.push({
record({
client: client.name,
config_path: client.configPath,
command: [],
@@ -168,9 +191,7 @@ export const readClientRegistrationStatuses = async (
});
continue;
}
statuses.push(
unavailableStatus(client.name, client.configPath, "missing"),
);
record(unavailableStatus(client.name, client.configPath, "missing"));
continue;
}
const registration = parseRegistration(raw, client);
@@ -178,21 +199,18 @@ export const readClientRegistrationStatuses = async (
registration.command,
...registration.args,
];
statuses.push(
const activation = registrationActivation(registration, client, parsed);
record(
configuredStatus(
client.name,
client.configPath,
command,
registrationAligned(
registration,
client,
currentCommandPath,
options.platform ?? process.platform,
clientServerForcedEnabled(parsed, PRODUCT_IDENTITY.mcpServerKey),
) &&
!clientServerListedDisabled(
parsed,
PRODUCT_IDENTITY.mcpServerKey,
activation === "enabled" &&
registrationAligned(
registration,
client,
currentCommandPath,
options.platform ?? process.platform,
) &&
(parsed.dialect !== "dsh" || dshDocument(parsed.document).ready) &&
policyBlock === undefined
@@ -200,9 +218,10 @@ export const readClientRegistrationStatuses = async (
: "stale",
policyBlock,
),
activation,
);
} catch (cause: unknown) {
statuses.push(
record(
unavailableStatus(
client.name,
client.configPath,
@@ -211,26 +230,31 @@ export const readClientRegistrationStatuses = async (
);
}
}
return statuses.sort((left, right) =>
compareUnicodeCodePoints(left.client, right.client),
return inspections.sort((left, right) =>
compareUnicodeCodePoints(left.status.client, right.status.client),
);
};
const registrationActivation = (
registration: z.output<typeof registrationSchema>,
client: SetupClient,
parsed: ClientConfigurationDocument,
): "enabled" | "disabled" =>
(client.format !== "grok" && registration.disabled === true) ||
(registration.enabled === false &&
!clientServerForcedEnabled(parsed, PRODUCT_IDENTITY.mcpServerKey)) ||
(client.format === "zcode" && registration.enable === false) ||
clientServerListedDisabled(parsed, PRODUCT_IDENTITY.mcpServerKey)
? "disabled"
: "enabled";
const registrationAligned = (
registration: z.output<typeof registrationSchema>,
client: SetupClient,
currentCommandPath: string,
platform: NodeJS.Platform,
forcedEnabled = false,
): boolean => {
const command = [registration.command, ...registration.args];
if (
(client.format !== "grok" && registration.disabled === true) ||
(registration.enabled === false && !forcedEnabled) ||
// ZCode's native flag is `enable`; other dialects leave that field opaque.
(client.format === "zcode" && registration.enable === false)
)
return false;
if (!isOwnedClientRegistrationCommand(command, currentCommandPath))
return false;
if (
+12 -39
View File
@@ -1,19 +1,10 @@
import { z } from "zod";
import { PRODUCT_IDENTITY } from "../identity.js";
import { isOwnedClientRegistrationCommand } from "./ClientRegistrationIdentity.js";
import { readClientRegistrationStatuses } from "./ClientRegistrationStatus.js";
import {
clientServerForcedEnabled,
clientServerListedDisabled,
effectiveClientServer,
parseClientConfiguration,
} from "./ClientConfigurationDocument.js";
import { supportedClients } from "./SupportedClients.js";
import { readClientRegistrationInspections } from "./ClientRegistrationStatus.js";
import { existingSkillDestinations } from "./SetupSkill.js";
import type { SetupAction } from "./SetupTypes.js";
import type { Result } from "../domain/result.js";
import { readRegularFileText } from "./RegularFileRead.js";
import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
/** Existing REA integrations selected for maintenance, without discovering new targets. */
@@ -61,41 +52,23 @@ export const existingMaintenanceScope = async (
environment: Readonly<NodeJS.ProcessEnv>,
resources?: ArtifactResourceScope,
): Promise<MaintenanceScope> => {
const registrations = await readClientRegistrationStatuses(home, entryPoint, {
environment,
...(resources === undefined ? {} : { resources }),
});
const supported = supportedClients(home, process.platform, environment);
const registrations = await readClientRegistrationInspections(
home,
entryPoint,
{
environment,
...(resources === undefined ? {} : { resources }),
},
);
const clients: string[] = [];
for (const registration of registrations) {
for (const { status: registration, activation } of registrations) {
if (
activation !== "enabled" ||
(registration.state !== "aligned" && registration.state !== "stale") ||
!isOwnedClientRegistrationCommand(registration.command, entryPoint)
)
continue;
const client = supported.find(({ name }) => name === registration.client);
if (client === undefined) continue;
const parsed = parseClientConfiguration(
await readRegularFileText(
client.configPath,
resources === undefined ? {} : { resources },
),
client.format,
);
const enabled = z
.object({
enabled: z.boolean().optional(),
disabled: z.boolean().optional(),
})
.parse(effectiveClientServer(parsed, PRODUCT_IDENTITY.mcpServerKey));
// Root disable lists override enabled entries and OMP's force-on allowlist.
if (
(enabled.enabled !== false ||
clientServerForcedEnabled(parsed, PRODUCT_IDENTITY.mcpServerKey)) &&
enabled.disabled !== true &&
!clientServerListedDisabled(parsed, PRODUCT_IDENTITY.mcpServerKey)
)
clients.push(client.name);
clients.push(registration.client);
}
const skillDestinations = await existingSkillDestinations(
home,
+7 -1
View File
@@ -4,13 +4,19 @@ import type {
} from "../AnalysisProvider.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import type { InspectEvmInterfaceInput } from "../../domain/evm/evmInterface.js";
import type { EvidenceSubjectTarget } from "../../domain/evidence.js";
import type { Result } from "../../domain/result.js";
/** Successful EVM interface recovery must identify the selected local file. */
export type EvmInterfaceExecution = Omit<AnalysisExecution, "subject"> & {
readonly subject: EvidenceSubjectTarget & { readonly format: "file" };
};
/** Replaceable offline interface-recovery engine, independent of wallets and chain clients. */
export interface EvmInterfacePort {
close?(): Promise<void>;
inspect(
input: InspectEvmInterfaceInput,
options?: ExecutionOptions,
): Promise<Result<AnalysisExecution, AnalysisError>>;
): Promise<Result<EvmInterfaceExecution, AnalysisError>>;
}
@@ -1,5 +1,6 @@
import { expect, it } from "vitest";
import { EvmInterfaceService } from "./EvmInterfaceService.js";
import type { EvmInterfaceExecution } from "./EvmInterfacePort.js";
import {
evmInterfaceExecution,
evmInterfaceFixture,
@@ -43,7 +44,8 @@ it.each(["carrier", "encoding", "digest", "no-subject"])(
},
};
const service = new EvmInterfaceService({
inspect: () => Promise.resolve(ok(execution)),
inspect: () =>
Promise.resolve(ok(execution as unknown as EvmInterfaceExecution)),
});
expect(
await service.inspect({
@@ -67,6 +67,7 @@ export class EvmInterfaceService {
!report.success ||
report.data.artifact.path !== input.data.path ||
report.data.artifact.encoding !== input.data.encoding ||
execution.subject?.format !== "file" ||
execution.subject?.path !== input.data.path ||
execution.subject.sha256 !== report.data.artifact.sha256
)
@@ -78,6 +78,51 @@ const update = (
});
describe("residual unknown registry reads and identity", () => {
it("seals mutation Evidence without freezing caller-owned inputs", () => {
const store = ledger();
const recordMutation = mutation("mutable-record-mutation");
const recorded = store.recordUnknown(
input("Does record mutation evidence stay stable?"),
recordMutation,
);
if (!recorded.ok) throw recorded.error;
const updateMutation = mutation("mutable-update-mutation");
const updated = store.updateUnknown(update(recorded.value), updateMutation);
if (!updated.ok) throw updated.error;
const atomicEvidence = evidence("mutable-atomic-evidence");
const atomicMutation = mutation("mutable-atomic-mutation");
const atomicallyRecorded = store.recordWithUnknown(
atomicEvidence,
input("Does atomic evidence stay stable?"),
atomicMutation,
);
if (!atomicallyRecorded.ok) throw atomicallyRecorded.error;
for (const callerEvidence of [
recordMutation,
updateMutation,
atomicEvidence,
atomicMutation,
]) {
expect(Object.isFrozen(callerEvidence)).toBe(false);
const retained = store.forAnalysis(callerEvidence.evidence_id);
if (retained === undefined) throw new Error("Missing retained Evidence");
const originalResult = structuredClone(retained.normalized_result);
expect(Object.isFrozen(retained)).toBe(true);
Reflect.set(callerEvidence, "operation", "caller mutation");
expect(
Reflect.set(
callerEvidence.normalized_result as object,
"label",
"caller nested mutation",
),
).toBe(true);
expect(retained.operation).not.toBe("caller mutation");
expect(retained.normalized_result).toEqual(originalResult);
}
});
it("merges record-only bundles without changing retained unknown history", () => {
const store = ledger();
const created = store.recordUnknown(
@@ -11,11 +11,7 @@ import {
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
import { EvidenceIntegrityError } from "../../domain/evidenceErrors.js";
import { UnknownRegistryError } from "../../domain/unknownRegistryError.js";
import {
immutableEvidence,
parseEvidence,
type Evidence,
} from "../../domain/evidence.js";
import { immutableEvidence, type Evidence } from "../../domain/evidence.js";
import {
createResidualUnknown,
updateResidualUnknown,
@@ -220,7 +216,7 @@ export class EvidenceLedger {
): Result<ResidualUnknown, EvidenceIntegrityError | UnknownRegistryError> {
let parsedMutation: Evidence;
try {
parsedMutation = parseEvidence(mutationEvidence);
parsedMutation = immutableEvidence(mutationEvidence);
} catch (cause: unknown) {
return err(
new EvidenceIntegrityError("Unknown mutation evidence is invalid", {
@@ -255,8 +251,8 @@ export class EvidenceLedger {
let evidence: Evidence;
let mutation: Evidence;
try {
evidence = parseEvidence(evidenceInput);
mutation = parseEvidence(mutationInput);
evidence = immutableEvidence(evidenceInput);
mutation = immutableEvidence(mutationInput);
} catch (cause: unknown) {
return err(
new EvidenceIntegrityError("Atomic Evidence validation failed", {
@@ -394,7 +390,7 @@ export class EvidenceLedger {
): Result<ResidualUnknown, EvidenceIntegrityError | UnknownRegistryError> {
let mutationEvidence: Evidence;
try {
mutationEvidence = parseEvidence(mutationEvidenceInput);
mutationEvidence = immutableEvidence(mutationEvidenceInput);
} catch (cause: unknown) {
return err(
new EvidenceIntegrityError("Unknown mutation evidence is invalid", {
@@ -0,0 +1,71 @@
import { expect, it } from "vitest";
import { TraceMap } from "@jridgewell/trace-mapping";
import { analyzeJavaScriptArtifactFiles } from "./JavaScriptArtifactAnalysis.js";
const HASH = "a".repeat(64);
it("retains batched TraceMap-resolved paths and null source positions", () => {
const sourceMapText = JSON.stringify({
version: 3,
file: "main.js",
sourceRoot: "../src/",
sources: ["cafe\u0301.ts", null, "other.ts"],
sourcesContent: ["export const value = 1;", null, "export {};"],
names: [],
mappings: "",
});
const analysis = analyzeJavaScriptArtifactFiles({
files: [
{
path: "dist/main.js.map",
container_sha256: HASH,
sha256: HASH,
bytes: Buffer.byteLength(sourceMapText),
inventory_artifact_id: "artifact_test",
kind: "source-map",
unpacked: false,
text: { included: true, value: sourceMapText },
},
],
containers: [],
text_bytes_read: Buffer.byteLength(sourceMapText),
invalid_utf8_files: 0,
});
const originals = analysis.source_maps[0]?.sources;
const original = originals?.[0];
const producerPath = new TraceMap(
{
version: 3,
file: "main.js",
sourceRoot: "../src/",
sources: ["cafe\u0301.ts", "", "other.ts"],
names: [],
mappings: "",
},
"file:///artifact/dist/main.js.map",
).resolvedSources[0];
if (producerPath === undefined)
throw new Error("TraceMap should resolve its single source declaration");
expect(originals?.[1]?.reference).toBeNull();
expect(originals?.[2]?.reference?.resolution).toEqual({
kind: "artifact-relative",
path: "src/other.ts",
});
expect(original?.reference).toMatchObject({
source_name: "cafe\u0301.ts",
source_root: "../src/",
map_path: "dist/main.js.map",
resolution: {
kind: "artifact-relative",
path: "src/cafe\u0301.ts",
},
});
expect(original?.reference?.resolution).toEqual({
kind: "artifact-relative",
path: decodeURIComponent(
new URL(producerPath).pathname.slice("/artifact/".length),
),
});
});
@@ -3,7 +3,7 @@ import { createHash } from "node:crypto";
import { moduleKindForJavaScriptReference } from "./JavaScriptArtifactPathResolution.js";
import { setImmediate } from "node:timers/promises";
import { resolveJavaScriptSourceMapReference } from "../../domain/javascript/javascriptSourceMapPaths.js";
import { resolveJavaScriptSourceMapReferences } from "../../domain/javascript/javascriptSourceMapPaths.js";
import { analyzeParsedJavaScriptStaticSourceSteps } from "../../domain/javascript/javascriptStaticAnalysis.js";
import {
analyzeParsedJavaScriptSemanticsSteps,
@@ -561,21 +561,27 @@ const collectSourceMapOriginals = (
"Source map sourceRoot must be a string or null.",
);
const root = typeof map.sourceRoot === "string" ? map.sourceRoot : null;
for (const [index, raw] of names.entries()) {
const sourceNames: (string | null)[] = [];
for (const raw of names) {
if (raw !== null && typeof raw !== "string")
return invalidSourceMap(
file,
"Source map contains a non-string source name.",
);
sourceNames.push(raw);
}
const references = resolveJavaScriptSourceMapReferences(
sourceNames,
root,
file.path,
);
for (const [index, reference] of references.entries()) {
const rawContent = Array.isArray(contents) ? contents[index] : undefined;
const content = typeof rawContent === "string" ? rawContent : null;
sources.push({
source_index: index,
declaration_index: declarationIndex++,
reference:
typeof raw === "string"
? resolveJavaScriptSourceMapReference(raw, root, file.path)
: null,
reference,
content,
content_sha256: content === null ? null : sha256(content),
});
@@ -22,6 +22,26 @@ const exampleMethod = () => {
};
describe("managed decompiler reconstruction import", () => {
it("reports UTF-8 byte totals while preserving UTF-16 text lengths", () => {
const text = 'return "é"; // 😀';
const method = exampleMethod();
const result = importManagedReconstructionEvidence({
...MANAGED_RECONSTRUCTION_IMPORT_EXAMPLE,
methods: [
{
...method,
reconstruction: { ...method.reconstruction, text },
},
],
});
if (!result.ok) throw result.error;
expect(result.value.normalized_result).toMatchObject({
summary: { total_text_bytes: 20 },
methods: [{ reconstruction: { text, text_length: 17 } }],
});
});
it("rejects reconstruction text hash drift", () => {
expect(() =>
importManagedReconstruction({
+23
View File
@@ -0,0 +1,23 @@
import type {
AnalysisExecution,
ExecutionOptions,
ProviderAvailability,
} from "../AnalysisProvider.js";
import type { PcapRequest } from "../../domain/pcap/pcapAnalysis.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import type { Result } from "../../domain/result.js";
/**
* Packet-capture analysis boundary over caller-supplied tshark/capinfos.
* Implementation details belong to the provider adapter.
*/
export interface PcapAnalysisPort {
/** Probe both tools without reading any capture file. */
inspectAvailability(signal?: AbortSignal): Promise<ProviderAvailability>;
/** Retry cleanup of retained owned commands and capture snapshots. */
close(): Promise<void>;
execute(
request: PcapRequest,
options?: ExecutionOptions,
): Promise<Result<AnalysisExecution, AnalysisError>>;
}
@@ -0,0 +1,36 @@
import type { PcapAnalysisPort } from "./PcapAnalysisPort.js";
import type { ExecutionOptions } from "../AnalysisProvider.js";
import {
pcapRequestSchema,
type PcapOperation,
} from "../../domain/pcap/pcapAnalysis.js";
import {
AnalysisCancelledError,
AnalysisInputError,
} from "../../domain/analysisErrorCore.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import type { Evidence } from "../../domain/evidence.js";
import { createPcapEvidence } from "../../domain/pcap/pcapEvidence.js";
import { err, ok, type Result } from "../../domain/result.js";
/** Shared CLI/MCP admission, execution, and Evidence composition for pcap. */
export class PcapAnalysisService {
constructor(readonly provider: PcapAnalysisPort) {}
/** Run one caller-selected pcap request and retain capture identity inline. */
async execute(
operation: PcapOperation,
input: unknown,
options?: ExecutionOptions,
): Promise<Result<Evidence, AnalysisError>> {
if (options?.signal?.aborted === true)
return err(new AnalysisCancelledError(operation));
const request = pcapRequestSchema.safeParse({ operation, input });
if (!request.success)
return err(new AnalysisInputError(operation, { cause: request.error }));
const executed = await this.provider.execute(request.data, options);
if (!executed.ok) return executed;
const execution = executed.value;
return ok(createPcapEvidence(request.data, execution));
}
}
+7 -1
View File
@@ -4,8 +4,14 @@ import type {
} from "../AnalysisProvider.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import type { InspectWasmArtifactInput } from "../../domain/wasm/wasmArtifact.js";
import type { EvidenceSubjectTarget } from "../../domain/evidence.js";
import type { Result } from "../../domain/result.js";
/** Successful WASM inspection must identify the selected local file. */
export type WasmArtifactExecution = Omit<AnalysisExecution, "subject"> & {
readonly subject: EvidenceSubjectTarget & { readonly format: "file" };
};
/** Replaceable offline WASM inspection engine, independent of target execution and network clients. */
export interface WasmArtifactPort {
/** Retry removal of retained workspaces; failures preserve ownership for another attempt. */
@@ -13,5 +19,5 @@ export interface WasmArtifactPort {
inspect(
input: InspectWasmArtifactInput,
options?: ExecutionOptions,
): Promise<Result<AnalysisExecution, AnalysisError>>;
): Promise<Result<WasmArtifactExecution, AnalysisError>>;
}
+83 -28
View File
@@ -22,6 +22,7 @@ import {
type NativeDmgInfo,
} from "./NativeDmgOwnership.js";
const HDIUTIL_COMMAND_TIMEOUT_MS = 120_000;
const VERIFY_SETTLE_DELAYS_MS = [250, 500] as const;
// APFS ejects and busy reporting can lag a detach attempt.
const DETACH_SETTLE_DELAYS_MS = [250, 500, 1000] as const;
const delay = (milliseconds: number): Promise<void> =>
@@ -44,22 +45,30 @@ export interface NativeDmgHost {
class NativeDmgCommandFailure extends ArtifactReaderFailure {
readonly processNotStarted: boolean;
readonly verifyDiagnostic: string | undefined;
readonly exitCode: number | null;
constructor(
reason: ArtifactReaderFailure["reason"],
failure: {
readonly reason: ArtifactReaderFailure["reason"];
readonly verifyDiagnostic?: string;
},
message: string,
readonly stdout: string,
options?: {
readonly cause?: unknown;
readonly processNotStarted?: boolean;
readonly exitCode?: number;
},
) {
super(
reason,
failure.reason,
message,
options?.cause === undefined ? undefined : { cause: options.cause },
);
this.processNotStarted = options?.processNotStarted ?? false;
this.verifyDiagnostic = failure.verifyDiagnostic;
this.exitCode = options?.exitCode ?? null;
}
}
@@ -84,7 +93,7 @@ const createSystemHost = (
const output = execFileOutputFailure(cause);
if (cause instanceof Error && cause.name === "AbortError")
throw new NativeDmgCommandFailure(
"cancelled",
{ reason: "cancelled" },
"DMG operation cancelled",
output?.stdout ?? "",
{ cause },
@@ -99,7 +108,7 @@ const createSystemHost = (
cause,
};
throw new NativeDmgCommandFailure(
commandFailureReason(cause, arguments_[0]),
classifyCommandFailure(cause, arguments_[0]),
`hdiutil ${arguments_[0] ?? "operation"} failed: ${describeCommandFailure(cause, arguments_)}`,
output?.stdout ?? "",
{ cause, processNotStarted: processNotStarted(cause) },
@@ -348,8 +357,7 @@ export class NativeDmgArtifactReader implements ArtifactReader {
}
async #attach(signal?: AbortSignal): Promise<void> {
await runChecked(this.host, ["verify", this.path], signal);
this.#provenance.push(command(["verify", this.path], ["read"]));
await this.#verify(signal);
const baseline = await this.#readInfo();
this.#baselineDevices = new Set(
baseline.images.flatMap((image) =>
@@ -431,6 +439,40 @@ export class NativeDmgArtifactReader implements ArtifactReader {
}
}
async #verify(signal?: AbortSignal): Promise<void> {
const arguments_ = ["verify", this.path];
const deadline = performance.now() + HDIUTIL_COMMAND_TIMEOUT_MS;
for (let attempt = 0; ; attempt += 1) {
if (signal?.aborted === true)
throw new NativeDmgCommandFailure(
{ reason: "cancelled" },
"DMG operation cancelled",
"",
{ cause: signal.reason },
);
try {
await runChecked(this.host, arguments_, signal, {
timeoutMs: Math.max(1, Math.ceil(deadline - performance.now())),
});
this.#provenance.push(command(arguments_, ["read"]));
return;
} catch (cause: unknown) {
if (!(cause instanceof NativeDmgCommandFailure)) throw cause;
this.#provenance.push(command(arguments_, ["read"], cause.exitCode));
const delayMs = VERIFY_SETTLE_DELAYS_MS[attempt];
// Retry only the producer's explicit transient diagnostic, before any mount.
if (
cause.verifyDiagnostic !== "Resource temporarily unavailable" ||
delayMs === undefined ||
deadline - performance.now() <= delayMs
)
throw cause;
await (this.host.delay ?? delay)(delayMs);
if (performance.now() >= deadline) throw cause;
}
}
}
async #readInfo(): Promise<NativeDmgInfo> {
const info = await runChecked(this.host, ["info", "-plist"], undefined, {
timeoutMs: HDIUTIL_COMMAND_TIMEOUT_MS,
@@ -501,14 +543,14 @@ const runChecked = async (
if (cause instanceof ArtifactReaderFailure) throw cause;
if (cause instanceof Error && cause.name === "AbortError")
throw new NativeDmgCommandFailure(
"cancelled",
{ reason: "cancelled" },
"DMG operation cancelled",
execFileOutputFailure(cause)?.stdout ?? "",
{ cause },
);
const output = execFileOutputFailure(cause);
throw new NativeDmgCommandFailure(
commandFailureReason(cause, arguments_[0]),
classifyCommandFailure(cause, arguments_[0]),
`hdiutil ${arguments_[0] ?? "operation"} failed: ${describeCommandFailure(
cause,
arguments_,
@@ -527,17 +569,17 @@ const runChecked = async (
stdout: result.stdout,
...(result.stderr === undefined ? {} : { stderr: result.stderr }),
};
const reason = commandFailureReason(failure, arguments_[0]);
const classification = classifyCommandFailure(failure, arguments_[0]);
const details = describeCommandFailure(failure, arguments_);
const unknownVerifyFailure =
arguments_[0] === "verify" && reason === "unavailable";
arguments_[0] === "verify" && classification.reason === "unavailable";
throw new NativeDmgCommandFailure(
reason,
classification,
unknownVerifyFailure
? `hdiutil verify failed; captured diagnostics do not establish the failure cause: ${details}`
: `hdiutil ${arguments_[0] ?? "operation"} failed: ${details}`,
result.stdout,
{ cause: result.cause ?? failure },
{ cause: result.cause ?? failure, exitCode: result.exitCode },
);
}
return { stdout: result.stdout, exitCode: 0 };
@@ -587,20 +629,36 @@ const processOutput = (cause: unknown, field: "stdout" | "stderr"): string => {
return typeof value === "string" ? value : "";
};
const commandFailureReason = (
const classifyCommandFailure = (
cause: unknown,
operation: string | undefined,
): ArtifactReaderFailure["reason"] => {
if (typeof cause !== "object" || cause === null) return "unavailable";
): {
readonly reason: ArtifactReaderFailure["reason"];
readonly verifyDiagnostic?: string;
} => {
if (typeof cause !== "object" || cause === null)
return { reason: "unavailable" };
const code = Reflect.get(cause, "code");
const exitCode = Reflect.get(cause, "exitCode");
if (typeof code === "number" || typeof exitCode === "number")
return operation === "verify" ? verifyFailureReason(cause) : "unavailable";
if (typeof code === "number" || typeof exitCode === "number") {
if (operation !== "verify") return { reason: "unavailable" };
const diagnostic = verifyFailureDiagnostic(
processOutput(cause, "stdout"),
processOutput(cause, "stderr"),
);
return {
reason: verifyFailureReason(diagnostic),
...(diagnostic === undefined ? {} : { verifyDiagnostic: diagnostic }),
};
}
if (code === "ENOENT") {
const syscall = Reflect.get(cause, "syscall");
return typeof syscall === "string" && syscall.startsWith("spawn")
? "unavailable"
: "io";
return {
reason:
typeof syscall === "string" && syscall.startsWith("spawn")
? "unavailable"
: "io",
};
}
if (
code === "EACCES" ||
@@ -613,17 +671,13 @@ const commandFailureReason = (
code === "EMFILE" ||
code === "ENFILE"
)
return "io";
return "unavailable";
return { reason: "io" };
return { reason: "unavailable" };
};
const verifyFailureReason = (
cause: unknown,
diagnostic: string | undefined,
): ArtifactReaderFailure["reason"] => {
const diagnostic = verifyFailureDiagnostic(
processOutput(cause, "stdout"),
processOutput(cause, "stderr"),
);
switch (diagnostic) {
// hdiutil follows AppleLanguages, so en-GB and en-AU spell it "recognised".
case "image not recognized":
@@ -656,11 +710,12 @@ const verifyFailureDiagnostic = (
const command = (
arguments_: readonly string[],
effects: ArtifactCommand["effects"],
exitCode: number | null = 0,
): ArtifactCommand => ({
tool: "/usr/bin/hdiutil",
arguments: [...arguments_],
tool_version: null,
executable_sha256: null,
exit_code: 0,
exit_code: exitCode,
effects,
});
+6 -2
View File
@@ -2,7 +2,6 @@ import * as t from "@babel/types";
import {
decodeValidatedSourceMapLeaves,
isBeforeSourceMapLeafStop,
resolveSourceMapSource,
} from "../javascript/sourceMaps/DecodedSourceMap.js";
import { sanitizeBrowserUrl } from "../domain/browserObservation.js";
@@ -21,6 +20,7 @@ import { webSourceMapsSchema } from "../domain/webBundleAnalysis.js";
import { createWebTextArtifact } from "../domain/webContentArtifact.js";
import { safeParseJson } from "../domain/safeJson.js";
import { sourceMapJsonText } from "../domain/sourceMapEnvelope.js";
import { resolveJavaScriptSourceMapUrl } from "../domain/javascript/javascriptSourceMapPaths.js";
import { jsonParts } from "../domain/jsonSerialization.js";
import {
inspectSourceMapValue,
@@ -867,7 +867,11 @@ const preflightResolvedSourceBytes = (
const sourceKey = source;
let resolved = sourcesForRoot.get(sourceKey);
if (resolved === undefined)
resolved = resolveSourceMapSource(source, map.sourceRoot, mapUrl);
resolved = resolveJavaScriptSourceMapUrl(
source,
map.sourceRoot,
mapUrl,
);
const bytes = Buffer.byteLength(resolved, "utf8");
if (bytes > SOURCE_MAP_RESPONSE_BYTES - resolvedBytes)
throw resolvedSourceLimitFailure();
+2
View File
@@ -24,6 +24,7 @@ import { registerManagedCommands } from "./cli/managedCommands.js";
import { registerFirmwareCommands } from "./cli/firmwareCommands.js";
import { registerAndroidCommands } from "./cli/androidCommands.js";
import { registerApktoolCommands } from "./cli/apktoolCommands.js";
import { registerPcapCommands } from "./cli/pcapCommands.js";
import { registerFlutterCommands } from "./cli/flutterCommands.js";
import { registerJebCommands } from "./cli/jebCommands.js";
import { registerAdbCommands } from "./cli/adbCommands.js";
@@ -105,6 +106,7 @@ export const createCli = (
registerManagedCommands(cli, logger, analysis.runProviderAnalysis);
registerAndroidCommands(cli, logger, environment);
registerApktoolCommands(cli, logger, environment);
registerPcapCommands(cli, logger, environment);
registerFlutterCommands(cli, logger);
registerJebCommands(cli, logger, environment);
registerAdbCommands(cli, logger, environment);
+69
View File
@@ -0,0 +1,69 @@
import { z } from "incur";
import { AnalysisError } from "../domain/analysisErrorBase.js";
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
import { PcapAnalysisService } from "../application/pcap/PcapAnalysisService.js";
import type { PcapAnalysisPort } from "../application/pcap/PcapAnalysisPort.js";
import { createPcapAnalysisProvider } from "../composition/pcap.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { PcapOperation } from "../domain/pcap/pcapAnalysis.js";
import type { Logger } from "pino";
import type { CliInstance } from "./types.js";
import { withCommandCancellation } from "./commandCancellation.js";
/** Expose the same explicit pcap requests through one-shot CLI commands. */
export const registerPcapCommands = (
cli: CliInstance,
logger: Logger,
environment: Readonly<Record<string, string | undefined>>,
provider: PcapAnalysisPort = createPcapAnalysisProvider(environment),
): void => {
const service = new PcapAnalysisService(provider);
const execute = (name: string, operation: PcapOperation, input: unknown) =>
withCommandCancellation((signal) =>
logCliCommand(logger, name, async () => {
const result = await service.execute(operation, input, { signal });
try {
await provider.close();
} catch (cause) {
if (cause instanceof AnalysisError) {
if (result.ok) cause.retainPartialObservation(result.value);
return projectAnalysisError(
result.ok
? cause
: analysisErrorWithCleanupFailure(
result.error,
cause,
operation,
),
);
}
throw cause;
}
return result.ok ? result.value : projectAnalysisError(result.error);
}),
);
cli.command(CLI_COMMANDS.inspectPcapClient, {
description:
"Inspect the caller-selected tshark and capinfos binaries without reading a capture",
args: z.object({}),
run: () =>
execute(CLI_COMMANDS.inspectPcapClient, "inspect_pcap_client", {}),
});
cli.command(CLI_COMMANDS.inspectPcapCapture, {
description:
"Inspect one pcap/pcapng capture: endpoints, DNS, TLS SNI, and protocol distribution",
args: z.object({
path: z
.string()
.min(1)
.describe("Local pcap or pcapng capture file to inspect"),
}),
run: ({ args }) =>
execute(CLI_COMMANDS.inspectPcapCapture, "inspect_pcap_capture", {
path: args.path,
}),
});
};
+2
View File
@@ -55,6 +55,8 @@ export const CLI_COMMANDS = Object.freeze({
inspectFirmwareRegions: "inspect-firmware-regions",
extractFirmware: "extract-firmware",
inspectAndroidPackage: "inspect-android-package",
inspectPcapClient: "inspect-pcap-client",
inspectPcapCapture: "inspect-pcap-capture",
identifyFlutterBuild: "identify-flutter-build",
inspectDartAot: "inspect-dart-aot",
inspectApktoolClient: "inspect-apktool-client",
+7
View File
@@ -0,0 +1,7 @@
import type { PcapAnalysisPort } from "../application/pcap/PcapAnalysisPort.js";
import { PcapProvider } from "../pcap/PcapProvider.js";
/** Construct one pcap provider bound to the caller-selected tools. */
export const createPcapAnalysisProvider = (
environment: Readonly<Record<string, string | undefined>> = process.env,
): PcapAnalysisPort => new PcapProvider({ environment });
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -218,7 +218,7 @@ export const APPLICATION_TOOL_CONTRACTS = [
name: "evaluate_reconstruction_coverage",
...toolContractMetadata("evaluate_reconstruction_coverage"),
description:
"Evaluate inline evidence-backed reconstruction coverage against one named boundary. Missing ownership or inventory is partial; stale, weak, truncated, skipped, or unresolved proof is unknown; contradictions, failed proof, missing owners, and authority routing fail closed.",
"Evaluate inline evidence-backed reconstruction coverage against one named boundary. Missing ownership or inventory is partial; stale, weak, truncated, skipped, unresolved, or conflicting tied verifier proof is unknown; contradictions, failed proof, missing owners, and authority routing fail closed.",
kind: "application",
inputSchema: reconstructionCoverageEvaluationInputSchema,
outputSchema: reconstructionClosureResultSchema,
+46
View File
@@ -0,0 +1,46 @@
import {
pcapInputSchemas,
pcapResultSchemas,
} from "../../domain/pcap/pcapAnalysis.js";
import type { ToolContract } from "../toolContractTypes.js";
import { evidenceResultOf } from "../toolOutputSchemaPrimitives.js";
import { toolContractMetadata } from "../toolEffects.js";
import {
analysisDetailSchema,
analysisDetailOutputSchemaOf,
} from "../analysisDetail.js";
import { pcapCaptureSummaryViewResultSchema } from "../../domain/analysisView/analysisView.js";
/** Packet-capture analysis served by caller-supplied tshark and capinfos. */
export const PCAP_TOOL_CONTRACTS = [
{
name: "inspect_pcap_client",
...toolContractMetadata("inspect_pcap_client"),
description:
"Inspect the caller-selected tshark and capinfos binaries: selected commands, selection sources, and versions. Both tools must be usable. Reads no capture file. REA never installs the Wireshark suite; select binaries with REA_TSHARK_COMMAND and REA_CAPINFOS_COMMAND (absolute) or from PATH.",
kind: "pcap-provider",
inputSchema: pcapInputSchemas.inspect_pcap_client,
outputSchema: evidenceResultOf(pcapResultSchemas.inspect_pcap_client),
examples: [{ title: "Check the analysis tools", input: {} }],
},
{
name: "inspect_pcap_capture",
...toolContractMetadata("inspect_pcap_capture"),
description:
"Request detail summary for capture identity/counts, then page retained facts with inspect_analysis_view kind pcap. Default detail complete preserves inline delivery. Inspect one saved pcap or pcapng capture through an owned snapshot: metadata, exact packet count, SHA-256, IPv4/IPv6 address counts, DNS questions and address values from the same single-question response, TLS SNI, protocol distribution and HTTP URIs. All derived inventories are retained within declared resource budgets; unknown DNS attribution and excluded credentials remain explicit. Linux requires util-linux prlimit; Windows uses owned Job Objects. Requires caller-supplied capinfos and tshark; REA never captures traffic.",
kind: "pcap-provider",
inputSchema: pcapInputSchemas.inspect_pcap_capture.extend({
detail: analysisDetailSchema,
}),
outputSchema: analysisDetailOutputSchemaOf(
pcapResultSchemas.inspect_pcap_capture,
pcapCaptureSummaryViewResultSchema,
),
examples: [
{
title: "Inspect a capture file",
input: { path: "/captures/session.pcap" },
},
],
},
] as const satisfies readonly ToolContract[];
+1
View File
@@ -33,6 +33,7 @@ export const TOOL_KINDS = [
"browser-provider",
"electron-provider",
"runtime-provider",
"pcap-provider",
"application",
"session",
] as const;
+2
View File
@@ -12,6 +12,7 @@ import { MANAGED_TOOL_CONTRACTS } from "./managed/managedToolContracts.js";
import { FIRMWARE_TOOL_CONTRACTS } from "./firmware/firmwareToolContracts.js";
import { ANDROID_TOOL_CONTRACTS } from "./android/androidToolContracts.js";
import { APKTOOL_TOOL_CONTRACTS } from "./apktool/apktoolToolContracts.js";
import { PCAP_TOOL_CONTRACTS } from "./pcap/pcapToolContracts.js";
import { FLUTTER_TOOL_CONTRACTS } from "./flutter/flutterToolContracts.js";
import { JEB_TOOL_CONTRACTS } from "./jeb/jebToolContracts.js";
import { ADB_TOOL_CONTRACTS } from "./adb/adbToolContracts.js";
@@ -46,6 +47,7 @@ export const TOOL_CONTRACTS = [
...MANAGED_TOOL_CONTRACTS,
...ANDROID_TOOL_CONTRACTS,
...APKTOOL_TOOL_CONTRACTS,
...PCAP_TOOL_CONTRACTS,
...FLUTTER_TOOL_CONTRACTS,
...JEB_TOOL_CONTRACTS,
...ADB_TOOL_CONTRACTS,
+2
View File
@@ -45,6 +45,8 @@ const EXECUTION_WORDING_EXCEPTIONS: Readonly<Record<string, string>> = {
search_android_classes: "states it does not execute the APK",
inspect_apktool_client:
"inspects the operator-selected apktool launcher; never installs it",
inspect_pcap_client:
"probes the operator-selected tshark and capinfos; never installs them",
inspect_jeb_client: "talks to a caller-started JEB; never launches it",
open_jeb_project: "the engine analyzes the file; REA does not execute it",
inspect_adb_client:
+10
View File
@@ -182,6 +182,16 @@ export const TOOL_EFFECTS: Readonly<Record<string, ToolEffects>> = {
launchesProcess: true,
writesFilesystem: true,
}),
inspect_pcap_client: effects({
mutatesSession: true,
launchesProcess: true,
writesFilesystem: true,
}),
inspect_pcap_capture: effects({
mutatesSession: true,
launchesProcess: true,
writesFilesystem: true,
}),
identify_flutter_build: effects({
mutatesSession: true,
writesFilesystem: true,
+7
View File
@@ -65,4 +65,11 @@ describe("analysis profile commitments", () => {
analysisProfileSchema.parse({ ...profile, digest: "0".repeat(64) }),
).toThrow(/digest/u);
});
it("projects known provider identity fields at construction", () => {
const liveProvider = { ...PROVIDER, transient_status: "starting" };
const profile = createAnalysisProfile(liveProvider, { loader: "default" });
expect(profile.provider).toEqual(PROVIDER);
expect(analysisProfileSchema.parse(profile)).toEqual(profile);
});
});
+7 -3
View File
@@ -7,13 +7,13 @@ import { jsonObjectSchema, type JsonValue } from "./jsonValue.js";
import { digestSchema } from "./../domain/digests.js";
/** Concrete provider identity committed by an analysis profile. */
export const committedProviderSchema = z.object({
export const committedProviderSchema = z.strictObject({
id: z.string().min(1),
name: z.string().min(1),
version: z.string().min(1),
});
const unsignedAnalysisProfileSchema = z.object({
const unsignedAnalysisProfileSchema = z.strictObject({
provider: committedProviderSchema,
parameters: jsonObjectSchema,
});
@@ -40,7 +40,11 @@ export const createAnalysisProfile = (
parameters: Readonly<Record<string, JsonValue>>,
): AnalysisProfileCommitment => {
const unsigned = unsignedAnalysisProfileSchema.parse({
provider,
provider: {
id: provider.id,
name: provider.name,
version: provider.version,
},
parameters,
});
return analysisProfileSchema.parse({
+3 -3
View File
@@ -1,6 +1,5 @@
import { expect, it } from "vitest";
import { resolveJavaScriptSourceMapReference } from "../javascript/javascriptSourceMapPaths.js";
import { functionDossierSchema } from "../hopperValues.js";
import { ghidraFunctionDossier } from "../ghidraValues.fixture.js";
import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "../../contracts/javascript/javascriptRuntimeReconciliationExample.js";
@@ -24,13 +23,14 @@ import {
completeWithinViewCoverage,
pageViewCoverage,
} from "./analysisViewCoverage.js";
import { sourceMapReference } from "../javascript/javascriptApplicationGraph.fixture.js";
import { javascriptModulePath } from "./javascriptApplicationView.js";
import {
inspectAnalysisViewInputSchema,
projectAnalysisView,
type AnalysisViewParent,
} from "./analysisView.js";
const layoutParent = (): AnalysisViewParent => {
const evidence = analysisViewLayoutEvidence();
return {
@@ -532,7 +532,7 @@ it("pages actual modules, including unknown and empty paths, without unrelated r
observations: original.observations.map((observation) => ({
...(kind === "source-module"
? {
source_map_reference: resolveJavaScriptSourceMapReference(
source_map_reference: sourceMapReference(
"",
null,
"renderer.js",
+28
View File
@@ -62,6 +62,12 @@ import {
captureSummarySchema,
} from "./webNetworkCaptureView.js";
import type { WebNetworkApiSurfaceAccumulator } from "./WebNetworkApiSurfaceAccumulator.js";
import {
pcapViewRequestSchema,
pcapViewPageSchema,
pcapSummarySchema,
projectPcapCaptureView,
} from "./pcapCaptureView.js";
import {
qlogViewRequestSchema,
qlogViewPageSchema,
@@ -196,6 +202,7 @@ export const analysisViewRequestSchema = z.discriminatedUnion("kind", [
apiSurfaceViewSchema,
captureViewRequestSchema,
qlogViewRequestSchema,
pcapViewRequestSchema,
dartAotViewRequestSchema,
]);
@@ -312,6 +319,7 @@ const viewResultBase = {
"inspect_managed_artifact",
"inspect_web_network_capture",
"inspect_android_package",
"inspect_pcap_capture",
"inspect_dart_aot",
]),
parent_digest: digestSchema,
@@ -384,6 +392,12 @@ export const androidPackageSummaryViewResultSchema = z.strictObject({
summary: androidPackageSummarySchema,
});
export const pcapCaptureSummaryViewResultSchema = z.strictObject({
...summaryViewResultBase,
parent_operation: z.literal("inspect_pcap_capture"),
summary: pcapSummarySchema,
});
/** Exact snapshot and pool-coverage carrier for selected AOT delivery. */
export const dartAotSummaryViewResultSchema = z.strictObject({
...summaryViewResultBase,
@@ -415,6 +429,7 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [
managedSummarySchema,
captureSummarySchema,
androidPackageSummarySchema,
pcapSummarySchema,
dartAotSummarySchema,
]),
}),
@@ -433,6 +448,14 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [
...viewResultBase,
parent_operation: z.literal("inspect_web_network_capture"),
}),
z.strictObject({
kind: z.literal("pcap"),
view: pcapViewRequestSchema,
page: pcapViewPageSchema,
source_coverage: z.enum(["complete", "partial"]),
...viewResultBase,
parent_operation: z.literal("inspect_pcap_capture"),
}),
z.strictObject({
kind: z.literal("facet"),
view: facetViewSchema,
@@ -562,6 +585,7 @@ export type UnsignedAnalysisView =
| Omit<Extract<AnalysisViewResult, { readonly kind: "wasm" }>, "view_digest">
| Omit<Extract<AnalysisViewResult, { readonly kind: "evm" }>, "view_digest">
| Omit<Extract<AnalysisViewResult, { readonly kind: "qlog" }>, "view_digest">
| Omit<Extract<AnalysisViewResult, { readonly kind: "pcap" }>, "view_digest">
| Omit<
Extract<AnalysisViewResult, { readonly kind: "capture" }>,
"view_digest"
@@ -686,6 +710,10 @@ export const projectAnalysisView = (
const projected = projectWebNetworkCaptureView(parent, view);
return projected.ok ? ok(sealAnalysisView(projected.value)) : projected;
}
if (parent.operation === "inspect_pcap_capture") {
const projected = projectPcapCaptureView(parent, view);
return projected.ok ? ok(sealAnalysisView(projected.value)) : projected;
}
if (parent.operation === "inspect_binary_layout") {
const layout = binaryLayoutSchema.safeParse(parent.normalizedResult);
if (!layout.success)
@@ -76,6 +76,7 @@ export const projectEvmInterfaceView = (
const report = parsed.data;
if (
parent.subject == null ||
parent.subject.format !== "file" ||
parent.subject.local_path !== report.artifact.path ||
parent.subject.digest.sha256 !== report.artifact.sha256
)
+327
View File
@@ -0,0 +1,327 @@
import { z } from "zod";
import { pcapResultSchemas } from "../pcap/pcapAnalysis.js";
import { analysisInputErrorFromIssues } from "../inputIssueProjection.js";
import type { AnalysisError } from "../analysisErrorBase.js";
import { err, ok, type Result } from "../result.js";
import {
analysisViewInputError,
type AnalysisViewParent,
type AnalysisViewRequest,
type UnsignedAnalysisView,
} from "./analysisView.js";
import {
completeWithinViewCoverage,
pageViewCoverage,
} from "./analysisViewCoverage.js";
const captureSchema = pcapResultSchemas.inspect_pcap_capture;
const fields = captureSchema.shape;
const pageShape = {
kind: z.literal("pcap"),
offset: z.number().int().nonnegative().default(0),
limit: z.number().int().positive().default(64),
};
export const pcapViewRequestSchema = z.discriminatedUnion("facet", [
z.strictObject({
...pageShape,
facet: z.enum([
"endpoints",
"dns-queries",
"tls-sni",
"protocols",
"http-uris",
"unattributed-addresses",
]),
}),
z.strictObject({
...pageShape,
facet: z.literal("dns-addresses"),
query: z.string().min(1),
}),
z.strictObject({
...pageShape,
facet: z.literal("diagnostics"),
stream: z.enum(["tshark", "capinfos"]),
}),
]);
export const pcapViewPageSchema = z.discriminatedUnion("facet", [
z.strictObject({ facet: z.literal("endpoints"), items: fields.endpoints }),
z.strictObject({
facet: z.literal("dns-queries"),
items: z.array(
fields.dns_queries.element.omit({ response_addresses: true }).extend({
retained_response_address_count: z.number().int().nonnegative(),
}),
),
}),
z.strictObject({ facet: z.literal("tls-sni"), items: fields.tls_sni_values }),
z.strictObject({
facet: z.literal("protocols"),
items: fields.protocol_distribution,
}),
z.strictObject({
facet: z.literal("http-uris"),
items: fields.http_request_uris,
}),
z.strictObject({
facet: z.literal("unattributed-addresses"),
items: fields.unattributed_dns_addresses,
}),
z.strictObject({
facet: z.literal("dns-addresses"),
items: fields.unattributed_dns_addresses,
}),
z.strictObject({
facet: z.literal("diagnostics"),
text: z.string(),
status: z.enum(["retained", "unknown"]),
}),
]);
export const pcapSummarySchema = captureSchema
.omit({
endpoints: true,
dns_queries: true,
tls_sni_values: true,
protocol_distribution: true,
http_request_uris: true,
unattributed_dns_addresses: true,
diagnostics: true,
})
.extend({
retained_counts: z.strictObject({
endpoints: z.number().int().nonnegative(),
dns_queries: z.number().int().nonnegative(),
tls_sni_values: z.number().int().nonnegative(),
protocol_distribution: z.number().int().nonnegative(),
http_request_uris: z.number().int().nonnegative(),
unattributed_dns_addresses: z.number().int().nonnegative(),
}),
diagnostics_code_units: z
.strictObject({
tshark: z.number().int().nonnegative(),
capinfos: z.number().int().nonnegative(),
})
.nullable(),
});
type Capture = z.output<typeof captureSchema>;
type PcapView = z.output<typeof pcapViewRequestSchema>;
type Page = z.output<typeof pcapViewPageSchema>;
const parseParent = (
parent: AnalysisViewParent,
): Result<Capture, AnalysisError> => {
const parsed = captureSchema.safeParse(parent.normalizedResult);
if (!parsed.success)
return err(
analysisInputErrorFromIssues(
"inspect_analysis_view",
parsed.error.issues,
parent.normalizedResult,
{ cause: parsed.error },
),
);
if (
parent.subject?.local_path !== parsed.data.target.path ||
parent.subject.digest.sha256 !== parsed.data.target.sha256
)
return err(
analysisViewInputError([
{
path: ["source", "subject"],
reason: "invalid_value",
message:
"Packet parent subject must identify the report's exact path and SHA-256.",
},
]),
);
return ok(parsed.data);
};
const summarize = ({
endpoints,
dns_queries,
tls_sni_values,
protocol_distribution,
http_request_uris,
unattributed_dns_addresses,
diagnostics,
...metadata
}: Capture) => ({
...metadata,
retained_counts: {
endpoints: endpoints.length,
dns_queries: dns_queries.length,
tls_sni_values: tls_sni_values.length,
protocol_distribution: protocol_distribution.length,
http_request_uris: http_request_uris.length,
unattributed_dns_addresses: unattributed_dns_addresses.length,
},
diagnostics_code_units:
diagnostics === undefined
? null
: {
tshark: diagnostics.tshark_stderr.length,
capinfos: diagnostics.capinfos_stderr.length,
},
});
const selectRows = (
capture: Capture,
view: Exclude<PcapView, { facet: "diagnostics" }>,
): Result<
{ page: Exclude<Page, { facet: "diagnostics" }>; total: number },
AnalysisError
> => {
const slice = <T>(items: readonly T[]) =>
items.slice(view.offset, view.offset + view.limit);
switch (view.facet) {
case "endpoints":
return ok({
page: { facet: view.facet, items: slice(capture.endpoints) },
total: capture.endpoints.length,
});
case "dns-queries":
return ok({
page: {
facet: view.facet,
items: slice(capture.dns_queries).map(
({ response_addresses, ...query }) => ({
...query,
retained_response_address_count: response_addresses.length,
}),
),
},
total: capture.dns_queries.length,
});
case "tls-sni":
return ok({
page: { facet: view.facet, items: slice(capture.tls_sni_values) },
total: capture.tls_sni_values.length,
});
case "protocols":
return ok({
page: {
facet: view.facet,
items: slice(capture.protocol_distribution),
},
total: capture.protocol_distribution.length,
});
case "http-uris":
return ok({
page: { facet: view.facet, items: slice(capture.http_request_uris) },
total: capture.http_request_uris.length,
});
case "unattributed-addresses":
return ok({
page: {
facet: view.facet,
items: slice(capture.unattributed_dns_addresses),
},
total: capture.unattributed_dns_addresses.length,
});
case "dns-addresses": {
const matches = capture.dns_queries.filter(
(query) => query.query === view.query,
);
if (matches.length !== 1)
return err(
analysisViewInputError([
{
path: ["view", "query"],
reason: "invalid_value",
message: "Select one unambiguous exact retained DNS question.",
},
]),
);
return ok({
page: {
facet: view.facet,
items: slice(matches[0]!.response_addresses),
},
total: matches[0]!.response_addresses.length,
});
}
}
};
/** Historical packet facts are paged without reopening the source or rerunning Wireshark. */
export const projectPcapCaptureView = (
parent: AnalysisViewParent,
view: AnalysisViewRequest,
): Result<UnsignedAnalysisView, AnalysisError> => {
if (view.kind !== "summary" && view.kind !== "pcap")
return err(
analysisViewInputError([
{
path: ["view"],
reason: "invalid_value",
message:
"Packet summary parents support summary and pcap fact pages.",
},
]),
);
const parsed = parseParent(parent);
if (!parsed.ok) return parsed;
const capture = parsed.value;
const shared = {
parent_evidence_id: parent.evidenceId,
parent_operation: "inspect_pcap_capture" as const,
parent_digest: parent.evidenceId.slice(3),
artifact: { path: capture.target.path, sha256: capture.target.sha256 },
limitations: [
...parent.limitations,
"Page totals count retained rows or UTF-16 diagnostic code units. Exhaustion does not prove whole-network completeness; source coverage and examined counts remain separate.",
],
unknowns: [
"Saved packet observations do not establish process identity, causality, encrypted payload contents or traffic missing from the recording.",
],
};
if (view.kind === "summary")
return ok({
kind: "summary",
view,
...shared,
summary: summarize(capture),
coverage: completeWithinViewCoverage(1, 1),
});
if (view.facet === "diagnostics") {
const value =
view.stream === "tshark"
? capture.diagnostics?.tshark_stderr
: capture.diagnostics?.capinfos_stderr;
const text = value?.slice(view.offset, view.offset + view.limit) ?? "";
return ok({
kind: "pcap",
view,
...shared,
source_coverage: capture.coverage,
page: {
facet: "diagnostics",
text,
status: value === undefined ? "unknown" : "retained",
},
coverage: pageViewCoverage(view.offset, text.length, value?.length ?? 0),
unknowns:
value === undefined
? [
...shared.unknowns,
"The parent did not retain diagnostics; an empty text page does not establish their absence.",
]
: shared.unknowns,
});
}
const selected = selectRows(capture, view);
return selected.ok
? ok({
kind: "pcap",
view,
...shared,
source_coverage: capture.coverage,
page: selected.value.page,
coverage: pageViewCoverage(
view.offset,
selected.value.page.items.length,
selected.value.total,
),
})
: selected;
};
@@ -103,6 +103,7 @@ export const projectWasmArtifactView = (
: {
path: parent.subject.local_path,
sha256: parent.subject.digest.sha256,
format: parent.subject.format,
},
)
)
+24
View File
@@ -52,6 +52,30 @@ it("preserves omitted relationship diagnostics and authenticates reported diagno
).toThrow("semantic identifier");
});
it("projects known provider fields and preserves open JSON payloads", () => {
const liveProvider = { ...PROVIDER, transient_status: "starting" };
const evidence = createEvidence(TARGET, liveProvider, {
operation: "inspect",
parameters: { caller_metadata: { note: "kept" } },
result: { producer_extension: { value: "kept" } },
analysisProfile: PROFILE,
environment: {
id: "fixture",
platform: "linux",
architecture: "x86_64",
isolation: "process",
},
locations: [{ kind: "address", address: "0x1000" }],
});
expect(evidence.provider).toEqual(PROVIDER);
expect(evidence.parameters).toEqual({ caller_metadata: { note: "kept" } });
expect(evidence.normalized_result).toEqual({
producer_extension: { value: "kept" },
});
expect(parseEvidence(evidence)).toEqual(evidence);
});
it("preserves identity and bytes when reusing an owned immutable result", () => {
const value = { nested: { values: ["observed", 1] } };
const mutable = createEvidence(TARGET, PROVIDER, {
+12 -9
View File
@@ -33,14 +33,14 @@ import {
} from "./relationshipCoverage.js";
/** Provider identity schema shared by evidence-bearing persistence formats. */
export const providerSchema = z.object({
export const providerSchema = z.strictObject({
id: z.string().min(1),
name: z.string().min(1),
version: z.string().nullable(),
});
const subjectSchema = z.object({
const subjectSchema = z.strictObject({
name: z.string().min(1),
digest: z.object({ sha256: digestSchema }),
digest: z.strictObject({ sha256: digestSchema }),
format: z.enum([
"hopper",
"analysis-database",
@@ -78,15 +78,18 @@ const subjectSchema = z.object({
/** Source location attached to an evidence observation. */
/** Source-location schema shared by evidence-bearing persistence formats. */
export const evidenceLocationSchema = z.discriminatedUnion("kind", [
z.object({ kind: z.literal("address"), address: z.string().min(1) }),
z.object({
z.strictObject({ kind: z.literal("address"), address: z.string().min(1) }),
z.strictObject({
kind: z.literal("address-range"),
start: z.string().min(1),
end: z.string().min(1),
}),
z.object({ kind: z.literal("artifact-path"), path: z.string().min(1) }),
z.object({ kind: z.literal("file-offset"), offset: z.number().int().min(0) }),
z.object({
z.strictObject({ kind: z.literal("artifact-path"), path: z.string().min(1) }),
z.strictObject({
kind: z.literal("file-offset"),
offset: z.number().int().min(0),
}),
z.strictObject({
kind: z.literal("file-offset-range"),
start: z.number().int().min(0),
end: z.number().int().min(0),
@@ -101,7 +104,7 @@ const evidenceAuthoritySchema = z.enum([
"analyst-inference",
]);
const executionEnvironmentSchema = z.object({
const executionEnvironmentSchema = z.strictObject({
id: z.string().min(1),
platform: z.string().min(1),
architecture: z.string().min(1),
+11 -8
View File
@@ -16,24 +16,27 @@ import {
} from "./residualUnknown.js";
import { freezeJsonSnapshot } from "./immutableJson.js";
const artifactManifestSchema = evidenceSchema.shape.subject.unwrap().pick({
digest: true,
format: true,
architecture: true,
});
const artifactManifestSchema = evidenceSchema.shape.subject
.unwrap()
.pick({
digest: true,
format: true,
architecture: true,
})
.strict();
const providerManifestSchema = providerSchema;
const environmentManifestSchema = evidenceSchema.shape.environment.unwrap();
const scenarioManifestSchema = z.object({
const scenarioManifestSchema = z.strictObject({
evidence_id: evidenceSchema.shape.evidence_id,
operation: z.string().min(1),
authority: z.literal("controlled-replay"),
});
const captureManifestSchema = z.object({
const captureManifestSchema = z.strictObject({
evidence_id: evidenceSchema.shape.evidence_id,
predicate_type: z.string().min(1),
});
export const evidenceBundleSchema = z.object({
export const evidenceBundleSchema = z.strictObject({
artifacts: z.array(artifactManifestSchema),
providers: z.array(providerManifestSchema),
environments: z.array(environmentManifestSchema),
+28
View File
@@ -119,6 +119,34 @@ const observe = (
);
describe("function comparison normalized identity", () => {
it("keeps differing generated names ambiguous and stable names mismatched", () => {
const named = (base: "0x1000" | "0x2000", name: string) =>
functionDossierSchema.parse({
...dossier("return 0;", base),
procedure: {
...procedure(base, name),
signature: "int helper(void)",
locals: [],
},
});
const generated = compareFunctions(
observe("1", named("0x1000", "sub_1000")),
observe("2", named("0x2000", "sub_2000")),
);
expect(generated.function_match.status).toBe("ambiguous");
expect(generated.status).toBe("unknown");
expect(
generated.dimensions.find(({ dimension }) => dimension === "identity"),
).toMatchObject({ status: "unknown" });
const stable = compareFunctions(
observe("3", named("0x1000", "firstFunction")),
observe("4", named("0x2000", "secondFunction")),
);
expect(stable.function_match.status).toBe("mismatched");
expect(stable.status).toBe("changed");
});
it.each(["sub_deallocate", "sub_deadbeef_handler", "fcn.dispatch"])(
"recognizes the explicit symbol %s without a generated-name prefix false positive",
(name) => {
+9 -12
View File
@@ -16,24 +16,21 @@ export const functionMatch = (
left: FunctionSnapshot,
right: FunctionSnapshot,
): FunctionComparisonResult["function_match"] => {
if (
!isAutoName(left.dossier.procedure.name) &&
left.dossier.procedure.name === right.dossier.procedure.name
)
const leftName = left.dossier.procedure.name;
const rightName = right.dossier.procedure.name;
const stableNames = !isAutoName(leftName) && !isAutoName(rightName);
if (stableNames && leftName === rightName)
return {
status: "matched",
method: "symbol",
left_name: left.dossier.procedure.name,
right_name: right.dossier.procedure.name,
left_name: leftName,
right_name: rightName,
};
return {
status:
left.dossier.procedure.name === right.dossier.procedure.name
? "ambiguous"
: "mismatched",
status: stableNames ? "mismatched" : "ambiguous",
method: "explicit",
left_name: left.dossier.procedure.name,
right_name: right.dossier.procedure.name,
left_name: leftName,
right_name: rightName,
};
};
@@ -6,6 +6,7 @@ import {
} from "./javascriptApplicationGraph.js";
import type { ApplicationNode } from "./javascriptApplicationGraphSchemas.js";
import type { ApplicationGraphEvidence } from "./javascriptApplicationEvidenceSchemas.js";
import { resolveJavaScriptSourceMapReferences } from "./javascriptSourceMapPaths.js";
export const APPLICATION_GRAPH_DIGESTS = {
package: "1".repeat(64),
@@ -22,6 +23,20 @@ export const completeCoverage = {
limits: [],
} satisfies ApplicationGraphEvidence["coverage"];
export const sourceMapReference = (
sourceName: string,
sourceRoot: string | null,
mapPath: string,
) => {
const reference = resolveJavaScriptSourceMapReferences(
[sourceName],
sourceRoot,
mapPath,
)[0];
if (reference == null) throw new TypeError("Expected a source-map reference");
return reference;
};
const extractor = (operation: string) => ({
name: "rea-test-fixture",
version: "1.0.0",
@@ -20,10 +20,9 @@ import {
inferredArtifactEvidence,
nodeByLabel,
runtimeEvidence,
sourceMapReference,
unknownEvidence,
} from "./javascriptApplicationGraph.fixture.js";
import { resolveJavaScriptSourceMapReference } from "./javascriptSourceMapPaths.js";
const createLargeNativeExportGraph = (): JavaScriptApplicationGraph => {
const properties = Object.fromEntries(
Array.from({ length: 1_001 }, (_, index) => [
@@ -438,7 +437,7 @@ describe("JavaScript Application Graph: source maps and observations", () => {
observations: [
{
label: "src/editor.ts",
source_map_reference: resolveJavaScriptSourceMapReference(
source_map_reference: sourceMapReference(
"webpack:///src/editor.ts",
null,
"dist/app.js.map",
@@ -535,7 +534,7 @@ describe("JavaScript Application Graph: source-map digest consistency", () => {
observations: [
{
label: null,
source_map_reference: resolveJavaScriptSourceMapReference(
source_map_reference: sourceMapReference(
"src/editor.ts",
null,
"dist/app.js.map",
@@ -1,5 +1,6 @@
import { posix } from "node:path";
import { domainToASCII, fileURLToPath, pathToFileURL } from "node:url";
import { TraceMap } from "@jridgewell/trace-mapping";
import { z } from "zod";
@@ -25,22 +26,64 @@ export type JavaScriptSourceMapReference = z.infer<
typeof javascriptSourceMapReferenceSchema
>;
/** Interpret a declaration once at the source-map ingestion boundary. */
export const resolveJavaScriptSourceMapReference = (
sourceName: string,
/** Interpret declarations together once at their source-map ingestion boundary. */
export const resolveJavaScriptSourceMapReferences = (
sourceNames: readonly (string | null)[],
sourceRoot: string | null,
mapPath: string,
): JavaScriptSourceMapReference => {
// TraceMap's producer prefixes sourceRoot before resolving the URL reference.
const reference = `${sourceRoot ? `${sourceRoot}/` : ""}${sourceName}`;
return {
source_name: sourceName,
source_root: sourceRoot,
map_path: mapPath,
resolution: resolveJavaScriptSourceMapPath(reference, mapPath),
};
): (JavaScriptSourceMapReference | null)[] => {
if (sourceNames.length === 0) return [];
const resolvedSources = resolveSourceMapUrls(
sourceNames.map((sourceName) => sourceName ?? ""),
sourceRoot,
"",
);
return sourceNames.map((sourceName, index) => {
if (sourceName === null) return null;
const reference = resolvedSources[index];
if (reference === undefined)
throw new TypeError(
`TraceMap did not resolve source declaration ${String(index)}`,
);
return {
source_name: sourceName,
source_root: sourceRoot,
map_path: mapPath,
resolution: resolveJavaScriptSourceMapPath(reference, mapPath),
};
});
};
/** Resolve a source declaration using the same pinned format implementation as map consumers. */
export const resolveJavaScriptSourceMapUrl = (
source: string | null,
sourceRoot: string | null | undefined,
mapUrl: string,
): string => {
const resolved = resolveSourceMapUrls([source ?? ""], sourceRoot, mapUrl)[0];
if (resolved === undefined)
throw new TypeError(
"TraceMap did not resolve its single source declaration",
);
return resolved;
};
const resolveSourceMapUrls = (
sources: string[],
sourceRoot: string | null | undefined,
mapUrl: string,
): readonly string[] =>
new TraceMap(
{
version: 3,
names: [],
sources,
...(sourceRoot == null ? {} : { sourceRoot }),
mappings: "",
},
mapUrl,
).resolvedSources;
/** Resolve a source-map URL reference without changing its recorded identity. */
const resolveJavaScriptSourceMapPath = (
source: string,
@@ -1,10 +1,11 @@
import { describe, expect, it } from "vitest";
import { TraceMap } from "@jridgewell/trace-mapping";
import { resolveJavaScriptSourceMapReference } from "./javascriptSourceMapPaths.js";
import {
createJavaScriptApplicationGraph,
createJavaScriptApplicationNode,
} from "./javascriptApplicationGraph.js";
import type { ApplicationNode } from "./javascriptApplicationGraphSchemas.js";
import { createHistoricalSourceGraph } from "../referenceSourceGraph.js";
import { compareUnicodeCodePoints } from "../unicodeCodePointOrder.js";
import { compareSourceToBundle } from "./sourceToBundleComparison.js";
@@ -12,7 +13,10 @@ import {
SOURCE_TO_BUNDLE_SIGNAL_WEIGHTS,
sourceToBundleComparisonResultSchema,
} from "./sourceToBundleComparisonSchemas.js";
import { artifactEvidence } from "./javascriptApplicationGraph.fixture.js";
import {
artifactEvidence,
sourceMapReference,
} from "./javascriptApplicationGraph.fixture.js";
const HASH = {
artifact: "1".repeat(64),
@@ -42,9 +46,6 @@ describe("historical source to bundle comparison", () => {
const second = compareSourceToBundle(input);
expect(first).toEqual(second);
expect(() =>
sourceToBundleComparisonResultSchema.parse(first),
).not.toThrow();
expect(first.summary).toEqual({
unchanged: 1,
modified: 0,
@@ -153,7 +154,7 @@ describe("source to bundle digest changes", () => {
observations: [
{
label: "display label",
source_map_reference: resolveJavaScriptSourceMapReference(
source_map_reference: sourceMapReference(
"webpack://fixture/./src/modified.ts",
null,
"dist/main.js.map",
@@ -202,6 +203,90 @@ describe("source to bundle digest changes", () => {
});
});
describe("source to bundle Unicode path collisions", () => {
it("keeps colliding canonical-equivalent current paths ambiguous", () => {
const historicalPath = "src/a\u0301\u0323.ts";
const currentPaths = ["src/a\u0323\u0301.ts", "src/\u1ea1\u0301.ts"];
const nodes = currentPaths.map((path, index) =>
moduleNode(`unicode-${index}`, { path }),
);
const result = comparePathToNodes(historicalPath, nodes);
const mapping = item(result, historicalPath);
expect(mapping).toMatchObject({
status: "unknown",
confidence: "unknown",
current_node_ids: [],
candidates: [
expect.objectContaining({
signals: expect.arrayContaining([
expect.objectContaining({ kind: "unicode-equivalent-path" }),
]),
}),
expect.objectContaining({
signals: expect.arrayContaining([
expect.objectContaining({ kind: "unicode-equivalent-path" }),
]),
}),
],
limitations: [
expect.stringContaining(
"filesystem paths with distinct Unicode spellings",
),
],
});
});
it("prefers a raw source-map path over canonically equivalent alternatives", () => {
const historicalPath = "src/cafe\u0301.ts";
const exact = sourceMapNode(
"exact",
"cafe\u0301.ts",
"../src/",
HASH.modified,
);
const equivalent = sourceMapNode(
"equivalent",
"caf\u00e9.ts",
"../src/",
HASH.removed,
);
const result = comparePathToNodes(historicalPath, [exact, equivalent]);
expect(item(result, historicalPath)).toMatchObject({
status: "modified",
current_node_ids: [exact.node_id],
});
});
it("finds Unicode-equivalent suffixes of absolute source-map URLs", () => {
const historicalPath = "src/caf\u00e9.ts";
const node = sourceMapNode(
"remote",
"cafe\u0301.ts",
"https://cdn.example/src/",
HASH.modified,
);
const result = comparePathToNodes(historicalPath, [node]);
expect(item(result, historicalPath)).toMatchObject({
status: "unknown",
current_node_ids: [],
candidates: [
expect.objectContaining({
current_node_id: node.node_id,
signals: expect.arrayContaining([
expect.objectContaining({
kind: "unicode-equivalent-path",
current_values: ["cdn.example/src/cafe\u0301.ts"],
}),
]),
}),
],
});
});
});
describe("source to bundle path syntax", () => {
it.each([
"src/modified#part.ts",
@@ -282,9 +367,60 @@ describe("source to bundle path syntax", () => {
}),
],
});
expect(
item(result, "src/modified.ts").candidates.flatMap(
({ signals }) => signals,
),
).not.toContainEqual(
expect.objectContaining({ kind: "unicode-equivalent-path" }),
);
});
});
describe("source-map resolver equivalence", () => {
it.each([
{
sourceRoot: "https://cdn.example/src/",
expectedSource: "https://cdn.example/src/a.js",
expectedResolution: { kind: "suffix", path: "cdn.example/src/a.js" },
},
{
sourceRoot: "https://cdn.example/src//",
expectedSource: "https://cdn.example/src/a.js",
expectedResolution: { kind: "suffix", path: "cdn.example/src/a.js" },
},
{
sourceRoot: "webpack://",
expectedSource: "webpack:///a.js",
expectedResolution: { kind: "suffix", path: "a.js" },
},
])(
"matches TraceMap for sourceRoot $sourceRoot",
({ sourceRoot, expectedSource, expectedResolution }) => {
const producerSource = new TraceMap(
{
version: 3,
file: "main.js",
sourceRoot,
sources: ["a.js"],
names: [],
mappings: "",
},
"file:///artifact/dist/main.js.map",
).resolvedSources[0];
const reference = sourceMapReference(
"a.js",
sourceRoot,
"dist/main.js.map",
);
expect(producerSource).toBe(expectedSource);
expect(reference.source_root).toBe(sourceRoot);
expect(reference.resolution).toEqual(expectedResolution);
},
);
});
describe("source to bundle complete metadata", () => {
it("accepts long source metadata and all reported limitations", () => {
const result = compareSourceToBundle({
@@ -328,6 +464,61 @@ describe("source to bundle complete metadata", () => {
});
});
const sourceMapNode = (
key: string,
sourceName: string,
sourceRoot: string,
digest: string,
) =>
createJavaScriptApplicationNode({
kind: "source-module",
identity: {
strategy: "source-map-original",
source_root: sourceRoot,
stability: "source-map-exact",
source_map_sha256: HASH.artifact,
original_source: sourceName,
source_sha256: digest,
},
observations: [
{
label: `${key} source-map spelling`,
source_map_reference: sourceMapReference(
sourceName,
sourceRoot,
`dist/${key}.map`,
),
properties: { content_available: true, content_sha256: digest },
evidence: artifactEvidence(HASH.artifact, `dist/${key}.map`),
},
],
});
const comparePathToNodes = (
sourcePath: string,
nodes: readonly ApplicationNode[],
) =>
compareSourceToBundle({
reference: historicalGraph("complete", [sourcePath]),
application: {
evidenceId: EVIDENCE_ID,
rootArtifactSha256: HASH.artifact,
graph: createJavaScriptApplicationGraph({
schema: "JavaScriptApplicationGraph",
root_node_ids: nodes.map(({ node_id: nodeId }) => nodeId),
nodes,
edges: [],
coverage: {
status: "complete",
truncated: false,
omitted_count: 0,
limits: [],
},
limitations: [],
}),
},
});
const historicalGraph = (
inventoryState: "complete" | "partial",
paths = [
@@ -258,7 +258,11 @@ const classifyMapping = (
confidence: "unknown",
currentNodeIds: [],
limitations: [
"Only weak basename or extension signals were available; no mapping was forced.",
candidates.some((candidate) =>
hasSignal(candidate, "unicode-equivalent-path"),
)
? "Unicode-equivalent paths are inferred candidates only; filesystem paths with distinct Unicode spellings may identify different files, so no mapping was forced."
: "Only weak basename or extension signals were available; no mapping was forced.",
],
};
return absenceComplete
@@ -15,6 +15,7 @@ export const SOURCE_TO_BUNDLE_SIGNAL_WEIGHTS = [
["source-map-original-path", 70],
["current-path-exact", 60],
["current-path-suffix", 40],
["unicode-equivalent-path", 20],
["basename-match", 20],
["language-extension", 5],
] as const;
@@ -24,6 +25,7 @@ const sourceToBundleSignalKindSchema = z.enum([
"source-map-original-path",
"current-path-exact",
"current-path-suffix",
"unicode-equivalent-path",
"basename-match",
"language-extension",
]);
+38 -1
View File
@@ -102,8 +102,12 @@ export const buildSourceToBundleCandidateIndex = (
for (const path of projection.paths) {
for (const suffix of path.allowSuffix
? pathSuffixes(path.value)
: [path.value])
: [path.value]) {
addIndexValue(byPathSuffix, suffix, projection.node.node_id);
const canonical = suffix.normalize("NFC");
if (canonical !== suffix)
addIndexValue(byPathSuffix, canonical, projection.node.node_id);
}
addIndexValue(
byBasename,
posix.basename(path.value),
@@ -126,9 +130,13 @@ export const candidateIdsForSource = (
source: SourceFile,
index: SourceToBundleCandidateIndex,
): string[] => {
const canonicalPath = source.path.normalize("NFC");
const groups = [
source.sha256 === null ? undefined : index.byDigest.get(source.sha256),
index.byPathSuffix.get(source.path),
canonicalPath === source.path
? undefined
: index.byPathSuffix.get(canonicalPath),
index.byBasename.get(posix.basename(source.path)),
];
const seen = new Set<string>();
@@ -220,6 +228,14 @@ const candidateSignals = (
signals.push(
signal("source-map-original-path", source.path, sourceMapPaths),
);
const unicodeEquivalentPaths = matchingUnicodePaths(
source.path,
current.paths,
);
if (unicodeEquivalentPaths.length > 0)
signals.push(
signal("unicode-equivalent-path", source.path, unicodeEquivalentPaths),
);
const otherPaths = current.paths.filter(
({ kind }) => kind !== "source-map-original",
);
@@ -286,6 +302,27 @@ const matchingPaths = (
)
.map(({ value }) => value);
const matchingUnicodePaths = (
sourcePath: string,
currentPaths: readonly CurrentPath[],
): string[] => {
const canonicalSource = sourcePath.normalize("NFC");
return currentPaths
.filter(({ value, allowSuffix }) => {
if (
value === sourcePath ||
(allowSuffix && value.endsWith(`/${sourcePath}`))
)
return false;
const canonicalValue = value.normalize("NFC");
return (
canonicalValue === canonicalSource ||
(allowSuffix && canonicalValue.endsWith(`/${canonicalSource}`))
);
})
.map(({ value }) => value);
};
const addJsonPath = (paths: CurrentPath[], value: unknown): void => {
if (typeof value === "string") addPath(paths, "observation-path", value);
};
+2 -1
View File
@@ -204,7 +204,8 @@ const buildReconstructionSummary = (
({ reconstruction }) => reconstruction.kind === "semantic-pseudocode",
).length,
total_text_bytes: methods.reduce(
(total, { reconstruction }) => total + reconstruction.text_length,
(total, { reconstruction }) =>
total + Buffer.byteLength(reconstruction.text, "utf8"),
0,
),
});
+83
View File
@@ -1,5 +1,11 @@
import { z } from "zod";
export interface OffsetDateTimeInstant {
readonly epochSecond: number;
/** Exact fractional second with insignificant trailing zeroes removed. */
readonly fraction: string;
}
/**
* ISO-8601 date-time that requires an explicit UTC offset.
*
@@ -19,3 +25,80 @@ export const offsetDateTimeSchema = z
/^(?:(?:\d\d[2468][048]|\d\d[13579][26]|\d\d0[48]|[02468][048]00|[13579][26]00)-02-29|\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\d|30)|(?:02)-(?:0[1-9]|1\d|2[0-8])))T(?:(?:[01]\d|2[0-3]):[0-5]\d(?::[0-5]\d(?:\.\d+)?)?(?:Z|(?:[+\x2d](?:[01]\d|2[0-3]):[0-5]\d)))$/u,
"Use an ISO-8601 date-time with an explicit UTC offset",
);
/** Parse an already validated offset timestamp without losing sub-millisecond precision. */
export const parseOffsetDateTimeInstant = (
value: string,
): OffsetDateTimeInstant => {
const match = /^(.*?)(?:\.(\d+))?(Z|[+\x2d]\d{2}:\d{2})$/u.exec(value);
if (match === null)
throw new TypeError("Expected a validated offset date-time.");
const [, dateTime, fractionalDigits, offset] = match;
if (dateTime === undefined || offset === undefined)
throw new TypeError("Expected a validated offset date-time.");
const wholeSecondMilliseconds = Date.parse(`${dateTime}${offset}`);
if (
!Number.isSafeInteger(wholeSecondMilliseconds) ||
wholeSecondMilliseconds % 1000 !== 0
)
throw new TypeError(
"Offset date-time is outside the supported time range.",
);
return {
epochSecond: wholeSecondMilliseconds / 1000,
fraction: fractionalDigits?.replace(/0+$/u, "") ?? "",
};
};
/** Convert a Date.now-style integer into the same exact comparison shape. */
export const offsetDateTimeInstantFromEpochMilliseconds = (
epochMilliseconds: number,
): OffsetDateTimeInstant => {
if (!Number.isFinite(epochMilliseconds))
throw new TypeError("Expected finite epoch milliseconds.");
if (Math.abs(epochMilliseconds) > Number.MAX_SAFE_INTEGER)
return { epochSecond: epochMilliseconds / 1000, fraction: "" };
const epochSecond = Math.floor(epochMilliseconds / 1000);
const millisecondsWithinSecond = epochMilliseconds - epochSecond * 1000;
return {
epochSecond,
fraction: fractionalSecondFromMilliseconds(millisecondsWithinSecond),
};
};
const fractionalSecondFromMilliseconds = (milliseconds: number): string => {
if (milliseconds === 0) return "";
const [coefficient = "", exponentText] = String(milliseconds)
.toLowerCase()
.split("e");
const exponent = exponentText === undefined ? 0 : Number(exponentText);
const point = coefficient.indexOf(".");
const decimalPosition =
(point < 0 ? coefficient.length : point) + exponent - 3;
const digits = coefficient.replace(".", "");
const fraction =
decimalPosition <= 0
? `0.${"0".repeat(-decimalPosition)}${digits}`
: decimalPosition < digits.length
? `${digits.slice(0, decimalPosition)}.${digits.slice(decimalPosition)}`
: `${digits}${"0".repeat(decimalPosition - digits.length)}`;
return (
fraction.includes(".") ? fraction.slice(fraction.indexOf(".") + 1) : ""
).replace(/0+$/u, "");
};
/** Compare instants exactly while treating absent fraction digits as zeroes. */
export const compareOffsetDateTimeInstants = (
left: OffsetDateTimeInstant,
right: OffsetDateTimeInstant,
): number => {
if (left.epochSecond !== right.epochSecond)
return left.epochSecond < right.epochSecond ? -1 : 1;
const length = Math.max(left.fraction.length, right.fraction.length);
for (let index = 0; index < length; index += 1) {
const leftDigit = left.fraction.charCodeAt(index) || 48;
const rightDigit = right.fraction.charCodeAt(index) || 48;
if (leftDigit !== rightDigit) return leftDigit < rightDigit ? -1 : 1;
}
return 0;
};
+110
View File
@@ -0,0 +1,110 @@
import { z } from "zod";
import { localPathStringSchema } from "../localPath.js";
import { digestSchema } from "../digests.js";
/**
* Caller intent for packet-capture analysis through caller-supplied tshark
* and capinfos. REA never installs Wireshark or its tools.
*/
export const pcapInputSchemas = {
inspect_pcap_client: z.strictObject({}),
inspect_pcap_capture: z.strictObject({
path: localPathStringSchema.describe(
"Local pcap or pcapng capture file to inspect",
),
}),
} as const;
/** Supported pcap-backed operations. */
export type PcapOperation = keyof typeof pcapInputSchemas;
/** Validated pcap requests carry their selected operation. */
export type PcapRequest = {
[Name in PcapOperation]: {
operation: Name;
input: z.infer<(typeof pcapInputSchemas)[Name]>;
};
}[PcapOperation];
/** Validate operation and input together so their types remain correlated. */
export const pcapRequestSchema = z.discriminatedUnion("operation", [
z.strictObject({
operation: z.literal("inspect_pcap_client"),
input: pcapInputSchemas.inspect_pcap_client,
}),
z.strictObject({
operation: z.literal("inspect_pcap_capture"),
input: pcapInputSchemas.inspect_pcap_capture,
}),
]);
const client = z.strictObject({
tshark_command: z.string().min(1),
tshark_source: z.enum(["environment", "path"]),
tshark_version: z.string().min(1).nullable(),
capinfos_command: z.string().min(1),
capinfos_source: z.enum(["environment", "path"]),
capinfos_version: z.string().min(1).nullable(),
});
const endpoint = z.strictObject({
address: z.string().min(1),
port: z.number().int().positive().nullable(),
packet_count: z.number().int().positive(),
});
const dnsQuery = z.strictObject({
query: z.string().min(1),
response_addresses: z.array(z.string().min(1)),
response_address_count: z.number().int().nonnegative(),
});
const protocolCount = z.strictObject({
protocol: z.string().min(1),
packet_count: z.number().int().positive(),
});
/** Portable pcap observations with explicit coverage limitations. */
export const pcapResultSchemas = {
inspect_pcap_client: z.strictObject({
client,
}),
inspect_pcap_capture: z.strictObject({
client,
target: z.strictObject({
path: localPathStringSchema,
bytes: z.number().int().nonnegative(),
sha256: digestSchema,
}),
/** File format as capinfos reports it (pcap vs pcapng). */
file_type: z.string().min(1).nullable(),
encapsulation: z.string().min(1).nullable(),
packet_count: z.number().int().nonnegative().nullable(),
capture_start: z.string().nullable(),
capture_end: z.string().nullable(),
capture_duration_seconds: z.number().nonnegative().nullable(),
snapshot_length: z.number().int().positive().nullable(),
/** Distinct endpoints with packet counts, bounded. */
endpoints: z.array(endpoint),
endpoint_count: z.number().int().nonnegative(),
/** DNS queries with resolved addresses, bounded. */
dns_queries: z.array(dnsQuery),
dns_query_count: z.number().int().nonnegative(),
/** TLS Server Name Indication values observed in handshakes. */
tls_sni_values: z.array(z.string().min(1)),
tls_sni_count: z.number().int().nonnegative(),
/** Protocol distribution (frame.protocols column counts). */
protocol_distribution: z.array(protocolCount),
protocol_count: z.number().int().nonnegative(),
/** HTTP request URIs seen on plain-text ports, bounded. */
http_request_uris: z.array(z.string().min(1)),
http_request_uri_count: z.number().int().nonnegative(),
http_uri_redaction_count: z.number().int().nonnegative(),
unattributed_dns_addresses: z.array(z.string().min(1)),
unattributed_dns_address_count: z.number().int().nonnegative(),
unattributed_dns_packet_count: z.number().int().nonnegative(),
coverage: z.enum(["complete", "partial"]),
diagnostics: z
.strictObject({ capinfos_stderr: z.string(), tshark_stderr: z.string() })
.optional(),
}),
} as const;
+34
View File
@@ -0,0 +1,34 @@
import { createEvidence } from "../evidence.js";
import { jsonObjectSchema, type JsonValue } from "../jsonValue.js";
import type { PcapRequest } from "./pcapAnalysis.js";
/** Domain facts needed by both successful and retained partial observations. */
interface PcapObservation {
readonly subject: Parameters<typeof createEvidence>[0] | null;
readonly provider: Parameters<typeof createEvidence>[1];
readonly result: JsonValue;
readonly rawResult: JsonValue | null;
readonly limitations: readonly string[];
readonly locations: NonNullable<
Parameters<typeof createEvidence>[2]["locations"]
>;
}
/** One owner commits packet aggregation provenance and caller intent into Evidence. */
export const createPcapEvidence = (
request: PcapRequest,
observation: PcapObservation,
) =>
createEvidence(observation.subject ?? undefined, observation.provider, {
operation: request.operation,
parameters: jsonObjectSchema.parse(request.input),
result: observation.result,
rawResult: observation.rawResult,
limitations: observation.limitations,
locations: observation.locations,
confidence:
request.operation === "inspect_pcap_capture" ? "derived" : "observed",
...(request.operation === "inspect_pcap_capture"
? { authority: "historical-reference" as const }
: {}),
});

Some files were not shown because too many files have changed in this diff Show More