refactor(status): one evidence store + zero-spawn status/dashboard (ADR-0063) - #252
Merged
Merged
Conversation
- Create src/lib/evidence.mjs with EvidenceRecord envelope (version: 1) - evidenceFile(kind, id) returns path to kind/sanitized-id.json under evidenceDir - writeEvidence() writes atomically, returns false on failure (never throws) - readEvidence() reads record if valid; computes ageMs, stale, invalidated - stableInputsKey(parts) generalizes live-check-evidence's stable/digest pattern - describeAge(ms) generalizes formatLiveCheckAge (just now/Xm/Xh/Xd ago) - Add evidenceDir() export to src/lib/paths.mjs (stateBase/agentic-kit/evidence) - All tests pass: 15/15 (round-trip, age/staleness/invalidation, edge cases)
…output
- Update module docs: sanitization is a NEW convention introduced here, not
matching existing patterns (live-check-evidence.mjs validates fixed ids;
ruflo-components/evidence.mjs takes full file paths, not ids)
- Add explicit sanitizeId function documentation
- Extend evidenceFile() jsdoc to document the sanitization convention
- Fix test to assert EXACT output: evidenceFile('k', 'a/b\c:d') ends with
'k/a_b_c_d.json', verifying filesystem safety for arbitrary ids like
@claude-flow/memory (which contain forward slashes)
- Add dedicated sanitization test case with traversal-shaped ids
- Tests now 16/16 (added sanitization verification test)
…s 2-7 touch anything Add tests/helpers/spawn-guard.mjs, a --import preload that patches node:child_process's spawn/execFile/execFileSync/spawnSync (plus execSync/fork for completeness) to append every call to a ledger file named by AK_SPAWN_LEDGER_FILE. No product code changes: a ledger seam inside exec.mjs's run() would under-count, since ~30 files spawn child_process directly. tests/kit/status-zero-spawn.test.mjs runs src/commands/status.mjs's collect() inside a sandboxed child Node process with the guard active and records the assertion as test.todo — it is currently RED (16 real spawns on this machine today: which x6, npm x7, node x2, ps x1) but test.todo keeps the gate green. Task 7 deletes the word todo once every spawn path above it is closed.
recordLiveCheck/readLiveCheck now route through evidence.mjs's writeEvidence/readEvidence instead of writing their own JSON file directly, moving storage from agentic-kit/live-checks/<id>.json to agentic-kit/evidence/live-check/<id>.json. Every exported name and its external behavior/return-shape stays pixel-identical for callers; only the module's internals changed. liveCheckDir() now points at the new location, since three test files rely on it to scope resets and write-failure sabotage to the real storage path — pointing it at the old path would silently stop testing what those assertions claim to test.
…d evidence directory
rufloComponentsEvidenceFile() now resolves through evidence.mjs's evidenceFile('ruflo-component', 'machine') instead of hand-joining a path beside maintenanceControlDir(), moving the cache from <stateBase>/agentic-kit/ruflo-components-evidence.json to <stateBase>/agentic-kit/evidence/ruflo-component/machine.json. readEvidenceCache/writeEvidenceCache/collectEvidence and the classify()/evidenceState() freshness logic are untouched, since this evidence kind never adopted the generic envelope's stale/invalidated computation.
…call Evidence-back rufloRuntimeNatives(): a plain ak status (refresh:false) reuses a still-fresh, matching native-runtime probe from the evidence store instead of always spawning a node subprocess per ruflo memory context; ak status --refresh (refresh:true, the default) still probes unconditionally. sync's native repair now records the same evidence after healing, so a plain status right after sync shows the repaired state without a second refresh.
…in status call detectHosts/hostInstallState/hostExecutable now reuse fresh (<=6h), inputs-matching evidence instead of always spawning `which`/`<bin> --version`; collectIntegrationFacts and status/sections/hosts.mjs thread `refresh` through so only a plain `ak status` (refresh:false) skips the probe. Every other caller (setup, about, x host, sync, x verify) keeps probing unconditionally, since none of them pass refresh at all. Any real probe from any caller still records evidence, so a later plain status shows the repaired/discovered state without a second refresh - mirroring the native-runtime pattern. That surfaced three pre-existing "command X is read-only" tests (status, sync --dry-run, verify) that didn't previously account for this shared probe cache; extend assertUnchanged with an optional ignore list and use it to keep those tests covering everything BUT the evidence store, rather than weakening what they check.
…call detect() caches its whole assembled facts object under one evidence record (mirroring Task 3's collectEvidence pattern), keyed on the desired intent, ownership target keys, and PATH. A plain `ak status` (collectDejaVuRows's refresh:false) reuses a fresh, matching record instead of running the binary/package/host-presence probes, doctor, and target observer. Every internal reuse of detect() - plan()'s own facts fallback and the operation executors' post-mutation re-verification calls - never passes refresh, so it keeps defaulting to true and always probes for real; a fresh status-cached record can never leak into apply()'s verification of its own mutations.
sync.mjs calls status.collect() internally twice (plan build, post-heal
convergence re-check), both before --dry-run's early return. On a cold
evidence cache these reached detectHosts/rufloRuntimeNatives/deja-vu
detect()'s probe-and-record branch, so a real or --dry-run sync wrote
evidence files as a side effect of merely reading current state.
Thread a record parameter (default true) through collect() -> detectHosts/
hostInstallState/hostExecutable/rufloRuntimeNatives/deja-vu detect(),
gating only the writeEvidence call, never the probe or its return value.
sync.mjs's two plan-computation collectFn() calls now pass record: false.
Also compute an accurate source ('status' vs 'status-refresh') in
status.mjs's collect() instead of hardcoding 'status-refresh' even on a
plain, non-refresh call.
versions/ruvector/ruvnet-brain/self sections silently dropped the refresh key already present in status.mjs's shared collect() ctx, so `ak status --refresh` had no effect on version-drift rows. Their libraries already have a correct TTL-cached force parameter (Ruling A/B); this only threads refresh into it at the four call sites.
globalRoot() (src/lib/paths.mjs) spawned `npm root -g` unconditionally on every call, at most once per process via its own in-process memo, but a fresh `ak status` invocation is a fresh process. It backs rufloRoot()/aqeRoot()/installedVersion() and dozens of their own callers throughout the codebase, so it needed its own evidence cache (kind 'npm-global-root', id 'machine', 24h TTL) rather than a new refresh/record parameter threaded through every caller. Both refresh and record default to false here, unlike every other evidence-gated check in this branch: globalRoot() is reached from far more contexts than status.mjs's own collect() tree (setup, sync, uninstall, heal, audit, and dozens of their tests), so a record- defaults-true design would make evidence writes a silent side effect of nearly any ak invocation. Persisting is the responsibility of the one caller that owns the refresh/record contract end-to-end. paths.mjs cannot import evidence.mjs's readEvidence/writeEvidence: evidence.mjs itself imports paths.mjs for evidenceDir(), and paths.mjs is almost always the first of the pair loaded, so the reverse edge deadlocks on evidence.mjs's own top-level `evidenceDir = paths. evidenceDir` line. A local envelope writes the identical on-disk shape instead, without creating that cycle.
listDaemons()'s processSweep() (src/lib/daemons.mjs) spawned `ps` (POSIX) or a PowerShell Get-CimInstance query (Windows) unconditionally on every call. This is genuinely time-sensitive information (a daemon can start or die within minutes), so it gets a short evidence TTL: kind 'daemon-sweep', id 'machine' (the sweep enumerates every process on the machine, not a per-cwd fact), 5 minutes. refresh/record/source thread through listDaemons() into processSweep() only; the registryWorkspaces()/pidfile-reading loop above it is already spawn-free and stays unconditional. refresh defaults to true so every non-status caller (sync's own daemon-reaping step, x/daemon-gc.mjs, cachedDaemonCount()) keeps its unconditional sweep; status/sections/ daemons.mjs's own collect() threads refresh/record/source from ctx.
claudeLauncherUnavailable() (src/lib/mcp.mjs) — the check the mcp section runs before it would ask sync to register/re-register claude-flow through ak's launcher — spawned `which ak` (and, when ak is present, `ak x ruflo-mcp --help`) unconditionally on every call. Evidence kind 'ak-launcher', id 'machine', 6h TTL, invalidated by a PATH change. refresh defaults to true (Ruling B: this was already a conditionally- probing check, gated by registerWouldWrite() in the mcp section, just with no gating concept of its own); status/sections/mcp.mjs's own collect() threads refresh/record/source from ctx into launcherCheck(). mcp-scopes.test.mjs's pre-existing direct-call test for this function has no HOME/XDG sandbox of its own and is about the probe DECISION, not evidence persistence (covered by the new ak-launcher-evidence.test.mjs), so it now passes record: false explicitly.
The providers section's hostManagementRows() called have(h.bin) for every non-enabled host to render its "Found, not managed" / "Not installed" row — the exact same "is this host's bin on PATH" fact providers.mjs's detectHosts() (via collectIntegrationFacts, already evidence-cached since the branch's hosts.mjs work) computes once per collect(), for every host regardless of enablement. Reusing integrationFacts.hosts[id].present removes a duplicate probe of an identical fact instead of adding a second evidence kind for it; have() stays as a fallback for a direct caller that passes no integrationFacts.
globalRoot() (src/lib/paths.mjs) now defaults refresh:false/record:false so its evidence write is never a silent side effect of the dozens of non-status contexts that call it bare. status.mjs's collect() is the one caller that actually owns this branch's refresh/record contract, so it warms globalRoot's in-process memo once, right after loadKitConfig(), with the real refresh/record — every other bare globalRoot() call below (natives, versions, providers, daemons, …) then reuses that memo for free, a plain `ak status` still persists evidence for a later process to reuse, and `--refresh` still forces a fresh read. Also updates the one --refresh help sentence, unchanged since it named only ruflo-component evidence: it now covers ruflo components, native runtime, host setup, deja-vu, version drift, and the rest of what a plain status call reuses from cache.
…rced A single fresh-sandbox collect() call can never show zero spawns (Ruling A requires a first probe on a cold cache), so the fixture now makes three collect() calls in the same child process/state dir, marking ledger boundaries between them: a cold-cache call (probes, sanity-checked non-empty), a warm-cache call (the real, enforced assertion — every spawn path this branch closed is now silent), and a refresh:true call (still probes everything again). The warm-cache call's only remaining, explained exception is versions.mjs's own `npm view <pkg>@<tag> version` registry lookups: driftReport()/selfDrift() only persist their kit.json TTL cache after at least one live fetch succeeds (deliberately, so a total npm outage is never mistaken for "confirmed unchanged" — see their own comments), and this harness's PATH is deliberately broken so npm can never succeed. Their caching is proven correct elsewhere, with an injectable, succeeding fetchLatest, in status-version-drift-refresh.test.mjs; versions.mjs itself is out of this task's scope to touch.
…tests Adds a regression test proving status.mjs's collect() is the sole persister of npm-global-root evidence: a bare globalRoot() call never writes, whether it runs before collect() (cold memo, no evidence yet), right after it (memo-hit), or later with a cold memo but warm evidence (cache-hit) — only collect()'s own record:true call, once, ever writes. Mutation-tested by temporarily disabling status.mjs's warming call and confirming the new test fails with the expected assertion, then reverting. Tightens status-zero-spawn.test.mjs's refresh:true assertion: it previously only checked the third call's ledger was non-empty, which passes on versions.mjs's unconditional npm view lookups alone even if --refresh silently stopped reaching globalRoot()/processSweep()/ claudeLauncherUnavailable() specifically. It now asserts each of this task's own re-probes (npm root -g, which claude/codex/opencode, which ak, ps -eo pid=,args=) by name. Mutation-tested by temporarily hardcoding refresh:false in the mcp section's launcherCheck() call and confirming the assertion fails naming exactly that offender, then reverting.
loadMaintenanceWorkspace saved lastView unconditionally on every call, and poll.mjs calls it every ~30s while the Maintenance tab is open, rewriting preferences.json even when the view hasn't changed. The client now tracks the last-saved view and only saves on an actual change; savePreferences also skips its own write when the computed next preferences equal what's already stored, as a server-side guard for any other caller.
tests/kit/maintenance-management-activity.test.mjs already imports createPreferencesStore and has its own Preferences (MNT-PRV-006/007) section; the dedup coverage belongs there instead of in a new, duplicate file.
collect({ refresh: false }) is genuinely spawn-free on a warm cache
(Tasks 1-7), so the dashboard's /api/status no longer needs execFile's
subprocess boundary to reach status.mjs's own collector. Replaces
shellOutStatus with inProcessStatus, which calls status.mjs's collect()
directly and adds back the two guarantees the subprocess used to give
for free: a 30s timeout (Promise.race-style, via setTimeout) so a hung
probe can no longer stall the server's event loop, and per-request cwd
threaded straight into collect() rather than any implicit process.cwd()
fallback. The fetchStatus injection seam's name and contract are
unchanged, so no existing test needed touching.
status.mjs's inline worst-level computation is extracted into an
exported worstLevel(rows), reused by both run()'s exit code and the
dashboard's new in-process provider (which has no CLI process to derive
an exit code from).
Tests: a real dashboard server started as a spawn-guarded child process
proves the in-process /api/status path is spawn-free on a warm cache and
byte-identical (rows/overall) to `ak status --json`; the two-poll-tick
dashboard-cost test proves a second 30s poll starts zero processes and
transfers no more data than the first; a cwd-honored regression test
(mutation-tested against providers/daemons) proves collect({ cwd }) is
never silently satisfied by process.cwd(), which matters now that the
dashboard can serve a request for a project other than its own launch
cwd.
…em/summary Decision 8's allow-list projected only catalog; the other four sections passed through GET /api/system's payload verbatim, which is where the remaining ~2.26 MB lived (storage+install+projects+consumers were 76% of a real-machine reconstruction). Each gets the same allow-list treatment catalog already had, keeping only the fields the System page's client code actually reads (a per-project stack detection, per-tool native-addon list, and full storage-tree metadata never render). GET /api/system and `ak system --json` are untouched. byLanguage stays on a project's loc despite being unread when languages is present: a carried-forward snapshot older than the languages field can still reach this endpoint, and the client falls back to byLanguage then. ADR-0025 and the machine-footprint/context-map DDD docs are updated to describe the extended projection.
Records what Tasks 1-11 of the evidence-store branch actually shipped: the shared evidence envelope and age rule, the nine evidence kinds, the record parameter, the npm-global-root design exception, the dashboard poll's spawn and write cost fixes, and the /api/system/summary projection. Amends ADR-0025, ADR-0048, ADR-0053, ADR-0055 and ADR-0058 with one-line Updated notes each; adds the matching bullet to docs/adr/README.md.
status --help's read-only claim was imprecise: a cold evidence cache means ak status may write its own local cache under <state>/agentic-kit/evidence/ on a first check, by design. UPGRADING.md gets a short note on the one-time cold-cache warmup after upgrading and the now-orphaned pre-branch storage locations (live-checks/, ruflo-components-evidence.json).
- host-health-evidence.mjs is an HMAC input-fingerprint helper host-readiness.mjs uses for its own in-memory cache invalidation, not itself a persisted store or the connection-check proof; ADR-0053's Updated line and ADR-0063 both named the wrong file/role. - every writeEvidence call actually populates inputs with the real hashed object; only live-check passes null. The prior text had this backwards. - eight kinds use the generic envelope directly, not nine; ten evidence-kind directories exist in total (adding ruflo-component and npm-global-root, which don't route through it). - --live is not free of the evidence store's refresh mechanics: its providers check calls collectIntegrationFacts with no refresh argument, defaulting to refresh:true and persisting host-setup evidence as a side effect. - UPGRADING.md's cold-cache note claimed an "unchecked" label that does not exist anywhere in the code; replaced with the real per-kind behavior (gated kinds probe silently, ruflo-component reads unknown, live-check rows are simply absent until first recorded).
Evidence for host-install-method, host-launch, host-setup, and daemon-sweep was recorded BEFORE a repair (the plan-time probe), and nothing re-probed after the repair landed — a host `ak sync` just installed, or a daemon it just reaped, could still read back as failing/stale for up to their TTL, and `ak sync`'s own converge proof could prove convergence against evidence it never refreshed. sync.mjs's `hosts` and `daemons` steps, setup.mjs's installEnabledAbsentHosts, x/host.mjs's installPickAbsentHosts, and x/daemon-gc.mjs now re-probe and re-record with `refresh: true, record: true` immediately after a successful install/repair/reap, each under its own `source` label. sync.mjs also gains `refreshPlanHosts`, a narrow sibling to the existing `refreshPlanDrift`, so a not-yet-stale but simply WRONG host-evidence row (one that changed since it was last recorded) can't hide a needed repair from the plan either. A blanket `refresh: true` on sync's own `collect()` calls was tried first and reverted: it forces every evidence-gated kind fresh, not just host/daemon evidence, which broke `--dry-run`'s touches-nothing contract for ruflo-component evidence (record:false cannot gate it — it doesn't route through the generic envelope) and several tests' `_setGlobalRootForTest` fixture, since `refresh: true` bypasses globalRoot's in-memory memo. The converge-proof `collect()` call now passes `record: true` (was `record: false`) so a cache-miss probe it triggers persists for a plain `ak status` to reuse right after.
status/sections/hosts.mjs's collect() accepted refresh/record but never
computed or passed a source, so a plain `ak status` run's host-install-method
and host-launch evidence writes always carried the functions' own hardcoded
fallback ('status-refresh') instead of an accurate label. It now computes
`source` the same way status.mjs's own collect() already does
(refresh ? 'status-refresh' : 'status') and threads it through both
deps.installState and deps.executable calls.
x/verify.mjs's verifyProviders() also called collectIntegrationFacts with no
source; it now passes source: 'verify'.
Ruling B and "The record parameter" claimed sync needed no change / that
sync's two internal collect() calls only differ on record. Both are now
false: sync gained refreshPlanHosts (a narrow, host-scoped forced refresh
ahead of the plan) and the converge-proof call's record flipped to true.
Corrected both sections to describe what actually shipped, including why a
blanket refresh:true on collect() was tried and reverted.
Also softened the dashboard warm/cold payload comparison: the test only
asserts a 10%-or-2KB budget, not byte equality (the byte-identical figure is
an observed data point on the measuring machine, not the enforced guarantee).
Replaced six source comments' ephemeral task/branch labels with an ADR-0063
reference or nothing: status/sections/providers-status.mjs ("Task 5"),
dashboard-server.mjs (two spots), heal.mjs ("(Task 4)"), paths.mjs ("out of
this task's scope to touch"); sync.mjs's own label was already replaced in
the preceding fix commit.
On win32 paths.mjs reads APPDATA for the kit config and never XDG_CONFIG_HOME, so the globalRoot persistence test's collect() call wrote the real %APPDATA%\agentic-kit\kit.json on Windows CI while passing on Linux and macOS. The test now redirects and restores both. spawn-env-guard gains the in-process twin of its child-env rules: a test that assigns an XDG_* base must also assign its Windows twin (XDG_CONFIG_HOME/APPDATA, XDG_STATE_HOME/LOCALAPPDATA). It failed on this file alone before the fix, so the leak now shows on every OS.
Three Windows-only breaks in the spawn-guard tests: - `--import` takes a module URL. A bare absolute path such as D:\...\spawn-guard.mjs fails with ERR_UNSUPPORTED_ESM_URL_SCHEME (protocol 'd:'), so every guarded child died at startup. Both launch sites now pass pathToFileURL(...).href. - The dashboard tests deleted the child's working directory right after kill(). Windows refuses to remove a running process's cwd, so a new stopGuardedDashboard() waits for the child to exit first (SIGKILL after 2s), mirroring the stock-gateway test's stop(). - The --refresh assertion named POSIX commands. On Windows have() resolves a binary without spawning, so the host and ak-launcher probes leave no ledger line, and the daemon sweep runs PowerShell. Windows now expects npm root -g and the PowerShell sweep; Linux and macOS keep the full list.
The branch 6a plan sits beside the program plan in docs/superpowers/plans, so ../ pointed one level too high and failed the internal link check.
execSync runs through the platform shell (cmd.exe on Windows), and the exact-four count was only ever observed on macOS. Asserting that spawnSync/execFileSync, execSync and fork each record keeps what the smoke test proves without assuming how many lines one shell call makes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Branch 6a of the remediation program (Wave 3). Builds one shared evidence-caching layer so
ak statusand its dashboard stop spawning subprocesses on every call, then uses it to close every remaining spawn path and swap the dashboard's/api/statusto compute in-process.src/lib/evidence.mjs: one envelope ({version, kind, id, source, checkedAt, inputsKey, inputs, result}) + one age rule, backing 9 evidence kinds (live-check, ruflo-component's storage location, native-runtime, host-setup/host-install-method/host-launch, companion-lifecycle, npm-global-root, daemon-sweep, ak-launcher).ak statuson a warm cache is now genuinely zero-spawn — enforced by a real test (tests/kit/status-zero-spawn.test.mjs, no longertest.todo)./api/statuscomputes in-process instead of shelling out to the CLI (measured: 14→0 spawns, byte-stable payload on a warm poll tick — the Review Focus item this whole branch was scoped around)./api/system/summarycut ~96% on a representative fixture (previously ~2.4MB on the real machine).ak syncnow re-records evidence right after a host/daemon repair, and force-refreshes host evidence before planning — closing a real bug the final whole-branch review caught: a repairak syncjust made could stay invisible to its own convergence proof (and to plainak status) for up to 6 hours.One deliberate design exception, documented in ADR-0063:
globalRoot()(npm root -g, insrc/lib/paths.mjs) uses a hand-written envelope, notevidence.mjsdirectly — a real circular import (evidence.mjsimportspaths.mjsforevidenceDir()), confirmed by reproduction.Full design record:
docs/adr/0063-evidence-store-and-refresh-vocabulary.md.Test plan
node scripts/run-tests.mjs unit— Node 26.4.0 and Node 22.22.3, both clean (~5600 tests, 0 fail)node scripts/run-tests.mjs ui— cleannpx tsc -p tsconfig.json,npx eslint(0 new warnings vs main),npx markdownlint-cli2,node scripts/build-check.mjsXDG_STATE_HOMEredirected): plainak statustwice in a row shows zero evidence-file changes on the second run (0.44s, cache reused);ak sync --dry-runwrites zero new evidence files🤖 Generated with Claude Code