Audit runs can leave .posthog-audit-checks.json in the user's project #1326

Problem

The wizard writes .posthog-audit-checks.json into the project root before the audit agent starts. No wizard code deletes it. The audit skills end with an instruction for the agent to run rm -f .posthog-audit-checks.json. The file is removed only if the agent runs that last step.

When the agent skips the step, the run still completes and the user keeps an untracked JSON file in their repo root, next to their own code. The next run does not break: the watcher ignores a leftover ledger and the seed replaces it. But the file is clutter, and a user can commit it by accident.

There is a second, smaller problem in the same path. On pi, a scoped rm of that file is logged as Denying bash command (not in allowlist) and captured as a bash denied event. Then the scoped-rm rule from #834 lets it run. The log and the telemetry say "denied". The command ran.

Evidence

Real-TUI sweep of the audit program on the wizard-workbench app audit/posthog-demo-3000. Harness pi (binding), sequence linear, model openai/gpt-5.6-sol.

• Refactor stack B2 head d938a43b (feat(programs): B2 surface shell — runProgram types and stub signatures #1307). The agent read step 6, uploaded the notebook, resolved write-report and upload-notebook, and published the handoff. It never ran the rm. After exit, .posthog-audit-checks.json (3929 bytes) was still in the project root.
• Release A top 200961f8 (refactor(agent): WIP publish the agent through entry modules #1303). The agent ran rm -f .posthog-audit-checks.json and the file was removed. The wizard log shows the misleading pair:

  [pi] → bash {"command":"rm -f .posthog-audit-checks.json"}
  Denying bash command (not in allowlist): rm -f .posthog-audit-checks.json
  [pi] ← bash: {"content":[{"type":"text","text":"(no output)"}]}
  

The same prompt, harness and model gave both outcomes. The cleanup depends on the agent.

Code on main (d8486dc1):

• src/lib/programs/audit/seed.ts:118-124: seedAuditLedger writes the ledger. src/lib/programs/audit/index.ts:26-29 and :58 call it before the agent starts. The wizard creates the file.
• src/lib/programs/run-agent-legacy.ts:81-95: runProgramAgent starts the ledger watcher and stops it in finally. Nothing removes the file.
• src/lib/programs/audit/ledger-watcher.ts:32: "A ledger an earlier run left behind stays ignored until this run writes." The code already expects leftovers.
• src/lib/runners/run-wizard.ts:223: the task stream reads the checks from the session (getAuditChecks(session)), not from the file. Nothing reads the file after the run.
• src/agent/runner/harness/pi/index.ts:741-749: pi deletes .posthog-events.json host-side after the run. There is no matching cleanup for the audit ledger.
• src/agent/runner/harness/pi/runtime-notes.ts:53: pi still tells the agent that bash is only for install, build, typecheck, lint and format commands. pi: prompt drift behind enforcement — agents self-censor rm, stall on schema lag, and trip fence false-positives #894 names this line as the reason agents skip rm. fix(pi): sync bash prompt with the fence #1000 would update it.
• src/agent/runner/harness/pi/security.ts:372-385: evaluateToolCall calls wizardCanUseTool first and applies isScopedFileRemoval after.
• src/agent/agent-interface.ts:523-528: wizardCanUseTool logs Denying bash command and captures bash denied before it returns the deny that pi then overrides.

Skill side, context-mill main:

• context/skills/audit/description.md:49 and context/skills/audit/references/6-report.md:284 tell the agent to delete the ledger.
• The same step is in the six audit-* leaf skills, for example context/skills/audit-events/references/4-report.md:21, and in context/skills/events-audit/references/6-dashboard.md:270.
• context-mill#306 (open) moves reports to publish_handoff and keeps the agent-driven delete.

The refactor stack has the same behavior. On d938a43b, src/programs/program-file-watchers.ts:16-58 owns the ledger for the run, but stop() only stops the watcher (src/programs/run-program.ts:706-708).

Prior work. #833 added a host-side cleanupArtifacts seam for .posthog-events.json. It was closed because it worked around pi blocking rm, and #834 fixed that instead. This case is different. rm works here, and the agent did not run it. The wizard also seeds this file, so the program is the file's owner. #898 and #901 fixed the misleading deny log and telemetry. Both were closed without merging, and main still has the old order. #612 (open) raises the same class of leak for /tmp scratch files that skills name themselves, and leans toward a skill-side fix because the wizard does not know those names. That concern does not apply here: the program declares the name in auditLedgerFile, so the wizard can delete it without learning skill filenames. #894 (open) would make the agent more likely to run the rm, but cleanup would still depend on the agent.

Suggested fix

1. Delete the ledger in the wizard after the agent run settles, on every exit path. On main, remove path.join(session.installDir, programConfig.auditLedgerFile) with force: true after ledger.stop() in the finally of runProgramAgent, and in the registerCleanup callback so aborts clean up too. On the stack, do it in startProgramFileWatchers().stop() or next to it in the run-program.ts finally. This covers audit, events-audit and the family leaves, because they all set auditLedgerFile.
2. In context-mill, remove the "delete .posthog-audit-checks.json" step from the audit skills, or mark it optional, once the wizard deletes the file. Then the skill and the wizard do not both own the file.
3. In evaluateToolCall, decide the scoped-rm rule before the deny is logged and captured. Keep the disallowedTools check. Log an allow line for the command that runs, and do not capture bash denied for it. fix(pi): stop recording allowed scoped rm as bash denied #898 has a small version of this change.