Inspector roadmap

[phodal]

Suggested evolution: from Session Inspector to Delivery Debugger

This PR establishes a strong first slice of the Inspector: a read-only delivery provenance workbench that brings Feature/Story intent, privacy-safe prompts, normalized tool activity, sessions, commits, and changed files into one review surface.

I think we should treat this as the beginning of an Inspector platform, rather than continuing to grow it as a larger Session Viewer.

Session Inspector
  → Delivery Provenance Inspector
  → Delivery Debugger
  → Compare / Eval Workbench
  → Harness Intelligence

What should remain inside this PR

The current scope is already a useful and coherent boundary:

• Feature Tree and Date navigation over the same evidence;
• provider-neutral NormalizedToolActivityV1;
• Prompt → Activity → Commit/File workbenches;
• chronological session turns with intermediate responses and collapsible tool groups;
• explicit distinction between declared, observed, candidate, and unmapped evidence;
• read-only continuation packets with no implied recovery authority.

In particular, we should preserve two trust boundaries:

1. Shared paths and time proximity are useful context, but do not prove that a session authored a commit.
2. A continuation packet is not a restored worktree, replayable checkpoint, or resumed native Agent session.

Live inspection, replay, autonomous recovery, and additional delivery systems should therefore remain follow-up slices.

Next architectural step: an Inspector Kernel

The next slice should extract the current report joins into a stable evidence model, for example:

DeliveryEvidenceGraphV1

Intent / Spec / Issue
        ↓ declares
      Story
        ↓ scopes
     Session
        ↓ contains
       Turn
        ↓ invokes
     ToolCall ── touches ──▶ File
                               ▲
                               │ changes
                             Commit
                               ↓ included-in
                         Pull Request
                               ↓ verified-by
                         Test / CI / Review

The key contract should be an explainable relation rather than a UI-local link:

interface EvidenceEdgeV1 {
  from: EntityRef;
  relation: string;
  to: EntityRef;

  assertion: "declared" | "observed" | "candidate";
  basis: Array<
    | "explicit-ref"
    | "provider-checkpoint"
    | "exact-path-overlap"
    | "time-window"
    | "semantic-overlap"
  >;
  confidence: "explicit" | "high" | "medium" | "low";
  resolver: {
    name: string;
    version: string;
  };
  review: "accepted" | "rejected" | "pending";
  sources: EvidenceRef[];
}

declared / observed / candidate should mainly describe edges. unmapped is better treated as a derived query state: an entity currently has no accepted relation.

This avoids mixing relation type, inference method, and confidence in one evidenceKind field.

NormalizedToolActivityV1 can remain an important projection for display, but should not become the final storage model. A future activity contract will likely also need orthogonal dimensions such as:

family    inspect | change | execute | verify | coordinate | deliver
operation search | read | edit | test | build | commit | push
resource  file | git | browser | process | test | issue | pull-request
effect    read | local-write | external-write | execute
actor     human | primary-agent | sub-agent | hook
outcome   success | failed | partial | cancelled

Inspector UX direction

The main lesson from Playwright Trace Viewer and browser/runtime DevTools is not “add more panels”; it is synchronized selection.

Selecting one action should eventually highlight its surrounding evidence together:

• originating user turn;
• preceding read/edit actions;
• affected files;
• test or runtime result;
• correlated commit;
• related finding or missing verification.

The most valuable immediate UI addition would therefore be an Evidence Drawer. Selecting any relation should answer:

• Why are these entities linked?
• Is the relation declared, observed, or inferred?
• Which exact facts support it?
• Which resolver and version produced it?
• Has a reviewer accepted or rejected it?
• What evidence would invalidate or strengthen the relation?

For long traces, the activity view should evolve through search, filters, virtualization, and semantic zoom rather than rendering every action at the same visual level:

L0  Session summary
L1  Inspect / Change / Verify / Deliver phases
L2  Individual normalized actions
L3  Bounded sanitized host evidence

Recommended follow-up slices

In priority order:

1. refactor(inspector): introduce DeliveryEvidenceGraphV1 and explainable EvidenceEdgeV1
2. feat(inspector): add synchronized selection and an Evidence Drawer
3. feat(inspector): add scalable search, filters, virtualization, and semantic zoom
4. feat(inspector): add side-by-side Session / Agent / Skill comparison
5. feat(inspector): add reviewable candidate mappings and versioned annotations
6. feat(inspector): connect test, CI, PR, review, and issue evidence
7. spec(inspector): define runtime-owned checkpoint and sandbox replay authority

I would prioritize Evidence Graph + Evidence Drawer + Compare before Live Mode or Resume.

Compare is where the Inspector begins to support Skill Eval and Harness improvement. It should be able to compare baseline/current/candidate runs and explain changes such as:

• fewer ineffective searches or retries;
• earlier first effective edit;
• smaller file-touch scope;
• stronger verification after the last edit;
• reduced time, tokens, or failed calls;
• improved delivery evidence coverage;
• new regressions or side effects.

Long-term positioning

Better Harness should not compete with Agent observability tools by merely displaying more Tool Calls. Its distinctive role can be:

A cross-Agent software-delivery evidence and debugging layer that connects intent, execution, code changes, verification, and delivery—and explains why a result is or is not trustworthy.

Entire-style checkpoints, OpenTelemetry/OpenInference traces, native Agent sessions, Playwright traces, GitHub, tests, and CI should become adapters into this model.

The Inspector then becomes the shared evidence data plane for:

• delivery provenance;
• Session analysis;
• Skill Eval;
• Contribution Graphs;
• Harness findings;
• guardrail suggestions;
• repeated-work and Skill-candidate discovery.

So the most important outcome after this PR is not another visualization. It is turning the current workbench into an Inspector Kernel with explainable evidence, synchronized debugging, comparison, and eventually capability-gated continuity.