Skip to content

feat(state): specify supplied inputs vs observed access and review scope (#174) - #176

Open
Adityakk9031 wants to merge 1 commit into
openprose:mainfrom
Adityakk9031:feat/execution-provenance-profile-174
Open

feat(state): specify supplied inputs vs observed access and review scope (#174)#176
Adityakk9031 wants to merge 1 commit into
openprose:mainfrom
Adityakk9031:feat/execution-provenance-profile-174

Conversation

@Adityakk9031

Copy link
Copy Markdown

Summary & Motivation

Resolves #174

In multi-cycle agent delegation and evidence observatory runs, coordinators delegate investigations, audits, and synthesis tasks to subagents. As documented in #174, a recurring audit friction is reconstructing what a worker was actually handed, what the host environment observed it access, and what review scope the worker attested to performing. Bound input tuples (input_fingerprints) were often conflated with semantic inspection, and auxiliary follow-up handoffs were vulnerable to omission or retroactive backfilling.

This PR introduces an optional, host-agnostic execution-provenance profile that defines the boundaries between three non-interchangeable planes of evidence without imposing cryptographic signatures, live messaging sockets, or domain-specific schemas.


Key Changes

  1. New Specification: Execution Provenance (skills/open-prose/state/execution-provenance.md)

    • Three Evidentiary Planes:
      • Supplied Inputs (Coordinator Intent): Immutable pre-dispatch handoff records (workspace/{node}/__delegate/{delegate}/handoffs/{seq}.json) tracking exact paths, content hashes, task identity, and instructions.
      • Observed Access (Host Telemetry): Empirical tool/IO events (observed_access.jsonl) captured by host adapters. Absent telemetry remains unknown.
      • Reported Review Scope (Worker Attestation): Worker return attestation in {id}-response.md with structured modes: full, partial (with section/line scope), integrity_check_only (hash verified), reused_prior, or not_read (with reason).
    • Permitted Follow-up Lineage: Sequenced handoff tracking (001.json, 002.json) ensures incremental inputs (e.g. adding input $D$ mid-run) are recorded contemporaneously before dispatch rather than backfilled from retrospective summaries.
    • Core Invariants:
      • A hash check or file read is not proof of understanding.
      • An agent report is not host authentication.
      • Missing telemetry must remain unknown.
      • Retrospective reconstruction must not be labeled contemporaneous.
  2. Filesystem & State Layout (skills/open-prose/state/filesystem.md, README.md)

    • Added handoffs/{seq}.json and observed_access.jsonl to the delegation directory layout.
    • Updated the "Who Writes What" table to establish write ownership: pre-dispatch records written contemporaneously before dispatch; review scopes written in responses; telemetry written by host adapters.
    • Linked execution-provenance.md in the state documentation table of contents.
  3. VM Execution Guidance (skills/open-prose/prose.md)

    • Documented the pre-dispatch recording step in the runtime delegation protocol and clarified sequence management for multi-turn follow-up handoffs.
  4. Inspector Provenance Auditing (packages/std/evals/inspector.prose.md)

    • Updated the extractor to pull handoff_provenance and observed_telemetry.
    • Updated the evaluator to detect unrecorded-handoff gaps (when worker outputs reference unrecorded inputs), flag supplied-unreviewed transparency, and maintain unknown status for missing host telemetry.
  5. Conformance & Fixture Tests (tests/open-prose/state/execution-provenance.test.ts)

    • Added 8 unit and conformance tests.
    • Implemented executable fixtures for the retestable scenario specified in Specify supplied inputs versus observed access and reported review scope in execution traces #174:
      • Fixture 1: Inputs $A, B, C$ supplied; worker reads slice of $A$, verifies hash of $B$, leaves $C$ unopened; input $D$ later supplied via permitted follow-up handoff. Evaluator verifies all 4 inputs, separates reported review scope from tool access, and flags $C$ as unreviewed.
      • Fixture 2 (Negative Fixture): Omitted follow-up handoff $D$. Evaluator flags an unrecorded-handoff gap rather than retroactively backfilling it from the worker's summary.
      • Fixture 3: Validates that absent host telemetry is classified as unknown rather than manufactured.

Verification

  • Ran pnpm vitest run tests/open-prose/state/execution-provenance.test.ts (8/8 tests passed).
  • Ran pnpm vitest run tests/open-prose/state/prose-state.test.ts (18/18 tests passed).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Specify supplied inputs versus observed access and reported review scope in execution traces

1 participant