Repository navigation
fix: credit interventions to their actual actor - #135
Conversation
Agent work on this PREstimated cost at least $1.00 · 52 responses · 27 sessions · 3.1 h wall time
Flags: 1 human correction · 1 large tool output · 6 repeated commands · 2 repeated failures · 60 repeated reads · 9 repeated skill loads · 6 tests edited after a failure · 1 session without usage records · 20 unpriced responses Details: snapshot, prices, coverage, counters
Token counters by model (native counter semantics; never added across semantics):
Generated tokens (reasoning and other output are disjoint; other output includes code, tool calls and replies):
Usage by work phase (this task's own responses):
Recorded activity by phase (observations, not separate token bills):
Selected rates (USD per million tokens). These rates value the report at the selected schedule date; they do not establish historical prices or subscription spending.
Other output and reasoning are priced without double counting inclusive native output. Missing rates remain unknown. Local measurement from native records; usage totals are not billing. Updated in place by |
|
Independent review of exact head abcee57 (round 1)
Coordinator adjudication: 1 accepted as required (verdicts are durable judge evidence; Muse's "harmless" missed the yes verdict). 2 accepted as required for completion notices, because acceptance criterion 2 covers notices regardless of harness: record the same evidence for Grok prompts and replay once; investigate the 46 non-matching Grok prompts and either match genuine notice shapes or state why they are not notices. Agent-sent Grok steers follow from the same fix. 3 accepted: update the PR body with full-replay numbers. Fix round requested; both reviewers will review the new head. |
abcee57 to
9f307d0
Compare
9f307d0 to
a16938b
Compare
|
Independent review of exact head a16938b (round 2)
Coordinator adjudication: 1 accepted as required (a user's own words must never be credited to an agent; anchor the notice match to the end of input modulo whitespace and |
dfd7671 to
49c3766
Compare
|
Dispatch rule for every remaining round on this PR (Luke, 2026-10-19 17:55 Warsaw): launch no new threads on cavallo. Run Muse Spark 1.3 xhigh (Muse Code) and every other reviewer or worker on bestia, or on rocky. Muse Code is available on bestia (login present since 16:59). Already-running cavallo reviewers may finish; nothing new starts there. |
|
Independent review of exact head 49c3766 (round 3)
Coordinator adjudication: no blocking findings. The low finding needs a user to type the exact closing sentence; tracked separately rather than another full round. Ready for merge approval, after #134: whichever merges second must rebase and take the next patch version (#134 is at 1.4.1, this PR also at 1.4.1). |
49c3766 to
3d8ae69
Compare
|
Independent review of the rebased head 3d8ae69 (onto #134, version 1.4.2)
Coordinator: ready to merge under Luke's approval. |
What: Credit intervention senders correctly, preserve grouped verdicts, and reject completion notices followed by user instructions.
Why: Prefix matching credited user prose to agents, while timestamp and canonical-id precedence could hide a stored yes.
So what: Round-3 review passed and Luke approved merging; this authorized rebase is pushed for the parent, with no merge or new reviewers started.
Closes #133. This advances the Objective through explicit ownership and visible evidence gaps across native harnesses.
Head:
3d8ae6971ba8ae7c50935a5e4213ed7a194a1208. Base:ed90f9577c0b48cfeb539688ba1370f14e4c4515(origin/main). This body supersedes earlier candidate proof and attribution totals.Rebase after #134 and exact candidate proof
49c3766e365fce0af524d56e5d61252dc3d212fd; new head:3d8ae6971ba8ae7c50935a5e4213ed7a194a1208; new main:ed90f9577c0b48cfeb539688ba1370f14e4c4515.versionctl prepare patch --reason 'Credit intervention senders, preserve grouped verdicts and retain unknown actor reasons'(preview and apply). It added Credit interventions to the right actor: steers, task notices and unknown root prompts #133 under 1.4.2 and updated VERSION and all three plugin mirrors.python3 -m unittest discover -s tests: 929 passed, 101.937 seconds.python3 -m compileall -q agent_observer tests scripts,git diff --checkandversionctl release-check: passed on the clean new head.mode=robackup of the live ledger was fully replayed on its copy using all native sources for intervention-bearing window sessions. 327/327 spectrum-review verdicts resolve, with the entire stored verdicts table unchanged. Recurrence is 64 yes / 259 no / 0 unclear, 323 canonical judgments and four superseded rows.db.init_dbcorrected all 23 and wrotepr_checks_pending_version=2. After full replay, all 23 recognized results remain pending andanalysis._failedreturns false for each; marker 2 remains set.The requested range-diff has no differences beyond version bookkeeping and changelog resolution. Excluding those five bookkeeping paths gives:
Full unfiltered range-diff:
1: 49c3766 ! 1: 3d8ae69 fix: credit intervention actors and group T3 steers (#133) @@ .claude-plugin/plugin.json @@ { "name": "agent-observer", -- "version": "1.4.0", -+ "version": "1.4.1", +- "version": "1.4.1", ++ "version": "1.4.2", "description": "Measure agent work across Codex, Claude Code, OpenCode and Grok Build from native records, and compare behavior by model and AgentsMD version.", "author": { "name": "toolboxmd", @@ .codex-plugin/plugin.json @@ { "name": "agent-observer", -- "version": "1.4.0", -+ "version": "1.4.1", +- "version": "1.4.1", ++ "version": "1.4.2", "description": "Measure agent work across Codex, Claude Code, OpenCode and Grok Build from native records, and compare behavior by model and AgentsMD version.", "author": { "name": "toolboxmd", @@ .grok-plugin/plugin.json @@ { "name": "agent-observer", -- "version": "1.4.0", -+ "version": "1.4.1", +- "version": "1.4.1", ++ "version": "1.4.2", "description": "Measure agent work across Codex, Claude Code, OpenCode and Grok Build from native records, and compare behavior by model and AgentsMD version.", "author": { "name": "toolboxmd", @@ CHANGELOG.md @@ # Changelog -+## [1.4.1] - 2026-10-19 ++## [1.4.2] - 2026-10-19 + +### Changed + +- Credit intervention senders, preserve grouped verdicts and retain unknown actor reasons + - ## [1.4.0] - 2026-10-19 + ## [1.4.1] - 2026-10-19 - ### Added + ### Changed ## README.md ## @@ README.md: person. They are never interventions and are counted as `policy_denials`. @@ README.md: person. They are never interventions and are counted as `policy_denia ## VERSION ## @@ --1.4.0 -+1.4.1 +-1.4.1 ++1.4.2 ## agent_observer/adapters/claude.py ## @@ agent_observer/adapters/claude.py: import jsonFinal behavior
related_ids; raw inputs remain available.verdict listretains original rows and exposescanonical_intervention,canonical_verdict,canonical_problem, andcanonical_group. Text output identifies a different canonical verdict.<runtime_info>block are accepted. A pasted notice followed by an instruction is ordinary input. Closed blocks cannot absorb prose after a closing tag through regex backtracking._upsert_submission. Its native first-input wrapper and complete runtime/linking/optional-tool-use suffix are parsed only as complete anchored shapes; prose after those blocks is retained. Claude/Codex source versions and Grok import version refresh older records once on a separately authorized sync. No transcript or new free-text sender data is stored.native_sdk_prompt_without_matching_origin; Grok usesnative_prompt_without_matching_origin. Reporting remains read-only. Treat pending gh pr checks (exit 8) as pending, not failed #132 analysis/failure classification is untouched.Review repairs and regression proof
Round 1 exposed orphaned verdict references and missing Grok evidence ingestion. Round 2 exposed incomplete notice boundaries and a judgment conflict policy that could suppress yes. The fixes live in the parser, canonical judgment owner and their regressions, rather than additional workflow rules.
Sanitized native-record regressions cover human/agent steers, tool denials, completion notices, Grok direct/wrapped notices and steers, aliases resolving/showing their canonical steer, recurrence/group/list agreement, superseded audit fields and replay idempotency. Added negative cases include a notice plus newline instruction, prose after runtime metadata, attempted extra closing tags, partial interruption boilerplate and a known human pasting an agent-steer note. Positive cases retain complete native suffixes and Codex's inherited turn id.
Against archived exact reviewed head
a16938bc3b1dc504d4f6731c54863d5e433375f2, the pasted-notice regressions fail withagent != user; superseded audit assertions fail on the missing problem field. A separate priority probe reports[no, no]for alias-yes/canonical-no at equal and newer-no timestamps; the repaired implementation reports[yes, yes]. With audit assertions omitted only in the disposable baseline copy, both timestamp cases fail at recurrence's verdict-count assertion. Production regression tests retain all audit assertions and also cover a newer yes.python3 -m unittest discover -s tests: 929 passed, 101.937 seconds on bestia after the rebase (including the seven fix: treat pending gh pr checks as pending (#132) #134 tests). Existing ResourceWarnings are non-failing.python3 -m unittest tests.test_intervention_actors: 11 passed after the final parser compatibility additions. Earlier broader focused run: 94 passed.python3 -m compileall -q agent_observer tests scripts,git diff --check, complete diff self-review: passed.versionctl release-check: passed for v1.4.2 on the exact head above. The already-published upstream v1.4.1 tag was fetched before validation; no tag was created.Full copied-ledger replay
Window: 2026-10-12T14:20+02:00 inclusive to 2026-10-19T14:20+02:00 exclusive, Warsaw time.
SQLite opened the live ledger with
mode=roand backed it up. The historical untouched baseline was read with the CLI archived from maineda2c8df41d75246ab11d0f7932cd0400f2f932c; its totals remain recorded here. A fresh post-rebase live-ledger copy independently revalidated the candidate totals and verdicts below. A separate copy received full native imports for every source backing intervention-bearing window sessions across all actors: 209 Claude files, 1,074 Codex files, 395 Grok directories and one OpenCode database. All sources were available. Existing mirrored T3 evidence was held constant; stored root reasons were backfilled. No live-ledger sync was run.Verdicts: 327/327 spectrum-review ids resolve, zero orphaned; the entire stored verdicts table is field-for-field unchanged. CLI listing returns 327 rows attached to canonical interventions. CLI show resolves the previously orphaned yes id
sub:claude:85509847-d096-4b83-bd7f-770a94c0b437with status ok. Private text is excluded from this record.Recurrence: 323 canonical interventions judged, 64 yes, 259 no, zero unclear, with four superseded judgments retained explicitly. The alias yes counts once.
Grok investigation previously inspected all 131 window prompts mentioning
Delegated task: 112 direct notices with attached runtime metadata, 18 notices inside a first-input envelope, and one non-terminal notification about a question. Native sibling context includes linking and tool-use metadata. All 130 terminal notices remain agent inputs. Compared with the previous reviewed replay, 11 Claude inputs with non-notice trailing prose now stay unknown rather than being claimed as agent; the user total is unchanged. Complete suffix parsing preserves those notice shapes without accepting a pasted notice followed by prose; partial interruption notes remain unknown when no author evidence exists.External unknown-actor cause and limits
The historical root identity gap is tracked in Chromeria #344, forward-repaired in #346. Of the original 570 unknown Claude root prompts, 548 provider references name assistant UUIDs and 22 have no matching inspected record. Observer preserves missing-identity reasons instead of guessing from the opener, text or child threads. Historical matching, deployment and new runtime acceptance are not claimed.
Version, delivery and cleanup
PR #134 merged after round-3 review as main
ed90f9577c0b48cfeb539688ba1370f14e4c4515, releasing 1.4.1. This branch was rebased onto that main and versionctl prepared its single pending patch transition as 1.4.1 → 1.4.2. CHANGELOG retains both 1.4.1 and 1.4.0, and all three plugin mirrors are 1.4.2. No version was released or tagged by this work.The Observer cost comment is republished from a separate temporary ledger for task
toolboxmd/agent-observer#135. Shared work, missing usage and unpriced usage remain visible limits; this is not a complete spend estimate.All task-owned copied ledgers, private replay artifacts and disposable source archives from the initial implementation and both review repairs were deleted after proof was recorded. Their deletion was verified. The live ledger was not modified.
Round-3 independent review passed on
49c3766e365fce0af524d56e5d61252dc3d212fd; Luke approved merging. The rebase changes only version bookkeeping and the changelog conflict, as the range-diff below shows. No new exact-head review is claimed and no reviewers were started. No merge, release, tag, plugin installation, deployment or live-ledger sync. Website/SEO impact: none; this changes local evidence/reporting, with README documenting the contract. The task worktree and branch remain for review; the eventual merger owns their retirement.Elon record
Requirements and who asked: Luke requires #133 and both repairs, then a behavior-preserving rebase after #134, version 1.4.2, full suite, fresh-copy verdict/marker-2 proof, cost republication, copy deletion and no merge or new reviewers.
Deleted: No verdict migration/deletion, raw-event deletion, guessed actors, fixture exclusions, extra service, child-thread redesign or #132 classification changes.
Bottleneck: Complete input boundaries and canonical judgment precedence determine whether reports preserve human authorship and durable yes evidence.
Checked myself: Round-3 review record, unchanged implementation hashes, full and scoped range-diff, 929-test suite, versionctl release-check, fresh copied-ledger native replay, 327 resolutions, unchanged 64/259 recurrence, marker-2 correction, pushed head, cost comment and copy deletion.