This repository was archived by the owner on Oct 13, 2026. It is now read-only.
Repository navigation
This repository was archived by the owner on Oct 13, 2026. It is now read-only.
Baseline proof misreads a passing suite (rc=19) and dispatchers cannot reach GitHub #144
Closed as not planned
Description
What: Prism's baseline proof reports a passing suite as passing, and dispatchers can reach the GitHub API.
Why: Job prism-mumnyqf1-504612d3 blocked with baseline_proof_failed: rc=19 on a suite that passes (311 tests OK on the same commit), and every dispatcher today logged error connecting to api.github.com, adding minutes per job before any worker started.
So what: An agent diagnoses both from the runner logs; until then the planner uses direct worker threads for bounded tasks.
Evidence (2026-09-29)
prism-mumnyqf1-504612d3(Procedure trigger audit: measure which operations procedures fire, then fix the worst agentsmd#164): blockedbaseline_proof_failed: rc=19on basefc1d8b2; the quoted output is only a PythonDeprecationWarningfromtests/test_shared_preferences.py:298followed by passing dots. Runningpython3 -m unittest discover -s testson the same commit in the same worktree:Ran 311 tests in 58.389s OK.- Every dispatcher briefing today (jobs for chromeria#68/Automatically release Model Router after versioned merges #69, marketplace#116, #116: Prism runtime on T3 only; models and limits from the T3 snapshot #119, agentsmd#164, #165) reported
gh issue viewfailing witherror connecting to api.github.comand fell back to a connector read. The user observed the #164 dispatcher running about 8 minutes without spawning a worker. - Of five Prism jobs submitted today, four blocked or looped: two on planner packet errors (missing
workspace;vpnot on PATH), one on this rc=19 misread, one onpr_not_open(Runner rejects completion after an authorized merge (pr_not_open) #143).
Elon record
- Requirements and who asked: the user (2026-09-29) asked why the #164 dispatcher took 8 minutes without a worker and why the
ghfailures were not investigated. - Deleted: retrying jobs unchanged; treating the connector fallback as good enough.
- Bottleneck: the dispatch and baseline stages, before any implementation starts.
- Checked myself: reran the #164 baseline suite locally (OK) and read the dispatcher briefings; I have not read the runner's proof classifier or the dispatcher sandbox configuration.
Acceptance criteria
- The baseline classifier uses the proof command's exit status; the #164 base classifies as pass.
- Dispatchers either have GitHub API access or the packet states that they do not, and they skip the failing call.
- Median time from submit to first worker start is reported for today's jobs and after the fix.
Non-goals
- Changing dispatcher models.
Blockers
None.
Required proof
Rerun of prism-mumnyqf1's baseline classifying as pass; a dispatcher gh issue view succeeding.
Activity
lukemaj commentedon Sep 29, 2026
Also (user, 2026-09-29): dispatcher
prism-mumo6jn1-c82f18bfthought for over 3 minutes without dispatching. Likely causes, not yet measured: (1) dispatchers run ongpt-6-lunaat max effort, far more than routing needs; (2) the AgentsMD global contract makes every agent, dispatchers included, orient at start (canonical-source hash, preferences, Project Direction,operations, repository status), and the briefings show dispatchers restating all of it. The dispatcher kit should give a routing-only packet and exempt the dispatcher from full startup orientation, leaving orientation to the worker it briefs. Measure: time from submit to first worker start per job, before and after.lukemaj commentedon Sep 29, 2026
Investigation (2026-09-29), from
~/.local/share/durable-runner/jobs.dband Codex session logsWhere the time goes. Dispatch turns (all on
t3:codex:gpt-6-luna@max): 6-11 min for most jobs (yesterday's chromeria-62 7.8 min, mr-132 6.3 min; today prism-mumlogix 10 min, mumlsvwv 8 min, mumm1j50 6 min, mumnqmwd 11 min, mumnyqf1 7 min, mumny937 2 min). Exceptions: mkt-112 38 s and ao-34 47 s. Longest silence per turn 4-39 s, so the dispatcher is working, not waiting. Baseline proof adds 40-75 s. The runner runs baseline after dispatch for T3 routes, so a broken packet is found only after the long dispatch.What the slow dispatcher does (prism-mumlogix session
rollout-2026-09-29T13-35-50-..., 11:35:50-11:45:39): 13 tool calls and 51 reasoning items before returning the action:ghfails: the session's sandbox isworkspace-writewith "Network access is restricted".bin/project-direction inspect, then reads operationsSKILL.mdand references (orchestration, Elon method algorithm) from the plugin cache.prism_statuson its own job.The fast mkt-112 dispatch returned its action with zero tool calls; its session made its first tool call five minutes later in a later turn.
Also found. 92 cached kits under
kits/linkedAGENTS.mdto the AgentsMD clone root, which since AgentsMD v14.0.0 holds repository rules, not the global contract (kits.py_write_agentsmd_linkresolves the OpenCode link once at build time). Re-pointed by hand toglobal/AGENTS.md(runbookcavallo/CHANGES.md); kits built after the move resolve correctly.Proposed fixes, in order of expected gain:
ghfrom its instructions; never let it fall back to tool discovery and web search.max; routing needs judgment but not maximum reasoning. Measure submit-to-first-worker time before and after each change.lukemaj commentedon Sep 29, 2026
Correction (2026-09-29): the rc=19 block was not a misread.
~/.local/share/durable-runner/outputs/prism-mumnyqf1-504612d3/baseline-proof.logshows the dispatcher-chosen proofpython3 -m unittest discover -s tests && bin/versionctl release-check: the suite passed (311 tests OK), thenrelease-checkexited 19 withTAG_CONFLICT: v13.5.0 already references different contentbecause the basefc1d8b2is a WIP commit whose VERSION still read 13.5.0. The runner classified it correctly. The defect is the choice of baseline proof: a final-candidate check (release-check) cannot pass on a WIP or pre-bump base. Fix direction: the baseline runs only the behavioral proof (tests), and release-check runs on the final candidate. My earlier claim of a misread passing suite was wrong.fix: self-contained dispatch packet, dispatcher network, baseline bef…
lukemaj commentedon Sep 29, 2026
Correction (2026-09-29), after reading RUNNER.md: (1) the baseline proof already runs before the first dispatch when the task carries
proof; today's packets had none, so the dispatcher chose one and the baseline ran after dispatch. (2) The runner already starts an independent reviewer on the completed head (#126); dispatchers asked the planner to route review only because the packets demanded it. Remaining genuine defects: dispatcher network (runtime modeautomaps to a Codex sandbox without network), dispatcher orientation and briefing cost, the missing-workspacedefault (toolboxmd/chromeria#68), and AgentsMD startup steps not scoped by role.fix: self-contained dispatch packet, dispatcher network, safe baselin…
fix: self-contained dispatch packet, dispatcher network, safe baselin…
Merge pull request #145 from toolboxmd/fix/144-routing-only-dispatch
lukemaj commentedon Sep 29, 2026
Status (planner, 2026-09-30): criteria 1 and 2 are met by PR #145 (merged, 0.42.0): exit-status classification is pinned in tests/test_issue144.py (the rc=19 was a real release-check TAG_CONFLICT, not a misread), and dispatch/review run with network. Criterion 3 has one timed sample per side (16m19s before, 26s after), not a median. The median now comes from
scripts/compare_dispatch.py(#146, PR #152, 0.44.0): run it over jobs submitted on 0.42.0 or later and post the median here, then close. Left open for that measurement.lukemaj commentedon Oct 1, 2026
Superseded by toolboxmd/chromeria#92: baseline proof and the runner's dispatch packet are deleted with the runner. Agent network access is covered by Chromeria children starting in full access (toolboxmd/chromeria#89).