Skip to content

fix: treat Cursor counter progress as watcher liveness - #2952

Closed
taimurrabuske wants to merge 7 commits into
kunchenguid:mainfrom
taimurrabuske:fm/fm-cursor-idle-false-wedge-t1
Closed

fix: treat Cursor counter progress as watcher liveness#2952
taimurrabuske wants to merge 7 commits into
kunchenguid:mainfrom
taimurrabuske:fm/fm-cursor-idle-false-wedge-t1

Conversation

@taimurrabuske

Copy link
Copy Markdown

Intent

Fix the watcher false possible-wedge alarms seen on healthy Cursor workers during long reasoning stretches. Scope this honestly: the fleet moved off Cursor to Codex on 2026-08-24, so this is currently a fix for a runtime nobody is using. Do the smallest correct thing: make a changed Cursor numeric token count or context percentage positive liveness evidence, including after the completed-turn age bound, while an unchanged counter and spinner-only redraw must still permit genuine wedge escalation. Match the progress property itself regardless of the spinner/status verb preceding the counter; do not enumerate a closed verb list. Live observed forms include Working, Thinking, Reading, and Running, and behavioral coverage must prove each known form is read plus prove a spinner-only redraw with a static counter still escalates. Keep the change Cursor-aware; do not raise the global idle threshold and do not build a general presentation-liveness framework. If the correct fix becomes large, stop for reconsideration. Follow Firstmate's one-owner, shared-tracked-material, test, shellcheck, documentation, and no-agent-coauthor conventions.

What Changed

  • Track consecutive Cursor token and context counters from recognized status/footer rows and reset wedge timing when either value changes, including after the busy-turn age bound.
  • Preserve wedge escalation for static counters, spinner-only redraws, missing fields, unrecognized layouts, ordinary pane content, and marker-write failures.
  • Clear Cursor progress markers during terminal cleanup and document and test the Cursor-specific liveness behavior across known status verbs.

Risk Assessment

✅ Low: The Cursor-only change is narrowly bounded, fails closed when structural or marker evidence is unavailable, and preserves wedge escalation for static numeric counters and spinner-only redraws.

Testing

After inspecting the commit scope, the targeted watcher and supervisor suites passed, and a manual production-watcher check demonstrated that genuine numeric progress suppresses the false alarm while static spinner-only redraw still emits a possible-wedge wake; all evidence was captured and the worktree is clean.

Evidence: Manual production-watcher evidence

Source: Manual production-watcher evidence

Advancing 59→60 tokens kept supervision active and reset the old wedge timer. Static 60-token spinner redraw emitted stale: test:fm-cursor-static (idle 501s, possible wedge, escalation 1).

CASE 1 — genuine numeric progress after the age bound
Rendered Cursor row: ⠋⠆ Working  60 tokens; footer: 7%
Previous persisted progress: token=59tokens context=7%
Watcher result: still supervising; no wake emitted
Watcher stdout: 
Wedge timer state: old 500-second timer was reset; a later unchanged poll started a fresh timer

CASE 2 — spinner-only redraw with static counters
Rendered Cursor row: ⠸⠆ Working  60 tokens; footer: 7%
Previous persisted progress: token=60tokens context=7%
Watcher stdout: stale: test:fm-cursor-static (idle 501s, possible wedge, escalation 1)
Evidence: Cursor watcher behavioral suite

Source: Cursor watcher behavioral suite

ok - signal_reason_is_actionable: benign absorbed, captain verbs and coalesced batches surfaced
ok - stale_is_terminal: terminal status surfaces, non-terminal and no-status are benign
ok - scan_captain_relevant_statuses lists only captain-relevant statuses
ok - classifier primitives: keyed decisions and activity phases, captain relevance, window-to-task, and overrides
ok - crew_is_provably_working: only working+run-step/pane is provable; idle/finished/parked/failed/unknown surface
ok - status_is_paused: only the leading paused verb matches, paused is not captain-relevant, and the two declared-wait verbs stay separable
ok - crew_absorb_class: working/paused/none from one read; crew_is_paused and crew_is_provably_working agree
ok - crew_worktree_written_since: real writes are evidence; no worktree, no anchor, quiet trees, .git churn and a mate's own home are not
ok - an empty FM_WORKTREE_WRITE_PRUNE widens the probe to the whole depth-bounded tree instead of disabling it
ok - an empty FM_WORKTREE_WRITE_PRUNE exported into the environment prunes nothing, widening the probe
ok - the worktree write probe is wall-clock bounded, and hitting the bound reads as no write evidence
ok - signal_crew_provably_working: benign only when every referenced crew is provably working
ok - a secondmate's status signal is never absorbed as provably working; crewmates are unaffected
ok - a no-verb signal whose crew is provably working is absorbed (no exit, no queue, suppressor advanced, beacon present)
ok - a bare turn-end whose crew is provably working (busy pane) is absorbed
ok - a bare turn-end whose crew is not provably working is surfaced (the swallowed-finish fix)
ok - a no-verb working: note whose crew is idle with no running pipeline is surfaced
ok - a secondmate's status note surfaces even while its own agent is busy
ok - a self-announced close never wakes its own home, and the next real note still does
ok - captain-relevant signal is surfaced (queue + exit) and marked surfaced
ok - a stale pane sitting on a terminal status is surfaced (queue + exit)
ok - a stale terminal-looking status is overridden and absorbed while a run is actively working, then wedge-escalated
ok - provably-working non-terminal stale is absorbed on first sight, then wedge-escalated past the threshold
ok - consecutive wedge escalations on the same pane accumulate and demand deep inspection at the threshold
ok - a pane becoming active again resets the consecutive wedge-escalation counter
ok - a busy worker below the turn-age bound remains working with no escalation
ok - a busy worker with a stable pane hash still escalates once its completed-turn age reaches the bound
ok - a busy worker whose pane hash changes every poll still escalates once its completed-turn age reaches the bound
ok - Cursor token or context progress resets the busy-turn wedge timer; spinner-only redraws do not
ok - an unwritable Cursor progress marker fails closed
ok - Cursor progress requires a changed numeric field shared by consecutive samples
ok - Cursor progress ignores convincing numeric text outside its reserved rows
ok - unrecognized Cursor progress structure returns a reasoned unknown
ok - touching a busy worker's completed-turn marker resets the age and prevents an old-age escalation
ok - repeated busy turn-age escalations reuse the existing escalation counter and demand deep inspection at the threshold
ok - the production default busy-turn-age bound is 3600s (5min under does not wedge, 66min over does)
ok - a busy pane under a declared pause is rechecked on the long cadence, and lifting the pause restores the wedge escalation
ok - a not-provably-working non-terminal stale is surfaced immediately (never left to wait out the timer)
ok - a declared pause is absorbed on first sight, then re-surfaced as a recheck past the threshold, never wedge-escalated
ok - exited declared-pause and captain-held panes use bounded pause cadence while a live decision gate still surfaces once
ok - a declared paused secondmate re-surfaces on the bounded normal-mode cadence
ok - a captain-held secondmate re-surfaces on the bounded normal-mode cadence
ok - a non-paused secondmate retains normal stale suppression
ok - a resumed secondmate clears pause and stale tracking before stale exemption
ok - unchanged stale hashes reclassify when a crew enters or leaves pause
ok - a declared pause is periodically rechecked against authoritative active-run state
ok - a paused status overridden by authoritative working preserves its wedge timer and escalates
ok - matching non-terminal stale suppressors repair missing or corrupt stale-since timers
ok - a quiet pane writing its own worktree is deferred, while one writing nothing still wedge-escalates on the unchanged schedule
ok - a write deferral re-surfaces once on the bounded pause cadence, so a churning worktree cannot stay invisible
ok - a secondmate's own home supervision churn is not crew write evidence, so a pane recording that home keeps the unchanged escalation schedule
ok - an idle-window timer repair drops a finished write-deferral chain, so the next deferral gets a fresh re-surface window
ok - both first-sight paths through a captain-relevant status drop a finished write-deferral chain with the idle window
ok - triage log capping handles wc byte counts with leading spaces
ok - a captured process-event result wakes a healthy watcher proactively, with no manual drain
ok - an unacknowledged process-event result re-drains until handling is acknowledged
ok - complete process-event queue keys map to distinct seen markers
WAKE_ACK_REQUIRED: after handling completes run bin/fm-wake-drain.sh --ack-through 1 --recovery-generation 3874892.1787571925.EF0f2z
ok - queue revalidation, proactive output, and marker commit serialize with drain
/home/taimur/.no-mistakes/worktrees/858142548832/01M0S74JGCB293DFCNRCAQ7GW7/bin/fm-push-transition-lib.sh: line 96: echo: write error: Broken pipe
tests/wake-helpers.sh: line 286: 3877752 Killed                     PATH="$dir/fakebin:$PATH" FM_HOME="$dir" FM_PROCEVENT_CLAIM_ROOT="$dir/claims" FM_CREW_STATE_BIN="$dir/fakebin/fm-crew-state.sh" FM_POLL=0.2 FM_SIGNAL_GRACE=1 FM_CHECK_INTERVAL=999999 FM_HEARTBEAT=999999 "$WATCH" > "$out"
tests/wake-helpers.sh: line 286: 3878979 Killed                     PATH="$dir/fakebin:$PATH" FM_HOME="$dir" FM_PROCEVENT_CLAIM_ROOT="$dir/claims" FM_CREW_STATE_BIN="$dir/fakebin/fm-crew-state.sh" FM_POLL=0.2 FM_SIGNAL_GRACE=1 FM_CHECK_INTERVAL=999999 FM_HEARTBEAT=999999 "$WATCH" > "$out"
ok - surfacing failures replay until post-handling acknowledgement
ok - marker failure exits through the shared wake owner, releases its lock, and replays later
ok - a heartbeat with no captain-relevant change is absorbed and backs off the cadence
ok - heartbeat backstop fail-safe surfaces a captain-relevant status the per-wake path missed
ok - the liveness beacon stays fresh while the watcher absorbs benign wakes (fm-guard never false-alarms)
ok - with .afk present the watcher reverts to one-shot so the daemon owns triage (no double-triage)
ok - AFK changed paused panes hand off plain stale identities for daemon-owned pause triage
Evidence: Supervisor marker-lifecycle suite

Source: Supervisor marker-lifecycle suite

ok - fm-afk-start.sh fails before daemon startup when the afk flag cannot be written
ok - fm-afk-start.sh ignores stale pidfile-only live pids
ok - fm-afk-start.sh reclaims stale daemon locks whose live pid identity no longer matches
ok - supervise daemon state root is scoped by FM_HOME
ok - routine signal self-handles
ok - captain-relevant status verbs escalate
ok - check + unknown escalate; heartbeat self-handles
ok - transient stale self-handles and records a persistence marker
ok - enriched stale wedges bypass status absorption without disturbing busy workers
ok - stale + terminal status escalates immediately
ok - paused reasons with captain phrases remain pause-classified
ok - a captain-held transfer classifies as pause, not as a wedge candidate
ok - handle_wake on a paused stale records a pause marker, drops the wedge marker, and does not escalate
ok - handle_wake records a declared pause from a routine signal for long-cadence rechecks
ok - a terminal signal clears pause and stale tracking across both supervisors
ok - housekeeping migrates a normal-watcher's declared pause into daemon tracking
ok - housekeeping clears an already-resumed watcher pause across both supervisors
ok - housekeeping seeds pause tracking from status without a watcher marker
ok - persistent stale escalates after threshold and clears its marker
ok - resumed (busy) stale clears its marker without escalating
ok - housekeeping re-surfaces a stale declared pause on the long cadence and resets its window
ok - housekeeping re-surfaces a forgotten captain hold on the long cadence and resets its window
ok - housekeeping clears a paused marker whose pane became busy again, without escalating
ok - housekeeping clears a paused marker once the crew is no longer declaring the pause
ok - housekeeping clears the pause marker once a captain hold is answered
ok - housekeeping moves an existing stale marker to pause before wedge escalation
ok - housekeeping moves a captain hold's existing stale marker to pause before wedge escalation
ok - housekeeping clears tracking when a crew leaves pause
ok - persistent herdr stale resolves the target from metadata and escalates
ok - herdr idle busy-footer stale clears through capture corroboration
ok - resumed herdr stale clears through backend-aware busy state
ok - persistent Orca stale resolves the terminal from metadata
ok - multiple escalations flush as a single batched digest
ok - batch flush measures max-delay from the first append, not the last
ok - catch-all scan escalates a missed terminal once, not twice
ok - handle_wake routes routine->self and captain->escalate
ok - INJECT_SKIP forces self-handle, bypassing captain-relevant classification
ok - is_wake_reason distinguishes watcher wake reasons from singleton-status stdout
ok - terminal-stale escalate removes its marker so housekeeping does not re-escalate
ok - captain signal escalate marks seen so the catch-all scan does not re-fire
ok - _collapse_newlines replaces newlines with literal separator
ok - afk flag absent: daemon does not inject, buffer preserved
ok - busy-guard defers injection when supervisor pane is busy
ok - marker detection: marker -> stay afk, no marker -> exit afk
ok - /afk invocation is exempt from afk exit (no self-cancel)
ok - should_exit_afk returns false when afk is not active
ok - strip_injection_marker removes the sentinel marker cleanly
ok - pane_input_pending detects partial input on the cursor line
ok - pane_input_pending: a blank unidentified cursor row defers (strict container-proof rule)
ok - pane_input_pending: only proven empty agent prompts pass
ok - fm_tmux_composer_state: a bare shell prompt ($/%/#/>) reads unknown, never empty (dead-shell injection safety)
ok - fm_tmux_composer_state: a bordered composer box and bare agent glyphs (❯/›) still read empty
ok - fm_tmux_composer_state: only matching edge borders form a composer box
ok - pane_input_pending preserves bright placeholder-like drafts in styled captures
ok - classify_signal dedupes against the catch-all scan seen marker
ok - classify_stale dedupes against the signal path seen marker
ok - AFK nonterminal working:+merged keeps wedge aging and re-escalates at bound
ok - genuine done: and merge-check events still escalate
ok - pane_input_pending: an idle bordered composer is NOT pending (afk-invx-i5)
ok - pane_input_pending: text inside a bordered composer is still pending
ok - submit-ACK confirms a submit when the composer returns to a bordered-empty box
ok - submit-ACK reports pending on a persistently swallowed Enter (type-once)
ok - max-defer on an empty stuck pane types once, alarms, and preserves the buffer
ok - max-defer flushes and clears the buffer on an empty bordered pane
ok - max-defer on a pending composer alarms without typing
ok - normal flush clears a stale wedge marker
ok - below MAX_DEFER: no inject, no alarm, buffer preserved
ok - max-defer does not flush or alarm while afk is inactive
ok - library mode: sourcing the daemon defaults FM_WEDGE_ALARM_EXEC to discard (no test can fire a real notification)
ok - wake helpers replace inherited notifier overrides with the safe recorder
ok - the discard seam suppresses every notifier, including command: (fires nothing)
ok - direct notifier helpers honor the discard seam, including command:
ok - osascript channel routes through the notifier seam with the summary (never a real notification)
ok - herdr channel routes through the notifier seam with the summary (never a real notification)
ok - command channel runs the captain command with the summary on $1 and on stdin
ok - command channel failures redact configured commands while logging their exit status
ok - unknown channel directives are redacted while the alarm keeps running
ok - off disables every active alert regardless of directive position (marker and tmux flash are unaffected)
ok - auto resolves to the macOS osascript notifier on Darwin (default-on)
ok - auto on a non-macOS platform selects no built-in OS channel (the marker or a configured command carries it)
ok - config/wedge-alarm selects every configured channel and skips comment and blank lines
ok - a failing channel logs and falls back to the next channel, never crashing the alarm
ok - a hung notifier is bounded, logged, and falls through to the next channel
ok - a backgrounded command notifier remains bounded until its process group is reaped
ok - a hung notifier override is bounded, logged, and proceeds to the next channel
ok - daemon shutdown stops and reaps the active notifier process group
ok - inject_wedge_alarm writes the marker AND emits the active alert even with no tmux status-line (herdr backend)
ok - in-process wedge throttle prevents alert spam when the marker cannot persist
ok - fm-send returns 3 with a non-error no-resend warning when confirmation stays pending
ok - fm-send exits non-zero when initial text send fails
ok - fm-send exits non-zero unless delivery is proven empty
ok - discover_supervisor_backend: override > TMUX_PANE > HERDR_ENV+HERDR_PANE_ID > tmux fallback
ok - discover_supervisor_target: override > TMUX_PANE > herdr '<session>:<pane-id>' composition > firstmate:0 fallback
ok - pane_is_busy: herdr native busy_state='busy' short-circuits without a capture fallback
ok - primary busy guard isolates rendered signatures by detected harness
ok - pane_is_busy: omitted backend defaults to tmux for Grok's isolated fallback
ok - pane_input_pending: dispatches through fm_backend_composer_state for backend=herdr
ok - inject_msg: herdr busy-guard defers before ever attempting a submit
ok - inject_msg: herdr composer-guard defers before ever attempting a submit
ok - inject_msg: herdr pane-gone check defers before any busy/composer/submit call
ok - inject_msg: dispatches busy-guard/composer-guard/submit through the herdr backend and succeeds on a confirmed empty composer
ok - inject_msg: defers on a dead-shell/unreadable composer (unknown), never typing the escalation into a shell
ok - inject_msg: unrecognized composer states defer by default

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 1 issue found → auto-fixed (3) ✅
  • ⚠️ bin/fm-watch.sh:337 - The progress-marker write result is ignored. If an existing marker records 59 tokens, the pane advances to 60 tokens, and this write fails, every later poll rereads 59 and treats the now-static 60 as fresh progress, indefinitely clearing wedge tracking. Require the marker update to succeed before returning positive liveness so an unwritable marker fails closed.

🔧 Fix: Fail closed on Cursor progress marker writes
1 error still open:

  • 🚨 bin/fm-watch.sh:338 - The required criterion limits positive liveness to a changed numeric token count or context percentage, but this compares the entire signature including field presence. For example, token=60tokens context=7% followed by the same 60 tokens with the context footer temporarily absent compares unequal and clears the wedge timer despite no numeric progress; alternating footer visibility can suppress escalation indefinitely. Compare numeric fields only when present in both samples, or preserve the last observed value for a temporarily absent field.

🔧 Fix: Require shared Cursor fields for progress
1 error still open:

  • 🚨 bin/fm-watch.sh:314 - The required evidence is Cursor's rendered numeric progress field, and the prior decision says field absence must never count as progress. However, this scans all 40 captured lines for any text such as 61 tokens (and similarly any context ... 8%), including ordinary response/tool content. A concrete sequence of a real Working 60 tokens sample followed by the progress row disappearing while displayed content says 61 tokens compares as progress and resets the wedge timer. Scope extraction to Cursor-owned status/footer rows without enumerating verbs, and cover disappearance with misleading numeric content.

🔧 Fix: Anchor Cursor progress to reserved footer structure
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

  • Inspected git diff 7b88520c055408a18f1476ecce08be60b2885fc9..26fc66c0329ab66169edcc17f8a3bc094f21f10e to map the authoritative intent to executable coverage.
  • tests/fm-watch-triage.test.sh
  • tests/fm-daemon.test.sh
  • Ran a hermetic two-case production bin/fm-watch.sh fixture and captured its real wake output to cursor-watcher-manual-transcript.txt.
  • Verified cleanup with git status --short.
✅ **Document** - passed

✅ No issues found.

⚠️ **Lint** - 1 warning
  • ⚠️ linter found issues (exit code 1)
✅ **Push** - passed

✅ No issues found.

@greptile-apps

greptile-apps Bot commented Aug 24, 2026

Copy link
Copy Markdown

Confidence Score: 4/5

The PR appears safe to merge from a runtime perspective, but its rigid Cursor UI parsing should be reconciled with the repository’s semantic-adapter policy.

Progress recognition depends on fixed row positions and presentation literals, making future presentation-only Cursor changes capable of reviving false wedge alarms.

Files Needing Attention: bin/fm-watch.sh

Reviews (1): Last reviewed commit: "no-mistakes(document): Clarify Cursor nu..." | Re-trigger Greptile

Comment thread bin/fm-watch.sh
Comment on lines +312 to +327
block=$(printf '%s\n' "$tail40" | awk '
{ line[NR] = $0 }
END {
for (i = NR - 1; i >= 5; i--) {
if (line[i - 4] ~ /^[[:space:]]*[^[:alnum:][:space:]]+[[:space:]]*$/ &&
line[i - 3] ~ /^[[:space:]]*.*Add a follow-up[[:space:]]+ctrl\+c to stop[[:space:]]*$/ &&
line[i - 2] ~ /^[[:space:]]*[^[:alnum:][:space:]]+[[:space:]]*$/ &&
line[i - 1] ~ /^[[:space:]]*[0-9]+ tasks?[[:space:]]*$/ &&
line[i] ~ /^[[:space:]]*Cursor .+ · [0-9]+([.][0-9]+)?[[:space:]]*%[[:space:]]+Run Everything[[:space:]]*$/ &&
line[i + 1] ~ /^[[:space:]]*[^[:space:]].* · [[:xdigit:]]+[[:space:]]*$/) {
printf "%s\t%s\n", line[i - 5], line[i]
exit
}
}
}
')

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Rigid Cursor layout coupling

Progress recognition now depends on fixed row positions and presentation literals such as Add a follow-up and Run Everything. This conflicts with the repository requirement to bind harness adapters to exposed semantics rather than rendered pixels, and makes presentation-only Cursor changes restore false wedge alarms and require repeated parser updates.

Context Used: If there is a VISION.md file at the root of the re... (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant