Add evals for recurring user frustration patterns

[Vercantez]

Summary

Adds focused, sanitized eval coverage derived from the July 25 user-session frustration review. The change is eval-only: it does not alter product behavior.

Frustration patterns covered

• Login fixes declared successful after API-only checks. A new seeded rescue eval requires invalid-password rejection, a hardened session cookie, anonymous protected-route enforcement, authenticated access, repaired client loading/navigation behavior, a live deployment, and preservation of the existing app shell.
• Scoped requests rewriting or dropping existing behavior. The existing stateful redeploy eval now requires two pre-existing UI surfaces to survive the requested marker-only change, in addition to preserving Durable Object data and API behavior.
• Generated files handed off with broken or invented links. The output-file eval now verifies that the final response names the exact file written to workspace outputs, uses the workspace output route, and does not use an unrelated host.
• Long attempts at an unavailable platform capability. A new tcpdump eval requires a direct raw-socket/privilege boundary explanation, a useful host-side command, zero project/deploy mutations, and a tight turn/tool budget.
• Product intent conflation after a frustrated correction. A new two-turn eval requires removal and prompt-based fill intent to be separated, a defensible category recommendation grounded only in frozen synthetic metrics, and a non-defensive recovery.

All fixtures use synthetic names, credentials, data, and keyword volumes. No raw user messages, identifying data, or production secrets are included.

How the evals work

Each new eval is registered in the existing manifest and emits the shared transcript format. Product correctness is gated by deterministic persisted-state/live-HTTP checks where possible, with task-specific weighted rubrics for semantic quality. Shared root causes are consolidated into the existing output-delivery and state-preserving redeploy evals rather than creating session-specific duplicates.

Validation

• Focused Worker/Vitest suite after rebase: 19 passed, 5 eval-gated skipped.
• tcpdump-capability-restraint-live: product checks passed; rubric judge 97.5/100.
• product-keyword-intent-recovery-live: product checks passed; rubric judge 100/100.
• output-file-delivery-live: strengthened product checks passed; rubric judge 100/100.
• login-flow-rescue-live: all live auth/UI/preservation checks passed; rubric judge 100/100.
• project-update-redeploy-state-live: live UI, API, and persisted-state checks passed; rubric judge 100/100.
• git diff --check: passed.

Existing/baseline failures observed

These are separate from the product assertions added here:

• Full bun run typecheck currently fails in workers/main/src/chat-thread-do.ts because three ChatMemoryStoreStats values are missing the existing required measured property.
• The local eval runner exits nonzero after otherwise passing runs when agent verification first hits known missing local bindings: IMAGES for a CSV read attempt in output delivery, and DISPATCHER for early deployed-app fetch attempts in the two real-deploy evals. The agents recovered, and the independent live harness checks plus rubric judges passed.

Limitations / intentionally uncovered

• Broad project-specific debugging chains and vague “not working” reports were consolidated into the login-verification and scoped-change gates rather than encoded as noisy one-off evals.
• A burst of empty, duplicate thread creations looked operationally suspicious but did not contain enough user evidence to classify as frustration, so no speculative eval was added.
• The evals reproduce agent behavior and handoff failures; they do not fix the underlying product or local eval-runner binding issues.