fix(web): serve SPA shell for dotted deep-link navigations

[ritvik-jentic]

Summary

Fixes a backend-only bug where browser navigations to a versioned API deep-link
(e.g. /app/workspace/<vendor>/<name>/1.0) returned raw {"detail":"Not Found"}
404 instead of booting the SPA. FastAPI's app.frontend(fallback="auto") treats
any /app/* path whose final segment has a file extension as a static-file
request, and API version segments contain dots (1.0, 2.1.3), so the route was
mis-classified as a missing file. Bookmarks and shared/copied links to versioned
API pages broke for anyone opening them fresh. (The CLI-emitted approve_url is
an API-namespace path handled by the #813 login redirect, not this shim.)

Related issue

Closes #647

Changes

• Add _SpaNavigationFallbackMiddleware, an innermost ASGI shim that swaps a
  404 for the SPA shell (200 + index.html) only for a genuine GET/HEAD
  browser navigation under /app/; everything else passes through untouched so
  app.frontend() stays authoritative for real files (304/206/Range/symlinks).
• Decide asset-vs-route by MIME type (recognized static-file extension via
  mimetypes), not a hardcoded prefix list, so a missing/renamed asset anywhere
  in the bundle (assets/… or a mount-root file like broker-openapi.json)
  still 404s rather than silently booting the app.
• Register the shim as the innermost user middleware so the rescued shell flows
  back out through request-id/telemetry (carries x-request-id, logged as the
  real 200, not the inner 404).
• Harden: RFC 7230-combine split Accept header lines; strip root_path on a
  segment boundary; strip Range/If-Range from the rescued shell (full 200,
  never 206/416); clean 404 (not 500) if index.html is missing mid-redeploy;
  idempotent mount that raises RuntimeError if the middleware stack is
  already built.
• Review-board hardening (2026-07-31): rescued shell is
  Cache-Control: no-store (the same URL answers 404 to non-navigation
  clients and the rescue path never revalidates, so keep shared caches out);
  dot-segment /app paths (/app/../x.1, encoded %2e%2e) are refused —
  never rescued; packaging smoke now exercises the dotted deep-link and the
  no-store/404 split against the installed wheel.

Testing

• ruff check, ruff format --check, mypy — clean.
• uv run pytest tests/unit → 2142 passed; tests/arch → 262 passed;
  make test-integration-sqlite → 541 passed.
• New tests: versioned deep-link boots the shell (1.0, 2.1.3); missing
  asset (incl. root-level .json/.svg) still 404s; Range on the shell → full
  200; rescued shell carries x-request-id; split Accept still boots;
  rescued shell is no-store while the framework-served shell keeps framework
  caching; dot-segment paths 404; missing index.html mid-redeploy → 404.
• make smoke-packaging (build UI + wheel, install in clean venv): dotted
  deep-link /app/workspace/stripe-com/stripe-com-api/1.0 serves the packaged
  shell with no-store, stays 404 for JSON clients, missing recognized assets
  404 — PASS.
• Red-team probes against the real combined app: 401/403 and API 404s pass
  through untouched (/api/... never rescued), //app, %00, backslash and
  non-/app paths never rescued, HEAD returns empty body with correct
  headers, websocket scopes ignored.

Checklist

• Commits follow Conventional Commits with a scope and are signed off (git commit -s, DCO)
• make check passes (lint, type check, secrets audit, arch tests)
• Tests added/updated for the change
• Docs updated where relevant (module docstrings; no separate plan doc)
• No secrets, credentials, or internal-only references included

[Manuel-Jentic]

Handover status (taking this over from Ritvik):

• Rebased onto current main (46a919a). One textual conflict with fix(auth): repair expired-token login race and add sliding web sessions #857's app.state.spa_mounted flag in mount_spa — resolved by keeping both (flag, then shim); no semantic overlap (the shim only rewrites 404s, the 401 on dashboard URLs returns raw JSON instead of redirecting to login #813/fix(auth): repair expired-token login race and add sliding web sessions #857 handler only touches 401s).
• Ran a 4-lens review board (red-team/ASGI security, correctness+architecture, product fit, code quality). No blocking findings; hardening applied on top (dea3acf):
  • Cache-Control: no-store on the rescued shell — the same URL answers 404 (JSON accept) or 200 HTML (browser accept) and the rescue path never honours conditional revalidation, so shared caches are kept out of it entirely. Framework-served shells keep framework caching.
  • Dot-segment /app paths refused — /app/../x.1 (decoded %2e%2e) is never a legitimate SPA route; it now stays an honest 404 instead of being rescued.
  • RuntimeError instead of assert for the mounted-after-stack-built guard (survives python -O).
  • Packaging smoke extended: make smoke-packaging now exercises the dotted deep-link, the no-store header, the 404-for-JSON split, and the missing-asset 404 against the installed wheel — ran locally end-to-end: PASS.
  • PR body corrected: the CLI approve_url is an API-namespace path handled by the 401 on dashboard URLs returns raw JSON instead of redirecting to login #813 login redirect, not this shim; and the dangling "plan doc" reference removed.
• Verified clean by probes: 401/403 and API 404s pass through untouched; //app, %00, non-/app paths never rescued; HEAD rescues with empty body and correct headers; missing index.html mid-redeploy degrades to a clean 404; idempotent double-mount no-ops.
• Full local suite green: 2142 unit / 262 arch / 541 integration-sqlite, ruff + mypy clean.

[Manuel-Jentic]

Parking this PR for now (merge-ready, CI green) pending a design decision with Manuel.

The open question: this shim is ~206 lines of ASGI logic (plus hardening/tests) to work around FastAPI's _is_frontend_navigation_request hard-rejecting dotted final segments — no framework knob avoids it (even fallback="index.html" runs the same check), and a catch-all route can't fire because the static mount swallows all /app/*. The alternative is a ~30-line UI-only route change (make :version not the final segment, e.g. .../:version/overview, or encode dots) — far less machinery, but it fixes only this route: anything routed via encodeURIComponent (e.g. toolkit ids) preserves dots and would silently re-trigger the class of bug, and it gives up the clean URL that mirrors the backend's /apis/:vendor/:name/:version identity triple.

Options on the table: merge the shim (class fix), switch to the URL-scheme change (instance fix, less code), and/or push the relaxation upstream to FastAPI so the shim can eventually be deleted.