feat: live R3F effort graph viz with SSE subscriptions - #223
Conversation
Add a Vercel-styled interactive Effort Graph visualization and the live channel it needs. - Mount GET /events SSE on the GraphQL live server (ready + generation) - Expose LiveSchemaReloader.subscribe for leak-free generation fans-out - Ship examples/effort-viz: R3F 2D scene, space-colonization growth physics, deterministic OKLCH effort colors, light/dark theme - Dogfoods ../../.flatbread-efforts via flatbread start --watch - Convenience script: pnpm play:efforts Change-Id: I85953a7374dcfea1e84c52dc48cc286e71165881
Frame the orthographic camera on settled simulation bounds so the graph fills the viewport, show labels on hover, and fetch rejected_by / mitigated_by / decision invalidates for richer live edges. Change-Id: I41dec139e1489ccb286314b1b51e6fcc9e1bbdaf
Drop invalidates/rejected_by/mitigated_by selections that the dogfood content graph does not currently expose on Decision/Risk, restoring a valid live GraphQL document. Change-Id: I2019b14f534695707452a27f02c2b11180cdf685
Use deterministic OKLCH samples in the kind legend instead of muted grey dots. Change-Id: Ia2294aac2e76fb152ab8a5ffb8b1aa747643c286
Address review findings on PR #223 for the Effort Graph viz: - Viewport: use 100dvh/svh shell, safe-area insets, and FitCamera re-fit on resize with chrome insets so the graph fills the canvas without top cutoff under the top bar. - Legend: node kinds, decision lifecycle states (proposed/accepted/ rejected/superseded), and relation-type samples with descriptions. - Canvas: encode edge kinds (dash/weight/opacity/arrows) via shared RELATION_META so chains match the legend. - Detail drawer: Notion-esque readonly MarkdownSurface from `_content.raw` (react-markdown + GFM), grouped Decision chain relations, lifecycle badges. API reserves editable/onChange for a future write surface. Tests: effort-viz physics + normalize body mapping (16 passing). Change-Id: Icebbdb7cd452ea9eff3dadb9c9e64bd83e58f7fa
…t name Journals the owner call through the typed effort-graph mutations rather than hand-edited frontmatter: - Finding (dead-end) recording why the crumb naming line does not hold: crumb reads as leftovers, collides with breadcrumb navigation, and hides the Effort primitive that the API, CLI, and skills already teach. Both prior Decisions listed exactly these as reversal criteria. - Decision "Keep Effort Graph as the product name", superseding Crumb Trail (and transitively Crumb Graph), accepted. - Both rename Issues resolved as wontfix, resolved_by the new Decision - the work is cancelled, not postponed. Change-Id: Id0da8e7252e90b4fee9fac2ea10b307fbe5943c8
… canvas The graph could not answer its most basic question: what kind of record is this? `nodeColor` hashed hue from the *effortId* and gave node kind only a +/-0.06 lightness nudge, so every record in an Effort cluster rendered as the same coloured circle. A per-kind palette already existed in `RelationLegend.tsx` and was dead code; the legend drew its six swatches from the broken function, so it showed six identical dots next to six different labels. Encoding - Hue now carries the primitive. Effort membership was already delivered by three channels - the force layout's shared centroid, a large labelled hub, and hub-tinted membership spokes - so it was spending the strongest nominal channel on the attribute that needed it least. - Shape repeats hue as a colour-vision backstop: amber/red and blue/violet partially merge under deuteranopia. Outlines are area-normalized in `lib/glyphs.ts` so a triangle and a square read at equal visual weight; otherwise size would imply a ranking nobody intended. - Effort hubs became neutral rings with a cluster-tinted core. The core also covers the membership spokes converging on the centre, which previously showed through the ring's hole as clutter. - Legend swatches and canvas geometry now derive from the same outlines and palette, so the key cannot drift from the render, and the legend lists only the relations the current generation contains. Lifecycle - `lib/lifecycle.ts` derives effective state from edges. A superseded Decision keeps `state: accepted` in frontmatter, so reading the field labelled retired reasoning as committed - the worst possible error on the record a reader most needs to get right. Retired records now fade with a struck-through label, and the drawer explains why its frontmatter disagrees. - Badges are keyed on `primitive:state`: a Risk's "accepted" means "we chose to live with this hazard" and must not borrow a Decision's reassuring green. Added the missing wontfix, mitigated, realized, and Effort status vocabulary. - Open blocker Issues wear an amber warning outline. Layout and camera - Added cluster separation to the force model. Generic node repulsion separates records but lets clusters interleave, and position is the channel carrying cluster identity. Cluster aggregates moved onto the scratch object, removing three map allocations per frame. - Effort titles anchor above their cluster's bounding box instead of on the hub, which sits at the centroid and put every label on top of the records it names. - Camera eases into place instead of cutting after a 2.4s delay, keeps re-fitting while the layout spreads, and yields permanently once the reader pans or zooms. Only the always-present legend is reserved in the fit; the transient drawer no longer costs a quarter of the canvas. Vocabulary - Aligned with the effort-graph glossary: Primitives not "node kinds", Relations not "Decision chain", relation groups named after the edges the CLI uses, and the relation descriptions rewritten to match the glossary's claims (`invalidates` says a record was *wrong*, not that it is "no longer valid"). - The header counts primitives and lifecycle rather than nodes and edges: half the edges are synthesised membership spokes, and "173 nodes" says a dot appeared where "3 proposed Decisions" says someone owes a call. - Relation rows render `directionHint` instead of a bare arrow glyph. Craft, a11y, and bug fixes - Node retraction never rendered: the scene resolved metadata from the current query result, so a deleted record vanished instantly while its edges withdrew gracefully. Metadata is now cached until the simulation drops the node. - `computeLineDistances()` ran inside `useFrame`, allocating a fresh `Float32BufferAttribute` per dashed edge per frame. Now recomputed only when the vertex count changes. - Selection committed on pointer-down while left-drag pans, so panning from a record opened the drawer. Now commits on pointer-up within an 8px slop. - Records gained an oversized invisible circular hit mesh sized from `viewport.factor`, guaranteeing a 44 CSS px target at any zoom. - Canvas is keyboard reachable: arrow keys walk a stable order, Enter opens, Escape closes, the camera follows focus, and moves are announced politely. - Type floor raised from 9-10px to 11-12px. Labels no longer change font weight on selection (it reflowed text under the pointer) and dropped their per-label backdrop blur. - Status pill has a fixed width and counts use tabular figures, so live updates cannot shift the header. - Theme follows `prefers-color-scheme` at runtime until the user picks a mode, instead of persisting the resolved system value as an explicit choice. - Reduced motion settles the layout before first paint rather than animating every spawn. - Markdown links only become in-graph navigation when the target resolves to a record; fragments and unresolved relative links stayed anchors instead of becoming silent no-op buttons. - Removed dead `ambientLight` (every material is unlit) and moved the atmospheric gradient off `body`, where an opaque app root hid it entirely. Tests: lifecycle derivation and glyph/palette invariants (31 passing). Change-Id: Ia53f416409d71f8f0866d25d6966d800e2ca97bf
Follow-up from an adversarial review pass on the encoding change. Functional bugs - Pointer hit radius was inverted. R3F hard-codes `viewport.factor` to 1 for orthographic cameras, so the "44 CSS px" calculation produced a constant 22 world units - a 176px target at the default zoom and 22px at the minimum, exactly backwards. It now divides by live `camera.zoom` (one world unit is `zoom` CSS px for an R3F ortho frustum) and caps the padding at half the resting gap between records, so a generous target can never swallow a neighbour and make selection depend on scene order. - The first click permanently disabled camera auto-fit. OrbitControls fires `start` on pointer-down before anything moves, so tapping a record counted as a pan and could strand records off-screen while the layout was still spreading. Takeover is now detected from raw wheel and drag-past-threshold input, which also avoids `change` events emitted by our own easing. - `BlockerRing` rotated unconditionally. It is the only motion left once the layout settles, so under `prefers-reduced-motion` the canvas never came to rest. Now frozen. - The reduced-motion warm-up ran 240 synchronous physics steps on every live generation, not just the first. With O(n^2) repulsion that is a multi-second freeze on a large graph - an accommodation worse than the animation it replaces. Now full only on first sync. - A failed refetch pinned the view to stale data forever: the generation marker advanced before the request. It now advances on success, rolls back on failure so the generation can be retried, and refuses to let an out-of-order response overwrite newer data. Encoding - The query is now assembled from schema introspection. Flatbread derives its schema from records on disk, so a relation field only exists once some record uses it - and the fixed document omitted `supersedes`/`superseded_by` on Findings and Constraints. A Finding has no state field, so supersession is its only retirement signal, and dropping it rendered retired evidence as live. Risk lineage and mitigation were likewise unreachable. - Constraint moved from a teal hexagon to a green bar. Blue Finding and teal Constraint collapse to near-identical cyan under tritanopia, and a hexagon is the least separable non-circle in the set, so both channels were weak at once. A bar also says "boundary" better than a hexagon did. - Levelled record chroma. At equal lightness, chroma was the only intensity variance, so Constraint at 0.11 read as less important than Risk at 0.17 - a ranking the datamodel does not make. - Resolved the hue collisions the Constraint move created: relation strokes keep one accent (h196, in the gap primitives leave between Constraint and Finding) for resolution, and invalidation drops its red, which collided with Risk. The "this was wrong" claim is carried more precisely by ghosting the invalidated record than by tinting the line pointing at it. - Retired records were near-invisible in light mode (~1.8:1 against white). White leaves far less headroom above a record's lightness than black leaves below it, so the light-mode lightness push shrank and opacity rose. - Legend line samples stopped drawing per-weight stroke widths. three.js `LineBasicMaterial` ignores `linewidth`, so the legend was advertising a distinction the canvas cannot render; weight now scales dash length, which is what actually varies. - The legend's blocker sample shows an Issue diamond inside the warning triangle. A triangle inside a triangle read as a Risk. Accessibility - Opening a record moves focus to the drawer heading and returns it to the canvas on close. Without this, Enter appeared to open something a screen reader user could never read - the body and relations stayed unreachable. Physics - Cluster separation buckets member indices (CSR layout) instead of scanning every node per cluster pair, and floors the centroid distance rather than relying on the integrator's step clamp to absorb a near-divide-by-zero. - A realized Risk now counts in the header. It is `settled` on the aliveness axis, so it was falling out of every total - and it is the record a reader most needs surfaced. CI - Wired `examples/effort-viz` into root `test` and `typecheck`. Its 46 tests and its TypeScript were previously never checked by any root script. Also journals the upstream writer defect this encoding works around (iss-writedecision-with-supersedes-leaves-the-superse--by624gyf21ex42sv): the create path for `supersedes` appends the reverse projection but never transitions the target Decision's state, while the `Supersede` retro-link mutation does. Deriving lifecycle from edges is correct regardless, since edges also cover legacy and hand-edited records. Change-Id: Ia2a4f16bc9c4883477b5e7134c1619c6a1a50883
…abels The primitive key is a two-column grid, and the gap between columns was barely larger than the gap between a glyph and its own label. A reviewer reading the panel paired each glyph with the label to its left and reported the whole key off by one. Proximity now groups each pair unambiguously. Change-Id: Id2bd3a2f719cf92437f05846f2a96a3cb8965e00
`LiveSchemaReloader` gained a required `subscribe` member when SSE generation events landed, but two test fakes were not updated. Both files therefore failed to compile, and because ts-node's `TSError` renders as an opaque `[Object: null prototype]` under Node 22, ava reported only "Non-error object" with no message - so `pnpm test` had been failing with no usable diagnostic. Root `typecheck` only covers `@flatbread/proof`, so nothing type-checks `packages/**` ahead of the suite and a compile error in a test file can only surface this way. Change-Id: I086d6a2884e9d0a3f222f678f529e5d80c5753f5
While FitCamera is easing toward a pending fit target, reader pan/zoom must clear that target immediately and stop writing the camera — takeover was only checked after the ease branch returned, so mid-ease input lost until the ease finished. Drop the world-space hit-padding cap (`MAX_HIT_PADDING_WORLD = 3`). At minZoom (~0.5) the 44 CSS px diameter needs ~44 world units of radius; the cap shrunk on-screen hits far below that floor. Hit scale is now `max(glyphRadius, (MIN_HIT_DIAMETER_PX / 2) / zoom)`. Change-Id: Iafbbf69f503a2f03fdb915083b3a9c7131765f70
Add supersedes/superseded_by to Risk RELATION_FIELDS (edges are the only retirement signal). When schema is null, select scalars only so the query does not hard-error against a live Flatbread schema that has not grown every optional relation yet. Cover Risk supersession, null-schema safety, and partial-schema filtering in query.test.ts. Change-Id: Id7f6c0c239530af70dca0a23ef527a117b01b76b
Treat deferred as a ResolveIssue resolution (settled, not open) so deferred blockers no longer get open-blocker treatment. Swap evidence directionHints to match record→Finding edges, drop the stale invalidation warning-hue comment, and align README glyph/retired-state wording with the encoding. Change-Id: Ibb90e54cc22e54f52909144ac47365c1709c1aa6
Keep the last successful schema probe across introspection blips so a transient SCHEMA_PROBE failure cannot fall back to an unsafe relation selection. Defer the generation pill and graph paint until a refetch commits, drop older-than-committed SSE responses before setNodes/setEdges, and roll the request watermark plus pill back on failure. Label SSE loss as Disconnected and query/schema failures as Error. Change-Id: If0309220fcfc9f4bac706215d19963d886724c73
Change-Id: Ia05f2dcfc4b75b660ef2263dc11fd1bfa6d75307
Change-Id: I535829f53973d9cbb2bfd9f8eabf6680e6147ace
Change-Id: I7000fd6518cff4b3b8ab4b26d0d4c60f70ce7cce
Change-Id: Iee592ae50c60892d7e43a3a5dd9d69b0876b3e0d
When relationship fields cannot be confirmed, keep showing records but label the connection Partial instead of Live so retirement links are not mistaken for complete. Split rejected overturn drawer copy, cover dangling superseded_by and rejected_by, and close out the cancelled Crumb rename Issues. Co-authored-by: Cursor <cursoragent@cursor.com> Change-Id: I063a0ae8113b6b75a6d3d152f51dc97aa21af7cc
…gibility-4df9 fix(effort-viz): make primitive and lifecycle legible at a glance
|
Rolling the stack tip into this base PR so the full effort-viz chain (including merged #228) can land on |
|
Queued — the merge queue status continues in this comment ↓. |
|
@Mergifyio queue |
Merge Queue Status
This pull request spent 4 minutes 8 seconds in the queue, including 3 minutes 35 seconds running CI. Required conditions to merge
|
There was a problem hiding this comment.
Review verdict
REQUEST_CHANGES — consensus HIGH on /events lifecycle (close() hang with live SSE; ready-before-subscribe race) plus independent HIGHs on untested live-schema fan-out and effort-viz EventSource orchestration.
Perspectives: correctness-and-contracts, test-coverage-robustness, cli-and-runtime, dx-and-examples, docs-and-positioning → judge (Grok 4.5 High / Composer 2.5).
Blocking findings (priority order)
GET /eventsready-before-subscribe race —lastSeen+readyare written beforereloader.subscribe. A commit in that window advances the schema with no listener and is never replayed (snapshot.generation <= lastSeencannot catch up). Client can stay stale until a later commit.RunningGraphqlServer.close()hangs with open SSE — each/eventsclient holds the socket (20s keepalives);close()onlyhttpServer.close()-waits for idle connections, so SIGINT/SIGTERM with effort-viz connected never settles.- Listener isolation untested — per-listener try/catch in
liveSchemais load-bearing for multi-subscriber fan-out; CI does not assert a throwing listener still delivers to peers. - effort-viz live client CI gap — production path is EventSource → refetch/probe/commit/rollback; tests only cover pure helpers. Hardcoded
http://localhost:5057also breaks non-default-p/FLATBREAD_PORT.
Coverage plan (must-have before merge)
- HIGH —
liveServerEvents.test.ts: connect race (commit between ready and subscribe still delivered / ready reports post-commit gen);server.close()completes with open/eventswithin a bounded timeout. - HIGH —
liveServerEvents.test.ts: rejectednotifyChangedproduces nogenerationframe; client abort unsubscribes and clears keepalive. - HIGH —
liveSchema.test.ts: throwing listener does not block peers; no generation-0 replay; rejected notify emits nothing. - HIGH —
useEffortGraphLive.test.ts: mocked EventSource ready/generation paint; GraphQL failure + watermark rollback; out-of-order ignored;onerrorstatus split. - MED —
graphql.test.ts/query.test.ts/normalize.test.ts: fetch error paths; mixed bare vs{ id }relation shapes; membership + self-loop drops.
Minimal fixes
- Subscribe first (or subscribe → re-read
reloader.generationintolastSeen→ emitready, buffering in-between commits). - Track SSE responses and
end/destroythem inclose()(orhttpServer.closeAllConnections()after stop-accept); addreq.on('error', cleanup). - Derive GraphQL/SSE from
NEXT_PUBLIC_FLATBREAD_GRAPHQL_URL(or document that-prequires matching client config).
Suggested follow-ups (out of scope)
- Document
/eventswire protocol in flatbread package docs (not only the example README). - Amend still-accepted
.flatbread-effortscopy that still says “Crumb Graph”. - Prefer event waiters over
setTimeoutsleeps in new live tests.
Reviewer scoreboard
| Reviewer | Verdict | Signal |
|---|---|---|
| correctness-and-contracts | REQUEST_CHANGES | HIGH |
| test-coverage-robustness | REQUEST_CHANGES | HIGH |
| cli-and-runtime | REQUEST_CHANGES | HIGH |
| dx-and-examples | COMMENT | MED |
| docs-and-positioning | COMMENT | MED |
Sent by Cursor Automation: Flatbread PR Review
| let lastSeen = reloader.generation; | ||
| res.write( | ||
| `event: ready\ndata: ${JSON.stringify({ generation: lastSeen })}\n\n` | ||
| ); |
There was a problem hiding this comment.
HIGH — ready-before-subscribe race. lastSeen is snapshotted and ready is written before reloader.subscribe (line 150). A commit in that window advances the schema with no listener; the client is told gen N while the server is already at N+1, and snapshot.generation <= lastSeen never replays the missed frame. If nothing else commits, the viz stays stale forever.
Minimal fix: subscribe first (or subscribe → re-read reloader.generation into lastSeen → emit ready, buffering any in-between commits). Cover with a connect-race case in liveServerEvents.test.ts.
| unsubscribe = reloader.subscribe((snapshot) => { | ||
| if (disconnected || snapshot.generation <= lastSeen) return; | ||
| lastSeen = snapshot.generation; | ||
| try { | ||
| res.write( | ||
| `event: generation\ndata: ${JSON.stringify({ | ||
| generation: lastSeen, | ||
| })}\n\n` | ||
| ); | ||
| } catch { | ||
| cleanup(); | ||
| } | ||
| }); |
There was a problem hiding this comment.
HIGH — open SSE blocks close(). Each /events client holds the HTTP socket open (plus 20s keepalives). RunningGraphqlServer.close() (≈259–268) only awaits httpServer.close(), which waits for idle connections — so SIGINT/SIGTERM with effort-viz connected never settles.
Also: cleanup listens for req/res close and res error, but not req.on('error', cleanup).
Minimal fix: track SSE responses and end/destroy them inside close() (or httpServer.closeAllConnections() after stop-accept). Add a bounded close()-with-open-SSE test.
| for (const listener of listeners) { | ||
| try { | ||
| listener(snapshot); | ||
| } catch (error) { | ||
| console.error('LiveSchemaReloader subscriber error:', error); | ||
| } | ||
| } |
There was a problem hiding this comment.
HIGH — listener isolation is load-bearing and untested. Per-listener try/catch is what keeps one bad /events subscriber from silencing the rest. Existing liveSchema.test.ts only covers happy-path notify + unsubscribe.
Minimal fix: positive two-subscriber commit + negative first-listener-throws; assert the second still receives and generation advances. Also lock the documented negatives: no generation-0 replay; rejected notifyChanged emits nothing.
| } from './query'; | ||
| import type { GraphEdge, GraphNode } from './types'; | ||
|
|
||
| const DEFAULT_GRAPHQL_ENDPOINT = 'http://localhost:5057/graphql'; |
There was a problem hiding this comment.
HIGH — live orchestration undertested; endpoint hardcoded. Production path is EventSource → maybeRefetch → probe/fetch/commit/rollback/reconnect, but CI only covers pure helpers in useEffortGraphLive.test.ts. Regressions in ordering, sticky-schema fallback, watermark rollback, or transport-loss status would not fail.
Hardcoding http://localhost:5057/graphql also silently breaks non-default -p / FLATBREAD_PORT (same pattern in lib/graphql.ts).
Minimal fix: mocked EventSource + graphqlFetch suite (ready/generation paint; GraphQL failure + watermark rollback; malformed/out-of-order frames; onerror status split). Derive endpoint from NEXT_PUBLIC_FLATBREAD_GRAPHQL_URL (SSE = same origin + /events) and document the override.
| import test from 'ava'; | ||
| import { mkdtemp, rm, writeFile, mkdir } from 'node:fs/promises'; | ||
| import { join, relative } from 'node:path'; | ||
| import filesystem from '@flatbread/source-filesystem'; | ||
| import markdownTransformer from '@flatbread/transformer-markdown'; | ||
| import { initializeConfig } from '@flatbread/core'; | ||
| import type { ConfigResult, LoadedFlatbreadConfig } from '@flatbread/core'; | ||
| import { startGraphqlServer } from './liveServer'; | ||
|
|
||
| interface Fixture { | ||
| dir: string; | ||
| postsPath: string; | ||
| postOne: string; | ||
| cleanup: () => Promise<void>; | ||
| } | ||
|
|
||
| const POST_ONE = (title: string) => `--- | ||
| id: post-1 | ||
| title: ${title} | ||
| --- |
There was a problem hiding this comment.
MED → covers happy path only. This suite exercises ready@0 and in-order generation after subscribe is live, but would stay green for the HIGH server bugs above: connect-time race, rejected-notify silence, client-abort cleanup, and server.close() with an open stream.
Minimal fix: add reject-no-frame, reconnect-at-N>0, connect-race, abort-cleanup, and bounded close()-with-open-SSE cases before merge.
| export async function graphqlFetch<T = unknown>( | ||
| query: string, | ||
| variables?: Record<string, unknown>, | ||
| endpoint: string = 'http://localhost:5057/graphql' |
There was a problem hiding this comment.
MED — fetch error paths and hardcoded port. graphqlFetch handles HTTP failures, GraphQL errors, 15s abort, and missing data, but none of those paths are tested. Default endpoint is also fixed to :5057.
Minimal fix: add lib/graphql.test.ts with mocked fetch for those four cases; share endpoint config with useEffortGraphLive via env/override.


Add a Vercel-styled interactive Effort Graph visualization and the live channel it needs.
What's included
GET /eventsSSE on the GraphQL live server (ready+generationframes) so clients can subscribe to content reloads without pollingLiveSchemaReloader.subscribefor leak-free generation fan-out (SSE unsubscribes on disconnect)examples/effort-viz: R3F orthographic 2D scene with space-colonization / slime-growth physics, soft repulsion displacement on add/remove, deterministic vibrant OKLCH colors per effort, light/dark theme.flatbread-effortsviaflatbread start --watchpnpm play:effortsHow to try
Open http://localhost:3000 — GraphQL on
:5057, live updates via/events.Tests
packages/flatbread/src/graphql/liveServerEvents.test.ts— SSE ready + generation framespackages/coreliveSchema subscribe coverageexamples/effort-vizphysics unit tests (pnpm --filter effort-viz test)Manual verification