Skip to content

feat(memory): WebGL2 3D graph for 50,000 nodes - #4

Open
dayton536 wants to merge 2 commits into
mainfrom
cursor/webgl-memory-graph-975c
Open

dayton536 wants to merge 2 commits into
mainfrom
cursor/webgl-memory-graph-975c

Conversation

@dayton536

Copy link
Copy Markdown
Owner

Problem and user impact

The 3D memory graph was a Canvas 2D loop: every frame projected and depth-sorted every node, stroked every edge, and hit-tested with a linear scan. The overview endpoint also stopped at 2,000 nodes and 6,000 edges, so larger archives were only ever a sample.

Change

This branch is cut from upstream oll4com/spaceapp main at f20c00a (1.1.9), not from this fork's older main.

  • WebGL2 renderer (memory-graph-webgl.ts, MemoryGraph3D.tsx): instanced disc nodes and batched lines, GPU depth, no new rendering library. Colors, sizes, and focus/neighbor highlighting follow the Canvas 2D view. Camera motion updates uniforms only.
  • Picking: screen-space uniform grid (memory-3d-pick.ts). A pointer query does not scan every node.
  • Labels: core and source labels first, collision avoidance, showTitles, with a hard cap of 96 measured and 40 drawn labels per frame.
  • Layout: the existing worker streams intermediate 3D positions while the simulation runs. Graphs at or below 4,096 nodes keep the exact solver, so memory-3d.test.ts stays deterministic. Larger graphs use a bounded deterministic grid. The graph is posted to the worker in chunks.
  • API: GET /api/admin/memory/graph/overview?limit= defaults to 2,000 and rejects values above 50,000. Edges scale at 3 per requested node, up to 150,000. redactedNode and the admin auth gate are unchanged. A non-default limit is part of the ETag.
  • Fallback: if getContext("webgl2") fails, or on webglcontextlost, the view uses the Canvas 2D renderer capped at 2,000 nodes / 6,000 edges and listens for webglcontextrestored.
  • Demo: /demo-workspace can synthesize a deterministic 2k, 10k, or 50k graph (up to 150k edges) without a backend.
  • Large responses: overview payloads above 1.5 MB are parsed in a worker and applied in chunks.

Verification

  • npm run check
  • npm test -w @space/web -- memory (52 passed)
  • npm test -w @space/api -- memory-graph (37 passed)
  • Contracts schema test for the overview limit (parses bounded memory graph)
  • Linux, macOS, and/or Windows coverage is listed. Measured on this Linux cloud VM (no integrated GPU). Headless Chrome 156 reports ANGLE (Google, Vulkan 1.3.0 (SwiftShader Device (Subzero) (0x0000C0DE)), SwiftShader driver). A full 50k/150k gl.finish loop on that software rasterizer did not return in the time available, so no GPU frame-time p95 is claimed from this host. Reproduce on a laptop with node apps/web/scripts/benchmark-memory-3d.mjs.
  • UI changes include real browser console/network evidence. Not captured here; the harness above is the frame-time path.
  • Every commit has a Signed-off-by: line (Developer Certificate of Origin 1.1), and the contribution is licensed under the MIT License.
  • No credentials, tokens, personal memory, customer data, private paths, or runtime state are included.
  • Changelog updated under Unreleased.
  • Security: redaction and admin authentication are unchanged. Rollback is reverting this commit; the default overview limit stays 2,000.

Measured numbers

Check Result
Pick query p95 at 50,000 nodes 0.062 ms (budget 4 ms)
Pick-index rebuild p95 at 50,000 nodes 1.54 ms
Overview limit=50000, 50,000 nodes / 150,000 edges 19.06 MiB (19,989,404 bytes)
Server-Timing memory-overview 482.8 ms
In-process inject, same payload 622.4 ms

Picking was recorded by tests/memory-3d-renderer.test.tsx (MEMORY_PICK_BENCH). The API size and time were recorded by tests/memory-graph-api.test.ts (MEMORY_OVERVIEW_BENCH) on this VM.

Closes oll4com#62.

Open in Web Open in Cursor 

oll4com and others added 2 commits October 28, 2026 19:01
…ble 4f01583e71 + d077b0ff0c; Node 22.23.4 base for CVE-2026-111497; acceptance for the tested tree with the Windows first-agent Zen exception, operator decision 54b65a84)
Replace Canvas 2D drawing in the 3D memory view with a dependency-free
WebGL2 renderer: instanced nodes, batched edges, and GPU depth. Hover
and click use a screen-space grid, titles stay bounded, and the force
layout streams from the worker while it converges. The overview route
accepts limit up to 50,000 nodes and 150,000 edges. Demo mode can
synthesize that graph. Without WebGL2, or after context loss, the view
falls back to the Canvas 2D renderer capped at 2,000 nodes.

Signed-off-by: Dayton Allen <daytonballen@gmail.com>
Co-authored-by: dayton536 <dayton536@users.noreply.github.com>
}
/** Repeats of one kind of message share a key: digits and dates fold, so "Daily backup missing for 2026-10-25" is one card. */
export function monitoringGuardianKey(message: string): string {
return message.split("\n")[0]!.replace(/\[[^\]]*\]/g, " ").replace(/\d+/g, "N").replace(/[^\p{L}\p{N} _:-]+/gu, " ")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[$750] WebGL renderer for the 3D memory graph: 50,000 nodes at interactive frame rates

2 participants