Skip to content

Share UTF-8 text snapshots and reduce text editing allocations - #115

Merged
elcritch merged 16 commits into
mainfrom
feat/shared-utf8-text-snapshot-v2
Sep 23, 2026
Merged

elcritch merged 16 commits into
mainfrom
feat/shared-utf8-text-snapshot-v2

Conversation

@elcritch

@elcritch elcritch commented Sep 22, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • Store NimKit text in shared immutable UTF-8 snapshots with sparse rune and line checkpoints. Rich text uses compact style IDs and byte/rune ranges, and layout receives borrowed source spans through FigDraw.
  • Create storage and TextView undo snapshots only when undo can use them. Transfer prepared snapshots into undo records, reuse the first snapshot for grouped edits, and keep marked-text delegate ordering and value-change notifications.
  • Apply syntax highlighting with one ordered run-table update and one style-table compaction. Retokenize safe single-line SynEdit edits locally; use a full scan when lexical state may cross the edited line.
  • Replace dense byte-to-rune maps and SynEdit's per-byte token cells with token ranges and monotonic UTF-8 endpoint cursors. Markdown code and blockquotes likewise map only token or inserted-prefix boundaries.
  • Keep rune-based editing ranges and expose TextByteRange for parser/layout endpoints, with explicit snapshot conversions. Snapshots provide rune indexing/iteration without materializing a sequence; toRunes() remains available when a decoded sequence is useful.
  • Add regression coverage for ordinary and gap storage, undo modes, UTF-8 and malformed bytes, CRLF, and nested blockquotes. Print diagnostic RSS for large styled text and undo modes.
  • Record before/after standalone memory profiles and the public API review in TODO-utf8runes.md.
  • Bump Merenda to 0.22.0 and update CHANGES.md.

Dependency

Depends on elcritch/figdraw#93: elcritch/figdraw#93

merenda.nimble points to FigDraw's feat/borrowed-utf8-text-spans branch with the existing siwin, sharedlib, and harfbuzz features. Atlas resolves that branch at ca08508. The FigDraw PR contains the borrowed-span API, compatibility fixes, an RSS benchmark, and FigDraw 0.42.0.

Verification

  • atlas-run tests: 4/4 runners passed after the public API additions. Focused atlas-run tests tnimkit also passed (1/1).
  • Standalone release/ARC profile versus parent dd933623 (median of three fresh processes): 8 MiB gap edit with undo disabled, +20,816 to +4,384 KiB RSS; enabled, +20,864 to +12,640 KiB; SynEdit tokenization of 8 MiB spaces, +81,968 to +8,208 KiB. Applying 2,048 styles over 1 MiB used +1,440 versus +1,312 KiB RSS, while operation time fell from about 0.336 to 0.006 seconds.
  • With the public snapshot API, iterating 4,194,304 runes added 0 KiB RSS after setup; explicit toRunes() added 16,432 KiB. These are process RSS observations, not allocation counts.
  • atlas-run tests --compile-only examples/all_compile.nim: 1/1 compiled.
  • nim r tests/nimkit/styledruns.nim: passed. An earlier standalone 8 MiB / 8,192-run sample reported borrowed spans +400 KiB RSS and owned runs +12,240 KiB after storage. RSS is diagnostic; the shared NimKit runner can report zero growth when it reuses resident pages.
  • FigDraw's native library build and four dynlib test files passed with Nim 2.3.1 in the dependency PR. No dynlib build was run for this Merenda update.

@elcritch elcritch changed the title Share immutable UTF-8 text snapshots in NimKit Share UTF-8 text snapshots and reduce text editing allocations Sep 23, 2026
@elcritch
elcritch force-pushed the feat/shared-utf8-text-snapshot-v2 branch from 51cc957 to 1cb60c4 Compare September 23, 2026 01:58
@elcritch
elcritch merged commit 6116bde into main Sep 23, 2026
12 checks passed
@elcritch
elcritch deleted the feat/shared-utf8-text-snapshot-v2 branch September 23, 2026 12:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant