[lexical-website][lexical-y] Bug Fix: Keep every unchanged word of a line whose text nodes an update makes again with several words changed

.changeset/lexical-y-remade-line-inline-node.md

@@ -0,0 +1,7 @@
+---
+'@lexical/y': patch
+---
+
+[fix] Stop throwing when an update makes again the nodes of a line holding an inline node whose `isInline()` reads the node's latest version
+
+Verify: removing a line's children and appending new ones made from their JSON, in a bound editor, where the line held an inline decorator or element whose class computes `isInline()` from `getLatest()` (as the playground's `EquationNode` does), writes the new line instead of throwing that the node does not exist in the active editor state.

.changeset/lexical-y-remade-text-unchanged-words.md

@@ -0,0 +1,7 @@
+---
+'@lexical/y': patch
+---
+
+[fix] Keep every unchanged word of a line whose text nodes an update makes again with several words changed, so a collaborator's concurrent delete between two changed words stays deleted
+
+Verify: an update that removes a line's text nodes and makes new ones with two or more words changed (an agent rewriting a block from its JSON) writes only the changed words. A word a collaborator deletes between two of them at the same time stays deleted, and shows once when the collaborator undoes the delete; a letter a collaborator types into an unchanged word between them stays in that word.

packages/lexical-website/docs/collaboration/y.md

@@ -800,10 +800,13 @@ person's:
   then edits them. Where the line's text changed, the text of nodes made in
   place of removed ones keeps the start and end it shares with the line's
   text as it was, by character, whatever nodes held it (as y-prosemirror
-  keeps a text run's): a block's text nodes made again from its JSON with
-  one word changed write that word alone, so a letter a collaborator types
-  into another word of the line at the same time stays in that word, and a
-  character a collaborator deletes there stays deleted. A replacement
+  keeps a text run's), and between two changes every run of unchanged
+  words, with the white space around it that the two texts share: a block's
+  text nodes made again from its JSON with some words changed write those
+  words alone, so a letter a collaborator types into another word of the
+  line at the same time stays in that word, and a character a collaborator
+  deletes there stays deleted (and is not shown twice when the collaborator
+  undoes the delete). A replacement
   replaces whole words, as below, and so does an insertion between two
   letters or digits of a word or a deletion inside one, so that an edit
   computed before it that replaces the old word takes nothing out of the new
@@ -813,8 +816,16 @@ person's:
   between the changed words and the kept text is written with them, unless
   it was a node of its own (a tab, a line break, a node of spaces). A word
   joined to a changed word through an inline node, with no space between, is
-  written again with it; text between two separate changes is written
-  again, and an inline node made there is new where it is put. Prefer
+  written again with it, and an inline node made between two changes is new
+  where it is put. The words kept are the longest run of words the two
+  texts share in the same order (where a word repeats, which of its
+  occurrences is kept can differ from what the update meant), wherever
+  that takes at most 2,048 word edits among the words both texts have, so
+  that the comparison stays within about 30 ms for a 5,000-word line.
+  Beyond that (a line made again with most of its words moved), only its
+  common start and end are kept. A word that is not kept is written again:
+  one the update moved past kept words, and beyond 2,048 edits, every word
+  between the first and the last change. Prefer
   these to `setTextContent`, which sets a node's text whole: such a change
   (and a change the browser makes to the DOM text, as an IME composition
   does) keeps the start and end its text shares with the node's old text,
@@ -992,19 +1003,31 @@ not merge as a user might expect:
   collaborator types in those items at the same time is lost.
 - **Text nodes made again with other text.** Where an update replaces a
   line's text nodes with new ones and the line's text changed (an AI rewrite
-  that makes a block's text nodes again with one word changed, or removes a
-  block's children and appends one new text node), a replaced node whose
-  text is not all there unchanged is written again whole: its characters are
-  deleted, and the new text is inserted. What a collaborator types into that
-  text at the same time, which the update's author has not seen, stays where
-  it was, so it lands outside the new text, before it (a letter typed at the
-  end of a word ends up in front of the rewritten text, out of its word),
-  and the collaborator's formatting of that text at the same time is not
-  kept. The concurrent typed text is kept, and every client shows the same
-  result. Edit the changed
-  words in place instead (`TextNode#spliceText`, the selection's
-  `insertText`; see [Edits made by code](#edits-made-by-code-agents-tools)):
-  only the changed characters are written.
+  that makes a block's text nodes again from their JSON with some words
+  changed, or removes a block's children and appends one new text node), the
+  unchanged text keeps its characters (see
+  [Edits made by code](#edits-made-by-code-agents-tools)), and each changed
+  word is written again whole: its characters are deleted, and the new word
+  is inserted. A word counts as changed when any character of it changed,
+  punctuation joined to it included (`t5z,` made again as `t5z`), and so
+  does a word joined to a changed word through an inline node. What a
+  collaborator types into a changed word at the same time, which the
+  update's author has not seen, stays where it was, so it lands next to the
+  new word, not in it (a letter typed at the end of the word ends up in
+  front of the new word), and the collaborator's formatting of the changed
+  word at the same time is not kept. So is a word the update moved past
+  the words it kept, and, where the words both texts have need more than
+  2,048 edits, everything between the first and the last change. A changed
+  word is written again even where only part of it changed: an insertion
+  between two letters of a word (`bbcc` made again as `bb xxcc`) writes
+  `bb` again, so a collaborator who deletes the old `bb` at the same time
+  and undoes the delete brings it back next to the new one, and `bb` shows
+  twice ([#9635](https://github.com/meta/lexical/issues/9635)), as when two
+  clients remove the same text and both undo (see Undo below).
+  The concurrent typed text is kept, and every client shows the same
+  result. Edit the changed words in place instead
+  (`TextNode#spliceText`, the selection's `insertText`): only the changed
+  characters are written.
 - **Delete wins.** Text typed into a container that a collaborator deletes at
   the same time is deleted with it, and so is a block moved into it.
 - **Last writer wins** for the fields of one link (URL and target change
@@ -1299,6 +1322,11 @@ reproduces it:
 - A comment can spread onto later words when a client types uncommented
   text inside it offline while a collaborator's deletes there drop the item
   that ended it: [#9564](https://github.com/meta/lexical/issues/9564).
+- A link a collaborator removes comes back when another collaborator
+  formats a range covering it at the same time; an update that makes a
+  line's text nodes again without the link while a stale edit formats a
+  range reaching past the link's text shows the same:
+  [#9597](https://github.com/meta/lexical/issues/9597).
 - Turning a paragraph inside a container that is itself inside a
   container (both not shadow roots) into a list removes the outer
   container and makes no list, in every `@lexical/y` editor (it runs

packages/lexical-y/DECISIONS.md

@@ -60,7 +60,7 @@ issues and docs that carry it.
 | D26 | **Undo favours people taking turns over concurrent races.** Where a fix makes undo correct for sequential edits at the cost of rare races between an undo and a simultaneous edit of the same text, the sequential case wins. | Taking turns is the common case; a race with a simultaneous edit of the same text is rare. Picked on 10-28, confirmed on 10-30. | `DECIDED` | 10-30 | [#9400] |
 | D27 | **Moves are a delete and a re-insert** at launch; the binding rewrites the cheaper side. Real moves within a container ([#9449], records after Yjs's unmerged `ContentMove` design) are paused and land last, backward compatible: a new layout version that older clients open read only. Move records leave tombstones, as every Yjs binding does. Moves across containers stay copies. | Every Yjs binding turns moves into delete plus insert; Yjs dropped its own move for complexity. Reliability of the rest came first. | `DECIDED` | 10-25 | [#9449], [#9455], [y.md: Limits][limits] |
 | D28 | **Text identity is fixed at the source.** While a binding is attached, core records text edits (`spliceText`, insert and remove, split and merge), and text entity plugins record a grow as a move and a remake as a retype. Text only `setTextContent` or the DOM changed is diffed. The binding adds no per-case heuristics. | No binding can recover intent an API dropped; special-case rules were fragile and regressed neighbouring cases. | `DECIDED` | 10-27 | [#9483], [y.md: Edits made by code][y-code] |
-| D29 | **Remade text keeps its common start and end by character**, whatever the node boundaries (y-prosemirror's `updateYText` rule), and never brings back a collaborator's delete. It replaces the word matching of [#9565]. | It is the industry-standard behaviour, and word matching glued words together and merged numbers oddly. | `DECIDED` | 10-30 | [#9565] |
+| D29 | **Remade text keeps its common start and end by character**, whatever the node boundaries (y-prosemirror's `updateYText` rule), **and between two changes every run of unchanged whole words** (the longest common run of words, by a linear-space shortest edit script over the words both texts have, up to 2,048 edits, about 30 ms for a 5,000-word line; past it, only the start and end; a word the update moved is written again; a word changed only in part is written whole, so a collaborator's undone delete of its unchanged part shows twice, [#9635]), and never brings back a collaborator's delete. It replaces the word matching of [#9565]. | It is the industry-standard behaviour, and word matching glued words together and merged numbers oddly. Keeping only the start and end wrote everything between two changed words again, so a collaborator's delete there came back, and showed twice after the collaborator's undo ([#9621]). | `DECIDED` | 10-30 | [#9565], [#9598], [#9621] |
 | D30 | **Diffed rewrites snap to whole words**; keystrokes and IME composition keep the exact diff. Scripts without spaces treat each character as a boundary. | Concurrent rewrites otherwise split words. Real word segmentation is a follow-up ([#9467]). | `DECIDED` | 10-26 | [#9400], [#9467] |
 | D31 | **A list item's nested list gets an item of its own.** `NodeStructureExtension`'s list rule prevents an item holding a nested list next to other content when it is written; the binding does not repair it afterwards. | That shape lost text in outdents, Backspaces and offline merges. Preventing it at write time follows D20. | `DECIDED` | 10-26 | [#9452], [y.md: Limits][limits] |
 | D32 | **List merging ignores `start`.** | Existing Lexical behaviour, intended. | `DECIDED` | 10-23 | [#9452] |
@@ -191,10 +191,13 @@ issues and docs that carry it.
 [#9561]: https://github.com/meta/lexical/pull/9561
 [#9582]: https://github.com/meta/lexical/issues/9582
 [#9593]: https://github.com/meta/lexical/issues/9593
+[#9598]: https://github.com/meta/lexical/pull/9598
 [#9599]: https://github.com/meta/lexical/issues/9599
 [#9603]: https://github.com/meta/lexical/issues/9603
 [#9605]: https://github.com/meta/lexical/pull/9605
 [#9607]: https://github.com/meta/lexical/pull/9607
 [#9608]: https://github.com/meta/lexical/issues/9608
 [#9609]: https://github.com/meta/lexical/issues/9609
 [#9610]: https://github.com/meta/lexical/issues/9610
+[#9621]: https://github.com/meta/lexical/issues/9621
+[#9635]: https://github.com/meta/lexical/issues/9635

packages/lexical-y/src/__tests__/unit/matrix/C9RemadeText.test.ts

@@ -50,6 +50,7 @@ import {
 import {describe, expect, test} from 'vitest';
 
 import {createYDoc, YCollabExtension} from '../../..';
+import {CATALOG_NODES, NODE_SHAPES} from '../../catalog/nodes';
 import {
   $createEquation,
   expectConverged,
@@ -332,3 +333,150 @@ describe('C9: a highlighted code line made again with one token changed', () =>
     }
   }
 });
+
+/** Every text in a JSON tree, with `changes` made in it. */
+function changed(
+  json: SerializedLexicalNode,
+  changes: readonly [string, string][],
+): SerializedLexicalNode {
+  const node = {...json} as SerializedLexicalNode & {
+    children?: SerializedLexicalNode[];
+    text?: string;
+  };
+  if (typeof node.text === 'string') {
+    for (const [from, to] of changes) {
+      node.text = node.text.replace(from, to);
+    }
+  }
+  if (node.children) {
+    node.children = node.children.map(c => changed(c, changes));
+  }
+  return node;
+}
+
+/** Makes the line's nodes again from their JSON, with `changes`. */
+function $rewriteAll(changes: readonly [string, string][]) {
+  const line = $line();
+  const json = line.getChildren().map(c => changed($deepJSON(c), changes));
+  for (const child of line.getChildren()) {
+    child.remove();
+  }
+  line.append(...json.map(j => $parseSerializedNode(j)));
+  $setSelection(null);
+}
+
+// #9621: with a word changed on each side (in the first and the last text
+// node), the words between the changes keep their characters too: the
+// peer's edit of `t2z` or `t3z` stays where it was made, and a delete it
+// undoes leaves the word once.
+describe('C9: a line made again from its JSON with a word changed on each side keeps a collaborator edit between them', () => {
+  const CHANGES: [string, string][] = [
+    ['t1z', 't8z'],
+    ['t4z', 't9z'],
+  ];
+  for (const [name, $shape] of SHAPES) {
+    for (const [editName, edit, kind] of [
+      ...EDITS,
+      [
+        'deletes and undoes the digit of',
+        (leaf: TextNode, at: number) =>
+          void leaf.spliceText(at + 1, 1, '', false),
+        'undo',
+      ] as const,
+    ]) {
+      for (const word of ['t2z', 't3z']) {
+        for (const timing of ['concurrent', 'unread'] as const) {
+          for (const ids of [
+            [1, 2],
+            [2, 1],
+          ]) {
+            test(`${name}: the peer ${editName} "${word}", ${timing} (ids ${ids})`, async () => {
+              const t = await setup(
+                () => {
+                  $getRoot().append(
+                    $createParagraphNode().append($t('Title')),
+                    $createParagraphNode().append(
+                      $t('t1z t2z '),
+                      ...$shape(),
+                      $t(' t3z t4z'),
+                    ),
+                  );
+                },
+                {ids, nodes: [...NODES, HashtagNode]},
+              );
+              const before = lineText(t.a);
+              t.offline();
+              if (timing === 'unread') {
+                t.b.update($edit(word, edit));
+                t.a.editor.update(() => $rewriteAll(CHANGES));
+                await t.sync();
+                await flush();
+                await t.sync();
+              } else {
+                t.a.update(() => $rewriteAll(CHANGES));
+                t.b.update($edit(word, edit));
+                await t.sync();
+              }
+              if (kind === 'undo') {
+                t.b.undo();
+                await t.sync();
+              }
+              let expected =
+                kind === 'undo'
+                  ? before
+                  : before.replace(word, edited(word, kind));
+              for (const [from, to] of CHANGES) {
+                expected = expected.replace(from, to);
+              }
+              const fresh = await freshReader(t.a, [...NODES, HashtagNode]);
+              expect([...t.peers, fresh].map(lineText)).toEqual(
+                [...t.peers, fresh].map(() => expected),
+              );
+              await expectConverged(t.peers);
+              expect(t.peers.flatMap(p => p.warnings)).toEqual([]);
+            });
+          }
+        }
+      }
+    }
+  }
+});
+
+// Every inline shape of the catalog (`catalog/nodes.ts`), its nodes made
+// again from their JSON with the words around it changed: the shape comes
+// back as it was, the same on every client. Reading the old line's layout
+// asked a removed node whether it is inline in the new state, which a class
+// that reads its latest version there (the playground's EquationNode)
+// refused with a throw.
+describe('C9: a line made again from its JSON around each catalog inline shape keeps the shape', () => {
+  const $json = (n: LexicalNode): SerializedLexicalNode => $deepJSON(n);
+  const CHANGES: [string, string][] = [
+    ['t1z', 't8z'],
+    ['t4z', 't9z'],
+  ];
+  for (const shape of NODE_SHAPES.filter(s => s.$inline !== undefined)) {
+    test(shape.name, async () => {
+      const t = await setup(
+        () => {
+          $getRoot().append(
+            $createParagraphNode().append($t('Title')),
+            $createParagraphNode().append(
+              $t('t1z t2z '),
+              ...shape.$inline!(),
+              $t(' t3z t4z'),
+            ),
+          );
+        },
+        {nodes: [...CATALOG_NODES]},
+      );
+      const json = (p: Peer) => p.read(() => $line().getChildren().map($json));
+      const expected = json(t.a).map(j => changed(j, CHANGES));
+      t.a.update(() => $rewriteAll(CHANGES));
+      await t.sync();
+      expect(json(t.a)).toEqual(expected);
+      expect(json(t.b)).toEqual(expected);
+      await expectConverged(t.peers);
+      expect(t.peers.flatMap(p => p.warnings)).toEqual([]);
+    });
+  }
+});