[ahmedriad1]

Description

Fixes both preprocessCustomTags problems reported in #706.

1. Self-closing tags. The open-tag regex also matched <cite id="a" />. When no </cite> followed, the tag was handled as an unclosed streaming container. If it ended a line, every later blank line in the document became <!---->, which split loose lists into one list per item. If a </cite> came later, the self-closing tag paired with it, and everything in between got padded.
2. Tags that follow text. A tag after other text on its line, such as He said: <verse>line one\nline two</verse> and left., got the blank-line sandwich. That ends the paragraph right after the opening tag, so the tag rendered empty and its body became a separate paragraph.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Performance improvement
• Refactoring (no functional changes)

Related Issues

Fixes #706

Changes Made

• Self-closing tags: they never match as opening tags, in either pass. Both passes share one open-tag pattern, openTagSource.

• Tags that follow text: they are left unchanged in both passes, as same-line tags already were. A tag keeps the fix(custom-tags): render Markdown inside custom tags with multiline content #484 container behavior only when it starts a block (createStartsBlock). It starts a block when only these come before it on its line (BLOCK_PREFIX_RE):

  • indentation;
  • list or blockquote markers;
  • closing tags.

  It also starts a block when only that same kind of run separates it from the closing tag of an earlier multi-line container that itself starts a block, as in <note>\na</note><note>\n**b**\n</note>. That container gets padded, so a blank line follows its closing tag. Containers are decided in document order, so for adjacent containers the result does not depend on the allowedTags order, in the closed and the streaming pass alike.

• Blank lines in the body: a tag that follows text is not padded even when its body has a blank line. Padding it can never put the body inside the tag, because the paragraph ends right after the opening tag, so it only emptied the tag. Leaving it alone also means streaming frames match the final render.

• Speed: line starts are found by binary search, and each prefix run is matched once. Container pairs are only collected when a tag that follows text comes right after a closing tag or a marker. Long single-line inputs and long chains of containers stay linear.

• Changeset: added a streamdown patch changeset.

Testing

• All existing tests pass
• Added new tests for the changes
• Manually tested the changes

20 new tests:

• 9 fail on main: unit tests for each pass, plus the two allowedTags: a self-closing custom tag at the end of a line turns every later blank line into <!----> (preprocessCustomTags) #706 repros as render tests.
• 11 pass on main and guard behavior that must not change:
  • a container tag that starts a bullet item, an ordered item or an indented line;
  • a container tag right after another container's closing tag: on its own line, at the end of the last body line, followed by a space, from a tag listed later in allowedTags, and while streaming;
  • the same output in either allowedTags order for adjacent containers;
  • a byte order mark at the start of the document.

Manual checks, comparing main and this branch:

• Real documents: 438 markdown documents from our app, in mode="static" and in streaming mode, including every character prefix of the 65 answers that contain tags. These are LLM answers that use a self-closing citation tag and inline quote tags, plus search chunks and docs.
  • Rendered HTML changes in 21 documents.
  • The changes are lists that no longer split, and stray newline text nodes between blocks that are gone.
  • In every changed document, this branch's output equals rendering with no preprocessing at all.
• Edge cases: about 250 hand-written cases covering:
  • attributes;
  • case;
  • list and blockquote prefixes;
  • adjacent and nested tags;
  • both allowedTags key orders;
  • literalTagContent;
  • fences.
• Order fuzzing: 200k random documents of adjacent containers, markers and closing tags give the same output in every allowedTags order.
• Timing: inputs built to stress the new check (200k characters: many tags on one line, long runs of closing tags, containers chained on one line) all stay linear. The worst is the chain, at 85 ms against main's 12 ms. Self-closing tags on one line drop from 500 ms on main to about 1 ms. The 438 real documents take about as long as on main.

Behavior changes beyond the issue

• Tag after an opening block tag on the same line: a custom tag right after an opening block tag, as in <div><note>\n**b**\n</note></div>, now keeps its body as raw HTML, as CommonMark does. Main padded it and parsed the body as markdown. On its own line, the tag still works as a container.

Not changed

• Paragraph continuation lines: a container tag that starts a line inside a running paragraph (Text\n<note>\n…) is still padded, as on main.
• Text after self-closing tags: the HTML parser ignores /> on non-void elements, so a self-closing custom tag still wraps the text after it, up to the end of its paragraph.

Checklist

• My code follows the project's code style
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation (no doc changes needed)
• My changes generate no new warnings or errors
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have created a changeset (pnpm changeset)
• All my commits are signed

[vercel]

@ahmedriad1 is attempting to deploy a commit to the Vercel Team on Vercel.

A member of the Team first needs to authorize it.