Close fenced div on CRLF input (fixes #113) - #149
Merged
Merged
Conversation
With CRLF line endings, a fenced div's closing ::: fence is not detected, so the div never closes and all following content is swallowed inside it. In the fenced_div continue callback (src/block.ts), the parser sets this.pos = m.endpos; the fence pattern (::::*)[ \t]*\r?\n matches through the \n, so on CRLF input m.endpos lands on the \n, one past the \r that getEol() records as starteol. The main loop then evaluates isBlank = (pos === starteol) as false, mistakes the fence line for a lazy paragraph continuation, and never closes the div. On LF the two positions coincide so it works by coincidence. Set this.pos = this.starteol, the position just before the line ending, correct for both LF and CRLF, mirroring the sibling code_block close path. LF output is byte-identical. Fixes jgm#113.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #113.
Problem
With CRLF line endings, a fenced div's closing
:::fence is not detected, so the div never closes and all following content is swallowed inside it.Buggy CRLF output ("after" trapped inside an unclosed div):
Correct output (the LF equivalent already produces this):
Root cause
In the
fenced_divcontinuecallback (src/block.ts), when the closing fence matches, the parser setsthis.pos = m.endpos. The fence pattern(::::*)[ \t]*\r?\nmatches through the\n, so on CRLF inputm.endposlands on the\n-- one position past the\rthatgetEol()records asstarteol. The main loop then evaluates the blank-line checkisBlank = (pos === starteol)as false, so the fence line is mistaken for a lazy paragraph continuation and the div is never closed. On LF inputm.endposandstarteolcoincide, so the check passes and it works by coincidence.Fix
Set
this.pos = this.starteol, the position just before the line ending, which is correct for both LF and CRLF. This mirrors the siblingcode_blockclose path and djot.js's\r?\nCRLF handling throughout the lexer. LF output is byte-identical; CRLF now closes the div correctly.Tests
Added a regression test in
src/html.spec.tsasserting the CRLF input closes the div (byte-identical to the LF output, which is also asserted unchanged). All existing tests pass (tsc + jest, 417 total).Thanks to @iorizu for the clear report.
This change was prepared with AI assistance and reviewed by me before submission.