FOP-3345: A substituted glyph's ToUnicode entry is the characters it stands for

[plutext]

Fixes FOP-3345.

MultiByteFont.performSubstitution maps characters to glyphs, runs GSUB, and maps the
glyphs back to characters, because layout works on characters. A glyph that substitution
produced has no character of its own unless the font's cmap happens to map one to it, so
mapGlyphsToChars minted a private-use code point for it (createPrivateUseMapping, from
U+E000). The painter hands that code point to the subset, and the ToUnicode CMap is built
from the subset's characters. So wherever a ligature, a contextual form or a decomposition
was applied, the PDF's text layer said U+E000 upward, or a presentation form, rather than
the letters: search, copy and paste and screen readers lose the text, and nothing in the
rendering shows it.

Measured on this branch, text extracted with pdftotext:

  Carlito, "fifty ti office"
    before   U+FB01 U+E000 "y " U+E001 " o" U+FB03 "ce"
    after    "fifty ti office"
  Noto Sans Arabic, U+0639 U+0644 U+064A U+0643 U+0645
    before   U+FEDC U+FEE2 U+E001 U+E000 U+FECB U+FEE0   (presentation forms, private use)
    after    U+0643 U+0645 U+E001 U+0639 U+0644 U+064A

Every word box is identical before and after: the code point used for layout is unchanged,
so no glyph, advance or position moves.

The association each glyph carries out of substitution already names the characters it
came from. MultiByteFont now records them per glyph (getGlyphMeaning) as it maps glyphs
back, CIDSet.getUnicodeSequences returns one string per selector (the recorded characters,
or the selector's own code point), and PDFFactory builds the CMap from those, which the
writer of FOP-3346 accepts: a ligature glyph publishes its letters as a bfchar with a
string destination, an Arabic contextual form its letter.

What it deliberately does not do. The second and later glyphs that a decomposition
produces from one character keep their private-use code point (the U+E001 above), because
a CMap cannot say that several glyphs share one character, and an empty destination reads
as U+FFFD, a space or a raw control character depending on the reader. A glyph seen with
two different meanings in one document publishes neither. And the stand-in glyph drawn for
a missing character records no meaning.

The private-use mapping itself is untouched, so MultiByteFont.hasPrivateUseSubstitutions()
(FOP-3337, AFP) answers as before.

This branch carries two other fixes it is built on: FOP-3346 (the CMap writer takes one
destination per selector) and FOP-3340 (an unsubstituted glyph maps back to the character
it came from), each with its own pull request.

MultiByteFontTestCase: a ligature records its characters, a contextual form its
character, an unsubstituted glyph nothing, the second glyph of a one-character cluster
nothing, a glyph with two meanings neither, the missing-character stand-in nothing.

Commits

Three commits: the first two are FOP-3346 (#114) and FOP-3340 (#109), which this change is built on and which have pull requests of their own; the third is this fix. Once those two are merged this branch will be rebased to the one commit.

🤖 Generated with Claude Code

[plutext]

Pushed 6968306, a correction to this change found on a corpus of real documents.

The first glyph of a multiple substitution recorded the source character. Cambria Regular's ccmp decomposes
precomposed letters: ά into alpha and a tonos mark, ü into u and uni0308. So its base glyphs were
published as the precomposed letters. A plain letter reaches the same glyph through the cmap, so every plain α,
o, e, u, c and A drawn with that glyph read accented, a regression against 2.11, which publishes the base
glyph's own character.

Where one character is split into exactly as many glyphs as its canonical decomposition (NFD) has characters,
each glyph now records its own piece: the base its plain letter, and the mark its combining character. That
includes U+0301 for Cambria's tonos, which 2.11 published as a private-use code point. Any other split keeps the
first-glyph rule, so the Arabic cases (a dotless base and its dots; Arabic letters have no canonical
decomposition) are unchanged.

The line à a é e ü u ά α ό ο ć c Å A in Cambria Regular, read with pdftotext:

• 2.11: à a é e ü u α α ο ο ć c A A
• this PR before: à̀ à é́ é ü̈ ü ά ά ό ό ć́ ć Å Å
• this PR now: à a é e ü u ά α ό ο ć c Å A, decomposed; identical to the source after NFC.

The drawing is byte-identical; only the ToUnicode CMap changes. MultiByteFontTestCase gains three cases, and
the two decomposition cases fail without the change. The fop-core suite gives 3684 tests and 0 failures, and
checkstyle is clean.

[skynavga]

@plutext Jason, Great work using Claude to ferret out long-standing bugs in FOP, especially as relate to complex scripts, fonts, and Unicode!