Skip to content

Preserve fully evidenced header-neutral table grids - #224

Merged
silverstein merged 5 commits into
masterfrom
codex/headerless-tables-20261013
Oct 13, 2026
Merged

silverstein merged 5 commits into
masterfrom
codex/headerless-tables-20261013

Conversation

@silverstein

@silverstein silverstein commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

Outcome

Add a default-off allow_headerless_tables option to convert_pdf_to_markdown.
When complete cell-rectangle geometry and source coverage support a grid but
do not establish a header row, the opted-in profile preserves escaped HTML
data cells rather than inventing Markdown header semantics.

Ordinary calls and explicit false retain the complete previous envelope and
renderer 1.21.0. Opted-in calls identify renderer 1.22.0 and retain a typed
header-role uncertainty gap and partial status. No new tool, OCR or model call.

Boundaries

  • Require complete rectangle census, exact consumed-run coverage, unique item
    assignment, full text-box containment and complete retained evidence.
  • Refuse ambiguous, overlapping, cross-cell, truncated or suspect evidence.
  • Sanitize unsafe controls/UTF-16, escape markup and neutralize URL-like text.
  • Emit only td cells, no headers, spans, inherited values or semantic blanks.
  • Preserve compact/proposal composition, source/share parity and public schemas.

Independent review found and closed a source-row omission and unsafe-text
sanitation bypass before this final candidate. Additional review accepted the
final bounded source and tests. The existing broader table-topology, scan and
equation limitations are not solved by this option.

Verification

  • Exact source candidate: 81b7400a9d7bd9b544357d836e917cce258cdd71.
  • Stonebook macOS, Darwin/x64, Node 22.23.2: nine focused/adjoining files,
    338 tests passed, zero failures. Includes actual synthetic PDF/MCP conversion,
    both optional flags, outside-grid/disjoint-grid retention, unsafe text,
    URL neutralization, source/schema/contract and neighboring table verifiers.
  • Syntax, whitespace checks and byte-identical source/share runtime modules pass.
  • Initial CI identified stale exact source fingerprints in the synthetic layout
    occurrence oracle. Regeneration and independent review verified that only
    two module size/hash records and their aggregate hash changed. All eight
    cases, expected facts, layout bindings and scoring rules are unchanged.
    The three affected Mac files then passed 20/20 tests. No test was waived.
  • Mac native partition: 99 passed, nine platform-conditional skips, zero failures.
  • Transactional share contract passed: 61 tools, 14 prompts, native raster image.
  • Two clean isolated MCPB builds were byte-identical, 1,586 files / 72,900,522
    bytes, SHA-256 bb4728da1390d39a5d2b27f5c3b992ec08b4cdc27b9f2dc70e0902d3a4497140.
  • Packed Darwin/x64 smoke passed; a separate synthetic packed-server probe
    verified the new HTML option, unchanged default envelope, both optional flags,
    source-byte preservation and invalid-argument rejection.
    Build/native checks ran at the preceding runtime-identical f8b97b5;
    the successor changes only the nonpackaged oracle metadata. The packed new-
    option probe was repeated successfully at final 81b7400.
  • All eight normal PR CI checks passed at final 81b7400. Six passed initially;
    the two third-shard runners lost communication without test conclusions or
    downloadable logs. One unchanged-source targeted rerun passed both jobs.
    The cause remains unproven and is separately tracked, not called a source fix.
  • Additional unfiltered Mac npm run test:all passed at final 81b7400,
    actual exit 0: 213 Vitest files / 4,158 tests passed, seven files / 109
    conditional tests skipped; native partition 99 passed / nine conditional
    skips / zero failures. No filters, budget changes or timing skips were used.
    Aggregate source barriers and final manual Git check confirm clean exact HEAD.

This is bounded source behavior and synthetic evidence, not installed desktop,
Windows, release or comparative benchmark acceptance. The historical default
text-only benchmark is unchanged and has not been rescored.

See Header-neutral tables for the option and limits.

@silverstein
silverstein merged commit f6a86c1 into master Oct 13, 2026
14 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant