Repository navigation
fix(html2md): retain complex tables and referenced fragments - #1336
lanyue-llk wants to merge 32 commits into
Conversation
Spider Bench A/B✅ All benchmark browser service runs completed; results are informational. Public HEAD: 184 / 240 rows; 38 / 48 sites produced rows; 10 unexpected empty sites. ✅ Deterministic fixture contract is clean. Common ancestor Public 48-site run · informational
Public-site content, timing, and memory are noisy. This single A/B run reports evidence only; site outcome counts explain missing rows without treating them as a deterministic regression. CPU and memory timelinesThese are bounded 40-point views of the same complete process-tree samples used by the HTML report. CPU uses 100% per occupied logical core; memory lines are RSS first and PSS second when complete PSS samples are available. Basexychart-beta
title "Base CPU"
x-axis "Elapsed seconds" [1.036, 1.537, 2.538, 4.039, 5.04, 6.541, 8.043, 9.044, 10.547, 11.549, 13.052, 14.054, 15.556, 16.558, 17.058, 18.06, 19.561, 20.562, 22.066, 23.569, 24.57, 26.072, 27.073, 28.575, 29.576, 31.078, 32.58, 33.581, 34.583, 35.083, 36.086, 37.588, 39.088, 40.089, 41.59, 42.591, 44.094, 45.094]
y-axis "CPU percent" 0 --> 200
line [23.98, 131.89, 119.88, 141.96, 133.96, 109.98, 89.91, 117.77, 123.73, 103.87, 93.85, 75.82, 37.94, 0, 4, 4, 2, 35.94, 73.85, 125.66, 99.99, 101.86, 113.86, 131.91, 109.93, 99.93, 111.94, 5.99, 145.92, 79.95, 29.95, 91.78, 99.9, 100.01, 101.89, 121.77, 79.79, 105.97]
xychart-beta
title "Base memory: RSS then PSS"
x-axis "Elapsed seconds" [0.036, 1.036, 1.537, 2.538, 4.039, 5.04, 6.541, 8.043, 9.044, 10.547, 11.549, 13.052, 14.054, 15.556, 16.558, 17.058, 18.06, 19.561, 20.562, 22.066, 23.569, 24.57, 26.072, 27.073, 28.575, 29.576, 31.078, 32.58, 33.581, 34.583, 35.083, 36.086, 37.588, 39.088, 40.089, 41.59, 42.591, 44.094, 45.094, 46.538]
y-axis "Memory MiB" 0 --> 500
line [13.07, 81.93, 99.28, 103.37, 137.92, 152.79, 161.22, 128.25, 150.24, 163.59, 144.41, 135.17, 148.13, 125.74, 121.86, 120.74, 120.76, 120.39, 128.62, 120.38, 145.16, 243.99, 183.54, 208.8, 136.61, 168.27, 132.58, 148.04, 147.27, 153.18, 140.94, 140.68, 142.68, 161.92, 161.92, 161.93, 164.42, 140.66, 149.04, 0]
line [9.62, 78.42, 95.66, 99.74, 134.3, 149.21, 157.6, 124.62, 146.61, 159.94, 140.76, 131.53, 144.9, 122.09, 118.21, 117.09, 117.11, 116.76, 124.92, 116.76, 140.78, 240.35, 180.17, 205.13, 131.68, 164.53, 128.8, 144.37, 138.79, 149.51, 137.33, 137.03, 139.03, 158.28, 158.28, 158.29, 160.78, 137.02, 145.63, 0]
HEADxychart-beta
title "HEAD CPU"
x-axis "Elapsed seconds" [1.035, 2.036, 2.537, 3.538, 4.539, 5.039, 5.539, 6.039, 7.04, 8.541, 9.543, 11.046, 12.046, 13.048, 14.551, 15.552, 16.554, 18.057, 19.059, 20.561, 21.563, 22.564, 24.066, 25.066, 26.568, 27.568, 28.568, 30.07, 31.071, 32.571, 33.573, 34.573, 36.075, 37.076, 38.078, 39.58, 40.582, 42.082]
y-axis "CPU percent" 0 --> 200
line [31.99, 95.89, 127.93, 93.94, 149.91, 122.03, 131.94, 124, 143.78, 63.97, 49.89, 111.97, 0, 31.95, 107.75, 87.96, 0, 125.76, 63.87, 93.79, 97.93, 123.86, 105.87, 132.01, 133.89, 110.06, 57.93, 27.92, 89.99, 19.99, 47.96, 123.9, 61.96, 5.99, 99.89, 147.85, 59.97, 135.85]
xychart-beta
title "HEAD memory: RSS then PSS"
x-axis "Elapsed seconds" [0.035, 1.035, 2.036, 2.537, 3.538, 4.539, 5.039, 5.539, 6.039, 7.04, 8.541, 9.543, 11.046, 12.046, 13.048, 14.551, 15.552, 16.554, 18.057, 19.059, 20.561, 21.563, 22.564, 24.066, 25.066, 26.568, 27.568, 28.568, 30.07, 31.071, 32.571, 33.573, 34.573, 36.075, 37.076, 38.078, 39.58, 40.582, 42.082, 42.837]
y-axis "Memory MiB" 0 --> 500
line [13.13, 84.43, 118.09, 114.46, 112.16, 131.06, 145.91, 147.95, 153.27, 144.38, 127.58, 137.21, 138.15, 111.17, 119.99, 152.31, 136.47, 127.55, 149.8, 118.35, 156.1, 167.92, 161.78, 213.26, 162.51, 147.89, 140.64, 128.52, 132.61, 138.43, 134.37, 142.11, 183.42, 138.56, 124.17, 146.55, 132.47, 126.68, 158.07, 0]
line [9.76, 80.84, 114.52, 110.87, 108.57, 127.32, 142.76, 144.39, 149.66, 140.81, 124.07, 133.62, 134.31, 107.54, 116.36, 148.5, 132.89, 123.92, 146.9, 114.72, 150.35, 164.48, 158.13, 209.67, 158.87, 144.27, 136.07, 124.9, 128.96, 134.83, 130.73, 138.47, 179.23, 134.9, 120.5, 144.32, 128.81, 123.02, 154.48, 0]
Deterministic fixture · required contractThe fixture exercises 15 routes: 8 are expected to emit rows and 7 intentionally exercise empty, loading, or timeout behavior.
Full HTML, JSON, CSV, logs, and page snapshots: workflow run and full The public-web A/B is informational. The exact HEAD fixture contract runs as a separate required CI check; timing and public-site content deltas remain non-blocking. |
CI Regression ReportSource CI run · source state at render:
Release regression — ✅ HEAD/base failures 0/0; raw binary +0.022795%Package and image size
Startup latency and PSS
Concurrency matrix
Frontend differential — ✅ 1,020/1,020 cases matched; 0 issues
Agent episodes · Moli vs Chromium — ✅ 8/8 Moli episodes passed; 0 failures
Runtime and CDP session contracts — ✅ 26 contract cases; 0 failures
CDP smoke — ✅ 53/53 groups passed; 625 scenariosWorkers: 4 · cumulative group time: 201.90 s · failed groups: 0
WebMainBench · 545 pages — ✅ 545/545 completed; 0 unexpected failures
Frozen HTML · Linux · Diagnostics: All artifact fields are parsed by the trusted default-branch renderer; missing or invalid inputs remain visible as unavailable. |
fd2db36 to
6ed184d
Compare
Sequential Navigation Soak A/B✅ HEAD completed the 200-navigation resilience and memory observation. One browser process, one target, and one CDP session navigate CSDN → SegmentFault → Huaban → example.com repeatedly. This run issued 200 Common ancestor Session resilience
A public-page failure is reported separately from an unrecoverable session. The soak requires all navigation attempts, zero failed recovery, zero lifecycle/network ordering violations, and complete resource evidence. Process-tree memory
HEAD memory by 50-navigation quarter
Raw reports and per-navigation boundary samples: workflow run and full Public-site timing and memory are observational. Use the A/B deltas and trend shape as evidence, not as a deterministic performance threshold. |
6ed184d to
22760a4
Compare
13a8b62 to
da9b680
Compare
ba33592 to
a863725
Compare
a863725 to
eb8ccd8
Compare
eb8ccd8 to
59c3c4d
Compare
|
Closing after the consolidation audit. This belongs to an earlier experimental Markdown stack and was not part of the measured fixed build. Current failure evidence does not justify carrying its broad table and fragment expansion into #1501, so it is intentionally not merged. |
5df2dd7 to
803de4b
Compare
问题
Markdown 管道表格无法表达合并单元格、嵌套关系和单元格中的块内容;丢弃文内链接目标也会破坏导航。代码转 HTML 后备路径时,交互控件的排除规则还曾与普通代码路径不一致。有锚点的呈现表会错误退回数据表 HTML。
一图看懂
修改
简单表格输出 GFM。复杂表格有表头、标题、数据角色、表头关系或真实跨度时保留作者结构;只有嵌套布局和块内容的包装表展开为普通块。role=presentation/none 同样展开,包括带锚点的情况。
仅保留实际被引用的片段目标,保证每个目标只出现一次;覆盖表格、标题及 pre/code 根节点。按 ID 优先、精确匹配后再解码的顺序处理。无需默认禁用文内导航。
普通转换与 HTML 后备复用媒体、表单、数学和代码控件的语义判断;格式化与转义由各自输出路径完成。
修正后备路径的 Copy 按钮污染。带链接的预格式化代码保留 HTML 链接、语言和物理换行;普通包装中的标题仍输出 Markdown 标题。
按 HTML 非负整数规则判断单元格实际跨度:非法值与默认跨度继续使用 GFM;rowspan=0 和真实合并单元格保留结构。
空 poster 不生成空图片;已经有块级标签的元素不再增加重复 div,CSS 引入的块边界仍保留。
列表、表格和选项节点自行管理结构边界,CSS 补充边界不再重复分段;适配器请求的行内边界在两条输出路径中都用空格分隔。
HTML 后备共享全局目标去重;图片保留被引用的 ID 与 title,别名放在链接开始标签之前。重新解析输出验证链接文字仍属于原链接。
原生 DOM 与浏览器 CLI 使用同一完整 HN 快照;CLI 实际输出严格比较,仅去除期望文件的末尾换行。
代码元素自身可以有补充 CSS 边界,其源码后代不应用这些边界。普通媒体名称优先使用 aria-label,再使用 title。列表的作者结构与计算盒分开处理:单个 CSS 块不改变列表松紧;角色匹配使用 HTML 空白,col/colgroup 不生成额外块。
列表先完成待输出正文,再判断松紧。嵌套列表位于末尾时保持紧凑;其后还有被空行分开的正文时,整个外层列表按松列表输出。
验证
cargo test -p moli-html2md --locked:398 项通过;Clippy 通过。非法跨度和空 poster 的反例均先失败后通过;另有 CSS 块边界与实时 textarea 的两种输出路径验证。复现
本 PR 的转换库测试可独立运行:
cargo test -p moli-html2md --locked。浏览器接入在 #1337,以下 CLI 与冻结页复现使用完整 stack。最小用例
跨两列的表头保持 colspan;文内链接与目标锚点同时保留。
实际输出:
真实 WebMainBench 页面
34b822a2…LinuxFocus 文章:保留嵌套表格内容与 colspan 关系。七个相关页面通过 browser-eval 的现有 runner 共用一次定向运行,7/7 正常导出,并核对上述内容;不是 545 页质量重测。查看冻结输入、实际输出与核对结果。
19bfc785;二进制 SHA-256:1c17bba8cb6b0a532144697411d13ec6a3872d1abad96d5872da0ded93d7b8b4。345ea093。0efaa4b49a45e320a27fe6e5a0b6aad5b57259fc3321ac3448519cacc74c537e,runner 会校验。保留原网址,代理返回冻结 HTML,外部资源禁止访问,无点击或 DOM 改写。环境准备与可复制命令
results.jsonl记录成功状态、错误和输入/输出哈希,execution.json记录二进制身份,artifacts/moli/保存实际 Markdown。定向运行只用于定位和回归,不进入正式榜单;报告中的历史成绩仍绑定原测量版本。依赖
直接基于 #1335。#1334 → #1335 → #1336 → #1337 → #1380