Repository navigation
feat(webfetch): support reusable seven-family native comparisons - #42
Conversation
1a7f2e3 to
da3f01b
Compare
|
Claude encountered an error after 3s —— View job I'll analyze this and get back to you. |
|
Claude encountered an error after 3s —— View job I'll analyze this and get back to you. |
|
Claude encountered an error after 3s —— View job I'll analyze this and get back to you. |
|
Claude encountered an error after 3s —— View job I'll analyze this and get back to you. |
|
Claude encountered an error after 3s —— View job I'll analyze this and get back to you. |
|
Claude encountered an error after 3s —— View job I'll analyze this and get back to you. |
|
Claude encountered an error after 3s —— View job I'll analyze this and get back to you. |
|
Claude encountered an error after 3s —— View job I'll analyze this and get back to you. |
|
Claude encountered an error after 3s —— View job I'll analyze this and get back to you. |
|
Claude encountered an error after 3s —— View job I'll analyze this and get back to you. |
|
Claude encountered an error after 3s —— View job I'll analyze this and get back to you. |
|
Claude encountered an error after 3s —— View job I'll analyze this and get back to you. |
lanyue-llk
left a comment
There was a problem hiding this comment.
审查结论:建议修正下列问题后再合入。
本 PR 将 Web Fetch 扩展为五浏览器、七类任务、每浏览器 430 题,并统一原生接口、正文评分和发布校验。共享任务投影与统一路由选择器是合适的抽象;保留动态任务分类也合理,但调用校验没有同步迁移。资源投影还把单个 URL 当成 srcset 列表解析,导致合法资源地址损坏。
确认两处功能回退,均做了基线对照:目标分支通过,PR 失败;在隔离副本中验证最小修复后,相关测试 63 项通过。现有相关测试分两组共 478 项通过、13 项跳过;启用四种真实本地浏览器后,资源、取消和文件测试 24 项通过、1 项失败,失败原因见第三条。进程清理失败后的隔离行为符合现有安全合同,不列为缺陷。
这次审查覆盖代码、调用合同及相关测试,不改变报告中独立逐结果复核 0/2,150 的范围。
|
Claude encountered an error after 3s —— View job I'll analyze this and get back to you. |
7607b18 to
2166c94
Compare
cd0b669 to
0598c6a
Compare
85f8524 to
caa269e
Compare
This reverts commit 8dd0a1a.
06b86f4 to
b61c880
Compare
The Web Fetch harness could not fairly run and explain a browser-native comparison across seven task families. Native Markdown, rendered HTML, embedded resources, direct files, retries, scoring, and failure review were handled by separate paths. The first implementation also stored one report's profile, rerun scope, release prose, and participant identities in Git.
This change makes those boundaries reusable:
report-assetswhen supplied. The root release registry is a legacy fallback, and the shared pricing catalog contains product facts rather than one run's wording.The published report remains on Artifact Site, with only a compact version/profile-hash archive binding in Git: https://artifact-site.lexmount.cn/v/IqGpMUYHWat-dw4n8usQttTptoPzboAo
Reproduce the 430-case definition
Use the report profile from the evidence package. It fixes the participants, ordered case inventory, run policy, executable identities, and report output without putting one report instance back into Git.
Assembly rejects a missing profile, stale source hash, mismatched case ownership, unregistered correction control, or report-local data embedded in the reusable authority.
Reproduce representative Browser Eval cases
These are real cases from the compiled 430-case definition, rather than fixtures made for this PR:
document-01: direct PDF body delivery. A method passes only when it returns readable document content.dynamic-104-mui-data-grid: native dynamic-page completion and structured content.replacement-ko-yonhap-sports: a live dynamic page that exercises the declared failure-retry path.risk-diverse-134: record ownership and body acceptance on a risk-controlled page.Run both Moli builds in alternating order so they see the same case sequence. Replace the two external files with the hash-bound files from the evidence package and point their generic Moli slots at the binaries under review.
The certified output retains every physical call and always uses the final declared attempt. The comparison audit checks the two binary identities, serial call order, native route, diagnostic log, output hash, and complete case matrix.
Validation
At merge, the focused architecture, scoring, report-content, pricing, and participant tests passed (83 tests and 14 subtests), all 240 evaluation/research/configuration JSON files parsed, Ruff F checks passed, and
git diff --checkpassed.Post-merge verification on main at
28534e24reran the affected evidence, native-route, report-interface, and method tests: 42 tests and 45 subtests passed. The same code rescored the four saved Browser Eval cases above without changing their verdicts:document-01: fail — Moli does not support the required readable PDF output.dynamic-104-mui-data-grid: pass.replacement-ko-yonhap-sports: fail — page loading/extraction timed out.risk-diverse-134: pass.This verification used saved, hash-bound outputs. It made no provider calls and did not alter the published results.