Skip to content

fix(webfetch): unify source validation and bound browser capture - #61

Open
lanyue-llk wants to merge 16 commits into
codex/notion-webfetch-fixes-currentfrom
codex/unified-webfetch-source-capture
Open

lanyue-llk wants to merge 16 commits into
codex/notion-webfetch-fixes-currentfrom
codex/unified-webfetch-source-capture

Conversation

@lanyue-llk

@lanyue-llk lanyue-llk commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Change

Reference checks and capture completion depended on the task family even when the task declared the same requirements. A stable record contract bypassed an explicit live-reference policy, and body/pattern references were rejected merely because no literal fact list was present. During real source collection, five browser workers also stalled beyond their declared page timeout.

The shared Web Fetch pipeline now:

  • Applies explicit live_exact policies across task kinds and contracts, rejecting missing required live text before provider calls. Existing frozen policies remain frozen.
  • Validates declared body regions, records, composite fields and patterns against both reference and independent rendered text. A fact list is optional when another content requirement supplies acceptance.
  • Runs each batch capture in a bounded worker, including browser transport and cleanup. A failed worker preserves that case's failure and diagnostics while the batch checkpoints other cases.
  • Excludes unrendered noscript and template fallback markup from painted-text observations. This fixes actual CSS/tracking text contamination observed on Amazon; DOM-mode observations preserve raw contents.
flowchart LR
    A[Task-declared source scope and requirements] --> B[Independent HTTP and browser capture]
    B --> C{Capture completed within budget?}
    C -->|Yes| D[Shared source and content validation]
    C -->|No| E[Retain acquisition failure and continue batch]
    D --> F[Freeze qualified input before provider calls]
Loading

The runtime's unused case-ID-specific fact formatter is retired in the companion library PR; historical fixtures now own their old formatting locally. No website-specific collector, scorer or repair script is added. Main-content authoring guidance explains why headings, footer text and field names alone cannot establish body coverage.

Validation

  • 201 focused tests and 323 subtests passed with real Chrome; one historical-input check skipped because its external source bundle is absent. The browser controls ran.
  • 110 source validation/freeze tests and 50 subtests passed after adding dashboard-value controls. A complete local suite passed 6,310 tests and 2,895 subtests; installed-wheel packaging and fixture smoke checks passed.
  • A real subprocess-hang control proves the batch returns an acquisition failure within the outer budget. Crash controls preserve the failing case.
  • Independent HTTP collection attempted all 200 dynamic/popular-site tasks: 192 HTTP 200 responses, six HTTP 403 responses and two transport failures. Browser acquisition covered 200 tasks; after a declared retry, 193 acquired content and seven retained failures; a separately scoped Tabulator fragment candidate succeeded. Captures remain private and acquisition success is separate from source qualification.
  • Existing frozen task inputs, provider attempts, published scores and report targets are unchanged. No provider calls are made.

Depends on #56. Companion acceptance change: lexmount/lexbench-webfetch#8. Per-case main-content input revisions are tracked there and remain incomplete; this PR fixes the shared capture and validation boundaries needed to qualify them.

Base integration

Includes #56 through e57c1059, with LexBench pinned to 071f8b68. The shared method catalog, external instances and localized task guides remain the report owners. After synchronization, 52 capture/freeze/observation tests and three subtests passed; 26 observation tests passed with real Chrome. The single missing historical-input fixture is separate from the browser controls. Static and instance-boundary gates passed.

@lanyue-llk lanyue-llk changed the title fix(references): exclude hidden fallback text from source observations fix(webfetch): unify source validation and bound browser capture Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant