Repository navigation
Investigate reliable content coverage for changing article post lists #9
Description
Problem
PR #5 retains five changing post-list cases: article-038/039/040/041/101. Their current acceptance checks cover required stable text and distinct attributed records with nonempty bodies, using count floors of 4/8/4/4/16. They do not establish completeness of each displayed post body.
The reviewer reproduced that all five cases pass when each qualifying post body is replaced with x; removing the records fails. See the review findings. This is a known limitation of the current contract, not a newly introduced regression.
Current scope decision
Retain the five lists and existing structural checks in PR #5. Do not describe a passing result as proof that every displayed post is complete. Do not replace these cases, lower existing thresholds, freeze historical post text, or add arbitrary minimum text lengths to make this limitation disappear. The user agreed to retain this scope for now; this issue does not assert reviewer approval or resolve the separate reference-material distribution concern.
Follow-up investigation
Prototype in the execution harness before changing this repository's contracts:
- Assess whether an independent contemporaneous page reference can be collected reliably and compared with provider output. A separate page load may return different posts, order, or visible text; source disagreement must not automatically become a fetch failure or a pass.
- Limit the reference to automatically displayed content. Do not click, expand, switch tabs, log in, or require hidden full-post text. Preserve legitimately short posts and quoted content.
- Keep the acceptance procedure reusable while allowing the observed records to change. Never use the evaluated provider's own output as its reference.
- Measure acquisition latency, failure rate, page disagreement, and operational cost on these five cases. Report whether the stronger check is practical before adopting it.
- If justified by that experiment, define generic, data-driven per-record coverage in this repository; keep provider adapters, reference acquisition, and run evidence in the harness or external storage.
Completion evidence
Demonstrate paired controls for complete displayed text, genuinely short posts, truncated text, missing records, wrong content, duplicates/padding, and differing or unavailable source views. Document how mismatched source views are reported as inconclusive. State the precise guarantees and remaining limits; do not claim universal completeness.
Closing this issue requires a supported implementation or an evidence-backed decision to retain structural-only coverage, not merely adding a text-length threshold. No per-run reference workflow is implemented or authorized by filing this issue.
Activity