Preserve sitemap and page publication dates - #503
Open
PrzemyslawKlys wants to merge 3 commits into
Open
PrzemyslawKlys wants to merge 3 commits into
PrzemyslawKlys wants to merge 3 commits into
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## v2-speedygonzales #503 +/- ##
=====================================================
+ Coverage 58.05% 58.19% +0.14%
=====================================================
Files 486 488 +2
Lines 36054 36108 +54
Branches 7249 7272 +23
=====================================================
+ Hits 20930 21013 +83
+ Misses 12757 12713 -44
- Partials 2367 2382 +15 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Source discovery needs reliable dates before it selects or orders pages. This adds sitemap entries with modification dates while preserving document order and keeping child sitemaps distinct from page URLs. The existing URL-only API shares that parser and excludes image-extension locations.
HtmlPublicationDates.Parsereads declared publication and modification dates from page metadata and matching Article/WebPage JSON-LD. It keeps those dates separate, rejects other-page identities, and retains unambiguous dates across matching graph records. Structured crawler metadata preserves these dates after article selection removes the document head.A private downstream consumer uses the APIs for bounded discovery and source chronology. Its package repin follows the HtmlTinkerX release containing these changes.
Validation: 17 discovery-parser tests and 29 publication/structured-metadata tests pass on each of net472, net8.0, and net10.0. Independent read-only reviews covered both contracts; all actionable findings were fixed and confirmed. Downstream selector integration and the full 122-test consumer suite pass against a local candidate package. A live source check also confirms that language-query URLs retain dates declared for their canonical article URLs. Repository CI runs on this PR.