Skip to content

Fix region scraper false positives and improve matching logic - #34

Merged
comnam90 merged 4 commits into
developfrom
fix/region-scraper-false-positives
Jan 21, 2026
Merged

Fix region scraper false positives and improve matching logic#34
comnam90 merged 4 commits into
developfrom
fix/region-scraper-false-positives

Conversation

@comnam90

@comnam90 comnam90 commented Jan 21, 2026

Copy link
Copy Markdown
Owner

Problem

The automated region maintenance workflow was creating false positive issues for Vault regions.

  1. Regex Ambiguity: The regex for "Core Regions" was accidentally matching "Non-Core Regions" if "Non-Core" appeared first in the HTML, leading to incorrect tier assignments.
  2. Matching Logic: Some regions like "India Central" (scraped) weren't matching "Central India" (repository) efficiently.
  3. Alias False Positives: Over-aggressive alias matching caused "Australia Southeast" to match "Australia East" simply because both shared the "Australia" alias.

Solution

  1. Regex Fix: Updated parseVaultFAQ to use a negative lookbehind (?<!Non-) to ensure "Core Regions" doesn't match "Non-Core Regions".
  2. Matching Improvements: Added logic to match regions if they contain the same words in any order (e.g., "India Central" matches "Central India").
  3. Alias Refinement: Restricted alias matching to be exact-only to prevent false positives.
  4. Asterisk Handling: Improved parseVaultFAQ to correctly handle asterisk (*) markers in the Veeam Vault documentation, ensuring correct edition/tier detection.

Testing

  • Added 4 new tests to scripts/test-scraper.js (Tests 14-17) covering regex fixes, word-order matching, alias false positive prevention, and asterisk handling.
  • All 17 tests passed (17/17).
  • Verified against live scraping execution.

- Fix regex in parseVaultFAQ to correctly distinguish between "Core" and "Non-Core" regions by using negative lookbehind.
- Improve findMatchingRegion logic with a new "same words, any order" priority (e.g., "India Central" matches "Central India").
- Tighten alias matching to exact matches only to prevent false positives like "Australia Southeast" matching "Australia East".
- Add regression tests for regex fix and matching logic improvements.
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jan 21, 2026

Copy link
Copy Markdown

Deploying veeam-data-cloud-services-map with  Cloudflare Pages  Cloudflare Pages

Latest commit: 2321a07
Status: ✅  Deploy successful!
Preview URL: https://b66fc601.veeam-data-cloud-services-map.pages.dev
Branch Preview URL: https://fix-region-scraper-false-pos.veeam-data-cloud-services-map.pages.dev

View logs

- Fix regex in parseVaultFAQ to correctly distinguish between "Core" and "Non-Core" regions.
- Fix parseVaultFAQ to correctly handle asterisk (*) in Core regions (indicating Foundation only).
- Improve findMatchingRegion logic with "same words, any order" matching.
- Tighten alias matching to prevent false positives.
- Add regression tests.
@comnam90

Copy link
Copy Markdown
Owner Author

I've added the missing aliases ('Africa South 1' and 'South America East 1') to the AWS region YAML files to ensure they match the scraped names correctly. This should resolve the false positives for these regions.

@comnam90

Copy link
Copy Markdown
Owner Author

I've also improved the parseVaultFAQ logic to correctly handle asterisk (*) markers in the Veeam Vault documentation. These markers indicate that only the Foundation edition is available in certain Core regions. The scraper now correctly identifies these and avoids incorrectly flagging the Advanced edition as missing or available when it is not. Additionally, I've refined the regex to prevent 'Non-Core Regions' from being incorrectly matched as 'Core Regions'.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes false positives in the automated region maintenance scraper (especially Vault tier parsing) and improves region name matching to reduce incorrect discrepancies.

Changes:

  • Updated Vault FAQ parsing to prevent “Core Regions” regex from matching inside “Non-Core Regions”, and to handle *-marked regions with restricted editions.
  • Improved findMatchingRegion to match regions with the same words in any order, and tightened alias matching to exact matches only.
  • Added new scraper regression tests and expanded AWS region aliases to support stricter matching.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
scripts/test-scraper.js Adds regression tests for Vault parsing and improved matching behavior.
scripts/scrape-veeam-regions.js Fixes Vault regex ambiguity; improves matching logic; tightens alias matching.
data/regions/aws/aws_sa_east_1.yaml Adds a specific alias to support stricter matching.
data/regions/aws/aws_af_south_1.yaml Adds a specific alias to support stricter matching.

Comment thread scripts/test-scraper.js Outdated
Comment thread scripts/test-scraper.js
Comment thread scripts/scrape-veeam-regions.js
- Rename regionA_Core to regionACore in tests

- Update comments to reflect exact alias matching

- Update PR description and test counts
@comnam90

Copy link
Copy Markdown
Owner Author

I've addressed the code review comments:

  • Renamed regionA_Core to regionACore in scripts/test-scraper.js.
  • Updated the comment in scripts/scrape-veeam-regions.js to correctly reflect exact alias matching.
  • Updated the PR description to verify 17/17 tests passing (including the 4 new ones).

@comnam90
comnam90 merged commit 3f0cdbd into develop Jan 21, 2026
4 checks passed
@comnam90
comnam90 deleted the fix/region-scraper-false-positives branch January 21, 2026 22:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants