[Feature]: Publish sample indexes upstream (WinUI Gallery + Community Toolkit) so find-ui stops scraping

[nmetulev]

Is your feature request related to a problem? Please describe.

winapp find-ui (#681) builds its corpus by scraping three upstream repos. The cost of that scraping is wildly uneven, and it comes down entirely to what each repo publishes:

Source	Publishes	Fetcher size
microsoft/microsoft-ui-reactor	reactor-search-index.json — metadata and sample code	7 KB (ReactorFetcher.cs)
microsoft/WinUI-Gallery	ControlInfoData.json — metadata only	43 KB (GalleryFetcher.cs)
CommunityToolkit/Windows	nothing published	54 KB (ToolkitFetcher.cs)

Reactor's fetcher is ~6x smaller for the same job, purely because the Reactor team publishes a purpose-built index. Gallery and Toolkit require us to reverse-engineer repo layout, split samples out of XAML pages, clean code-behind, fold #if branches, re-balance tags, and truncate — roughly 1,450 lines of parsing that exists only to reconstruct data the upstream repos already have.

This is not just a size problem. It produces wrong output:

• 9 of 321 Gallery scenarios (~3%) emit structurally malformed XAML that fails to parse. Verified against both the current corpus and the original winui-search snapshot in microsoft/win-dev-skills — identical results, so this is inherent to scraping, not a porting bug:

  xamlresources-3, appbarbutton-5, swipecontrol-4, swipecontrol-5, listview-8,
  infobadge-2, selectorbar-3, themeshadow-1, richtextblock-3
  

  Example — gallery-appbarbutton-5, a short, non-truncated sample:

  <AppBarButton Icon="Save" Label="Save" Click="AppBarButton_Click">
    <AppBarButton.KeyboardAccelerators>
      <KeyboardAccelerator Modifiers="Control" Key="S"/>
    <AppBarButton.KeyboardAccelerators/>
  </AppBarButton></AppBarButton.KeyboardAccelerators></AppBarButton>

• Every upstream layout change breaks us. CacheVersion.cs is on version 18, and the bump history is almost entirely upstream churn (e.g. "17" — WinUI-Gallery moved + reformatted its samples).
• Toolkit control names are maintained by hand in a 34-entry SampleOverrides table, so a newly added Toolkit sample gets a guessed name until someone notices.

Since find-ui is aimed at AI agents, a snippet that looks authoritative but doesn't compile or parse is worse than no result at all.

This also isn't a winapp-only problem: every WinUI agent/tool that wants sample data re-scrapes these same repos and independently re-derives these same bugs.

Describe the solution you'd like

Submit PRs to the two upstream repos so each publishes a machine-readable sample index, the way microsoft-ui-reactor already does.

1. CommunityToolkit/Windows — lowest effort, biggest win.

The Toolkit already generates exactly the index we need. ToolkitSampleMetadataGenerator (in CommunityToolkit/Tooling-Windows-Submodule) stitches together:

• YAML frontmatter on every components/*/samples/*.md (title, description, keywords, category, subcategory)
• > [!SAMPLE XSample] markers linking docs to sample classes
• [ToolkitSample(id:, title:, description:)] attributes on the sample classes

…into ToolkitSampleMetadata at build time, with source-generator diagnostics when the frontmatter is malformed. It just never leaves the compiler — it only materializes inside the gallery app's binary.

Ask: also emit that metadata as a JSON artifact (repo file, release asset, or docs-site output). No new data modeling and no new validation — the assembly step already exists and is already enforced.

2. microsoft/WinUI-Gallery — additive schema change.

WinUIGallery/SampleSupport/Data/ControlInfoData.json already carries UniqueId, Title, Subtitle, Description, ApiNamespace, RelatedControls, and Docs[]. What's missing is the sample code, which lives in separate Samples/{UniqueId}/{SampleDefinition} .txt bundles.

Ask: inline those bundle contents (--- header / --- xaml / --- c#) into ControlInfoData.json, or emit a sibling index that does. The repo already maintains ControlInfoDataSchema.json, so there's a natural place to version the addition.

Outcome. Either change makes that source Reactor-shaped: its fetcher collapses from ~40-50 KB to ~100 lines, and the malformed-output class of bug disappears by construction, because nothing is being guessed.

Additional context

• Blocked on nobody — this is follow-up work after feat: add winapp find-ui — WinUI control & sample search (Gallery + Toolkit + Reactor) #681 merges. feat: add winapp find-ui — WinUI control & sample search (Gallery + Toolkit + Reactor) #681 should ship the local mitigations (validate snippets at the corpus boundary; parse [ToolkitSample(...)] instead of stripping it) independently of upstream timelines.
• Suggested order: open the Toolkit issue/PR first — their data is already assembled and validated, so it's the shortest path to a proof point that helps the Gallery conversation.
• Once an upstream index exists, GalleryFetcher.cs / ToolkitFetcher.cs and most of CacheVersion.cs's bump history can be deleted outright.
• Prior art within this codebase: ReactorFetcher.cs + DATA_SOURCES.md in microsoft/win-dev-skills, which states it directly — "The Reactor team owns a purpose-built search index we consume directly, so there is no scraping or per-file cleaning."

[nmetulev]

Additional justification: this is also what makes find-ui work behind a proxy

Follow-up after profiling find-ui's network behaviour from a sandboxed agent environment. The upstream-index ask is stronger than "cleanup" — it is what makes a live corpus refresh possible at all in the environments this tool targets.

raw.githubusercontent.com is commonly blocked; api.github.com is not

raw.githubusercontent.com        FAIL (20s timeout)
github.com/.../raw/...           FAIL (redirects to raw.githubusercontent.com)
api.github.com  (contents+tree)  HTTP 200, ~200ms
codeload.github.com              HTTP 200

Standard allowlist pattern: api.github.com permitted (gh CLI, integrations, Copilot), raw.githubusercontent.com not. Agent sandboxes commonly look exactly like this — and find-ui is aimed at agents.

Every fetcher depends on raw for content, so all three sources are dead in that environment. See #704 for the winapp-side mitigation.

The scrape cannot move to api.github.com — but a published index can

api.github.com is rate-limited to 60 requests/hour unauthenticated. Measured against the real corpus:

Source	Cold-fetch requests	Reason
Gallery	up to 433	1 ControlInfoData.json + 111 pages + up to 321 SampleDefinition bundles
Toolkit	~105	1 tree + ~26 controls × (md + xaml + xaml.cs + csproj)
Reactor	1	one published index
Total	~540	against a 60/hr budget

A straight host swap fails for nine consecutive hours. Requiring a GitHub token to look up a button sample isn't acceptable for this command.

With a published index per source, a cold fetch is 3 requests total — comfortably inside even the unauthenticated budget, over the host that is allowlisted. Reactor already demonstrates this: its single-request design is the only one of the three that would keep working behind such a proxy.

So the ask to Gallery and Toolkit isn't only "delete ~1,450 lines of our scraping and fix 9 malformed samples" — it's "make WinUI sample data reachable for tooling that runs in restricted networks." That framing may also land better upstream, since it benefits every consumer of their samples, not just this CLI.