Skip to content

[Feature]: find-ui is unusable behind proxies and in agent sandboxes (raw.githubusercontent.com blocked) — bake and embed the corpus #704

Description

Is your feature request related to a problem? Please describe.

winapp find-ui (#681) fetches its entire corpus from raw.githubusercontent.com at runtime. That host is blocked in many corporate networks and in most agent sandboxes — which is the environment find-ui is explicitly designed for.

Measured from a sandboxed agent environment:

raw.githubusercontent.com        FAIL (20s timeout)
github.com/.../raw/...           FAIL (redirects to raw.githubusercontent.com)
api.github.com  (contents+tree)  HTTP 200, ~200ms
codeload.github.com              HTTP 200

This is the standard allowlist pattern: api.github.com is permitted (gh CLI, integrations, Copilot), raw.githubusercontent.com is not.

Effect on each source today:

Fetcher Index request Content requests Works behind this proxy?
Gallery raw raw no
Reactor raw no
Toolkit api.github.com tree ✅ raw no — dies after the tree

So find-ui is entirely non-functional in these environments. To its credit it fails loudly on a fully cold cache (❌ No WinUI control data is available… connect to the internet and run the command once, exit 1), so this is a functionality gap rather than a correctness bug — but it means the command cannot serve its primary audience wherever that host is filtered.

Note: with a partially warm cache it fails silently instead — see the --source/--id finding raised on #681 (H4), where a source that failed to load is reported as a genuine "no match". The two compound badly.

Describe the solution you'd like

Bake the scenario corpus at build time and embed it, while keeping the network refresh.

The original winui-search in microsoft/win-dev-skills did exactly this — Data/gallery-scenarios.json, toolkit-scenarios.json, reactor-scenarios.json, auto-baked by a winui-search update command, with BackgroundUpdater.cs refreshing from GitHub out of band. The port to winapp dropped all of it in favour of always-fetch (CacheVersion.cs: "Unlike the upstream winui-search tool, find-ui ships NO embedded scenario snapshot").

Restoring it should be embedded snapshot as a floor, network refresh as the ceiling — not one or the other — so we don't trade a proxy bug for a staleness bug.

Size is not an obstacle. Brotli, measured on the real corpus:

gallery scenarios.json : 575 KB → 57 KB  (10.1x)
toolkit scenarios.json : 106 KB → 12 KB  (9.0x)

70 KB embedded for a corpus that today costs hundreds of HTTP requests on every cold start.

This single change resolves four separate problems:

  1. Works offline / behind a proxy — zero network required for a first run.
  2. Makes corpus bugs visible. 9 of 321 Gallery scenarios currently emit structurally malformed XAML (xamlresources-3, appbarbutton-5, swipecontrol-4, swipecontrol-5, listview-8, infobadge-2, selectorbar-3, themeshadow-1, richtextblock-3). Baked in CI, validating them becomes a build gate with a reviewable diff instead of a silent per-user runtime event.
  3. Simplifies the cache. SearchProvider.cs is 173 code lines here vs 113 upstream (+53%), because removing the guaranteed fallback pushed "what if there's no data" into every caller: TTL-ignoring stale re-read, ControlsDataUnavailableException, allowCoreOnly, the one-shot fetch notice, dual-engine memoization. Upstream's whole load path is 11 lines.
  4. Removes cold-start latency — currently a multi-hundred-request fetch on first use.

Additional context

Why we can't simply switch to api.github.com. It's reachable where raw is blocked, but it's rate-limited to 60 requests/hour unauthenticated. Measured request counts for a cold fetch:

Source Requests
Gallery 1 (ControlInfoData.json) + 111 pages + up to 321 SampleDefinition bundles = up to 433
Toolkit 1 tree + ~26 controls × (md + xaml + xaml.cs + csproj) ≈ 105
Reactor 1
Total ~540 against a 60/hr budget

A host swap would fail for nine consecutive hours. Authenticating raises it to 5,000/hr, but requiring a GitHub token to look up a button sample isn't acceptable for this command. raw.githubusercontent.com is not rate-limited, which is precisely why the scraping design works — but only where that host is reachable.

Relationship to #703. Once Gallery and Toolkit publish sample indexes, a cold fetch becomes 3 requests total, which fits comfortably inside even the unauthenticated 60/hr api.github.com budget — making live refresh viable through the host that is allowlisted. The two issues reinforce each other: this one gives a working floor now; #703 makes the refresh path work behind a proxy too.

Not recommended: a rawapi.github.com/contents fallback would rescue Reactor (1 request) and Gallery's ControlInfoData.json, but cannot rescue a 433-request scrape — partial fix only, and unnecessary once #703 lands. A codeload.github.com tarball is 1 request and works, but downloads the whole repo.

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions