download.sh: retry transient MIT fetch failures (fail fast on expected 404) - #24
Merged
Merged
Conversation
…d 404 The ESP 1.29.2 release build failed when both `curl --head` probes to acdl.mit.edu timed out (~134s each) and download.sh fell through to its not-found branch. MIT was reachable -- it was a transient runner->MIT blip. Route the probe and download through mit_curl(): --retry / --retry-connrefused ride out transient errors (timeouts, 5xx, refused connections) and --connect-timeout 30 keeps a hung endpoint from stalling the job for minutes. A 404 is deliberately NOT retried (curl --retry skips it) -- a given version lives in exactly one of PreBuilts/ or archive/, so a miss in the other is expected and should fail over immediately. Also drops the redundant `-O ... -o` (which logged "Got more output options than URLs"). Verified: ./download.sh 129 completes end-to-end (probe -> download -> all three 129 patches, exit 0); the expected 404 on archive/ fails fast (~0s). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The ESP 1.29.2 release build failed at
Run download.sh: bothcurl --headprobes toacdl.mit.edutimed out (~134s each), so the script hit its not-found branch and exited 1. MIT was actually up — it was a transient runner→MIT connectivity blip (a re-run succeeded immediately).Change
Route both the location probe and the download through a
mit_curl()helper:--retry 5 --retry-delay 5 --retry-connrefused— ride out transient errors (timeouts, 5xx, refused connections).--connect-timeout 30— a hung endpoint fails in 30s instead of stalling for minutes.--retryskips it), by design: a given ESP version lives in exactly one ofPreBuilts/orarchive/(MIT moves releases toarchive/as they age out), so a 404 from the other directory is expected and should fail over immediately rather than burn retries.-O … -o(which loggedGot more output options than URLs).Test plan
bash -n download.shclean./download.sh 129end-to-end: probe → download → all three 129 patches applied, exit 0archive/ESP129…) fails fast, no retry (~0s, exit 22)PreBuilts/ESP129…) succeeds immediatelyContext: surfaced while cutting ESP 1.29.2 (the release carrying the FXC-8846 OuterShell fix, #23).
🤖 Generated with Claude Code
Note
Low Risk
Build-script-only change to download reliability; no auth, data, or application runtime paths affected.
Overview
Hardens MIT ESP tarball fetching in
download.shso CI no longer fails on transientacdl.mit.edutimeouts while still failing over quickly when a version is only inarchive/(expected 404).A new
mit_curl()helper wraps probe and downloadcurlcalls with--connect-timeout 30,--retry 5, and related flags for timeouts/5xx/refused connections; 404s are not retried, so PreBuilts vs archive probing stays immediate. The script picks the winning URL from HEAD checks, downloads once with-o, and drops the invalid-O … -ocombination. Comments and log/error text clarify one-location-per-version semantics and “unreachable after retries” vs hard not-found.Reviewed by Cursor Bugbot for commit 37d4633. Bugbot is set up for automated code reviews on this repo. Configure here.