You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The dashboard workflow runs the validation suite against each environment in sequence, and that loop is essentially the entire runtime. From the most recent run (26 minutes end to end):
25m 48s Run the test suite against every target
30s everything else — checkout, report generation, website build, deploy
Six targets at roughly four minutes each, one after another, on one runner. Splitting them into a matrix job — one target per runner, each uploading its raw/$t.jsonl as an artifact, with a single dependent job that downloads them all, generates the report and deploys — would bring the wall clock down to about the slowest single target plus overhead: roughly 6-8 minutes instead of 26.
dashboard.yaml already carries a ponytail: note anticipating this ("sequential loop; split into a matrix job with artifact merging if total runtime ever approaches the 6h job limit"). The 6h limit is not the reason to do it — the feedback loop is. A four-hour-old dashboard is fine; waiting 26 minutes to see whether a change to the report generator worked is not.
continue-on-error per target, since a nonzero pytest exit is normal here — the report is the artifact.
Only the final job should touch gh-pages; it force-pushes as a single commit.
Keep timeout 45m per target, which caps a hung environment.
Deliberately not done in #120: it restructures the job graph and the deploy step, and it cannot be validated without a full run, so it wants its own review.
The dashboard workflow runs the validation suite against each environment in sequence, and that loop is essentially the entire runtime. From the most recent run (26 minutes end to end):
Six targets at roughly four minutes each, one after another, on one runner. Splitting them into a matrix job — one target per runner, each uploading its
raw/$t.jsonlas an artifact, with a single dependent job that downloads them all, generates the report and deploys — would bring the wall clock down to about the slowest single target plus overhead: roughly 6-8 minutes instead of 26.dashboard.yamlalready carries aponytail:note anticipating this ("sequential loop; split into a matrix job with artifact merging if total runtime ever approaches the 6h job limit"). The 6h limit is not the reason to do it — the feedback loop is. A four-hour-old dashboard is fine; waiting 26 minutes to see whether a change to the report generator worked is not.Points to get right:
targets.ini, not hardcoded:read_targets()is the single source of truth as of Replace the website with a daily cross-environment test dashboard built around promotion drift #120. AfromJSONmatrix fed by a small setup job that prints the target list would keep that property.continue-on-errorper target, since a nonzero pytest exit is normal here — the report is the artifact.gh-pages; it force-pushes as a single commit.timeout 45mper target, which caps a hung environment.Deliberately not done in #120: it restructures the job graph and the deploy step, and it cannot be validated without a full run, so it wants its own review.