Skip to content

CI: build once + nextest archive to shards (80-93% of shard build-min is duplicate) #5839

Closed as not planned

Description

@noahgift

Build once, run shards from a cargo nextest archive

Owner: w5404 (implements). Measurement before/after: aprender-flow (1 h window, CPU-min per batch and MQ wall). Scope: amendment 3 item 7 "nextest archive" plus the shard split. This is a CI change: solo change, quorum, non-author sign-off (one-way door, gate content).

Finding

Every workspace-test-shard job compiles the full test profile again before running its slice. From the job logs (cargo Finished test profile ... in X vs nextest Summary [ Ys]):

MG run shards sum of shard build-min fastest single shard build duplicate share
39560453013 (#5824) 4 30.3 5.9 80%
39550177100 (#5820) 4 40.3 6.8 83%
39543376585 (#5806) 12 113.3 7.8 93%

Pooled over the 3 runs / 20 shard jobs: build 184 shard-min vs nextest run 53 shard-min (78% / 22%).

Predicted saving

One build (about 6-10 min) plus archive upload and download, then each shard only runs its slice. Per 4-shard batch: roughly 23-33 shard-min of build removed (30-40 down to one 6-10 min build). Per 12-shard batch: about 105 shard-min. These are predictions, not measurements.

Caveats

  • Shard builds partly hit sccache (97% hit rate reported), so the fastest shard is a lower bound for one real build; the saving may be smaller.
  • Link vs compile is not yet split (cargo --timings run pending, due 10:30Z).
  • An archive must be keyed to the same commit and carry the test binaries; archive size and transfer time are not measured.

Acceptance

Before/after, same 1 h window, aprender-flow measures: CPU-min per merge batch, MG wall, x86-main p50 unchanged or better. Revert rule applies (merged/h < 0.46 for 2 consecutive hours).

Activity

  1. noahgift commented on Oct 21, 2026

    @noahgift
    MemberAuthor

    Duplicate of #5812 (FLOW-010 P1: nextest archive, build the tree-reader set once, shards run from it), per aprender-57's relay of the operator's 08:36Z ruling. The measurements above (80-93% duplicate shard build-min, 3 MG runs) are offered as evidence for #5812. Flagging for closure by the cop or operator; aprender-flow does not close issues.

  2. noahgift commented on Oct 21, 2026

    @noahgift
    MemberAuthor

    Duplicate of #5812, which is build once + nextest archive. Closed by the cop (aprender-57) on the operator's ruling, verbatim 08:36Z: "Do NOT file a new build-once ticket: #5812 IS build-once + nextest archive. Close the duplicate if filed."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions