Context
Every automatic source is wired and every one can be run from the admin, but somebody has to press a button.
Achievements should keep up with Boost without anyone remembering to.
Two things from the earlier engine ticket are needed to read this one. A backfill grants whatever a source
supports and the site is missing, and structurally cannot remove anything, which is the only reason it is
allowed to run unattended; removing grants is the separate two-way reconcile, which stays a deliberate
human action. And every real run is already recorded in a sync log - one row per source per run, saying
what changed, when, and whether a person or an automated job started it.
The site already has a weekly release pipeline that refreshes exactly the data these sources read - commits,
library metadata, authors and maintainers. That is the right place to hang the backfill: the sources are freshest
immediately after those steps run, and there is no separate schedule to own or monitor.
Sequence this last in the ingestion half of the phase. By the time it lands, every source has been reviewed
on its own and exercised by hand through the admin, so switching it to unattended is a change of trigger rather
than a change of behaviour.
Two details decide where the calls go. The pipeline runs several of these refresh tasks, and those tasks also run
standalone. So the sweep belongs at the end of the pipeline, after everything it reads has been refreshed - while
a task that runs on its own and changes source data needs its own scoped call, covering only the sources whose
data it actually touched.
Scope
- Add a Backfilling achievements action to the release pipeline, last in the list.
- In the authors-and-maintainers task, backfill only the three library sources whose upstream data that task just
changed. A blanket backfill would also sweep the commit and review tables it never touches.
- Tag the pipeline's runs so the sync log distinguishes the weekly job from a person pressing an admin
button. Support reads that log when a member asks where their badge went, and "a scheduled sweep did it"
and "an admin did it" are different answers.
- In the commits task, record why there is deliberately no backfill call: it runs as a step of the release
pipeline, which sweeps everything once at the end, so calling it there too would walk the whole commit table
twice per release.
- Tests asserting the wiring. The pipeline's existing command invocations have no coverage, so add it for the new
calls rather than following that pattern.
Out of scope
- Reconciling on a schedule. Nothing unattended is allowed to delete grants; reconciliation stays a deliberate
admin or shell action.
- Giving anything automatic a way to override the engine's refusal to prune a source that yielded nothing at
all. That override exists for a human who knows the emptiness is real, and it stays that way.
- Any new schedule of its own.
Acceptance criteria
Risks & considerations
This is the change that makes badges appear in production without anyone pressing anything. Everything it runs
is additive - the backfill command cannot delete a grant or revoke a badge - so the failure mode is over-granting rather than data loss.
Context
Every automatic source is wired and every one can be run from the admin, but somebody has to press a button.
Achievements should keep up with Boost without anyone remembering to.
Two things from the earlier engine ticket are needed to read this one. A backfill grants whatever a source
supports and the site is missing, and structurally cannot remove anything, which is the only reason it is
allowed to run unattended; removing grants is the separate two-way reconcile, which stays a deliberate
human action. And every real run is already recorded in a sync log - one row per source per run, saying
what changed, when, and whether a person or an automated job started it.
The site already has a weekly release pipeline that refreshes exactly the data these sources read - commits,
library metadata, authors and maintainers. That is the right place to hang the backfill: the sources are freshest
immediately after those steps run, and there is no separate schedule to own or monitor.
Sequence this last in the ingestion half of the phase. By the time it lands, every source has been reviewed
on its own and exercised by hand through the admin, so switching it to unattended is a change of trigger rather
than a change of behaviour.
Two details decide where the calls go. The pipeline runs several of these refresh tasks, and those tasks also run
standalone. So the sweep belongs at the end of the pipeline, after everything it reads has been refreshed - while
a task that runs on its own and changes source data needs its own scoped call, covering only the sources whose
data it actually touched.
Scope
changed. A blanket backfill would also sweep the commit and review tables it never touches.
button. Support reads that log when a member asks where their badge went, and "a scheduled sweep did it"
and "an admin did it" are different answers.
pipeline, which sweeps everything once at the end, so calling it there too would walk the whole commit table
twice per release.
calls rather than following that pattern.
Out of scope
admin or shell action.
all. That override exists for a human who knows the emptiness is real, and it stays that way.
Acceptance criteria
grants
Risks & considerations
This is the change that makes badges appear in production without anyone pressing anything. Everything it runs
is additive - the backfill command cannot delete a grant or revoke a badge - so the failure mode is over-granting rather than data loss.