feat(ci): show the live runner containers on the product's CI tab - #63
Merged
Conversation
Runner state was two numbers — "1 of 1 runner slot(s) live" — and nothing else. The containers behind them were only visible over SSH with `docker ps --filter label=watchtower.managed=ci-runner`, which is a poor answer to "is my runner actually up, and is it running the settings I just saved". (The design doc already promised container ids on `ci.getRunnerStatus`; they were never implemented.) `ci.getRunnerStatus` — and with it every response carrying a CiRepoDto — now includes the repo's runner containers: name, short id, Docker state and uptime, image, and the runner's id at GitHub. A container whose spec hash no longer matches the repo's settings is flagged stale, which the CI tab badges "settings changed": the operator can see that a saved change has not reached the running runner yet and that it is queued to be retired, instead of wondering why nothing happened. The rows are read off the host's containers and their Watchtower labels rather than from any new database state — the containers are the state (docs/ci-runners/design.md), so a runner left behind by a previous Watchtower process shows up exactly as it is. The snapshot is taken after each repo's reconcile pass rather than from the listing that opened it, so it reflects what that pass just reaped, retired and spawned.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Runner state in the UI was two numbers — "1 of 1 runner slot(s) live" — plus
the last orchestration error. The containers behind those numbers were only
visible over SSH with
docker ps --filter label=watchtower.managed=ci-runner, which is a poor answerto "is my runner actually up, and is it running the settings I just saved".
docs/ci-runners/design.mdalready promised container ids onci.getRunnerStatus; they were never implemented. Now they are, along with thetable that renders them.
What's new
ci.getRunnerStatus— and with it every response carrying aCiRepoDto, whichis what the CI tab actually polls — gains
runners[]: per container the name,short id, Docker state and status, start time, image, and the runner's id at
GitHub. The CI tab renders them in a
DataListbelow the slot count, with theGitHub id linking to that runner in the repository's Actions settings.
A container whose spec hash no longer matches the repo's current settings is
flagged
staleand badged settings changed. That closes the loop on therecycling added in #62: an operator who ticks "allow docker socket" can see
that the change has not reached the running runner yet and that it is queued to
be retired, rather than wondering why nothing happened.
How the data is obtained
Read off the host's containers and their Watchtower labels — no new database
state, no runner table. The containers are the state (per the design doc), so
a runner left behind by a previous Watchtower process shows up exactly as it
is, and the UI cannot drift from reality.
The snapshot is taken in the reconcile loop after each repo's pass, not from
the listing that opened it, so it reflects what that pass just reaped, retired
and spawned — the operator's next poll shows the runner that was created rather
than the one that is already gone. It sits outside the pass's try/catch, so a
failed reconcile still leaves an accurate picture. A snapshot failure is not a
reconcile failure: it logs at debug and leaves the previous list in place.
Notes for the reviewer
GET /containers/jsonper repo per reconcile interval (default15s), filtered by label. Local socket call; the loop already makes three
unfiltered ones per pass.
DockerContainerInfogainsCreated(Unix seconds — an integer on/containers/json, unlike the RFC3339 string the inspect endpoint returns),which is where uptime comes from.
status.Updateisoptimistic after a spawn, the snapshot is observed. Documented on the DTO
rather than papered over.
torn down, so "disabled" doesn't look like "already gone".
rpc-schema.jsonis regenerated; the diff is large becauseCiRepoDtoisinlined into every method response that returns it.
Verification
Release build clean, 0 warnings.
rpc-schema.jsonregenerates identically(the CI staleness check will pass). Backend unit tests for the new projection
pass — including that an unlabelled container counts as stale, since the loop
is going to retire it.
The frontend is not typechecked locally: there is no Node installed on this
machine (
npm run buildin CI runsgenerate:rpc+tsc --noEmit+vite build, so CI is the gate). Likewise the Docker-dependent backend suites can'trun here — no Docker daemon on this box.