Skip to content

feat(stacks): resolve GPU intents to an NVIDIA reservation too (ADR-0032) - #73

Merged
swimmesberger merged 1 commit into
mainfrom
feat/nvidia-gpu-intent
Aug 29, 2026
Merged

feat(stacks): resolve GPU intents to an NVIDIA reservation too (ADR-0032)#73
swimmesberger merged 1 commit into
mainfrom
feat/nvidia-gpu-intent

Conversation

@swimmesberger

Copy link
Copy Markdown
Owner

Draft — this amends ADR-0031's decision 3, so it's a design call for you before it's a code review.

ADR-0031 skipped NVIDIA, and the reason it gave is right: mapping /dev/nvidia* isn't enough, because the container also needs the toolkit-injected user-space driver. What it didn't follow through on is that the supported NVIDIA route is itself an intent:

deploy:
  resources:
    reservations:
      devices:
        - driver: nvidia
          count: all
          capabilities: [gpu, video]

No paths, no GIDs, nothing host-specific. That fits ADR-0031's host-neutral model better than render nodes do — so NVIDIA was excluded from the transcoding feature on the hosts most likely to be transcoding boxes.

Two things made the exclusion misleading, not just incomplete

NVIDIA is usually invisible to the probe. It only appears under /sys/class/drm when nvidia-drm is loaded — common on desktops, not guaranteed on the headless hosts that hold the cards. So the "needs the container toolkit" note fired where it was least needed, and a real NVIDIA server got:

No mappable host GPU was detected — 'transcoder' gets no GPU devices on this host.

True, unhelpful, and with no hint that a working route exists.

The toolkit's presence is knowable. It registers itself as a Docker runtime, which GET /info reports. That matters because emitting a reservation without it doesn't degrade — Compose fails the entire deploy with could not select device driver "nvidia" with capabilities: [[gpu]]. So NvidiaUsable requires both the card and the runtime; a card without the toolkit reserves nothing and says why.

What changed

  • No new row, no new control. stack_gpu_mappings already means "give this service the host's GPUs"; resolution now picks the mechanism per host — render nodes plus group_add where they exist, a reservation where NVIDIA does. A stack moving from an Intel host to an NVIDIA one changes nothing.
  • HostGpuProbe reports /dev/nvidiactl (exists whenever the kernel driver is loaded) and reads the daemon's runtimes.
  • ComposeOverrideFile emits the reservation. capabilities includes video — the toolkit only injects the encoder libraries when asked, so the usual [gpu] or compute,utility spelling leaves CUDA working and NVENC failing to open. Video transcoding is the motivating workload, so that one isn't optional.
  • The GPU-less note no longer fires on an NVIDIA host; the Settings read-out names the toolkit state.

Reviewer notes

  • Runtime-neutral per ADR-0010: ServiceDeviceMappings.NvidiaGpus says the service wants the vendor's GPUs; only ComposeOverrideFile knows Compose spells that as a device reservation. Kubernetes says nvidia.com/gpu.
  • Compose honours deploy.resources.reservations.devices outside Swarm, unlike most of deploy:. Genuinely surprising, so it's commented at the emitter rather than left looking like a mistake.
  • count: all matches ADR-0031's choice for render nodes; single-card selection would be device_ids and is deferred for the same reason.
  • One bug the tests caught: a C# raw string literal carries the source file's line endings into the emitted document, so the block wrote CRLF on a Windows checkout while every other line in that file appends '\n'. The emitter now writes \n explicitly.

Verification

dotnet build Watchtower.slnx clean (0 warnings). Application tests: 1900 passed, 23 failed — the same 23 fail on the base commit with these changes stashed (ACME/certificate X509 chain building on this Windows box), so no regressions; +13 new tests. Frontend generate:rpc + typecheck clean, build succeeds. rpc-schema.json regenerated — the only diff is the new nvidia object on stacks.hostGpus.

Not verified against real NVIDIA hardware, same as the rest of the GPU feature. The emitted block is the documented Compose form; the video capability comes from observed toolkit behaviour rather than assumption, but a real Docker host with the toolkit would be the honest end-to-end check before this leaves draft.

…032)

ADR-0031 skipped NVIDIA, and its reason was right: mapping /dev/nvidia*
is not enough, because the container also needs the toolkit-injected
user-space driver. What it did not follow through on is that the
supported NVIDIA route is itself an intent - a Compose reservation
carries no paths and no GIDs, so it fits the ADR's host-neutral model
better than render nodes do, not worse.

Two things made the exclusion actively misleading rather than merely
incomplete. NVIDIA is usually invisible to the probe, because it only
appears under /sys/class/drm when nvidia-drm is loaded - common on
desktops, not guaranteed on the headless hosts that hold the cards. So
the "needs the container toolkit" note fired where it was least needed,
and a genuine NVIDIA server was told "No mappable host GPU was detected",
with no hint that a working route exists. And the toolkit's presence is
knowable: it registers itself as a Docker runtime, which GET /info
reports.

- HostGpuProbe reports /dev/nvidiactl, which exists whenever the kernel
  driver is loaded, and reads the daemon's runtimes; NvidiaUsable needs
  both, because a reservation on a host without the toolkit does not
  degrade - Compose fails the whole deploy
- DeviceMappingPlan resolves the existing intent per host: render nodes
  plus group_add where they exist, a reservation where NVIDIA does. No
  new row, no new control - a stack moving between an Intel host and an
  NVIDIA one changes nothing
- ComposeOverrideFile emits deploy.resources.reservations.devices with
  capabilities [gpu, video]. "video" is not optional here: the toolkit
  only injects the encoder libraries when asked, so the usual [gpu]
  spelling leaves CUDA working and NVENC failing to open
- The GPU-less note no longer fires on an NVIDIA host, and the Settings
  read-out names the toolkit state

The emitter writes its lines with explicit \n rather than a raw string
literal, which would carry the source file's CRLF into the generated
document on a Windows checkout - caught by the render test.
@swimmesberger
swimmesberger marked this pull request as ready for review August 29, 2026 13:47
@swimmesberger
swimmesberger merged commit b1279d3 into main Aug 29, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant