Skip to content

feat(stacks): map host devices into stack containers from the UI (ADR-0030) - #71

Merged
swimmesberger merged 3 commits into
mainfrom
wt/device-mapping-stack-ui-71210e
Aug 28, 2026
Merged

feat(stacks): map host devices into stack containers from the UI (ADR-0030)#71
swimmesberger merged 3 commits into
mainfrom
wt/device-mapping-stack-ui-71210e

Conversation

@swimmesberger

Copy link
Copy Markdown
Owner

What

Host devices (e.g. /dev/dri/renderD128 for GPU transcoding) can now be mapped into a stack's containers from the stack's Settings tab, without the devices: key ever appearing in the product's repository compose file.

  • ADR-0030 documents the decision, including the deliberate precedence inversion: on a container-path collision the Watchtower mapping wins over the repo's entry, because device paths are per-host facts.
  • StackDeviceMapping entity + stack_device_mappings migration, keyed by compose service name so rows survive redeploys.
  • DeviceMappingPlan — the runtime-neutral policy half (ADR-0010 seam rule, mirroring ImagePinPlan): a mapping naming a service the resolved project lacks becomes a deploy-log warning, never a failed deploy.
  • ComposeOverrideFile.Render gains a third plan and emits a devices: list in Compose's host:container[:permissions] string form. Compose merges devices: by container path, so override entries append to the repo's own. A stack with no mappings renders a byte-identical override to before.
  • stacks.getDevices / stacks.setDevices RPCs. The set is replaced atomically (the stacks.setEnv shape), validated — absolute paths, rwm permission subset, duplicate host paths and duplicate container targets refused — and audit-logged, since mapping a host device is a host-access grant.
  • Settings-tab "Device mappings" editor (service / host device / optional container path / optional access), saved by the existing Save button only when actually edited.
  • The deploy log names every applied device and every skipped mapping.

Why

Device paths are host-specific: which render node exists differs per machine, while the compose file is shared by every stack of the product (ADR-0026). Committing them breaks hosts without the device; env-var pass-through still forces the stanza into the repo. The ADR-0012 generated override is the one place that can add them per stack without touching the repository.

Reviewer notes

  • Out of scope by decision: template-level defaults and NVIDIA gpus/CDI (deploy.resources.reservations.devices is a different mechanism).
  • ADR numbered 0030: 0029 is claimed by the blue-green ADR on its own branch.
  • rpc-schema.json is regenerated (154 methods).

Testing

  • New DeviceMappingPlanTests + ComposeOverrideFileTests device cases; full suite green (1896/1896).
  • Verified live against the dev instance: editor round trip (save → RPC returns stored row with defaulted container path → reload rehydrates), and the validation refusals (relative path, bad permissions, duplicate target) — each errors without wiping stored rows.
  • dotnet ef migrations has-pending-model-changes is clean; frontend tsc + lint clean.

…-0030)

Device paths like /dev/dri/renderD128 are host-specific, so they must not
live in the product's shared compose file. Store them per (stack, service)
in Watchtower instead and render them as a devices: block in the ADR-0012
generated override on every deploy — Compose merges devices: by container
path, so entries append to whatever the repository declares and a per-host
value can deliberately override a repo default.

- StackDeviceMapping entity + stack_device_mappings migration
- DeviceMappingPlan: runtime-neutral policy (ADR-0010 seam), warns instead
  of failing when a mapping names a service the resolved project lacks
- ComposeOverrideFile renders devices: after image/environment; unchanged
  byte-for-byte for stacks with no mappings
- stacks.getDevices / stacks.setDevices RPCs; the set is replaced
  atomically, validated (absolute paths, rwm permissions, duplicate
  targets) and audit-logged as a host-access grant
- Settings-tab "Device mappings" editor riding the existing Save button
- Deploy log names every applied device and every skipped mapping
…oy time (ADR-0031)

Literal device paths were still one notch too concrete for the GPU case:
renderD128 is host-specific, and mapping the node is not enough — the
container user must carry the node's owning group, whose GID differs per
host. Store a host-neutral per-service intent instead and resolve it on
the deploying host.

- HostGpuProbe: short-lived helper container (backup helper image) with
  /dev and /sys bind-mounted read-only; reads each render node's vendor,
  driver, PCI address and owning GID from sysfs; cached 5 min; failure
  degrades to "no GPUs" and never fails a deploy
- StackGpuMapping entity + stack_gpu_mappings migration; replaced
  atomically by the same stacks.setDevices call and audit entry
- DeviceMappingPlan resolves intents to every mappable node plus
  supplementary group ids; NVIDIA nodes are skipped with a note (the
  toolkit, not a device mapping, is the working route); a GPU-less host
  is a note, not a warning
- ComposeOverrideFile renders group_add (quoted GIDs) after devices;
  Compose appends group_add, so repository entries survive
- stacks.hostGpus RPC + Settings-tab "GPU passthrough" editor showing
  what the probe detected on this host
@swimmesberger

Copy link
Copy Markdown
Owner Author

Second commit adds ADR-0031: "map host GPU(s)" as a per-service intent, resolved by probing the Docker host at deploy time.

  • HostGpuProbe runs the backup helper image with /dev + /sys read-only and reads each render node's vendor/driver/PCI address/owning GID from sysfs (deterministic, no heuristics). Cached 5 min; a probe failure degrades to "no GPUs found" and never fails a deploy.
  • The intent maps every mappable render node and emits group_add with the node's GID — closing the "device mapped but VAAPI still fails" permissions trap. NVIDIA nodes are deliberately skipped with a deploy-log note (toolkit/CDI is the working route; a bare device node fails inconsistently).
  • A GPU-less host is a note, not a warning — same stack deploys everywhere, GPU where one exists.
  • Same atomic stacks.setDevices replace + audit entry now carries both lists; new stacks.hostGpus powers the Settings read-out of what the intent would resolve to.

Verified live against the dev instance: the probe found Docker Desktop's virtio-gpu render node end to end (helper container → sysfs parse → UI read-out), and the intent round-trips through the Settings save. 1910/1910 tests green, model-changes check clean.

…ck-ui-71210e

# Conflicts:
#	docs/decisions/README.md
@swimmesberger
swimmesberger merged commit 38c16df into main Aug 28, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant