A self-hosted inference engine item, so the external node has a real drop-in alternative

[justinmerrell]

A customer needing a language model has three ways to get one, and two of them are already expressible:

1. A Musher inference endpoint — an external node whose address happens to be ours.
2. Any third-party OpenAI-compatible service — the same external node, a different address.
3. An engine they run themselves — a SERVICE component on a gpu.* or accelerator.* Compute Profile.

The third needs no spec change at all. Blueprint §4.3 already says of those families that economy through premium "run entry inference to frontier training". What is missing is an item.

Why it is worth building

It makes path 3 a real alternative rather than a theoretical one, and it is what demonstrates that the external shape is not a euphemism for "our endpoint". A self-hosted engine publishing outputs with the same three resourceTypes is a drop-in substitute for the external node:

# swap this…
models:
  componentRef: ./components/models.yaml   # spec.external
  size: null

# …for this, and the consumers wires do not change
engine:
  componentRef: ./components/vllm.yaml     # a SERVICE
  size: gpu.performance.large

The consumer’s connections are keyed by its own input names and name fromRole/fromOutput, so as long as the engine publishes baseUrl and apiKey-shaped outputs carrying dev.musher.llm.base-url and dev.musher.llm.api-key, nothing else in the blueprint moves. That substitutability is the property worth proving with a fixture-like item rather than asserting in a doc.

Shape

• A SERVICE component, pinned image, one endpoint, a readiness probe.
• Outputs: the private base URL (valueFrom: DERIVED), a served-model name (DECLARED), and a token if the engine takes one — a generated input republished with valueFrom: INPUT, which is exactly what that member is for.
• A gpu.* profile, and placement pins for the accelerator if the engine needs a specific one.

Open questions for whoever picks it up

• vLLM or Ollama. vLLM is the closer fit for an OpenAI-compatible surface; Ollama is friendlier and already the thing open-webui’s abandoned default pointed at.
• Model weights. The image does not carry them. Whether they are pulled at boot onto a volume, or the item asks for a model id and accepts a cold first request, is the real design decision and it is bigger than the component document.
• Which gpu.* tier is actually offered — blueprint §4.3 makes a grammatical slug no guarantee of an available one, and an unavailable profile is a capability failure at deploy.