Skip to content

Model drop-in: no model names outside adapters; conformance suite for every kind - #131

Merged
jayleaton merged 6 commits into
mainfrom
refactor/model-dropin
Oct 28, 2026
Merged

jayleaton merged 6 commits into
mainfrom
refactor/model-dropin

Conversation

@jayleaton

Copy link
Copy Markdown
Owner

Summary

A model of a kind LocalRouter already serves is now one adapter plus one config entry. MCP, the OpenAI routes and the voice loop name no model. The audit, the contract each kind follows and the ranked gaps are in docs/dev/MODEL-DROPIN.md.

client ── MCP tool / OpenAI route / voice loop        (kind + capability, never a model name)
              │
         routes.resolve ── config `defaults` ── engine.check (declared limits: 400 before load)
              │
         scheduler ── worker protocol ── adapter (engine.Entry | cmd | openai_server)
Gap (risk) Before After
Voice clone (high) daemon required Fish's 10 codebook rows and 44.1 kHz clips clone_codebooks and clone_rate tool options; the Jarvis template sets clone_codebooks: 10
Voice loop speech (high) voice.speech_models defaulted to ["fishaudio/drama-3", "kitten-tts-2"] in code: Kitten beat a newly configured TTS default defaults to empty: the text_to_speech default speaks; a box that wants a chain lists it (Jarvis and DGX-02 already do)
Model-manager prompt (medium) fixed roster: "Drama 3, the voice (21 GiB); Whisper … 120 GiB" {models} rendered from config: role, id and needed memory of each model
STT languages (medium) Whisper's list hard-coded in /voice and the companion languages option: listed in /v1/models / list_models, refused outside it; /voice and the companion read it (Whisper's list as offline fallback)
Names in descriptions (low) MCP instructions said "Drama 3 hears it…"; decide.Qwen; edit-pack hint on every refusal model-neutral wording; ChatDecider; hint only for image_edit

The MCP tools/list stays a stable, kind-level list. A tool with no capable model is still listed and refused cleanly with the models named. That is the existing contract tests/mcp.zig pins, and a swap leaves the list unchanged. The voice loop already offers its chat model only the tools a model can serve.

The new conformance suite also found a latent bug, fixed here. MCP decide with no model could fail with "state is required" after the decision had already run. api/decide.zig put the model into a shallow copy of the caller's map, and the put could move the entries out from under the caller.

Evidence

  • Conformance (tests/dropin.zig, new). For every kind, set A and set B differ in ids and, where a fake exists, in adapter:
    • tools/list and /v1/capabilities tools are byte-identical across the two sets;
    • /v1/models lists exactly the configured models;
    • all six OpenAI routes and the MCP job tools work with and without a model named;
    • no-edit and no-i2v models are refused cleanly;
    • defaults alone moves an unnamed request to the other model;
    • a realtime voice turn works on either set;
    • the voice loop speaks with the TTS default unless speech_models names a chain;
    • declared languages are listed and enforced before load.
  • Zig, DGX-02 checks clone (/root/lrclone, CPU only): fmt exit 0, build exit 0, Build Summary: 22/22 steps succeeded; 297/297 tests passed (227 unit, 70 integration).
  • Companion (npm run type-check && npm run lint && npm test, dev box, because the Davit host was below its disk floor): all exit 0, Tests 422 passed | 15 skipped. Mobile code is unchanged (it sends no language); CI checks it.
  • Before/after on the live DGX-02 config:
    • Method: main 87ea48b and this branch, both built in the checks clone, each served the box's own config.json with keep_loaded off. Nothing loaded and the service stayed stopped.
    • tools/list, /v1/models, list_models, list_voices and /v1/capabilities are identical except one line:
      - "description": "How to say it (delivery, emotion, pace), for a model whose list_models entry says instructions: true; others refuse it. Drama 3 hears it as a bracketed direction before the text: [whispering, nervous].",
      + "description": "How to say it (delivery, emotion, pace), for a model whose list_models entry says instructions: true; others refuse it. The model's own template puts it into the text, such as a bracketed direction before it: [whispering, nervous].",
    • /v1/skills/model-manager (the voice prompt's roster) now comes from config, and its sizes are more accurate (image 42 not 35 GiB, video 52 not 40):
      - The machine has about 120 GiB for models: you, the chat model (24 GiB); Drama 3, the voice (21 GiB); Whisper, the ears (4 GiB); the image model (35 GiB); the decision model (12 GiB); the video model needs about 40 GiB.
      + The machine has about 119 GiB for models: you, the chat model qwen3.8-27b (22 GiB); the voice fishaudio/drama-3 (21 GiB); the ears whisper-large-v3-turbo (4 GiB); the image model qwen-image-2.1-turbo (42 GiB); the video model minimax-h3 (52 GiB); the decision model knowline-4b (12 GiB).

Merge Danger

Door: two-way

There is no wire-shape change, no model change and no download. One tools/list description is reworded, and two new optional fields (languages, and the clone options) appear only when configured. Reverting the merge restores the old behaviour.

Blast Radius: voice

  • A box that relied on the old built-in speech chain without listing voice.speech_models now speaks with its text_to_speech default. Jarvis and DGX-02 list it, so nothing changes there.
  • The voice prompt's roster text changes; it is accurate now, but voice calls see different wording.
  • A clone on a box without clone_codebooks accepts any consistent row count. The Jarvis template keeps 10.

🤖 Generated with Claude Code

jayleaton and others added 6 commits October 28, 2026 22:47
…roster from config

- voice.speech_models defaults to empty: the text_to_speech default speaks alone
- voice cloning: clone_codebooks and clone_rate tool options instead of Fish's 10 rows at 44.1 kHz
- the model-management prompt's roster is rendered from config (ids, roles, needed memory)
- transcription tools declare languages: listed, refused outside them, /voice narrows its picker
- model-neutral MCP synthesize_speech instructions description; decide.Qwen -> ChatDecider
- docs/dev/MODEL-DROPIN.md: the audit, the contract and the ranked gaps

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tests/dropin.zig runs the same checks on two model sets (other ids; testpattern vs the cmd speech worker for speech
and transcription; text-only image and video models in set B): MCP tools/list and /v1/capabilities' MCP tools are
identical, /v1/models lists exactly the configured models with their kinds and capabilities, every OpenAI endpoint,
/v1/systemone and MCP tool works with the model named and picked by defaults, a missing capability is refused naming
the model and what it can do, changing only defaults moves unnamed requests, and the voice loop takes either set's
speech and transcription models and runs a turn on them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d by config

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…edit pack only for image_edit

MCP decide reads the questions again after the job: a put on a shallow copy of its map could move the
entries out from under it ("state is required" after the decision had run, seen by tests/dropin.zig).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jayleaton
jayleaton marked this pull request as ready for review October 28, 2026 16:20
@jayleaton
jayleaton merged commit 020816c into main Oct 28, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant