Skip to content

Add LM Studio as a keyless local model provider. - #184

Open
sobiya-22 wants to merge 1 commit into
andrewyng:mainfrom
sobiya-22:feat/lm-studio-provider
Open

Add LM Studio as a keyless local model provider.#184
sobiya-22 wants to merge 1 commit into
andrewyng:mainfrom
sobiya-22:feat/lm-studio-provider

Conversation

@sobiya-22

@sobiya-22 sobiya-22 commented Jul 26, 2026

Copy link
Copy Markdown

Fixes #122

Summary

Adds LM Studio as a first-class keyless local model provider, following the same pattern as Ollama.

  • Register lm_studio in the provider registry with default endpoint http://localhost:1234/v1
  • Wrap OpenAIProvider with placeholder key lm-studio and configurable base_url
  • Keyless verification via GET /v1/models (no auth header)
  • Live model discovery and liveness gating for lm_studio:* entries in the model picker
  • Conservative capability defaults for local models (same as Ollama)
  • Provider-specific Settings help text and gallery ordering

LM Studio models are not added to matrix.py — users load arbitrary GGUF models locally. Models can be discovered after Detect (when the local server is running) or added manually via the existing "Add another model…" field.

Test plan

  • pytest tests/test_provider_router.py tests/test_provider_verify.py tests/test_settings.py tests/test_server.py
  • Start LM Studio, load a model, enable local server
  • Settings → LM Studio → Detect → card shows "✓ Running"
  • Configure Models → suggested loaded models appear → tick one → use in composer as lm_studio:<model>
  • Send a chat message and confirm completion works

Mirrors the Ollama pattern so users can run open-weight models via
LM Studio's OpenAI-compatible localhost API without cloud keys.
@LifetimeVip

Copy link
Copy Markdown

Support

@zalex666

zalex666 commented Aug 3, 2026

Copy link
Copy Markdown

Nice work on this — and apologies, I opened #434 for the same feature before spotting this PR (should have searched first). Laying out the delta here so maintainers can pick a direction easily.

#434 follows the same core pattern (keyless descriptor + OpenAIProvider at <root>/v1, liveness-gated picker entries via /v1/models) and additionally covers:

  • Keyless first-Detect persistence. With the endpoint field untouched, a passing Detect stores no profile, and _lm_studio_models() gates on a stored profile — so the live model list and suggestions stay empty until the endpoint is manually edited. (Add LM Studio as a local model provider (and fix keyless-provider discovery) #206 described the same latent bug for Ollama.) Fixed GUI-side (Detect always persists for keyless providers) plus backend (is None gate so an empty stored profile still probes the default URL) — that's what makes "Detect → suggested models appear" work at the default URL.
  • model_ready and the default-model handoff gate on actual liveness (_provider_available), so a dead local server no longer reads as ready and no longer blocks a working provider from taking the default.
  • Probe/verify check the response shape (data/models list), not just any 2xx — a non-LM-Studio service on :1234 no longer passes Detect.
  • Liveness-cache invalidation on endpoint changes; an explicitly-cleared endpoint verifies the default instead of the stored URL; failed saves surface in the GUI instead of showing "✓ Tested & saved".
  • The lobe-icons brand mark (this branch adds lm_studio to PROVIDER_ORDER but not PROVIDER_LOGOS, so the card renders the monogram fallback), a README mention, and a recommended_model for the post-Detect auto-add.
  • Rebased on current main (this branch is showing conflicts at the moment).

Validated #434 live against LM Studio: tool-calling round trips on two models incl. JIT load and a thinking model's reasoning_content streaming.

Zero attachment to which PR lands — if maintainers prefer this one, everything above can be cherry-picked here and I'll close #434.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Support LM Studio as a local model provider

3 participants