Skip to content

Discover OSS/MLflow model capabilities and persist bounded metadata #332

Description

@dgokeeffe

Problem

The OSS/MLflow provider currently relies mainly on a static validated model cohort. AI Gateway foundation-model metadata can expose additional chat-completions-only models, reasoning capability, and context information, but the data is partial and must not leave clients uncapped or route native Claude/Gemini/GPT endpoints through MLflow.

Expected behavior

  • Discover MLflow chat-completions-only foundation models dynamically.
  • Project live capabilities onto matching UC model-service IDs.
  • Persist reasoning, context-window, and output-limit metadata for Pi and OpenCode.
  • Exclude endpoints that expose native Anthropic, Gemini, or Responses APIs.
  • Deduplicate malformed/repeated endpoint entries.
  • Keep static GLM/Kimi behavior when metadata is unavailable.
  • Complete partial/unknown limits with a conservative bounded pair.
  • Keep SSE repair and cache diagnostics in separate changes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions