Skip to content

Let callers supply discovered model capabilities instead of re-resolving from _REGISTRY #2

Description

@jay-m-dev

Problem

_sampling_kwargs() (agentic_core/services/llm_service.py:155-169) is the only place that decides what sampling control goes on the wire, and it re-resolves capabilities from the model string:

caps = get_capabilities(model_str)
if caps.supports_temperature:
    return {"temperature": config.temperature}
if caps.supports_effort and config.effort:
    return {"reasoning_effort": config.effort}
return {}

ModelConfig (agentic_core/schemas/agent.py:31) has no field for capabilities, so a caller that has already determined a model's real capabilities has no way to pass them in. Core looks the model up in _REGISTRY again and ignores what the caller knew.

_REGISTRY currently holds five entries — claude-opus-5, claude-opus-4-8, claude-opus-4-7, claude-fable-5, claude-sonnet-5 — and everything else falls through to DEFAULT_CAPABILITIES (supports_temperature=True). Substring matching covers point releases (claude-opus-5-1 matches claude-opus-5) but not new families.

Why this now produces wrong wire calls

ASAREE reads Anthropic's GET /v1/models, which returns a real per-model capability tree (effort.supported plus a supported flag per level). The live ladders are genuinely more granular than the registry's single hardcoded list:

Model Effort levels (live API)
5-series, opus-4-7, opus-4-8 low / medium / high / xhigh / max
sonnet-4-6, opus-4-6 low / medium / high / max (no xhigh)
opus-4-5 low / medium / high
haiku-4-5, sonnet-4-5 effort.supported: false

For any model outside those five registry entries — claude-opus-4-6, claude-sonnet-4-6, and every model Anthropic ships from here on — the UI now shows an Effort control sourced from the live API, the user picks a level, and core sends temperature instead. The effort choice is silently dropped, or the call 400s if the model genuinely rejects temperature.

The same wall blocks live OpenAI discovery. OpenAI's GET /v1/models returns no capability data at all (id/object/created/owned_by/shutdown_date only, confirmed by live probe), so a consumer has to maintain its own table for the gpt-5.x / o* lines. With no way to hand that table to core, the picker would offer reasoning_effort while core sent temperature — so the work can't be done consumer-side today.

Proposed change

Add an optional capabilities override to ModelConfig and have _sampling_kwargs() prefer it, falling back to get_capabilities(model_str) when it's absent:

caps = config.capabilities or get_capabilities(model_str)

Backward compatible — nothing that omits the field changes behaviour — and model_capabilities stays the default oracle for callers that have nothing better.

Alternative considered

Keep adding _REGISTRY entries for each new model family. Rejected: it means re-tagging agentic-core every time a provider ships a model, which is the treadmill live discovery exists to get off. It also can't work for OpenAI, where the registry would have to encode a request-mode rule (temperature is legal precisely when reasoning_effort is none on the 5.1+ families) that a per-model capability struct can't express.

Notes

  • The module docstring's warning that litellm's get_supported_openai_params is not a reliable oracle still holds — this issue is about letting a caller with a better oracle use it, not about trusting litellm.
  • No cross-provider capability standard exists: Anthropic returns a full capability tree, Azure Foundry's capabilities field only ever holds {"chat_completion": "true"} (a modality, not a sampling contract), and OpenAI returns nothing. Azure would keep using get_capabilities() under this change.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions