Skip to content

[Feature] mcode provider add should support context and output limits for custom/local providers #161

Closed

Description

@mrefaie

Problem

I configure a local llama.cpp server as a custom provider with:

mcode provider add --name my-provider --base-url https://example.com/v1 \
  --api-format openai-completions --model my-model \
  --api-key-env MCODE_PROVIDER_API_KEY --use

There is no CLI way to set the model's context window (ctx) or output limit at creation time.

This matters because model discovery (ModelDiscoveryClient in packages/local-runtime-v2/src/service/model-system/connectivity/discover-models.ts) only discovers modelId/displayName from GET /v1/models or /models — it does not probe num_ctx / n_ctx / context_length. At resolve time planCustomProviderResolution() in packages/local-runtime-v2/src/service/model-system/resolution/model-resolver-byok.ts falls back to BYOK_FALLBACK_MODEL_LIMITS (200_000 ctx / 16_384 output) when modelConfig.limit is unset. So a local model started with e.g. --ctx-size 32768 is silently resolved with the wrong window unless the user edits config.yaml manually.

Current chain

  • packages/tui/src/cli/program.ts (provider add): only --name, --base-url, --api-format, --model <id> (repeatable string[]), --api-key-env, --use.
  • packages/tui/src/cli/provider-command.ts (McodeProviderCliRequest): models: readonly string[], mapped as models.map((modelId) => ({ modelId })).
  • Downstream already supports limits: McodeProviderModelInput.limit?: { context?: number; output?: number } in packages/tui/src/provider/contract.ts, consumed by modelsFromInputs() / normalizeModelLimit() in packages/local-runtime-v2/src/service/model-system/management/service-input.ts and customProviderLimits() in model-resolver-byok.ts.

So this is a CLI-surface gap only.

Proposal

Extend mcode provider add with e.g.:

mcode provider add --name my-provider --base-url http://localhost:8080/v1 \
  --api-format openai-completions --model my-model \
  --api-key-env MCODE_PROVIDER_API_KEY \
  --context-limit 32768 --output-limit 4096 --use
  • Validate as positive safe integers, consistent with existing normalizeModelLimit() / updateLocalModelSelection() validation.
  • Document semantics for repeatable --model: simplest is "same limits apply to all listed models".
  • Consider whether provider update / existing updateUserModelParameters() path also needs CLI exposure, or if add-time support is sufficient for v1.
  • Update README.md + README_ZH.md + docs/examples.md (English primary per CONTRIBUTING.md).

Acceptance

  • mcode provider add ... --context-limit N --output-limit M persists custom_provider.<key>.models.<id>.limit = { context: N, output: M } in <dataDir>/config.yaml.
  • mcode provider list --json / provider view shows the configured limits.
  • Invalid (zero/negative/non-integer) values are rejected with a clear error.
  • Existing behavior without the new flags is unchanged.

Activity

  1. hetaoBackend commented on Sep 18, 2026

    @hetaoBackend
    Collaborator

    Implemented in #169. Local validation and all three platform CI checks passed, but GitHub currently blocks merging pending the required independent approval. The PR has not merged yet.

    mcode provider add now accepts --context-limit and --output-limit, for example --context-limit 32768 --output-limit 4096. Both require positive safe integers and can be supplied independently. The same limits apply to every repeated --model.

    The values persist in each model's limit configuration and appear as contextLimit and maxOutputTokens in mcode provider list --json. Omitting the flags preserves existing defaults. The English/Chinese READMEs and examples now document the behavior; provider-update support remains outside this change.

    Verified with the built CLI using an offline HTTP fixture: YAML/JSON readback, repeated models, each flag independently, invalid input rejection, and unchanged no-flags behavior. Full local verification passed all 14 macOS gates, with startup/smoke and BYOK tests rerun before the merge attempt. Keeping this issue open until the PR merges; no new npm release has been published as part of this change. Thanks for the detailed proposal.

  2. hetaoBackend commented on Sep 18, 2026

    @hetaoBackend
    Collaborator

    Update: #169 has now merged as 149d764, and this issue was closed by the merge. The approval blocker mentioned above no longer applies.

    The fix is now in main. Local built-CLI tests and Linux/macOS/Windows CI passed. This source merge does not itself publish a new npm version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions