Skip to content

OTel spans that report the provider host (api.<provider>.<tld>) are billed at zero cost #7772

Description

@Anuj7411

Problem

OpenTelemetryMapper sets the span provider directly from the OTel PROVIDER attribute (provider = value.getStringValue()), then CostService.findModelPrice looks up pricing by PROVIDERS_MAPPING.get(provider). That mapping keys on the canonical litellm_provider name (cerebras, xai, deepseek, ...).

When an OpenAI-compatible SDK is pointed at a provider via its base URL, some instrumentations report the host rather than the canonical name, so the span carries PROVIDER = "api.cerebras.ai" (or api.x.ai, api.deepseek.com, api.groq.com, ...). None of those hosts are in PROVIDERS_MAPPING, so buildModelPrice returns null, the model price is dropped, and the span is billed at DEFAULT_COST (zero). Cost tracking and the per-evaluation spend budget silently under-report.

This is not specific to any one provider; every OpenAI-compatible entry (the merged xai / deepseek / moonshot / sambanova and the in-review cerebras / nebius / etc.) shares the gap.

Proposal

Normalize api.<provider>.<tld> host aliases to the canonical provider centrally, as a resolver step in the OTel ingestion path (alongside the existing ElasticInferenceServiceResolver and GoogleProviderResolver in OpenTelemetryMapper), or as a normalization step in CostService.findModelPrice before the PROVIDERS_MAPPING lookup. A single host-stripping rule (api. prefix + public-suffix tail -> registered canonical key, when the stripped label matches a known provider) fixes every provider at once and stays consistent as new providers are registered, rather than adding one-off host aliases per PR.

Repro sketch

// bare cerebras model, provider reported as the host
CostService.calculateCost("llama-3.3-70b", "api.cerebras.ai", usage, null); // -> 0 (dropped)
CostService.calculateCost("llama-3.3-70b", "cerebras", usage, null);        // -> expected non-zero

Surfaced by a Baz review comment on #7731. Filing separately so the fix lands centrally instead of scope-creeping a provider-registration PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions