Skip to content

feat(cost): config-level rates override for models missing from _RATES - #65

Open
Joi Ito (Joi) wants to merge 1 commit into
microsoft:mainfrom
Joi:feat/config-rates-override
Open

feat(cost): config-level rates override for models missing from _RATES#65
Joi Ito (Joi) wants to merge 1 commit into
microsoft:mainfrom
Joi:feat/config-rates-override

Conversation

@Joi

Copy link
Copy Markdown

Problem

Models absent from the hardcoded _RATES table in _cost.py make compute_cost() return None, so every usage.cost_usd for those models is null. Today this affects all claude-fable-* sessions: in my logs, 129 of 129 fable responses have cost_usd: null. microsoft/amplifier#303 documents the family-detection side of the same gap. Any private, fine-tuned, or newly released model hits this until a code change ships.

Solution

A new rates provider config key supplies rates for models the built-in table does not know:

providers:
  - module: provider-anthropic
    config:
      rates:
        claude-fable-5:
          input: 10.00        # USD per 1M tokens, same units as _RATES
          output: 50.00
          cache_read: 1.00    # optional, defaults to 10% of input
          cache_write: 12.50  # optional, defaults to 125% of input
        claude-fable-*:       # trailing-* glob covers dated ids
          input: 10.00
          output: 50.00
  • Lookup order: exact override, longest matching trailing-* glob override, then _RATES. Config rates take precedence over the built-in table.
  • Cache-field defaults (10% / 125% of input) are the ratios every existing _RATES row uses.
  • Values are parsed via Decimal(str(x)). No float arithmetic touches the cost path.
  • None still means unknown model with no rate anywhere. Unknown is never silently zero.
  • Invalid entries are skipped with a warning, matching the provider's lenient config parsing (_config_int / _config_float style). They never fail the mount.
  • The speed='fast' 2x multiplier logic is unchanged and still applies only to _FAST_ELIGIBLE_MODELS.

_RATES itself is untouched.

Relationship to #58

Complements it. #58 adds fable/mythos rows to _RATES and should still land for out-of-box fable pricing. This PR covers the general case: any model missing from the table, without waiting for a release. The override row shape and the fable example rates here match #58.

Tests

tests/test_rates_override.py adds 21 tests: parse validation (missing/invalid/unknown fields, non-mapping input), derived cache defaults, Decimal precision (no float artifacts), override hit for an unlisted model, precedence over _RATES, unknown model still None, glob matching (exact beats glob, longest prefix wins), fast-multiplier interaction, and three integration tests through _convert_to_chat_response with config-supplied rates.

Local run (uv, amplifier-core 1.6.0 from PyPI): 499 passed, 1 skipped, and 10 pre-existing failures/errors that reproduce identically on clean main in the same environment (test_tool_repair, test_validation, test_behavioral, test_cost_usd_json_serializable::test_usage_model_stores_decimal_internally). No regressions from this change; all cost tests (test_cost.py + new file, 41 tests) pass.

🤖 Generated with Claude Code

Models absent from the hardcoded _RATES table make compute_cost() return
None, so every usage.cost_usd on such models is null. A new 'rates'
provider config key maps model ids (exact or trailing-* glob) to
{input, output, cache_read, cache_write} in USD per 1M tokens. Overrides
take precedence over _RATES; lookup order is exact override, longest
matching glob override, then the built-in table. Missing cache fields
default to 10% / 125% of input, the ratios every _RATES row uses.
Values are parsed via Decimal(str(x)) so no float arithmetic touches
the cost path; unknown models still yield None. Invalid entries are
skipped with a warning, matching the provider's lenient config parsing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant